Comparison

Same workload. Different broker.

On the tested synthetic workload, predictive scheduling beats RabbitMQ and Redis FIFO, while Harbinger has higher broker cost. The v4 follow-up isolates aging's contribution and tests a stronger scheduling baseline.

Summary

Two questions, two answers

−21.7%mean latency vs RabbitMQ FIFO
−25.5%mean latency vs Redis FIFO
−57.6%mean latency with the predictor as a RabbitMQ sidecar
5,293/sHarbinger's peak throughputRabbitMQ 7,371, Redis 21,182

Scheduling outcome

A win against the tested FIFO baselines

On the same production-flow workload, predictive routing cut mean latency by more than 20% against both FIFO brokers and the short-job median by about 90%. The gate passed.

Broker cost

Harbinger the broker loses

With empty handlers it delivers about 5,300 messages per second, 72% of RabbitMQ and a quarter of Redis, and falls behind at 20,000 per second.

Method

The setup

Identical producers, handlers, workers, arrival schedule and seeds. Only the broker and its policy change. The comparison was pre-registered and frozen before any registered run.

The eight arms of the scheduling comparison
Arm	Policy
Harbinger FIFO	One level
Harbinger static, tuned	A well-configured static priority map
Harbinger predictive	Per-key predictor, strict priority, default aging (primary arm)
Harbinger predictive, weighted	Predictor + level weights [8, 3, 1] + pausing aging
RabbitMQ FIFO	Classic queue, manual ack, prefetch 1
RabbitMQ priority	Max priority 3, the same static map, no aging
RabbitMQ predictive	Priorities from a sidecar running Harbinger's own predictor
Redis FIFO	Reliable list queue: LPUSH, BLMOVE to a per-worker list, LREM

Ground rules

AWS us-east-1a in one placement group. Broker on a c8i.xlarge (4 vCPU), clients on a c8i.2xlarge (8 vCPU), with a fresh broker container for every arm, seed and rate.
Versions: RabbitMQ 3.12.1 (Erlang 25.3), Redis 7.0.15. Default settings except those listed.
Nothing is durable on any broker: Harbinger is in memory, RabbitMQ uses transient messages with no publisher confirms, and Redis has no RDB or AOF.
RabbitMQ runs with prefetch 1, which is how its priorities reorder anything at all, and which also limits its throughput.
Retries follow each broker's own rules: near the head of RabbitMQ's queue, the tail of Redis's, and the original priority on Harbinger.

Question 1

Scheduling outcome

The production flow: nine job types, a flash sale and a mid-run cost shift, five seeds, 300 s of arrivals and 4 workers. The primary arm is Harbinger predictive.

Mean latency, median across seeds (s). Lower is better.

Harbinger FIFO

7.32 s

Redis FIFO

7.01 s

RabbitMQ FIFO

6.64 s

Harbinger predictive

5.30 s

RabbitMQ priority

4.15 s

Harbinger weighted

3.14 s

RabbitMQ + sidecar

2.84 s

Grey bars are FIFO. Bars with a dot are Harbinger's own routing. The paired changes are medians of per-seed differences, so they differ slightly from the ratio of the medians.

The gate. Primary arm: Harbinger predictive. Paired by seed, bootstrap 95% intervals.
Criterion	Estimate [95% interval]	Result
P1 mean vs RabbitMQ FIFO (≤ −15%)	−21.7% [−26.2, −18.4]	Pass
P2 short-job median vs RabbitMQ FIFO (≤ −30%)	−89.5% [−98.8, −75.5]	Pass
P3 mean vs Redis FIFO (≤ −15%)	−25.5% [−29.5, −22.6]	Pass
P4 short-job median vs Redis FIFO (≤ −30%)	−90.7% [−99.0, −78.0]	Pass
G1 all-message P99 vs RabbitMQ FIFO (≤ 1.25×)	1.10× [1.07, 1.12]	Pass
G2 export max wait vs RabbitMQ priority (≤ 1.0×)	0.20× [0.18, 0.23]	Pass
G3 no lost work	Every message completed	Pass
G4 throughput vs RabbitMQ FIFO (≥ 0.98×)	1.00× [0.99, 1.00]	Pass

What it shows

The gate passes by less than expected. The pre-registration expected −25% to −28%, and against RabbitMQ it is −21.7%. Part of the gap is Harbinger's own FIFO arm running 11% slower than RabbitMQ's, just outside the ±10% bar.
The idea transfers: on this workload the RabbitMQ sidecar cut mean latency by 57.6% against its FIFO, versus Harbinger predictive at −21.7% and weighted at −52.4% against RabbitMQ FIFO.
This original cross-broker experiment does not isolate aging. RabbitMQ priorities never age, while Harbinger's default aging promotes waiting long jobs. Separate v4 experiments now measure the cost of that protection within Harbinger.

v4 follow-up: aging's cost is now isolated

A preregistered synthetic production-flow experiment lowers Harbinger's mean another 37.4% [35.1, 39.8] by turning aging off, reaching −57.6% vs its own FIFO. That supports aging as the mechanism behind the sidecar gap. The matching −57.6% figures are not a paired equivalence test: the experiments use different host layouts and baselines. Without aging, P99 and long-class maximum wait each rise 13.6% vs FIFO. Separate Huawei replay also recovers gains, but its 20–25-minute runs cannot establish starvation freedom over hours.

Aging isolation results ↗Tail costs and real-trace isolation →

Protecting long jobs

Static priorities without aging greatly increase long-job waits here. The longest export waited 101 s on RabbitMQ priority, 5.2× FIFO. Harbinger's predictive routing held that wait to a fifth of it. Finite observed waits do not establish general starvation freedom for either policy.

Longest export wait, median across seeds (s)

RabbitMQ FIFO

18.3 s

RabbitMQ + sidecar

20.9 s

RabbitMQ priority

101.0 s

Secondary criteria, reported but not gated
Criterion	Estimate [95% interval]	Result
S1 Harbinger FIFO vs RabbitMQ FIFO, mean (±10%)	+11.0% [+10.4, +11.6]	Fail
S2 Harbinger FIFO vs Redis FIFO, mean (±10%)	+5.6% [+4.9, +6.2]	Pass
S3 RabbitMQ predictive vs RabbitMQ FIFO, mean (≤ −15%)	−57.6% [−58.6, −56.8]	Pass
S4 RabbitMQ predictive vs RabbitMQ priority, mean (≤ +10%)	−26.3% [−31.5, −21.1]	Pass
S6 Harbinger weighted vs RabbitMQ FIFO, mean (≤ −15%)	−52.4% [−53.4, −51.0]	Pass
S7 Harbinger weighted vs RabbitMQ FIFO, P99 (≤ 1.25×)	1.21× [1.20, 1.24]	Pass
S8 RabbitMQ priority vs RabbitMQ FIFO, export max wait (≤ 2×)	5.23× [4.79, 5.55]	Fail, expected

Question 2

What each broker costs

Empty handlers, 64-byte payloads, 4 producers and 4 consumers, 30 s runs with a 5 s warm-up and five repetitions. This tests the broker itself, with no scheduling in play.

Peak sustained delivery rate, median (messages per second)

Harbinger

5,293

RabbitMQ (prefetch 1)

7,371

Redis

21,182

At a fixed rate

Offered 5,000 msg/s. Kept up means no loss, at least 99% of the rate delivered, and sends on time.
Broker	Kept up	P50	P99
Harbinger	5/5	0.30 ms	10.6 ms
RabbitMQ	5/5	0.12 ms	0.31 ms
Redis	5/5	0.10 ms	0.31 ms

Harbinger is at its limit. Two of its five runs built a queue (P50 91 ms and 462 ms, P99 446 ms and 1,227 ms). The other three stayed at 0.30 ms P50.

What it shows

Harbinger is the slowest of the three, as the pre-registration expected: about 72% of RabbitMQ's throughput and a quarter of Redis's.
Its latency matches the others only at 1,000 msg/s. The pre-registration expected the same sub-millisecond range at 5,000 msg/s, and that held in only three of five runs.
RabbitMQ's undelivered messages at peak rate need care. It accepted about 56,000 publishes per second but delivered about 7,400, and 8–14% of messages were still undelivered when the 60 s drain ended. Without publisher confirms the benchmark cannot tell whether they were still queued, so they count as not delivered. Every fixed-rate RabbitMQ run lost nothing.

Conclusions

Taken together

The scheduling idea is separable from Harbinger's broker. The same predictor lowers mean latency on RabbitMQ by more than half and beats a hand-tuned RabbitMQ priority map by 26%.
In the production flow broker cost is small. Handlers take milliseconds to seconds at about 116 messages per second, far below Harbinger's limit, so it shows only as the 11% gap in the FIFO arms.
If handlers are trivial and the rate is tens of thousands per second, this benchmark says Redis or RabbitMQ will serve better today. Harbinger fell behind at 20,000 msg/s.
Prioritising short jobs has a long-job cost. In this workload RabbitMQ static priority's longest export waited 5.2× longer than under FIFO; v4 Harbinger aging-off runs separately measure higher P99 and long waits.

Limits of the comparison

One synthetic workload, one cloud provider and region, two instance types.
The broker comparison and throughput measurements are the original v3 results. V4 adds separate baseline and aging studies, not a new RabbitMQ/Redis rerun.
Default broker settings, except those listed. RabbitMQ prefetch 1 limits its throughput and is noted wherever it matters.
Harbinger's scheduling arms ran the benchmark build of the broker, which has observation hooks. The broker-cost test ran the production server.
Non-durable modes only. Kafka, Pulsar, managed cloud queues and durable modes were out of scope.

All AWS results ↗Pre-registration ↗

Keep reading

ExperimentsHow each experiment steered the design, including the ones that failed.BenchmarksWhat we measured on a laptop, on AWS and on real traces, and what it supports.

Glossary

Ack: Short for acknowledgement. A consumer tells the broker it finished a message, and here it also reports how long the work took. Learn more ↗
Aging: A message that has waited too long is moved up one priority level, so low-priority work is delayed but never stuck forever. Learn more ↗
Backlog: Messages that have arrived but are still waiting because workers can't keep up.
Bootstrap: A statistics method that re-samples your own results many times to see how much they could vary by chance. Learn more ↗
Broker: The server in the middle. Producers hand messages to it, and consumers pull messages from it. Learn more ↗
Cold start: The period before the predictor has seen enough finished jobs for a key to trust its estimate. Cold keys use the default middle tier.
Confidence interval: A range that likely contains the true value. A 95% interval that sits entirely below zero means the improvement is unlikely to be luck. Learn more ↗
Consumer: A program that pulls messages from the queue and does the work they describe. Learn more ↗
Control: A test case designed so the thing being measured shouldn't help. If it still shows a gain, something is wrong. Learn more ↗
Dead-letter queue (DLQ): Where messages go after they fail too many times or expire, so they can be inspected instead of retried forever. Learn more ↗
Decision tree: A model that makes a prediction by answering a chain of yes/no questions about the input. Learn more ↗
EWMA: An exponentially weighted moving average. A running average that gives recent values more weight, so it adapts when things change. Learn more ↗
FIFO: First in, first out. Messages are served strictly in arrival order, like a single supermarket line. Learn more ↗
Gate: A pass/fail test agreed before running an experiment. Predictive routing is only allowed if every gate passes.
Gradient boosting: A heavier machine-learning model that combines many small decision trees. Used here as a stand-in for "a much more complex model". Learn more ↗
gRPC: A fast way for programs to call each other over a network. Harbinger's producers and consumers talk to the broker with it. Learn more ↗
Guardrail: A metric that must not get much worse, even if the main metric improves. Tail latency and lost work are guardrails here.
Head-of-line blocking: One slow item at the front of a line holds up everything behind it, however quick those items are. Learn more ↗
Histogram: A count of how many times each range of values occurred. Here: how many jobs took 1–2 ms, 2–4 ms, and so on. Learn more ↗
Key: What Harbinger groups messages by when learning: the producer, optionally plus a job_type label (and size). Each key gets its own duration history.
Latency: How long a message takes from being sent to being finished, including time spent waiting in the queue. Learn more ↗
Level weights: Instead of always serving the highest level first, each level gets a share of worker time (for example 8 : 3 : 1). Lower levels still make steady progress.
MAE: Mean absolute error: on average, how far a prediction is from the real value, in the same units (here, milliseconds). Learn more ↗
Mean: The ordinary average. It is pulled up by a few very slow items, so it shows the overall experience better than the median does. Learn more ↗
Median: The middle value: half the jobs are faster, half slower. It describes a typical job and ignores rare outliers. Learn more ↗
Message queue: Software that holds messages (jobs to do) until a worker is free to handle them, so senders don't have to wait. Learn more ↗
Multi-level queue: Several queues stacked by priority. The scheduler always serves the highest non-empty one first. Learn more ↗
Non-preemptive: Once a job starts it runs to the end. The scheduler can reorder waiting jobs but can't interrupt one that's running. Learn more ↗
Opt-in: Off by default. You have to switch it on deliberately.
Oracle: A reference that cheats by knowing each job's true cost in advance. It shows what perfect knowledge would give, not what's achievable.
P95 / P99 (tail latency): P99 is the time that 99% of messages beat. It describes the slowest 1%, the "tail", which matters because users notice the slow ones. Learn more ↗
Pausing aging: Aging that holds off promoting a message while the level above is still working through earlier promoted messages. It stops a long backlog from pushing everything to the top.
Pre-registration: Writing down the hypotheses, setup and pass/fail rules before running the experiment, so results can't be quietly reinterpreted afterwards. Learn more ↗
Predictor: The part of the broker that estimates how long a message will take, using what earlier messages with the same key took.
Producer: A program that creates messages and sends them to the queue. Learn more ↗
Quantile: A cut point that splits sorted values into equal groups. Harbinger uses them to decide where one tier ends and the next begins. Learn more ↗
Round-robin: Take turns between queues in a fixed rotation, regardless of what's in them. Learn more ↗
Seed: A starting number for a random generator. Same seed, same random-looking data, so a run can be repeated exactly. Learn more ↗
Serverless: A cloud style where you upload small functions and the provider runs them on demand. The Azure Functions trace records one such platform's invocations. Learn more ↗
Shadow mode: The predictor runs and learns, but its guesses are only recorded, never used. Safe way to try it on real traffic.
Shortest job first: Serve the quickest jobs first. It lowers the average wait but makes long jobs wait longer. Learn more ↗
Slowdown: Latency divided by the job's own work time. A 1 ms job that waits 1 s has a slowdown of 1000.
Spread gate: If a key's slowest runs (p99) are more than 16× its typical run (median), its cost is too erratic to trust, so it takes the default tier.
Static priority: A hand-written rule that fixes each job type's priority in advance and never changes with what's observed.
Throughput: How many messages get finished per second. Learn more ↗
Tier: A priority level in the queue. L0 is served first, then L1, then L2. Learn more ↗
Tier accuracy: How often a message's predicted tier matches the tier its real duration would have earned.
TTL: Time to live. After this long a message is considered expired and dropped. Learn more ↗
Worker: A process that takes one message at a time and runs the work for it.