/blog

12 min read

Backpressure is a feature: notes on bounded queues

An unbounded queue doesn't remove load, it postpones the outage. How to choose a bound and what to do when it's full.

Every queue in a system is a promise about how much work you are willing to hold. An unbounded queue makes that promise to everyone, which means it keeps none of them once traffic outruns the consumers.

Pick the bound from latency, not memory

If a consumer handles 500 jobs a second and callers give up after 2 seconds, a queue deeper than about 1,000 only holds work nobody is waiting for any more.

max_depth = throughput_per_sec * acceptable_wait_sec
          = 500 * 2
          = 1000

When the queue is full

  • Reject fast. Return 429 or 503 with Retry-After so callers back off.
  • Shed the oldest. For live data, the newest item is usually the one worth keeping.
  • Block the producer only when the producer is yours and can wait.

Watch the right number

Queue depth alone lies. Graph the age of the oldest item; it rises before depth does, and it is the number your users feel.

$ subscribe

Want the next one?

Leave your email and new posts from Stack & Heap come to your inbox.

$ subscribe --email

Get new posts by email

Leave your address and new posts from Stack & Heap come to your inbox.

Rather write back? hello@stackandheap.example