10-attempt, 10-minute-TTL Retry Cycle before dead-lettering to an error queue
Queue consumers fail two different ways: transiently (a DB blip, a network hiccup) or permanently
(a duplicate delivery, a payload with no body). A permanent failure should never be retried — it’ll
never produce a different outcome. A transient failure should be retried, but not forever and not
instantly (hammering a struggling downstream dependency makes things worse). declareQueueAndDLX()
provisions every Queue as three physical RabbitMQ queues (main, -retry, -error) instead of
tracking retry state in the application (e.g. a retry_count column plus a cron sweep): a nack
dead-letters the message to -retry, which holds it for a 10-minute TTL before dead-lettering it
back to main for another attempt. RabbitMQ’s own x-death header counts attempts; getDeathCount()
reads it, and once it reaches 10, nackOrDeadLetter() moves the message to -error instead —
terminal, for manual inspection — rather than cycling forever. Permanent failures skip this cycle
entirely and ack immediately.
Considered options
- App-managed retry accounting (a
retry_countcolumn, a cron job re-publishing due rows) — rejected: reimplements scheduling RabbitMQ already does natively via TTL + dead-letter-exchange, and adds a DB write on every retry just to track state the broker already tracks viax-death. - Unbounded retry (no error queue, nack forever) — rejected: a permanently-broken message (bad schema, a bug in the consumer) would cycle every 10 minutes indefinitely, with no signal that it needs human attention.