Technical Article

At-Least-Once Delivery and Consumer Idempotency

At-Least-Once Delivery and Consumer Idempotency

A background job can finish its work, update the database, and trigger an external action, yet appear in the queue again minutes later, even though nobody called retry and the application reported no error.

Consider a worker processing 𝘫𝘰𝘣-123: it persists a result, sends a notification, and then the process dies just before acknowledging the delivery, leaving the completed side effect without a recorded ACK.

Why Messages Are Delivered Again

The reason is that completing the business operation and acknowledging the message are separate events, and a crash between them leaves the broker unable to know whether the worker actually finished. Under at-least-once delivery, when a visibility timeout expires or a broker requeues an unacknowledged delivery after a connection loss, another worker may receive the same job and repeat an operation that already happened.

Durable Completion Records

The consumer therefore needs a durable record of completion, keyed by a stable operation identifier, so that redelivery can recognize completed work and avoid repeating database changes or external actions.

External Actions and Ambiguous Outcomes

That record must also account for failures around external calls: storing a local success flag before calling a provider can suppress unfinished work, while storing it only afterward can leave an ambiguous outcome.

When the outcome is ambiguous, use provider-supported idempotency keys or reconciliation against the provider's authoritative state, then acknowledge only when processing has reached a safe, recoverable point.

The queue guarantees a chance to process the message again. The consumer must decide whether there is anything left to do, because delivery confirmation is never the same thing as effect completion.

AI Disclosure: The ideas and content are my own. I use LLMs to improve clarity and polish my writing.