Webhook Ingestion and the Inbox Pattern

A webhook can arrive multiple times, out of sequence, or after a network timeout, even when the upstream provider operates normally. If an HTTP endpoint treats every incoming POST as a synchronous, definitive command, standard network delivery characteristics mutate into critical business bugs.

Consider receiving a payment confirmation before the corresponding subscription creation event. Applying each webhook in order of arrival risks regressing or corrupting entity state, even though both messages are perfectly authentic.

Authenticity and the Durable Inbox Contract

The initial contract governs authenticity: validating cryptographic signatures over the exact raw request bytes according to provider rules and verifying timestamp tolerance windows before inspecting or trusting any field in the payload.

Once authenticated, the API must persist the raw payload alongside a unique event or delivery identifier into a durable inbox table before returning a 2xx HTTP response. An acknowledgment signifies durable receipt, not completed business side effects.

Asynchronous Execution and Idempotency

Business execution takes place downstream: background workers poll pending inbox records, execute idempotent domain logic, and persist the transition state. Downstream messaging must leverage a reliable relay mechanism to prevent dual-write inconsistencies between persistence and queue submission.

This decoupling transforms error handling: transient downstream failures can be retried internally with exponential backoff without relying on provider retries, while duplicate deliveries are recognized and handled safely at the storage layer.

Handling Out-of-Order Delivery

Because network delivery order never guarantees causal order, consumers evaluate monotonic version numbers or timestamps when available. In ambiguous cases, workers fetch current state directly from the provider's authoritative API before applying mutations.

A resilient integration decouples three distinct architectural decisions: verifying message authenticity, acknowledging durable receipt, and deciding whether the payload should alter domain state. Conflating all three within a synchronous HTTP handler leaves system correctness hostage to network volatility.