Schema Evolution in Event-Driven Systems

The deployment completed without errors, the producer service is healthy, and API metrics look normal. Yet an un-updated background worker immediately starts throwing exceptions the moment it consumes the first event published by the new release.

The bug originates in a seemingly minor change: the document.processed event previously carried file_id, but the new producer sends document_id, while an older consumer version still relies on the former field name to process jobs.

The Monorepo Compilation Fallacy

In a monorepo, modifying a shared type and watching CI tests pass creates a false sense of security. While the entire codebase compiles together in Git, services in production are never deployed instantaneously or atomically.

A second temporal dimension is frequently overlooked: message brokers retain events published days before deployment. Even an up-to-date consumer must remain capable of reading legacy schemas during standard message retention windows, dead-letter re-drives, or historical replays.

The Expand and Contract Pattern for Events

A safe schema migration begins with the expand phase: producers publish the new field without removing the old one, defining explicit interpretation rules and ensuring consumers tolerate additive properties without failing schema validation.

In the transition phase, consumers are deployed to read document_id, maintaining backward-compatible fallback to file_id. Only after all active consumer instances and historical backlog messages have cleared does removing the legacy property become safe.

Contract Testing Across Asynchronous Boundaries

This lifecycle necessitates cross-version contract testing rather than verifying only the latest schema in isolation. Adding fields can break strict schema parsers, and semantic changes often necessitate versioned schemas or distinct event types.

Backward compatibility in event-driven architectures is a function of time, not merely code state. Event contracts must hold across asynchronous temporal boundaries, connecting producers, consumers, and retained messages across differing versions.