Knowledgebase

Designing Event Schemas Print

  • dataengineering, data, guide, howto, solution, zillionkinghost, hosting, support
  • 0

Structuring what flows.

WHAT EVERY EVENT SHOULD CARRY

A unique identifier The event type When it occurred The entity it concerns The payload A schema version

WHY THE IDENTIFIER MATTERS

It is what makes consumers idempotent.

WHY THE VERSION MATTERS

Consumers must handle events produced before a change.

WHAT EVENT DESIGN APPROACHES EXIST

  • Notification: something happened, with an identifier to look up
  • State transfer: the full new state
  • Delta: what changed

WHAT NOTIFICATION SUITS

Loose coupling, where consumers fetch what they need.

WHAT STATE TRANSFER SUITS

Consumers that must not depend on the source being available.

WHAT TO PREFER FOR ANALYTICAL CONSUMPTION

State transfer, since it is self-contained and replayable.

WHAT SCHEMA EVOLUTION RULES TO FOLLOW

Adding optional fields is safe Removing or renaming fields is not Changing types is not

WHAT A SCHEMA REGISTRY PROVIDES

Central schemas, with compatibility checked before producers publish.

WHY THAT MATTERS

It prevents a producer breaking every consumer.

WHAT TO NEVER PUT IN AN EVENT

Sensitive data that consumers do not need.

WHY

Events are retained and widely readable.


Was this answer helpful?
Back

Are you happy with your experience? Leave us a review on Trustpilot.


Trustpilot