The available systems.
WHAT LOG-BASED PLATFORMS PROVIDE
Durable, ordered, replayable streams with several independent consumers.
WHAT THEY SUIT
High throughput Several consumers of the same data Replay requirements
WHAT MESSAGE BROKERS PROVIDE
Queues with routing, acknowledgement and delivery guarantees.
WHAT THEY SUIT
Task distribution Request-response patterns Complex routing
WHAT THE DIFFERENCE MEANS PRACTICALLY
A log retains and replays; a queue delivers and removes.
WHAT MANAGED SERVICES PROVIDE
The same capability without operating the platform.
WHAT THEY COST
Per-throughput charges, and less control.
WHAT STREAM PROCESSING FRAMEWORKS PROVIDE
Stateful processing with windowing, joins and guarantees.
WHAT LIGHTWEIGHT OPTIONS PROVIDE
Processing in your own application, with the platform's client library.
WHAT THAT SUITS
Modest processing, without a separate cluster.
WHAT TO ESTABLISH BEFORE ADOPTING STREAMING
Whether batch every few minutes would serve.
WHY
Streaming infrastructure carries substantial ongoing operational cost.
WHAT TO PLAN FOR
Retention, which determines replay capability and storage cost.
