When data is processed.
WHAT BATCH PROCESSING IS
Processing accumulated data at intervals.
WHAT STREAMING IS
Processing each record as it arrives.
WHAT BATCH PROVIDES
Simplicity Easy reprocessing Efficient use of resources Straightforward correctness
WHAT IT COSTS
Latency: results are as fresh as the last run.
WHAT STREAMING PROVIDES
Low latency Continuous results
WHAT IT COSTS
Substantially more complexity Harder correctness, particularly around ordering and late data Harder debugging and reprocessing
WHAT TO CHOOSE
Batch, unless latency genuinely requires otherwise.
WHY
Most requirements described as real-time are satisfied by processing every few minutes.
WHAT TO ASK
What decision changes if this is an hour old rather than a second old?
WHAT MICRO-BATCHING IS
Frequent small batches, approximating streaming with batch simplicity.
WHAT IT SUITS
Most cases between the two.
WHAT TO AVOID
Building a streaming system for a requirement nobody stated.