Computing over flows.
WHAT STATELESS PROCESSING IS
Each record processed independently: filtering, transforming, routing.
WHAT STATEFUL PROCESSING IS
Processing depending on previous records: counting, aggregating, joining.
WHY STATE IS THE HARD PART
It must survive failure, be distributed correctly, and not grow indefinitely.
WHAT WINDOWING IS
Grouping events into bounded intervals for aggregation.
WHAT THE WINDOW TYPES ARE
- Tumbling: fixed, non-overlapping
- Sliding: fixed, overlapping
- Session: bounded by inactivity
WHAT EVENT TIME MEANS
When the event actually occurred.
WHAT PROCESSING TIME MEANS
When your system saw it.
WHY THEY DIFFER
Network delays, buffering, retries, and devices that were offline.
WHY THAT MATTERS MOST
Aggregating by processing time produces wrong results when data is delayed.
WHAT A WATERMARK IS HERE
An assertion that no events older than a point are expected.
WHAT IT CONTROLS
When a window is considered complete.
WHAT LATE DATA IS
Events arriving after their window closed.
WHAT THE OPTIONS ARE
Discard them Keep windows open longer Emit a correction
WHAT TO DECIDE
Which, per use, deliberately.
