Returning large collections.
WHY PAGINATION IS MANDATORY
An endpoint returning everything works until the data grows, then fails entirely.
WHAT THE APPROACHES ARE
Page number and size Offset and limit A cursor pointing at a position
WHAT PAGE AND OFFSET APPROACHES PROVIDE
Simplicity, and the ability to jump.
WHAT THEY COST
Slowness at high offsets Items skipped or repeated when data changes between pages
WHY THAT SECOND POINT MATTERS
Synchronisation jobs silently miss records.
WHAT CURSOR PAGINATION PROVIDES
Consistent results as data changes, and constant performance.
WHAT IT COSTS
No jumping to an arbitrary page.
WHAT TO CHOOSE FOR SYNCHRONISATION
Cursors.
WHAT TO ALWAYS SET
A default page size, and a maximum.
WHY A MAXIMUM
Without one, a caller requests everything and takes the service down.
WHAT TO RETURN ALONGSIDE
Whether more exists The next cursor, or the total where affordable
WHY THE HEDGE ON TOTALS
Counting a large filtered set is expensive.
WHAT FILTERING SHOULD SUPPORT
The fields callers actually need.
WHAT TO AVOID
Arbitrary query languages in parameters Filters with no index behind them
WHAT SORTING SHOULD SUPPORT
A defined set of fields, with direction.
WHAT TO ENSURE
A stable tiebreaker, or pagination becomes inconsistent.
