Scripted integration.
WHAT THIS COVERS
Fetching data from an API, or sending data to one, on a schedule.
WHAT TO BUILD IN
Timeouts Retries with increasing delay, for temporary failures Respect for rate limits Validation of responses
WHAT TO DO ABOUT AUTHENTICATION
Read credentials from configuration, never from code.
WHAT TO HANDLE
The service being unavailable Responses in an unexpected format Partial results Pagination
THE PAGINATION POINT
Many APIs return results in pages.
Taking only the first page and assuming it is everything is a common and silent error.
WHAT TO RECORD
What was fetched and when, so you can resume or avoid repeating.
WHAT TO AVOID
Fetching everything every time, where incremental is possible Hammering a service on failure
WHAT TO DO ON REPEATED FAILURE
Stop, log clearly, and notify.
WHAT TO TEST
Behaviour when the service returns an error.
WHAT NOT TO DO
Assume the service will always behave as documented.
