The summary.
ESTABLISH REQUIREMENTS AND MAGNITUDES BEFORE PROPOSING A DESIGN
Rough numbers are what determine whether the design needs anything beyond the obvious.
Start with the simplest thing that meets them, then identify where it would fail and address that, stating the trade-off.
STATELESSNESS IS WHAT MAKES HORIZONTAL SCALING POSSIBLE
Keep state in shared storage, not in the instance.
And scale what is actually the constraint, measured. It is usually the database — indexes, query improvement, read replicas, then caching.
FAILOVER THAT HAS NEVER BEEN EXERCISED FREQUENTLY DOES NOT WORK
Design for partial failure, degrade gracefully rather than failing entirely, and stop calling a failing dependency — continuing prolongs the outage and can spread it.
YOU CANNOT HAVE CONSISTENCY AND AVAILABILITY DURING A NETWORK PARTITION
Decide explicitly which data requires which guarantee. Money and inventory need strong consistency; counts and feeds do not.
Design so transactions are needed within one boundary only.
ASSUME MESSAGES ARRIVE TWICE AND OUT OF ORDER
Delivery guarantees almost always permit duplicates, so make processing safe to repeat.
PASS A CORRELATION IDENTIFIER THROUGH EVERY CALL
Without it, related events across services cannot be connected. And measure latency across the distribution — averages conceal a poor experience for a proportion of users.
