Structuring an automated suite.
WHAT THE SHAPE DESCRIBES
Many unit tests, fewer integration tests, few end-to-end tests.
WHY THAT SHAPE
Cost, speed and reliability all worsen as tests move up.
WHAT UNIT TESTS COST
Milliseconds, and they rarely fail spuriously.
WHAT END-TO-END TESTS COST
Seconds or minutes each, and they fail for reasons unrelated to defects.
WHAT AN INVERTED PYRAMID PRODUCES
A suite taking hours, failing intermittently, that developers learn to ignore.
WHY IGNORING IS THE REAL FAILURE
A suite nobody trusts provides nothing while consuming effort.
WHAT TO PUSH DOWN
Any test that can be written at a lower level.
WHAT BELONGS AT THE TOP
A small number of critical user journeys, end to end.
WHAT NUMBER IS REASONABLE
A handful, not hundreds.
WHAT THE MIDDLE LAYER VERIFIES
Components integrating correctly: database access, external calls, messaging.
WHAT TO SUBSTITUTE AT THAT LAYER
External systems you do not control.
WHAT THE TROPHY SHAPE ARGUES
More weight on integration tests, since they catch realistic problems.
WHAT TO TAKE FROM BOTH
Test at the lowest level that gives real confidence.
WHAT TO MEASURE
Suite duration, and failure rate unrelated to defects.
