Tests that fail unpredictably.
WHAT A FLAKY TEST IS
One passing and failing without the code changing.
WHY THEY ARE SERIOUS
They destroy trust in the entire suite.
WHAT HAPPENS ONCE TRUST IS LOST
Failures are rerun rather than investigated, and real defects pass unnoticed.
WHAT CAUSES FLAKINESS
Timing and fixed waits Shared state between tests Test order dependence Real time and dates Randomness Network and external dependencies Concurrency
WHAT TO DO WHEN ONE IS FOUND
Investigate it, rather than rerunning.
WHY
The cause is frequently a genuine race condition in the code.
WHAT THAT MEANS
A flaky test may be reporting a real defect intermittently.
WHAT TO NEVER DO
Add automatic retries to hide it.
WHY
It conceals both the flakiness and the defect.
WHAT TO DO IF IT CANNOT BE FIXED IMMEDIATELY
Quarantine it, visibly, with an owner and a deadline.
WHAT TO TRACK
Flakiness rate per test.
WHAT TO DELETE
Tests that are chronically flaky and low value.
WHY DELETION IS ACCEPTABLE
A test nobody trusts is worse than no test.
