Knowing whether something worked.
WHAT TO DO BEFORE CHANGING ANYTHING
Record the current position.
Without a baseline you cannot judge.
WHAT TO CHANGE
One thing.
WHY
Several changes at once mean you never know which worked.
HOW LONG TO MEASURE
Long enough for the difference to exceed normal variation.
For a small business, frequently a month or more.
WHAT TO COMPARE AGAINST
The same period previously, and normal variation.
WHAT TO BE CAREFUL WITH
Seasonality Anything else that changed Small numbers
WHAT SMALL NUMBERS MEAN
A difference across few records tells you very little.
WHAT TO RECORD
What you changed, when, what you expected, and what happened.
WHY
So you learn, and so you do not repeat experiments.
WHAT TO DO IF THE RESULT IS UNCLEAR
Say it is unclear.
That is a legitimate finding, and better than a false conclusion.
WHAT TO DO WITH A CLEAR RESULT
Keep it, and test the next thing.
