Knowing if it succeeded.
WHAT TO DECIDE BEFORE BUILDING
What you expect to change, and by how much.
WHY BEFORE
Afterwards, any outcome can be justified.
WHAT TO MEASURE
Whether people use it Whether they continue using it Whether it produced the intended outcome
WHY CONTINUED USE MATTERS MOST
Initial use measures curiosity; repeated use measures value.
WHAT METRICS COMMONLY MISLEAD
Total registrations Page views Feature usage without context
WHY REGISTRATIONS MISLEAD
People register and never return.
WHAT TO PREFER
Measures of the outcome the customer wanted.
WHAT EXAMPLES LOOK LIKE
Orders completed rather than pages viewed Time to complete a task rather than clicks
WHAT RETENTION SHOWS
Whether the product delivers lasting value.
HOW TO EXAMINE IT
Groups of customers who started at the same time, tracked over time.
WHY GROUPED
It separates changes in the product from changes in who joined.
WHAT TO DO ABOUT A FEATURE NOBODY USES
Establish why: unknown, unusable, or unwanted.
HOW
Ask, and watch people try it.
WHAT TO DO ABOUT EACH CAUSE
- Unknown: improve visibility
- Unusable: improve the design
- Unwanted: remove it
WHY REMOVING MATTERS
Unused features cost maintenance and complicate everything.
WHAT TO REVIEW
Whether each release produced what was predicted.
