Ordering results by relevance.
WHAT LEARNING TO RANK DOES
Learns an ordering from examples of what users preferred.
WHAT TRAINING SIGNALS EXIST
Explicit relevance judgements Clicks and engagement Purchases or conversions
WHAT MAKES CLICKS PROBLEMATIC
Position bias: results shown higher are clicked more regardless of relevance.
WHAT THAT MEANS
Naive training on clicks teaches the model to reproduce the existing ranking.
WHAT ADDRESSES IT
Position-bias correction, or deliberate randomisation.
WHAT FEATURES TYPICALLY MATTER
Textual match between query and item Item quality and popularity Personalisation signals Freshness Business rules, applied separately
WHY BUSINESS RULES SEPARATELY
Promotions and availability are constraints, not learned relevance.
WHAT TO EVALUATE WITH
Ranking metrics reflecting position.
WHAT TO TEST ONLINE
Always, since offline evaluation cannot capture responses to unseen orderings.
WHAT TO MONITOR
Queries returning nothing useful Queries where nothing is clicked
WHY
They indicate gaps in the catalogue or failures in matching.
WHAT TO PROTECT
The ability to explain why an item ranked where it did.
