Finding relevant documents.
WHAT THE COMPONENTS ARE
Crawling: discovering and fetching content
Parsing and extraction
Indexing: building a structure for fast lookup
Query processing Ranking Serving results
WHAT AN INVERTED INDEX IS
A mapping from each term to the documents containing it.
WHY THAT STRUCTURE
It makes finding documents containing a term immediate, rather than scanning everything.
WHAT ANALYSIS DOES BEFORE INDEXING
Splits text into terms Normalises case Reduces words to a common form Removes or retains very common words
WHY THAT MATTERS
Query terms must be analysed identically, or nothing matches.
WHAT RANKING MUST BALANCE
Term relevance Document importance Freshness User context
WHAT THE CLASSICAL RELEVANCE MEASURES DO
Weight terms by how often they appear in a document against how common they are overall.
WHAT SEMANTIC SEARCH ADDS
Matching by meaning rather than exact terms, using learned representations.
WHAT HYBRID SEARCH DOES
Combines both, which usually outperforms either alone.
WHAT TO MEASURE
Whether users find what they wanted.
