Choosing what the model sees.
WHY IT MATTERS
Irrelevant features add noise, cost and opportunity for leakage.
WHAT SELECTION METHODS EXIST
Filtering by statistical relationship with the target
Wrapping: evaluating subsets by model performance
Embedded: methods that select as part of training
WHAT REGULARISATION PROVIDES
Selection as a side effect, by driving some parameters to zero.
WHAT FEATURE IMPORTANCE SHOWS
How much a model relies on each feature.
WHAT IT DOES NOT SHOW
Causation, or importance independent of the model.
WHY THAT MATTERS
Correlated features share importance arbitrarily, which misleads.
WHAT PERMUTATION IMPORTANCE MEASURES
Performance loss when a feature is shuffled.
WHY IT IS MORE TRUSTWORTHY
It measures reliance on actual information, not model internals.
WHAT THE CURSE OF DIMENSIONALITY MEANS
Data becoming sparse as features increase, so more examples are needed.
WHAT TO PREFER
Fewer, well-chosen features.
WHY ALSO PRACTICALLY
Every feature must be available, correct and maintained in production.
WHAT TO REMOVE
Features unavailable at prediction time Features that will not exist in future Features adding nothing measurable
