Preparing inputs.
WHAT NUMERICAL FEATURES MAY NEED
Scaling, for distance and penalty-based methods Transformation, where distributions are skewed Binning, occasionally
WHAT TREE METHODS DO NOT NEED
Scaling.
WHAT CATEGORICAL FEATURES REQUIRE
Encoding into numbers.
WHAT ENCODINGS EXIST
One column per category Ordinal, where order is meaningful Target-based, using the outcome statistics per category Hashing, for very high cardinality
WHAT TARGET ENCODING RISKS
Leakage, since it uses the outcome.
WHAT PREVENTS THAT
Computing it within cross-validation folds.
WHAT HIGH CARDINALITY CAUSES
Very wide data, and categories seen too rarely to learn from.
WHAT TO DO
Group rare categories, and handle unseen ones explicitly.
WHAT MISSING VALUES REQUIRE
A decision, per feature.
WHAT THE OPTIONS ARE
Fill with a statistic Fill with a sentinel value Add an indicator that it was missing Let the model handle it, where supported
WHY THE INDICATOR MATTERS
Missingness is frequently informative in itself.
WHAT DATES REQUIRE
Decomposition into useful parts: day of week, month, time since an event.
