Learn · Module 02 · Classical machine learning · Lesson 1 of 8
Preparing data
Most of the work in real ML happens before any model is trained. Clean, well-shaped data beats a clever model on messy data.
- Fill missing values (the median resists outliers) and consider a 'was missing' flag.
- Scale numeric features for gradient-based and distance-based models; trees don't need it.
- One-hot encode categories, and fit every preparation step on training data only.
In plain words
Models only understand numbers, and they're sensitive to how those numbers are shaped. Before training you usually need to: fill in missing values, put numeric features on a similar scale, turn categories like city names into numbers with encoding, and split the data so the test set stays truly unseen.
Worked example: scaling a feature
Ages in a dataset: 20, 30, 40, 50, 60. Standardising subtracts the mean and divides by the standard deviation, so the feature is centred on 0 with a spread of 1.
- 1Mean
(20 + 30 + 40 + 50 + 60) ÷ 5 = 40
- 2Standard deviation
Square root of the average squared distance from the mean.
√((400 + 100 + 0 + 100 + 400) ÷ 5) = √200 = 14.14
- 3Standardised values
Each age becomes (age − 40) ÷ 14.14.
−1.41, −0.71, 0, 0.71, 1.41
Why bother? Gradient descent and distance-based methods like k-means treat every feature's numbers as equally important. Without scaling, a feature measured in rupees would drown out one measured in years. Tree-based models don't need it, because they only compare values within a feature.
Worked example: a missing value
Monthly incomes in thousands: 30, 45, missing, 60, 1200. The mean of the known values is 333.75, pulled up by one very large income. The median is 52.5. Filling the gap with the median gives a far more typical value. It's also worth adding a 'was missing' flag column, because missingness itself can carry information.
Encoding categories
| city | city_Hyderabad | city_Pune | city_Chennai |
|---|---|---|---|
| Hyderabad | 1 | 0 | 0 |
| Pune | 0 | 1 | 0 |
| Chennai | 0 | 0 | 1 |
One-hot encoding gives each category its own 0/1 column. Numbering the cities 1, 2, 3 instead would wrongly tell the model that Chennai is 'more' than Hyderabad. For categories with thousands of values, like product IDs, use embeddings or group rare values together.
You standardise a feature using the mean and standard deviation of the whole dataset, then split into train and test. What's wrong?
What to remember
Split first, then fit every preparation step on the training set and apply it to the rest. Libraries like scikit-learn's Pipeline make this automatic. Look at your data before modelling: distributions, outliers, missing values and duplicates explain a surprising number of 'model' problems.
Common mistakes
- Fitting the scaler or imputer on the whole dataset before splitting, which leaks test information.
- Encoding categories as 1, 2, 3, which implies an order that doesn't exist.
- Filling missing values with the mean when outliers pull it far from a typical value.
Answer in your own words, out loud or on paper, before you open the model answer.
How would you prepare a messy tabular dataset before training a model?
I'd start by looking at the data: types, distributions, missing values, duplicates and obvious errors. Then split into training, validation and test sets before anything else, by time if the data is time-ordered. On the training set only, I'd fit the preparation steps: impute missing numbers with the median and add a 'was missing' flag, scale numeric features if the model is gradient- or distance-based, and one-hot encode categories, grouping rare ones. I'd wrap it all in a pipeline so exactly the same steps apply to validation, test and production data. Finally I'd check for leakage, like features that are only known after the outcome.
- When does feature scaling matter, and when doesn't it?
- How would you encode a categorical feature with 10,000 possible values?
Go further
- DocsPreprocessing datascikit-learn
- DocsCommon pitfalls and recommended practicesscikit-learn