epochs

Learn · Module 02 · Classical machine learning · Lesson 1 of 8

Preparing data

Most of the work in real ML happens before any model is trained. Clean, well-shaped data beats a clever model on messy data.

4 minute read

In short
  • Fill missing values (the median resists outliers) and consider a 'was missing' flag.
  • Scale numeric features for gradient-based and distance-based models; trees don't need it.
  • One-hot encode categories, and fit every preparation step on training data only.

In plain words

Models only understand numbers, and they're sensitive to how those numbers are shaped. Before training you usually need to: fill in missing values, put numeric features on a similar scale, turn categories like city names into numbers with encoding, and split the data so the test set stays truly unseen.

Worked example: scaling a feature

Ages in a dataset: 20, 30, 40, 50, 60. Standardising subtracts the mean and divides by the standard deviation, so the feature is centred on 0 with a spread of 1.

  1. 1
    Mean
    (20 + 30 + 40 + 50 + 60) ÷ 5 = 40
  2. 2
    Standard deviation

    Square root of the average squared distance from the mean.

    √((400 + 100 + 0 + 100 + 400) ÷ 5) = √200 = 14.14
  3. 3
    Standardised values

    Each age becomes (age − 40) ÷ 14.14.

    −1.41, −0.71, 0, 0.71, 1.41

Why bother? Gradient descent and distance-based methods like k-means treat every feature's numbers as equally important. Without scaling, a feature measured in rupees would drown out one measured in years. Tree-based models don't need it, because they only compare values within a feature.

Worked example: a missing value

Monthly incomes in thousands: 30, 45, missing, 60, 1200. The mean of the known values is 333.75, pulled up by one very large income. The median is 52.5. Filling the gap with the median gives a far more typical value. It's also worth adding a 'was missing' flag column, because missingness itself can carry information.

Encoding categories

citycity_Hyderabadcity_Punecity_Chennai
Hyderabad100
Pune010
Chennai001

One-hot encoding gives each category its own 0/1 column. Numbering the cities 1, 2, 3 instead would wrongly tell the model that Chennai is 'more' than Hyderabad. For categories with thousands of values, like product IDs, use embeddings or group rare values together.

Your turn

You standardise a feature using the mean and standard deviation of the whole dataset, then split into train and test. What's wrong?

What to remember

Split first, then fit every preparation step on the training set and apply it to the rest. Libraries like scikit-learn's Pipeline make this automatic. Look at your data before modelling: distributions, outliers, missing values and duplicates explain a surprising number of 'model' problems.

Common mistakes

  • Fitting the scaler or imputer on the whole dataset before splitting, which leaks test information.
  • Encoding categories as 1, 2, 3, which implies an order that doesn't exist.
  • Filling missing values with the mean when outliers pull it far from a typical value.
Check yourself

Answer in your own words, out loud or on paper, before you open the model answer.

How would you prepare a messy tabular dataset before training a model?

I'd start by looking at the data: types, distributions, missing values, duplicates and obvious errors. Then split into training, validation and test sets before anything else, by time if the data is time-ordered. On the training set only, I'd fit the preparation steps: impute missing numbers with the median and add a 'was missing' flag, scale numeric features if the model is gradient- or distance-based, and one-hot encode categories, grouping rare ones. I'd wrap it all in a pipeline so exactly the same steps apply to validation, test and production data. Finally I'd check for leakage, like features that are only known after the outcome.

More to think about
  • When does feature scaling matter, and when doesn't it?
  • How would you encode a categorical feature with 10,000 possible values?

Go further

Was this lesson clear?