To answer yes or no, compute a score, then squash it into a probability between 0 and 1.
4 minute read
In short
Logistic regression scores with w·x + b, then sigmoid turns the score into a probability.
It learns by minimising log loss, which punishes confident mistakes hardest.
The decision threshold is a separate choice, set by the cost of each kind of mistake.
In plain words
Many questions have yes or no answers: will this customer buy, is this email spam? Logistic regression computes a score z = w·x + b, just like linear regression, then passes it through the sigmoid function, which turns any number into a probability.
score z
−4
−2
0
2
4
sigmoid(z)
0.018
0.119
0.500
0.881
0.982
Big positive scores become close to 1, big negative scores close to 0, and a score of 0 gives exactly 0.5.
Worked example: will a lead buy?
A sales team describes each lead by two numbers: demos attended and support complaints. A trained model has weights w = [0.8, −0.5] and bias b = −1.
1
3 demos, 1 complaint
z = 0.8×3 − 0.5×1 − 1 = 0.9
p = sigmoid(0.9) = 0.711
2
1 demo, 2 complaints
z = 0.8×1 − 0.5×2 − 1 = −1.2
p = 0.231
3
5 demos, no complaints
z = 0.8×5 − 0 − 1 = 3.0
p = 0.953
The weights tell a story: each demo raises the chance of buying, each complaint lowers it.
How it learns: log loss
The loss for one example is −ln(p) when the true answer is yes. It rewards confident right answers and punishes confident wrong ones very hard.
model says p(yes)
0.9
0.5
0.1
loss when the truth is yes
0.105
0.693
2.303
Your turn
For a lead, the score z is exactly 0. What probability does the model give?
Where you'll see this
You still have to choose a threshold: predict 'yes' above 0.5, or above 0.8 if false alarms are costly. The last layer of most neural network classifiers is logistic regression in disguise, and softmax is the same idea extended to many classes.
Common mistakes
Using squared error as the loss. Log loss is the right choice and gives cleaner gradients.
Treating the 0.5 threshold as fixed. The best threshold depends on the cost of each kind of mistake.
Calling it a regression model in the sense of predicting numbers. It predicts probabilities for classification.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
How does logistic regression differ from linear regression?
Both compute a weighted sum of the features, but linear regression outputs that number directly, while logistic regression passes it through a sigmoid to squeeze it between 0 and 1, so it can be read as a probability of the positive class. Because of that, the loss changes too: linear regression uses mean squared error, logistic regression uses log loss, which heavily penalises confident wrong answers. Each weight in logistic regression changes the log-odds of the outcome. To make a decision you pick a threshold, often 0.5 but ideally chosen from the costs of false positives and false negatives. It's simple, fast and interpretable, which is why it's still a strong baseline.
More to think about
Why use log loss instead of mean squared error for classification?
How would you choose the classification threshold?