Learn · Module 03 · Neural networks · Lesson 4 of 7
Optimisers
Gradient descent says which way to step. Optimisers decide how big each step should be, and they make a large difference.
In short
- SGD steps by learning rate × gradient and is easily thrown by noise.
- Momentum averages past gradients, smoothing noise and speeding up on consistent slopes.
- Adam scales each weight's step by its gradient history; AdamW is the transformer default.
This lesson is part of Pro
The first two lessons of every module are free. Pro opens every lesson and every project.
See Pro plansFrom ₹119 a month. No auto-renewal.