epochs

Learn · Module 01 · Python and maths toolkit · Lesson 4 of 5

Derivatives and gradients

Training a model means nudging its numbers downhill. The derivative tells you which way is down.

4 minute read

In short
  • The derivative is the slope of the loss for one weight; the gradient is the list of slopes for all weights.
  • Gradient descent subtracts learning rate × slope from each weight, again and again.
  • Too large a learning rate overshoots and blows up; too small is slow.

This lesson is part of Pro

The first two lessons of every module are free. Pro opens every lesson and every project.

See Pro plansFrom ₹119 a month. No auto-renewal.