epochs

Paths · 8 modules · 43 lessons · 5 projects · about 19 hours

Research Engineer

Turn papers into working code: the maths, neural networks, transformers and training at scale.

Who it's for

Strong programmers who want to work on models themselves.

By the end, you can
  • Implement attention, tokenizers and training loops from scratch
  • Reason about training memory, compute and scaling laws
  • Run experiments and evaluations you can trust

The route

Modules in order, with a project after the lessons it builds on. Mark lessons as done and tick project steps, and this page tracks your progress. Sign in to keep it on every device.

  1. Module 01 · 5 lessons · about 19 minPython and maths toolkitVectors, probability and gradients, with NumPy.Not started
  2. Module 03 · 7 lessons · about 25 minNeural networksFrom one neuron to backprop, by hand, with small numbers.Not started
  3. Project · Intermediate · 3–5 hours · ProA neural network in NumPyWrite the forward pass, backpropagation and training loop yourself, and prove your gradients are right.5 steps
  4. Module 04 · 3 lessons · about 11 minDeep learning in PyTorchTensors, autograd and training loops: the tools real ML work uses.Not started
  5. Project · Intermediate · 3–4 hours · ProRecognise handwriting with PyTorchTrain a small convolutional network that reads handwritten digits, and learn the PyTorch workflow every ML team uses.5 steps
  6. Module 05 · 4 lessons · about 18 minTokens and embeddingsHow text becomes numbers a model can work with.Not started
  7. Project · Beginner · 2–3 hours · FreeBuild a tokenizerTeach a program to split text into tokens the way GPT-style models do, by learning merges from a book.5 steps
  8. Module 06 · 6 lessons · about 26 minTransformersAttention, the full architecture, and how the next token is chosen.Not started
  9. Project · Beginner · 2–3 hours · FreeA tiny language modelGenerate new names by predicting one letter at a time, the same idea as an LLM predicting tokens.4 steps
  10. Module 07 · 8 lessons · about 29 minTraining and fine-tuningPretraining, LoRA, preference tuning and the data behind them.Not started
  11. Project · Advanced · 3–4 hours · ProFine-tune a language model with LoRATeach a real language model a new style by training less than 0.2% of its weights, on a laptop CPU.5 steps
  12. Module 11 · 5 lessons · about 16 minEvaluationDecide what good means, then measure it without fooling yourself.Not started
  13. Module 16 · 5 lessons · about 19 minML from scratchWrite k-means, attention and a tokenizer from scratch.Not started