Paths · 8 modules · 43 lessons · 5 projects · about 19 hours
Research Engineer
Turn papers into working code: the maths, neural networks, transformers and training at scale.
Who it's for
Strong programmers who want to work on models themselves.
By the end, you can
- Implement attention, tokenizers and training loops from scratch
- Reason about training memory, compute and scaling laws
- Run experiments and evaluations you can trust
Not startedStart the path →
The route
Modules in order, with a project after the lessons it builds on. Mark lessons as done and tick project steps, and this page tracks your progress. Sign in to keep it on every device.
- Module 01 · 5 lessons · about 19 minPython and maths toolkitVectors, probability and gradients, with NumPy.Not started
- Module 03 · 7 lessons · about 25 minNeural networksFrom one neuron to backprop, by hand, with small numbers.Not started
- Project · Intermediate · 3–5 hours · ProA neural network in NumPyWrite the forward pass, backpropagation and training loop yourself, and prove your gradients are right.5 steps
- Module 04 · 3 lessons · about 11 minDeep learning in PyTorchTensors, autograd and training loops: the tools real ML work uses.Not started
- Project · Intermediate · 3–4 hours · ProRecognise handwriting with PyTorchTrain a small convolutional network that reads handwritten digits, and learn the PyTorch workflow every ML team uses.5 steps
- Module 05 · 4 lessons · about 18 minTokens and embeddingsHow text becomes numbers a model can work with.Not started
- Project · Beginner · 2–3 hours · FreeBuild a tokenizerTeach a program to split text into tokens the way GPT-style models do, by learning merges from a book.5 steps
- Module 06 · 6 lessons · about 26 minTransformersAttention, the full architecture, and how the next token is chosen.Not started
- Project · Beginner · 2–3 hours · FreeA tiny language modelGenerate new names by predicting one letter at a time, the same idea as an LLM predicting tokens.4 steps
- Module 07 · 8 lessons · about 29 minTraining and fine-tuningPretraining, LoRA, preference tuning and the data behind them.Not started
- Project · Advanced · 3–4 hours · ProFine-tune a language model with LoRATeach a real language model a new style by training less than 0.2% of its weights, on a laptop CPU.5 steps
- Module 11 · 5 lessons · about 16 minEvaluationDecide what good means, then measure it without fooling yourself.Not started
- Module 16 · 5 lessons · about 19 minML from scratchWrite k-means, attention and a tokenizer from scratch.Not started