Reasoning From Scratch round 8, the finale!
This last lecture in the series covers distillation as an efficient alternative to the previous reinforcement learning training runs.
(Planning to take a brief break, but I will be back!)
00:00 Introduction and series recap
01:07 Why distill a reasoning model?
02:51 Soft vs. hard distillation
11:06 On-policy vs. off-policy distillation
18:09 Preparing data for reasoning distillation
22:35 Loading the DeepSeek R1 MATH dataset
28:04 Inspecting reasoning traces and final answers
31:27 Tokenization and chat templates
35:58 Pre-tokenizing the training dataset
39:01 Sequence lengths and the 2,048-token limit
42:38 Creating training and validation splits
45:32 Loading the Qwen3-0.6B base model
46:24 Computing answer-token log probabilities
54:10 Cross-entropy loss for distillation
57:25 Training and validation loss helpers
1:02:03 Implementing the distillation training loop
1:07:26 Running the standalone training script
1:09:41 Plotting training and validation losses
1:11:31 Evaluating checkpoints on MATH-500
1:14:09 Comparing teachers and distillation results
1:16:34 Future directions for reasoning models
1:19:41 Bonus exercises and next steps