12 tutorials· Ordered reading path
Reinforcement learning teaches agents to make decisions by trial and error. This reading path starts with the simplest tabular methods and builds toward deep RL and policy optimisation.
Each tutorial implements the algorithm from scratch in Python, so you can run, break, and rebuild every component. Start at the top and work down — each post assumes you have read the ones before it.
Implement Q-learning from scratch on FrozenLake. The Bellman equation, temporal difference learning, epsilon-greedy exploration, and tabular RL in action.
A Q-learning agent teaches itself tic-tac-toe through self-play. Tabular Q-learning in adversarial games, two agents bootstrapping toward optimal play.
Two approaches to solving MDPs on FrozenLake: value iteration and Q-learning. The Bellman equation, DP sweeps, and why Q-learning needs no model.
Build Monte Carlo tree search in Python and measure it against alpha-beta at the same node count. Four phases, one formula, no evaluation function, and it wins.
Watch a Q-learning agent game a naive reward: it farms a bonus tile forever and never reaches the goal. Then two fixes and a detector, from scratch in Python.
Move beyond Q-tables to neural network function approximation. DQN from scratch in PyTorch with experience replay and target networks.
The Cross-Entropy Method: a derivative-free algorithm that solves CartPole by evolving a population of policies. No backpropagation, no value functions.
Simulated annealing solves CartPole-v1 with 4 parameters and 800 episodes. The simplest derivative-free RL algorithm: perturb, evaluate, keep if better.
The REINFORCE algorithm in pure NumPy on CartPole: forward pass, backprop, RMSProp, and why policy gradients optimise the policy instead of a value function.
Build A3C from scratch in PyTorch: a critic cuts REINFORCE's gradient variance, and asynchronous workers train it without a replay buffer, on CartPole.
REINFORCE, TRPO, and PPO from scratch in PyTorch on CartPole. Why vanilla policy gradients are unstable, how trust regions fix it, and why PPO won.
Build GRPO from scratch in Python. The group-relative advantage that replaced PPO's critic in DeepSeek-R1, measured on a bandit, a maze and a verifier.
Get notified when new tutorials and deep-dives are published. No spam, unsubscribe anytime.
Common questions about this topic