skip to content

Search
← All concepts

#reinforcement-learning

Learning from reward signals.

5 chapters, oldest era first: one idea traced across the books.

  1. The Language Machines 2017–2022 Ch. 13: RLHF Bradley–Terry reward models and the PPO objective with a KL penalty — InstructGPT.
  2. From Frames to Worlds 2018–2019 Ch. 3: Predicting the Next Frame World-model prehistory: learn a latent simulator, then act inside it.
  3. The Language Machines 2022–2026 Ch. 20: Reasoning Models STaR, process rewards, o1, RLVR, and DeepSeek-R1 — test-time compute as the new scaling axis.
  4. From Frames to Worlds 2024 Ch. 15: Video Models Become Game Engines Action-conditioned generation: Genie, GameNGen, Oasis, and DIAMOND.
  5. From Frames to Worlds 2025–2026 Ch. 16: World Models as Foundation Models Genie 2 and 3, Cosmos, V-JEPA 2 — robotics and embodied simulation as the endgame.