#generative-modeling
Learning p(x) to sample new data.
8 chapters, oldest era first: one idea traced across the books.
- The Language Machines 1948–2003 Ch. 1: From Shannon to n-grams Language modeling as next-symbol prediction, perplexity, and Bengio's first neural language model.
- From Noise to Image 2006–2013 Ch. 1: The Generative Problem What modeling p(x) means, density versus sampling, and why image generation stalled until deep learning's revival.
- From Noise to Image 2006–2026 Appendix: Timeline and Reading List A one-page dated timeline of image generation and every paper cited in this book.
- From Noise to Image 2013–2015 Ch. 2: Variational Autoencoders The ELBO, the reparameterization trick, and the first neural networks that could imagine — blurrily.
- From Noise to Image 2014 Ch. 3: The Adversarial Game The GAN minimax objective, its JS-divergence interpretation, and the failure mode called mode collapse.
- From Frames to Worlds 2014–2015 Ch. 1: Why Video Is Hard Dimensionality, temporal coherence, and the blurry-MSE lesson that frames the whole book.
- From Noise to Image 2015–2020 Ch. 8: Diffusion Foundations Forward noising, the ELBO reduced to a simple denoising loss, and score functions — the math under everything modern.
- From Noise to Image 2016–2017 Ch. 5: Pixels One at a Time Autoregressive factorization, PixelRNN/PixelCNN, and VQ-VAE's discrete codebooks.