skip to content

Search
← All concepts

#autoregressive

Generation by next-token factorization.

7 chapters, oldest era first: one idea traced across the books.

  1. From Noise to Image 2016–2017 Ch. 5: Pixels One at a Time Autoregressive factorization, PixelRNN/PixelCNN, and VQ-VAE's discrete codebooks.
  2. The Language Machines 2018–2020 Ch. 7: The GPT Lineage Decoder-only scaling from GPT-1 to GPT-3, and in-context learning as an emergent phenomenon.
  3. From Noise to Image 2021 Ch. 12: DALL-E and GLIDE Discrete-token text-to-image versus text-conditioned diffusion — the fork in the road.
  4. From Frames to Worlds 2021–2022 Ch. 4: Tokens for Video VQ tokenization plus autoregressive transformers: VideoGPT, CogVideo, and Phenaki's variable-length generation.
  5. From Noise to Image 2022 Ch. 14: The Year Images Went Mainstream DALL-E 2, Imagen, Parti, and Midjourney: unCLIP, giant text encoders, cascades, and autoregression at scale.
  6. From Noise to Image 2024–2026 Ch. 20: The Current Frontier Autoregression strikes back: VAR, native multimodal generation inside LLMs, and instruction-following images.
  7. From Frames to Worlds 2024 Ch. 12: Long Video and Consistency Character and scene persistence, AR–diffusion hybrids, and streaming generation.