skip to content

Search
← All concepts

#compute-economics

Cost, energy, and supply of compute.

9 chapters, oldest era first: one idea traced across the books.

  1. The Engine Room 1958–2009 Ch. 1: The Hardware Lottery Ideas win when hardware fits: why neural networks waited for GPUs — the book's thesis.
  2. The Engine Room 2012–2026 Ch. 20: The Co-evolution Thesis The master mapping: GTX 580 to AlexNet, V100 to the transformer, A100 to GPT-3, H100 to GPT-4-class, GB200 to reasoning — and what the next silicon implies.
  3. The Engine Room 2015–2024 Ch. 3: TPUs and the Accelerator Cambrian Systolic arrays, TPU v1 through v5, Cerebras and Groq — specialize versus generalize.
  4. From Frames to Worlds Cross-era Ch. 18: Evaluating and Monetizing Motion FVD's blind spots, VBench, physics tests, and the economics of the most expensive pixel in AI.
  5. The Language Machines 2020–2022 Ch. 10: Scaling Laws Power laws, compute-optimal training, the Chinchilla re-balance, and the emergence debate.
  6. The Language Machines 2022–2023 Ch. 14: ChatGPT and the Open Explosion The product moment, then the LLaMA leak, Alpaca, Llama 2, and Mistral.
  7. The Engine Room 2022–2024 Ch. 11: Training Runs in Practice Loss spikes, node failures, checkpointing, and MFU — what the OPT, BLOOM, and Llama 3 logbooks reveal.
  8. From Frames to Worlds 2024–2025 Ch. 10: The Commercial Race Gen-3, Kling, Veo, Pika, Luma: capability versus cost per second of footage.
  9. The Engine Room 2024–2026 Ch. 19: The Economics of Test-Time Compute Inference scaling as the new capex: train once versus think longer, and the batch economics of reasoning.