The Engine Room

Every model breakthrough in the other three books rode a silicon breakthrough — AlexNet on two gaming GPUs, the transformer on V100s, GPT-3 on A100 clusters, reasoning models on racks of GB200s. This book tells that story directly: the hardware, the numerics, the parallelism, the memory tricks, the fine-tuning and quantization methods, and the inference engines that successively raised the ceiling on what could be trained and served.

It ends where the whole series ends: the co-evolution thesis — models are shaped by the machines they run on, and the machines are shaped back.