skip to content

Search
← All concepts

#kernels

IO-aware and compiler-optimized GPU code.

4 chapters, oldest era first: one idea traced across the books.

  1. The Engine Room 2007–2014 Ch. 2: GPUs Learn to Think CUDA, AlexNet on two gaming GPUs, and cuDNN — graphics chips become math engines.
  2. The Engine Room Cross-era Ch. 4: The Memory Wall The memory hierarchy, HBM, the roofline model, and why LLM inference is bandwidth-bound.
  3. The Engine Room 2022–2024 Ch. 12: FlashAttention and the Kernel Era Online softmax, tiling, and IO-complexity: software catching up to silicon.
  4. The Engine Room 2022–2025 Ch. 17: Inference Engines The KV-cache memory formula, multi-query and grouped-query attention, PagedAttention's block-based memory management, speculative decoding, and continuous batching.