Concepts
Every chapter in the AI books is tagged with the concepts it teaches. Pick one to follow it chronologically across all four books, from its first paper to the current frontier.
Paradigms
#generative-modeling 8 chapters Learning p(x) to sample new data. #gan 5 chapters Adversarial generator/discriminator training. #vae 3 chapters Latent-variable models trained via the ELBO. #diffusion 16 chapters Iterative denoising / score-based generation. #autoregressive 7 chapters Generation by next-token factorization. #flow-matching 1 chapters Learning velocity fields along noise→data paths. #consistency-models 1 chapters Few-step generation via consistency constraints. #world-models 6 chapters Learned simulators of environments and dynamics. #reinforcement-learning 5 chapters Learning from reward signals.
Architecture
#transformers 6 chapters The transformer family of architectures. #attention 7 chapters Attention mechanisms and their variants. #recurrent-networks 2 chapters RNN/LSTM sequence models. #state-space-models 1 chapters S4/Mamba-style linear-time sequence models. #mixture-of-experts 1 chapters Sparse expert routing. #embeddings 3 chapters Vector representations of tokens and concepts. #tokenization 4 chapters Discretizing text/images/video into tokens. #latent-space 9 chapters Operating in learned compressed spaces.
Training
#pretraining 7 chapters Large-scale self-supervised training. #fine-tuning 7 chapters Adapting pretrained models to tasks or subjects. #peft 2 chapters Parameter-efficient fine-tuning (LoRA, adapters). #instruction-tuning 2 chapters Supervised fine-tuning on instruction–response data. #rlhf 2 chapters Preference-based alignment (RLHF, DPO, GRPO). #alignment 5 chapters Making models follow intent safely. #optimization 5 chapters Optimizers, schedules, and training dynamics. #training-stability 4 chapters Normalization, initialization, loss-spike engineering. #scaling-laws 15 chapters Compute/data/parameter ↔ performance relationships. #data-curation 2 chapters Corpora, filtering, dedup, synthetic data. #distillation 3 chapters Compressing teachers into students (knowledge or step).
Systems
#hardware 9 chapters GPUs, TPUs, accelerators, memory and interconnect. #kernels 4 chapters IO-aware and compiler-optimized GPU code. #distributed-training 5 chapters Data/tensor/pipeline parallelism. #mixed-precision 1 chapters FP16/BF16/FP8 numerics. #memory-efficiency 7 chapters ZeRO, checkpointing, offloading. #quantization 2 chapters Low-bit weights and activations. #inference-optimization 8 chapters Fast, cheap generation at serving time. #compute-economics 9 chapters Cost, energy, and supply of compute.
Capabilities
#language-modeling 5 chapters Predicting text sequences. #in-context-learning 2 chapters Learning from the prompt without weight updates. #text-to-image 5 chapters Image generation from text. #conditioning 6 chapters Steering generation with signals (text, pose, depth, guidance). #video-generation 13 chapters Generating moving pictures. #temporal-consistency 6 chapters Coherence across frames and time. #multimodal 6 chapters Models spanning text/image/video/audio. #long-context 1 chapters Extending usable context windows. #retrieval 1 chapters Retrieval-augmented generation and grounding. #evaluation 11 chapters Metrics, benchmarks, human preference measurement. #open-source 5 chapters Open-weights releases and their ecosystems.