skip to content

Search
← All concepts

#fine-tuning

Adapting pretrained models to tasks or subjects.

7 chapters, oldest era first: one idea traced across the books.

  1. The Engine Room 2015–2025 Ch. 16: Distillation Soft targets and temperature, DistilBERT, and distilling reasoning traces.
  2. The Language Machines 2018 Ch. 6: Pretrain, Then Fine-tune ELMo, ULMFiT, GPT-1, and BERT: masked versus causal objectives.
  3. The Language Machines 2019 Ch. 9: One Model, Many Tasks T5's text-to-text framing and BART — and why decoder-only eventually won.
  4. The Engine Room 2019–2023 Ch. 14: PEFT and LoRA Adapters, prompt and prefix tuning, the low-rank update, and QLoRA's NF4.
  5. The Language Machines 2021–2022 Ch. 12: Instruction Tuning FLAN and T0 task mixtures, and self-instruct bootstrapping.
  6. From Noise to Image 2022–2023 Ch. 16: Personalization and LoRA Textual inversion, DreamBooth, and low-rank adaptation applied to diffusion — the fine-tuning economy.
  7. From Noise to Image 2023 Ch. 15: Control ControlNet's zero-convolution trick, adapters, and reference images — from prompting to specification.