skip to content

Search
← All concepts

#embeddings

Vector representations of tokens and concepts.

3 chapters, oldest era first: one idea traced across the books.

  1. The Language Machines 2013–2014 Ch. 2: word2vec and the Geometry of Meaning Skip-gram, negative sampling, and king minus man plus woman.
  2. The Language Machines 2020–2024 Ch. 17: Retrieval and Grounding RAG, RETRO, and hallucination: knowledge in weights versus knowledge in indexes.
  3. From Noise to Image 2021 Ch. 11: CLIP: Bridging Text and Image The InfoNCE contrastive loss, a shared embedding space for words and pictures, and why it unlocked text-to-image.