skip to content

Search
← All concepts

#recurrent-networks

RNN/LSTM sequence models.

2 chapters, oldest era first: one idea traced across the books.

  1. The Language Machines 1997–2015 Ch. 3: RNNs, LSTMs, and seq2seq Gating equations, the bottleneck vector, and machine translation as generation.
  2. The Language Machines 2014–2015 Ch. 4: Attention Alignment scores and softmax weighting — the fix for the bottleneck, and the seed of everything after.