Multi-head attention. See the big picture.

Follow the sentence “the cat sat down” through two attention heads. Predict, explore and explain one step at a time.

Inspect queries, keys, dot products, scaling, softmax, causal masking, value mixing, concatenation and the output projection.

This lesson focuses on self-attention, one part of a Transformer. Sign in with approved Transformers access or request access.