# Multi-head attention. See the big picture.

Follow the sentence “the cat sat down” through two attention heads. Predict, explore and explain one step at a time.

Inspect queries, keys, dot products, scaling, softmax, causal masking, value mixing, concatenation and the output projection.

This lesson focuses on self-attention, one part of a Transformer. Sign in with approved Transformers access or request access.

Canonical page: https://21ifm.com/ai/transformers/multi-head-attention

## Explore

- [Transformers overview](https://21ifm.com/ai/transformers)
- [Multi-head attention walkthrough](https://21ifm.com/ai/transformers/multi-head-attention)
- [Backpropagation with real numbers](https://21ifm.com/ai/backpropagation)