Neural Ninjas · Agentic AI Lab
Self-Attention: Theatre Stage & Solar System
Sentence: "The cat sat on the mat" — pick a word, pick a head, watch attention move.
🎭 Spotlights follow attention
Brighter spotlight = higher attention weight from the token in the gold box
Head 0 — local (looks at neighbor)
Head 1 — subject tracking ("cat")
Head 2 — broad / diffuse
What this showsEach actor's spotlight brightness is that word's attention weight — the floor grid marks fixed positions, same idea as positional encoding.
🌞 The sun is whichever word is asking
Closer orbit = higher attention weight to that word
Head 0 — local
Head 1 — subject tracking
Head 2 — broad / diffuse
What this showsEvery planet keeps orbiting — that's just motion. Its distance from the sun is the real signal: attention weight.