Neural Ninjas ¡ Transformer Architecture
Tower of Tokens
One atomic step at a time. Nothing moves until you say so.
Encoder â reading the sentence
The word stays the same. Its internal vector doesn't.
Click "Next step" to begin â one thing happens at a time.
Decoder â writing the translation
Same idea, plus it keeps checking back on the encoder
peeking at encoder: â
Run the Encoder tab first, then click "Next step" here.
𧊠Multi-head attention â with real numbers
Every ray is a head. Every label is an actual weight.
Press "Run attention" â rays stay on screen so you can actually read the weights.