`clowns wear shoes`

Query token

clownswearshoes

Available context

EncoderCausal

Divide scores by √2

Here, d_k = 2 because each query and key vector has two features (dimensions). This is the vector size, rather than the number of tokens.

1. Compare keys. 2. Normalize scores.
Key token	Key	Score	Weight
`clowns`	`[1,0]`
`wear`	`[0,1]`
`shoes`	`[1,1]`

3. Combine values

clowns

× [2,0]

wear

× [0,2]

shoes

× [1,3]

New representation

Fixed illustrative vectors. These are not learned language features.