Begin with the architecture

The explorer starts with the input token, embedding, 26 transformer blocks, and output. The blue bar in each block is the mean absolute z-score of its strongest 69 MLP units for the selected word. Orange diamonds mark global self-attention layers; they are not activation values.

Select a block to open its self-attention path, measured gated MLP stage, residual stream, and 96 × 72 unit field. Start with L12/U646, the first frozen winner, then use the word search to compare all 320 recorded words.

What is measured

These are saved target-token activations from a pretrained Gemma model. The controls choose which measurement to display. They do not run a new prompt, disable a unit, or change the model’s answer. A category mean is a descriptive response profile, not a probability, accuracy, or causal effect.

Read the experiment Open observatory on its own Download manifest