Stanford paper maps the circuit behind in-context learning in small language models

A Stanford paper titled "A Circuit for In-Context Learning" finds that small language models store each label's class in label tokens, then match new sentences against those stored classes.

The lookup activates only when labels repeat. Rohan Paul, who reported on the paper, says it tested Qwen3-4B, SmolLM3-3B and Llama-3.2-3B, with accuracy rising from about 20% with one example per class to 96% with six.

A separate report of the same Stanford research says more examples cannot teach a small model a pattern it does not already notice, such as the third letter of a word.