Interesting work from Anthropic.
They reframe MLP neurons as a stronger basis for interpretability than commonly assumed.
It is standard practice to study circuits with transcoders, because individual MLP neurons are considered too polysemantic to interpret.
They find that many active transcoder features in late layers are close copies of single MLP neurons. They measure this across 13 open-source transcoders.
These neuron-aligned features are also the only ones with a stable link to the underlying MLP. The other features change more between training runs, don't match the MLP's Jacobian, and move less from their random initialization.
That makes MLP neurons a stronger starting point for interpretability than usually assumed. It also means the non-neuron transcoder features need better validation before they are used to explain model behavior.
https://arxiv.org/abs/2610.24521
Chat with Paper: https://academy.dair.ai/papers/comparing-sparsified-mlps-and-transcoders-many-transcoder-features-are-neurons-i-2610.24521