Goodfire finds neurons in Qwen3.6-35B-A3B are sparse and interpretable
AIGoodfire reports that ablating the 92.2% weakest MoE neuron activations in Qwen3.6-35B-A3B has only modest effects, a 2.9% increase in language model loss. The team says most neurons have one dominant computational mode and can be described in natural language, with descriptions predicting firing with 95% accuracy against random text.
AIWhy it matters.
The paper tests whether neurons are a usable analysis unit by ablating 92.2% of weak activations and describing neurons in natural language, giving a concrete method for interpretability work.
Source details
Goodfire Research · goodfire.comPublished · added here
Original title: Neurons as an Interpretable Basis