Goodfire reports neurons in Qwen3.6-35B-A3B are sparse and interpretable
Overview
Goodfire Research reports that neurons in the Qwen3.6-35B-A3B model are sparse and interpretable.
The company says ablating the 92.2% weakest MoE neuron activations raises language model loss by only 2.9%. It also says most neurons have one dominant computational mode that can be described in natural language, and that those descriptions predict firing with 95% accuracy against random text. These findings come from Goodfire's own first-party research and have not been independently verified.
Written by AI from the articles below · updated Nov 10, 4:56 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Goodfire ResearchOfficialPickGoodfire finds neurons in Qwen3.6-35B-A3B are sparse and interpretable
AIGoodfire reports that ablating the 92.2% weakest MoE neuron activations in Qwen3.6-35B-A3B has only modest effects, a 2.9% increase in language model loss. The team says most neurons have one dominant computational mode and can be described in natural language, with descriptions predicting firing with 95% accuracy against random text.
Heat Trend
Not enough continuous observations to show a trend yet.