Goodfire and Nick Cammarata say most neurons in large AI models are interpretable
Article timeline
The articles in this story. Times are ET.
Max Zeff@ZeffMaxXGoodfire says most neurons in trillion-parameter models are now legibleAIFormer OpenAI interpretability researcher Nick Cammarata says agentic tools now make most neurons in models up to trillions of parameters legible, versus about 2% in his earlier OpenAI project. He says many famous tangled neurons were misunderstood and that ambitious interpretability may be solved in time, possibly soon. The post links to a Goodfire page.
Andrew Curran@AndrewCurran_XAndrew Curran says interpretability may be solved in time for superintelligenceAIAndrew Curran reposts a researcher who co-authored early mechanistic interpretability work at OpenAI and had been pessimistic about the field. The researcher now says that with agentic tools, the vast majority of neurons in models up to trillions of parameters are legible, and that interpretability may be solved in time. Curran calls this a massive positive update on the future, saying he had given up on interpretability before superintelligence arrived.
Goodfire@GoodfireAIOfficialPickGoodfire says neural networks are far more interpretable than early researchers thoughtAIGoodfire says AI models are far more interpretable than previously believed, and that this is good news for interpretability. Nick Cammarata, who co-authored the early Zoom In paper on polysemanticity and superposition, now argues that the vast majority of neurons in models up to trillions of parameters are legible. He says ambitious interpretability may be solved in time, possibly soon, as automated scientists improve.
roon@tszzlXRoon says interpretability breakthroughs could make alignment normal scienceAIRoon says Nick Cammarata's interpretability breakthroughs may be remembered as a major moment in scientific history. The linked Goodfire post says modern agentic tools find most neurons in models up to trillions of parameters legible, versus about 2% in Cammarata's earlier OpenAI project.