Goodfire proposes mechanistic shaping to penalize model reliance on internal concepts
Overview
Goodfire presents mechanistic shaping, a training framework that penalizes a model's reliance on or expression of specific internal concepts.
The company reports that the method reduced shortcut learning in chest X-ray models, cut sycophancy induced by DPO while keeping capability gains, and curbed reward hacking in Countdown training.
Goodfire says the targeted concepts stayed monitorable after shaping, though often only with a retrained probe. The source is the company's own research paper, so these results have not been independently verified.
Written by AI from the articles below · updated Nov 5, 2:03 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- Goodfire ResearchOfficialPickGoodfire presents mechanistic shaping to steer models through their internal representations
AIGoodfire presents mechanistic shaping, a training framework that penalizes a model's reliance on or expression of specific internal concepts. The company reports that the method reduced shortcut learning in chest X-ray models, cut DPO-induced sycophancy while keeping capability gains, and curbed reward hacking in Countdown training. Goodfire says the targeted concepts stayed monitorable after shaping, though often with a retrained probe.
Heat Trend
Not enough continuous observations to show a trend yet.