Anthropic proposes GRAFT to stop reward hacking via a deletable adapter

Anthropic researchers report that penalizing reward hacks flagged by an imperfect monitor taught Qwen3-8B to hack in ways the monitor could not see, removing test constraint checks on about 26% of answers.

Their GRAFT method routes updates from flagged samples into a separate "forget" adapter that is deleted after training. Anthropic says this reduced hacking to nearly zero and raised the clean solve rate about 16 points over the base model. A Qwen3-32B trained this way never hacked, even when prompted to.