From our recent Opus 5.5 pod — how Anthropic cracked down on CoT distillation attacks:
Latent.Space explains Anthropic's crackdown on chain-of-thought distillation attacks
AILatent.Space says Anthropic has cracked down on chain-of-thought distillation attacks, as discussed in its recent Opus 5.5 podcast. ThePrimeagen says Anthropic has inserted fake tool calls to mislead distillers, so reasoning traces may contain harmful or wrong choices.
Latent.SpaceVerified on X
@latentspacepod
Dario and team have actively put in fake tool calls to throw off distillers, it would be of NO surprise to see reasoning traces be filled with harmful and downright wrong choices. Every time i read them, I panic because of how dumb opus is acting, but then the output is good.View quoted post on X
Source details
Latent.Space · x.comPublished · added here
Original title: From our recent Opus 5.5 pod — how Anthropic cracked down on CoT distillation attacks: