OpenAI describes training activation classifiers to detect misaligned model behavior
AIOpenAI says it has added activation classifiers, or probes, that run on all assistant completions starting with GPT-5.6. The probes use a model's internal activations to flag misaligned or undesirable activity, can stop a policy-violating turn midway, and are more robust to jailbreaks than external text classifiers.
AIWhy it matters.
The post explains how OpenAI's activation probes detect misaligned activity, including training choices, transfer limits, and jailbreak robustness, useful for understanding white-box safety monitoring.
Source details
OpenAI Alignment Research Blog · Alex Makelov, Xiaojun Xu, Sundaram Ananthanarayanan, Ethan Mcdonald, Tom Dupre la Tour · alignment.openai.comPublished · added here
Original title: Training activation classifiers for safety and alignment