OpenAI proposes pair-matched and post-control metrics to complement g-mean² for monitorability
Overview
OpenAI's Alignment Research Blog proposes two metrics to complement g-mean², a dataset-level score of how well a monitor discriminates target behavior.
The first, a pair-matched metric, compares models only on task instances both have valid behavior for. The second, a post-control integration metric, estimates how much bad output remains after monitor-gated rejection.
The post says a monitor's score on one model's realized dataset can be confounded by differences in which tasks each model produces valid behavior on.
Written by AI from the articles below · updated Oct 20, 12:53 PM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
- OpenAI Alignment Research BlogOfficialOpenAI proposes pair-matched and post-control metrics to complement g-mean² for monitorability
AIOpenAI's Alignment Research Blog proposes two metrics to complement g-mean², a dataset-level score for how well monitors discriminate target behavior. A pair-matched metric compares models only on task instances supported for both, and a post-control integration metric estimates how much bad output remains after monitor-gated rejection. The post says a monitor's score on one model's realized dataset can be confounded by differences in which tasks each model produces valid behavior on.
Heat Trend
Not enough continuous observations to show a trend yet.