Anthropic has placed 6 safety evaluators from the outside nonprofit METR inside the company, with access to internal systems, tools and staff roughly equal to that of its own risk teams.
Six METR staff have company hardware, office space, and internal access “roughly on par” with Anthropic’s own risk assessment teams.
METR will publish its own verdicts on how risky Anthropic’s models and practices are, and Anthropic cannot block them, only redact a few predefined categories such as customer data, a decision METR can publicly challenge.
Other evaluators, such as SecureBio, start at the lowest of 3 new access tiers, where Anthropic still controls what they see and must approve what they publish.