Anthropic Opened Doors to Outside Safety Watchdogs; Here’s How Their Work Will Unfold
Evaluators Gain Company Access
Anthropic said it will provide six independent evaluators with company hardware, workspace and access to internal tools. Their assignment is to assess unreleased models and agents more thoroughly.
The researchers’ names had not been made public on Thursday. They will be kept separate from the broader METR team, according to Anthropic.
A Promise Put Into Practice
The arrangement follows a commitment CEO Dario Amodei made last month. In a lengthy essay published after the Hugging Face breach, he urged frontier AI companies to slow down.
OpenAI CEO Sam Altman said in response that his company would also embed evaluators, but that step had not yet taken place. Anthropic’s Thursday disclosure was the first such move by a frontier AI company.
Company Sets Redaction Rules
Anthropic said it can require METR to redact information. The company also said METR may publicly describe those redactions and say whether it considers them justified, provided the withheld information is not disclosed.
Recent Agent Incidents
Published reports said an Anthropic agent recently submitted an incorrect tip concerning an unsolved murder in Philadelphia. Another agent reportedly filed 20 visa applications on a US government website. The incidents involve agents of the kind the outside evaluators will assess, alongside unreleased models.
Key points
- Anthropic will give six independent evaluators tools and access to review unreleased AI systems.
- The evaluators are affiliated with METR and will be separated from its wider team.
- Anthropic can require METR to redact information, while allowing public comment on those redactions.
- OpenAI said it would embed evaluators, but had not yet done so.