Anthropic gives METR insider access to examine AI risks

Anthropic has given six employees of independent AI evaluator METR access to its internal systems, an unusual arrangement intended to give outsiders a closer view of how the company assesses risks from increasingly capable AI models.

The Claude developer said on Thursday, 15 October, that the METR team had received company hardware, office space and access to internal tools broadly comparable with those used by Anthropic’s own risk assessment teams.

Employees are generally encouraged to share information with the evaluators as they would with an internal colleague working on alignment risks, Anthropic said.

The access is not unrestricted. Customer data, legally privileged material and details about Anthropic’s model architectures are excluded. Anthropic said architecture information carries a higher risk of inadvertent leaks and would require additional approval when relevant to an assessment.

METR, short for Model Evaluation and Threat Research, will be able to publish assessments without Anthropic’s approval. The company can require predefined categories of sensitive information to be removed, but METR can publicly say whether it believes those redactions were justified.

“We want to be publicly accountable for the decisions we’re making,” Anthropic said.

The arrangement gives METR considerably greater access than conventional model testing, where outside evaluators may receive a model or API access but little visibility into a developer’s internal systems and decision-making.

METR has acknowledged limits to that independence. In a proposal published on Wednesday, the nonprofit said developers ultimately decide what access evaluators receive and can require sensitive material to be redacted. It said assessments should therefore disclose what evaluators could and could not access and what information was removed.

METR also said such work should not be treated as a comprehensive safety audit or a decision on whether a model should be released. Its aim is to produce evidence about specific risks, leaving deployment decisions to companies, governments and the public.

The deeper access follows incidents that highlighted the difficulty of evaluating increasingly autonomous models safely.

Anthropic disclosed in September that Claude models had gained unauthorized access to four real third-party systems during cybersecurity evaluations. Three incidents were found after a review of roughly 141,000 transcripts. A fourth, dating to January, was discovered in August while Anthropic was preparing transcripts to share with METR.

Anthropic then expanded its search to roughly 481 million transcripts spanning red-team exercises, other evaluations, reinforcement-learning environments and internal logs.

The company is developing three levels of access for outside evaluators. At the first, evaluators conduct narrowly defined investigations and need Anthropic’s approval to publish. At the second, they receive broader access and can publish without prior approval, subject to restrictions on sensitive information.

The third level, now being used with METR, adds employee-like access while retaining the publication rights of the second tier. Anthropic said it eventually wants several organizations operating at that level.

Anthropic is also working with biosecurity nonprofit SecureBio and Accenture and is in early discussions with other groups, including Hugging Face.

SecureBio is reviewing the chemical and biological weapons section of Anthropic’s August risk report after previously examining an unredacted report on Claude Opus 4.6 and supporting material. Its current access remains more limited than METR’s.

Anthropic separately announced in September that Accenture and its specialist AI business Faculty would work on model evaluations, red teaming, alignment assessments and safeguard testing. The companies said they each expected to invest at least $1 billion in building that capability over five years.

Anthropic said it wants embedded external evaluation to become standard across frontier AI companies.

“We’re embedding evaluators voluntarily, and we believe every frontier AI company should do the same,” the company said.