Latest
  1. Anthropic publishes guidance on building AI safety evaluations, with figures from its own tests
  2. Pentagon renames its Chief Digital and AI Office the Super Intelligence Office
  3. Donor software firm Mythos sues Anthropic over its Mythos model name
  4. OpenAI launches ChatGPT Wallet so its dots agents can complete purchases
  5. Cohere completes its AISLE acquisition and adds AI code-review company cubic
  6. Firm founded by Alice Huang listed restricted Nvidia servers in Chinese filings, Bloomberg reports
  7. Senate Banking chair asks Jensen Huang and Howard Lutnick to testify on Nvidia
  8. Tasmania’s AI data centres would each run their own community benefit fund

Latest / Frontier models and safety

ResearchFrontierInternationalConfirmed

Anthropic publishes guidance on building AI safety evaluations, with figures from its own tests

See how this story connects

Source: AnthropicAlso reported by Unite.AI

Anthropic published recommendations on 13 November 2026 for anyone who builds or relies on AI safety evaluations, the tests that inform its decisions on whether a model is ready to deploy. It says its single-turn safety evaluations are saturated: every recent model scores between 94% and 98% on them through its API, so they now serve mainly as regression checks, while multi-turn tests vary much more. It recommends preregistering the analysis, running at least 100 items for a complex multi-turn evaluation and 1,000 for a single-turn one, checking AI graders against human labels, and reporting confidence intervals, which exceed 10 percentage points for some of its multi-turn evaluations. In one romance-scam evaluation Claude Opus 5 scored 98% and Opus 5.5 94%, a difference that was not statistically significant. Testing Claude Fable 5.1, results on the raw API and on claude.ai with its system prompt differed by up to 34 percentage points.

Why it matters

Developers and evaluators who compare model safety scores have a lab’s own figures on how far results move with sample size, grader and system prompt.

When
13 Nov 2026
ID
WTA-N-2026-0897
Who
Anthropic

Summary written with AI by wheresthe.ai, from Anthropic and the reporting credited above. Checked against the source on 13 Nov 2026.

Read the original at Anthropic

Source copy: saved on 13 Nov 2026 to check this story and not republished, because the text belongs to its publisher. Document the source offers: document 1 (PDF). Also held by the Internet Archive.

Cite this story

WTA-N-2026-0897: “Anthropic publishes guidance on building AI safety evaluations, with figures from its own tests.” wheresthe.ai, 13 Nov 2026. https://wheresthe.ai/d/intl-anthropic-publishes-guidance-on-building-safety-evaluations-2026-11/ (source: Anthropic).

Spotted an error? Email [email protected].