Knowledge Engine

METR estimates Anthropic's capability gains have roughly doubled in pace, but most of the evidence is redacted

·no read data

What happened

On 5 November 2026 METR, a nonprofit that evaluates frontier AI systems, published “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026”. The work was done by METR staff, about 2.5 full-time equivalentsfull-time equivalent A way of counting staff effort in which, for example, two people each working half time count as one full-time worker., between 12 August and 29 October 2026, with the final report submitted on 3 November. The engagement looked at three questions: whether the growth rate of Anthropic’s internal capabilities has sped up, how much of any speed-up is due to AI, and whether there are near-term bottlenecks.

Anthropic’s Responsible Scaling PolicyResponsible Scaling Policy Anthropic's published framework that ties how it develops and releases AI models to the risk levels those models are judged to reach. lists AI R&D acceleration as a key risk indicator, which is the backdrop for the exercise. METR says it did not pay special attention to the policy’s thresholds. On the same day METR published a separate review of the automated R&D section of Anthropic’s August 2026 Risk Report, in which it agreed that the models assessed there posed low catastrophic risk but judged that an early-warning threshold for doubled acceleration had plausibly already been crossed by mid-July.

What is claimed vs. what is verified

METR’s headline estimate comes from AECI, which it describes as Anthropic’s version of the Epoch Capabilities IndexEpoch Capabilities Index A single score, developed by the research group Epoch AI, that combines results from many AI benchmarks to track overall model ability over time., a single score combining many benchmarksbenchmark A fixed set of tasks used to score and compare AI systems.. Including unreleased internal models, dated by when most Anthropic employees could use them, METR estimates growth since October 2025 at 1.9 times the earlier rate, with a 95 percent confidence intervalconfidence interval A range of values, calculated from data, that is meant to show how precisely a quantity has been estimated. of 1.5 to 2.8. Different modelling choices gave between 1.1 and 2.9 times. Most METR staff believe Anthropic is improving on well-specified tasks at about twice its March 2024 to October 2025 rate, and would not be surprised by anything from 1.3 to 3 times.

On causes, METR says it is confident that faster growth in training compute did not drive the acceleration, and puts the share of October 2026 acceleration due to AI at perhaps 80 percent, with substantial uncertainty. It found no clear bottlenecks in algorithms, compute or data but says its view changed several times and it is not confident. Staff polls reported self-assessed uplift of about 12 times; METR calls these very loose evidence and says they are not acceleration rates.

What cannot be verified by anyone outside is most of the underlying evidence. METR states that “Anthropic redacted a majority of the evidence we found most informative”, including algorithmic progress estimates, bottleneck details and forecasts. Anthropic says the redactions remove information that could accelerate other developers. Anthropic also cancelled a planned check of compute allocation partway through. METR worked on Anthropic devices, received free tokenstoken A small chunk of text, often a word or part of a word, that a language model reads or writes as one unit., did not sample from non-public models, and says it took no payment. The independent coverage found for this post (Resultsense, Unite.AI) summarises the report; neither adds outside expert analysis or reproduces the numbers.

Why it matters

The report is an outside estimate, based on internal data, of how fast a frontier lab’s own capability growth has been changing, a question usually answered only by the lab itself. If the estimate holds, capability gains that once took two years arrive in about one, which compresses the time available for safety testing and policy responses. It also sets a precedent for how outside evaluation with company-controlled redaction works in practice.

Open questions and other views

METR lists reasons it might be overestimating: non-AI factors could explain all or most of the acceleration, gains might be confined to well-specified tasks, and Anthropic’s historical baseline may have been unusually slow. It also lists reasons it might be underestimating, including that its index could understate real-world impact. Benchmark saturation, with the newest internal model above 50 percent on all 15 benchmarks, may make future measurement harder.

Supporters of this kind of engagement can point out that a lab opening internal data to an unpaid outside group, and allowing it to say what was redacted, is more than the public otherwise gets. Critics of the arrangement focus on the redactions: Resultsense, in its own commentary, presents them as the main lesson for policy, since conclusions resting on hidden evidence ask the public to trust the evaluator. METR itself says the redactions leave the public relying on its judgment more than would be ideal. Whether this model of oversight is adequate is unresolved.

Sources

  • METR, “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026”, 5 November 2026. https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/
  • METR, “Review of the ‘Risks from automated R&D’ section in Anthropic’s Risk Report (August 2026)”, 5 November 2026. https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review/
  • Resultsense, “METR: AI has likely doubled Anthropic’s pace of model improvement”, 6 November 2026. https://www.resultsense.com/news/2026-11-06-metr-anthropic-ai-rd-acceleration/
  • Unite.AI, “METR Estimates Anthropic’s Model Capability Growth Has Nearly Doubled”, 5 November 2026. https://www.unite.ai/metr-estimates-anthropics-model-capability-growth-has-nearly-doubled/

Check yourself

Answers are checked in your browser. Nothing is recorded or sent anywhere.

  1. What does the 1.9x figure in METR's report measure?

  2. What could Anthropic do under the terms of the engagement?

  3. How does METR treat the staff survey showing about 12x self-reported uplift?

  4. A reader says the finding is worthless because Anthropic controlled what METR could see. Which response fits the report best?

Questions without answers

  1. If an evaluator can see evidence the public cannot, what would make you more or less willing to accept its judgment on that evidence?
  2. The headline estimate rests on benchmarks that favour well-specified, easily graded tasks. What kind of measurement would tell you whether real research is speeding up in the same way?

Back to the archive