METR’s Anthropic Assessment: AI Is Now Doubling the Pace of AI R&D

METR spent eleven weeks inside Anthropic and found it is likely improving its models about twice as fast as before, mostly thanks to its own use of AI. In “12 months to the singularity,” Matthew Berman walks through the evaluator’s November 5, 2026 report, Independent Assessment of AI R&D Acceleration at Anthropic in October 2026, and explains why it makes him nervous. This post covers the video and the full 53-page report behind it, including the caveats, redactions and canceled compute audit that a 20-minute video only has time to touch.

TLDW

METR, an independent AI evaluator, estimates that Anthropic’s benchmark progress has run at about 1.9x its historical rate since October 2025, and guesses that perhaps 80% of today’s acceleration comes from Anthropic’s own use of AI in research. It found no clear bottleneck that would stop that acceleration from compounding over the next 12 months, which is where the video’s title comes from. Berman reads this as evidence that recursive self-improvement has begun. The report itself is far more hedged, and much of the evidence behind its conclusions was redacted by Anthropic.

Thoughts

What the public cannot see in this report matters as much as the 2x number. METR says plainly that Anthropic “redacted a majority of the evidence we found most informative” for its risk conclusions, that the report “does not achieve ‘transparency about evidence,’” and that the public therefore has to rely on METR’s judgment “much more than would be ideal.” On top of that, the original contract included verifying how Anthropic splits compute between capabilities and safety, and Anthropic removed that part midway, before METR reached any conclusion. Berman ties the cancellation to Anthropic’s upcoming IPO. That is his speculation; Anthropic’s stated reason was keeping the engagement focused on risk assessment, with compute verification to be revisited later. Either way, METR’s own AEF-1 checklist marks “sufficient methodological details to allow for independent review” and “did not have authority to redact results to conceal concerning findings” as not fulfilled. To be fair to both parties, METR also says access “surpassed our expectations” and that it is “extremely unlikely Anthropic was actively trying to mislead us.” The honest summary is that the first serious outside look at an intelligence explosion in progress is a well-run audit whose most important evidence the public is not allowed to check.

Berman’s boldest reading is that METR found “nothing between us and the singularity.” The report’s sentence is narrower: METR did not see evidence for significant bottlenecks in algorithmic progress, training compute, experiment compute or data “that will take effect in the next 12 months.” The same paragraph adds that its understanding of bottlenecks “changed several times,” that it is “not confident in our current assessment,” and that it “may be missing important bottlenecks.” This came from a team of 2.5 full-time equivalents working through a narrow, mediated information pipeline, and Anthropic redacted METR’s analysis of several specific bottlenecks. Not finding a wall is not the same as there being no wall. Still, METR’s own conclusion needs no exaggeration to be alarming: “there is a significant risk that further capabilities progress could lead to even faster acceleration over the coming year.” When the most cautious party in the room says that, the 12-month framing earns some of its weight.

Berman mentions near the end that Anthropic keeps having to make its benchmarks harder, and treats it as a sign the models are “too smart.” The appendix shows a bigger problem: the measuring stick is breaking. Model 4, an unreleased internal model, scores above 50% on all 15 benchmarks it was tested on, where Opus 4.6 scored below 50% on many of them just nine months earlier. METR writes that this “makes us somewhat nervous about our ability to measure AECI now and in the future.” The headline estimate also depends on modeling choices: swapping the link function moves it anywhere from 1.1x to 2.9x, and including non-Anthropic frontier models drops it to about 1.4x, partly because Anthropic was behind the frontier before late 2025 and so started from a low base. The 2x figure is robust across most variants, but the instrument that produced it is saturating at exactly the moment it matters most.

The most interesting idea in the middle of the video is what happens when code stops being the constraint. One researcher told METR the team is “now only bottlenecked on ideas & compute, no longer bottlenecked on code.” Berman’s answer is that research taste may matter less than we think: with enough compute, a model can simply test everything. The interview quotes back him up. Researchers describe an “80% reduction in time between idea and experiment,” discovering “very promising directions ~entirely because people can spam 10 ideas” at an agent, and trying many things quickly, “most of which won’t work.” That shifts the bottleneck onto compute. Berman calls the researcher’s line “a little bit of contradiction” with METR, but there is none: METR said the acceleration was not driven by faster growth in training compute, not that compute stopped mattering. And how Anthropic allocates compute is exactly the part of the audit that was canceled.

The video says METR isolated AI as the cause, “not because Anthropic is hiring more people… It’s none of that.” The report is more careful. It is confident only that faster training compute growth is not the explanation. Beyond that, “there was not a consensus among Anthropic’s own staff and we did not find decisive evidence,” the late 2025 jump was “likely influenced by non-AI factors,” and the 80% figure is a guess “given with substantial uncertainty.” The interview table makes the split visible: of eleven researchers, two clearly support a 2x or greater speedup from Claude, three lean yes, four are unclear, and two say no. One thinks hiring is the biggest reason, and another puts AI’s share at 15 to 20% and doesn’t believe self-reported productivity. Slack polls put self-reported uplift at about 12x, which METR calls “only very loose evidence.” The case for AI-driven acceleration is real, but it is a balance of evidence, not proof.

Key Takeaways

METR staff worked inside Anthropic from August 12 to October 29, 2026, and published their public report on November 5. No money changed hands; Anthropic supplied laptops and free tokens.
Most METR staff on the project believe Anthropic is improving its models on well-specified tasks at about twice the March 2024 to October 2025 rate, and would not be surprised by anything from 1.3x to 3x.
On AECI, Anthropic’s version of the Epoch Capabilities Index, benchmark growth accelerated 1.9x (95% confidence interval 1.5 to 2.8x) after a breakpoint estimated at October 7, 2025. That is as much benchmark progress in the past year as from Claude 2.1 to Sonnet 4.5.
METR’s estimate is above the 1.7x in the Haiku 5.5 system card mainly because it includes four internal-only models, the newest of which finished training in September and October 2026.
METR guesses that about 80% of the October 2026 acceleration is due to AI, while saying the late 2025 acceleration was likely influenced by non-AI factors whose details were redacted.
The one firm attribution finding: the speedup was not caused by an acceleration in training compute scaling. The evidence for that was redacted.
Self-reported uplift from Anthropic staff Slack polls reached about 12x for the best internal model in October 2026, but METR treats this as very loose evidence and notes that personal speedup is not the same as acceleration.
AI is not yet making highly significant, fully autonomous contributions to the most important parts of model training, but researchers describe significant “signs of life.”
METR found no significant bottleneck in algorithms, training compute, experiment compute or data for the next 12 months, while saying it is not confident in that assessment.
Anthropic redacted most of the evidence METR found most informative, and canceled a planned verification of how it splits compute between capabilities and safety.
New internal models make a permanent change in the trend look more likely than a one-time jump at Mythos Preview, the hypothesis earlier Anthropic system cards had floated.
Benchmarks are saturating: Model 4 scores above 50% on all 15 benchmarks used to measure it, which METR says makes measuring progress harder from here.
This was not the “employee-like” access program Anthropic announced on October 15. Anthropic could approve or reject every data request, and METR never had direct access to Anthropic’s systems.

Chapters

0:00 What METR Was Allowed to See

Berman opens by declaring that recursive self-improvement has begun and sets the report against the recent calls from AI lab leaders to pace the frontier. He first describes METR’s access as nearly unfettered, then corrects himself: this was not the employee-like access program the labs promised, Anthropic could approve or deny any request, and it controlled what could be published. He says the findings are striking even so.

2:47 The Headline: Roughly 2x, Mostly From AI

He reads METR’s central estimate: Anthropic is likely improving its models on well-specified tasks at about twice the March 2024 to October 2025 rate, with a plausible range of 1.3x to 3x. He explains well-specified tasks as the code-heavy work around AI R&D, and highlights METR’s guess that about 80% of the current acceleration is due to AI, even though late 2025’s jump likely had non-AI causes.

4:25 The Illustrative Chart and the Opus 4.5 Moment

Berman walks through METR’s illustrative chart, which he calls fake but realistic: a historical trend line, a counterfactual line without AI, and an orange line for excess growth from AI that bends upward at the end of 2025. He links that bend to the moment Opus 4.5 changed how developers wrote code. The segment then breaks for a sponsor read.

6:28 Why Compounding Is the Worry

The concern, in METR’s words, is that acceleration from AI “could start to compound.” Berman connects it to the intelligence explosion described in Leopold Aschenbrenner’s Situational Awareness essay and to compute efficiency figures he attributes to Anthropic’s IPO filing, which are not part of METR’s report. He also points to Google’s AlphaEvolve, which found efficiency gains in Google’s own infrastructure.

8:27 “We Did Not Identify Any Clear Bottlenecks”

Berman reads the line he finds most alarming: METR saw no clear bottlenecks or steep diminishing returns that would stop compounding acceleration, and believes there is a significant risk of even faster acceleration over the coming year. He calls it “a wild statement.” He also quotes the benchmark finding that Anthropic made as much progress in the past year as between October 2023 and October 2025.

9:41 The AECI Curve and Ruling Out Compute

On the AECI chart, Berman traces a roughly linear line through 2025, a bend at Opus 4.5, and continued steepening after Mythos Preview, which he describes as the acceleration itself accelerating. METR’s fit is more modest: two straight-line trends joined at a single breakpoint, with no clear sign of continuously accelerating growth in AECI, though it says redacted evidence makes that more plausible. He notes METR received “convincing evidence” that the speedup did not come from faster growth in training compute, while Anthropic redacted the details. He adds that Anthropic struggled to serve its models earlier this year.

11:25 What Anthropic Researchers Told METR

METR’s lines of evidence included interviews, self-reported uplift, headcount, lines of code, experiment usage and reflections from leadership. Berman calls the interviews informal but a good signal, then reads anonymous quotes: a greater than 2x speedup, AI acceleration larger than all other effects, and “I don’t think four is crazy” for years of progress per year.

12:41 Bottlenecked on Ideas, and Signs of Life

One researcher said the team is now bottlenecked only on ideas and compute, not code. Berman argues that picking research directions may be the last human edge, but with enough compute a model can test everything and needs less intuition. He reads METR’s finding that AI is not yet making fully autonomous contributions to the most important parts of training, while researchers describe “signs of life,” and mentions recent math results from OpenAI.

15:01 How the Engagement Worked, and the Canceled Compute Audit

METR sent 2.5 full-time equivalents starting in mid-August to answer three questions: has growth accelerated, how much is due to AI, and are there bottlenecks. Anthropic could unilaterally approve or reject requests, and the two sides often compromised on data that was informative, not extremely sensitive and easy to share. Anthropic also cut a planned verification of its compute split between capabilities and safety. Berman links the cut to the IPO and adds, “Interpret that how you will.”

18:03 Internal Models, Saturated Benchmarks and What Comes Next

From the appendices, Berman highlights the unreleased Models 2, 3 and 4, which make a permanent trend change look more plausible than a one-time jump at Mythos Preview. He also notes that benchmarks are being saturated and that researchers say models are becoming useful for autonomous experimentation. He closes by saying the support of an independent evaluator is what unnerves him, and points to new research on models writing weights directly.

Notable Quotes

“We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI.”
METR, on how much of Anthropic’s speedup comes from its own use of AI

“That is a wild statement. They’re basically saying we’re off to the races.”
Matthew Berman, reacting to METR finding no clear bottlenecks to compounding acceleration

“Now only bottlenecked on ideas & compute, no longer bottlenecked on code.”
An anonymous Anthropic researcher, from METR’s interview notes

“Like the intuition will help save money, save compute, but ultimately if you could just go out and test everything, like that’s the same thing.”
Matthew Berman, on whether AI needs research taste

“The public has to rely on METR’s judgment much more than would be ideal, given the weight of the questions answered here.”
METR, on the effect of Anthropic’s redactions

“This lack of benchmarks which are difficult for Model 4 makes us somewhat nervous about our ability to measure AECI now and in the future.”
METR, on benchmark saturation in the report’s appendix

“And where I start to get especially nervous is when we stop being able to understand AI completely.”
Matthew Berman, closing the video

Watch Matthew Berman’s full video here, and read METR’s full report here.