Profoundd checked a Nov. 7 YouTube video by Matthew Berman, “12 months to the singularity” (also shown under the title “The most important report in AI right now”), against the report it describes, METR’s Nov. 5 assessment of AI research at Anthropic, and against Anthropic’s own filing for its initial public offering (IPO), its first sale of stock to the public. Disclosure: this article was drafted by Claude, an AI model made by Anthropic, the company the report studies.
AI is now helping build the next generation of AI fast enough to show up in the numbers, according to METR, a Berkeley nonprofit that evaluates AI systems. After 11 weeks with internal data from Anthropic, METR estimates the company is improving its models on well-defined tasks at about twice the pace of 2024 and 2025, and guesses that roughly 80% of that speed-up comes from researchers using AI. METR gave both numbers with wide uncertainty and says Anthropic redacted “a majority of the evidence we found most informative,” so the public has to rely on METR’s judgment. The report arrived weeks before Anthropic’s planned stock listing, which the AP says could be the largest ever, and two months after Anthropic’s chief executive, Dario Amodei, urged the industry to “slow the pace at which we improve the capabilities of AI models.” Anthropic co-founder Sam McCandlish wrote on X that many of METR’s conclusions match his own: “AI is speeding up our research, but it’s hard to say exactly how much.”
METR is confident of one thing: the speed-up did not come from a faster rise in the computing power, or compute, used to train Anthropic’s models. That does not mean Anthropic used less compute. Its IPO filing shows compute and infrastructure costs up 382% in a year, and says the value of its own staff’s Claude use, at list prices, grew more than 250-fold from January to September 2026. Anthropic says it believes the compute it devotes to that internal use compounds its gains, “continuously accelerating our pace of development.”
The claims, checked
- Recursive self-improvement “has already begun.” In the sense Anthropic itself uses; not yet in the fully automatic sense. Anthropic’s filing says “each new version of a Claude model directly accelerates development of the next” and calls this “closing the loop.” Amodei wrote in September that this is “starting to happen across the industry.” METR says AI “is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process,” though researchers described “signs of life.”
- Anthropic is improving its models at “around twice the rate” Confirmed as METR’s central estimate, which Berman reported with its range. “Most METR staff” hold the estimate, with “internal disagreements on the team,” and METR “would not be surprised” by anything from 1.3 to 3 times, a range Berman read aloud. METR’s benchmark index shows 1.9 times the earlier pace, with a 95% confidence interval (the range the true value likely falls in) of 1.5 to 2.8.
- “It’s not because Anthropic is hiring more people, getting better AI researchers, having more compute. It’s none of that.” Mostly right about today’s speed-up; “none of that” goes further than METR. METR “confidently” ruled out a faster rise in training compute and does “not think the other factors we investigated would explain the sustained acceleration.” It puts AI’s share at “perhaps ≈80%,” “given with substantial uncertainty.” Berman himself relayed METR’s view that non-AI factors likely drove the late-2025 jump. Anthropic’s headcount rose about 170% in the year to Sept. 30, and two of the 11 researchers METR quoted credited hiring or other causes more than AI.
- “the compute wasn’t a bottleneck” Matches METR’s findings, which cover training compute and the next 12 months; the speed-up still ran on fast-growing compute. METR found no faster rise in training compute behind the speed-up and “did not see evidence for significant bottlenecks” in training or experiment compute for the next year, though it is “not confident” in that and Anthropic redacted its analysis of specific bottlenecks. The compute spent running Claude for Anthropic’s own staff grew fast, and the filing says past compute shortfalls “impacted both product development and deployment.”
- “Now only bottlenecked on ideas and compute. No longer bottlenecked on code.” Supported by most of the researchers METR quoted and by Anthropic’s own account. The words are one researcher’s, but most of the 11 interviewees described code or experiments becoming far cheaper, including one who doubted AI was the main cause of the speed-up. Anthropic’s explainer on AI building AI says the work “now costs almost nothing in human time, even if it still has costs in compute,” and adds that “human code review has become a new bottleneck.”
- “proof from a third-party evaluator that had near unfettered access” The evaluator and its unusual access are real; “near unfettered” and “proof” go further than METR. Early in the video Berman noted that Anthropic could approve or deny requests. METR calls the access “unprecedented,” but Anthropic “could unilaterally approve or reject our requests” and redact the public report, which METR says “does not achieve ‘transparency about evidence.’”
- “The compute efficiency was 67x. Then Q1 2026 break even. Q2 2026 it is 2x.” Right in shape, read from the original filing. Anthropic’s Oct. 26 filing charts revenue per dollar of compute, which it labels “compute efficiency”: 0.69 at the end of 2025 (the captions’ “67x” is evidently “.67x”), 1.40 in early 2026, the first quarter above break-even, and 2.08 in mid-2026. A Nov. 2 amendment added 1.72 for the quarter ended Sept. 30, a drop Anthropic blames on $5.0 billion of short-term compute that “costs significantly more.” The ratio counts revenue, which Anthropic books at full price on sales through cloud partners, so it measures sales as well as engineering.
- “There is literally no blocker between what is happening now and what artificial super intelligence or recursive self-improvement looks like.” The one-year core is METR’s finding, held with low confidence; superintelligence is beyond the report. Berman read METR’s sentence aloud: it found no bottleneck that would stop faster acceleration in the next 12 months and calls further acceleration “a significant risk.” METR adds that “we may be missing important bottlenecks” and that its view “changed several times.” The public report makes no forecast about superintelligence; METR’s forecast for a future model was redacted.
What METR found
METR’s team, the equivalent of 2.5 full-time staff, worked from Aug. 12 to Oct. 29 through datasets and documents Anthropic shared; it never had direct access to Anthropic’s systems. Anthropic supplied benchmark scores for unreleased models, training-compute data, staff surveys, counts of code changes and about 15 interviews, which METR says “surpassed our expectations.”
The main evidence is a benchmark index: Anthropic’s version of the Epoch Capabilities Index, which combines scores on many standardized tests into one number. Since about Oct. 7, 2025, that index has risen 1.9 times faster than before, “implying that Anthropic has made roughly two years of historical progress over the last year.” Other reasonable statistical choices give anywhere from 1.1 to 2.9. When other companies’ models set the earlier trend, the figure is 1.4; METR says that could mean Anthropic was partly catching up from behind, or could understate the change, and it gives reasons for either reading. METR’s 1.9 is higher than the 1.7 in Anthropic’s own system card for its Haiku 5.5 model mainly because METR included unreleased internal models. Anthropic’s newest internal model scores above 50% on every benchmark it was tested on, which METR says makes it “somewhat nervous about our ability to measure” progress, though the result “appears reasonably robust” to that problem. METR warns the gains may be concentrated in tasks that are easy to grade. It also says redacted evidence “gave us some indication that acceleration could be above 2x and increasing over time.”
The evidence that AI caused the speed-up is softer. In Slack polls, Anthropic staff put their own speed-up from the newest internal model at about 12 times on average; METR calls that “only very loose evidence,” because the answers came from an unrepresentative group and with little thought. METR’s interviews were “centered around a 2x acceleration,” and it rated them “mildly in favor” of a doubling. Outside Anthropic, METR’s own 2025 experiment found experienced open-source programmers took 19% longer with early-2025 AI tools while believing they had been sped up by 20%. Its February 2026 follow-up found AI cutting task time by about 18% for returning programmers and 4% for newly recruited ones, with ranges that include no effect; METR called the estimate “likely a bad proxy” because of who dropped out of the study, and said programmers are likely “more sped up from AI tools now.” METR’s companion review found that Anthropic’s August public risk report said AI was speeding its research by “less than a factor of 2” while its private figures implied 1.97, and that the public report “overall understates the level of acceleration.” METR agreed with Anthropic that risk from its July models was “low,” and said most researchers at both organizations lean toward a doubling not yet having occurred by July.
Researchers say the limit moved from code to ideas
The interviews describe what changed. One researcher described an “80% reduction in time between idea and experiment.” Another said: “People don’t try [redacted] things bc it takes 1-2 days to implement per idea, but now its ~free ...” A third said the team had found “very promising directions ~entirely because people can spam 10 ideas at [agent-scaffold] and see what sticks,” where the redacted name refers to Anthropic’s software for running AI agents. Anthropic’s explainer says the human advantage, “for now, is research taste and judgment,” meaning the choice of which problems matter and which results to trust. METR’s interviewees described the same pattern: “Claude lets researchers try many things quickly, most of which won’t work.” More experiments need more compute to run them, and the researcher who said code was no longer the limit named compute alongside ideas.
Speeding up one step moves the bottleneck to the next, a rule computer designers have known since Gene Amdahl described it in 1967. Anthropic says it has seen that happen: “human code review has become a new bottleneck.” Not everyone credited AI with the speed-up. One interviewee said, “Don’t believe in self-reported productivity increase,” and guessed AI explained 15 to 20% of it; another named hiring as “probably the biggest reason.” Anthropic’s filing says Claude wrote more than 90% of the contributions to its codebase in September 2026 and that each engineer’s daily code output was more than 45 times the 2024 average; those are the company’s own figures.
Was compute the bottleneck?
METR made two separate findings about compute, and Berman read both aloud. The first is about cause: METR is “confident that the acceleration was not due to an acceleration in the scaling of training compute,” meaning the computing power for training did not start growing faster. Anthropic redacted the details. The second is about limits: METR “did not see evidence for significant bottlenecks in algorithmic progress, training compute, experiment compute, or data that will take effect in the next 12 months,” but says its understanding of them “changed several times.” METR notes that its other research points both ways, including “our estimates of future compute bottlenecks point to limits on scaling,” a 2025 METR-affiliated study of industry-wide compute growth through 2030.
Neither finding covers the compute spent running AI. That is where Anthropic’s spending grew: the value of internal Claude use passed the company’s monthly payroll in April 2026, excluding stock pay, the filing says. Anthropic also says it has “consistently operated in a compute-constrained environment where customer demand exceeds available compute,” and that “when customer demand or research needs exceed” its long-term contracts, it buys short-term compute at a premium. So compute is scarce at Anthropic, and the speed-up came with heavy spending on running AI, though not from a faster rise in training compute. Anthropic says Claude “has helped write and iterate on compute multipliers that are central to efficient inference,” its term for techniques that get the same answers from less computing.
The public cannot check how Anthropic divides its compute. METR’s contract originally included verifying how much compute goes to making models more capable versus making them safe. METR reports that Anthropic canceled that part midway, saying it wanted to keep the work focused on risk assessment for now and would revisit it later. Berman suggested the IPO might be the reason; METR gives only Anthropic’s stated reason. McCandlish defended the redactions: “We redacted a lot, and I accept that this makes it harder for others to check METR’s work,” he wrote, adding that the details are “commercially sensitive” and risk “handing out a roadmap for others to go faster.” He said Anthropic will publish its own analysis “in the coming weeks.”
An old argument: more chips, or better ideas
Whether AI progress is limited by computing power or by ideas is an old argument. In 1965 the statistician I.J. Good described an “intelligence explosion”: a machine smart enough to design better machines would set off a runaway loop. In 2019 the computer scientist Rich Sutton wrote “The Bitter Lesson,” arguing that “general methods that leverage computation are ultimately the most effective, and by a large margin.” Seventy years of AI, in his account, rewarded more computing over clever human design.
The record also shows the same computing buying more every year. OpenAI researchers Danny Hernandez and Tom Brown found in 2020 that the compute needed to train an image classifier to a 2012 benchmark level fell 44-fold by 2019, halving about every 16 months. DeepMind’s 2022 Chinchilla model used the same compute as its 280-billion-parameter Gopher model (parameters are the numbers, or weights, a model computes with) but put it into a smaller model trained on more data, and it beat Gopher across the board. Epoch AI, a research group that tracks AI trends, estimated in 2024 that the compute needed to reach a given level of language-model performance halves about every eight months, though it also found that more compute contributed more to progress than better algorithms did. DeepSeek reported in December 2024 that its V3 model took 2.788 million GPU-hours (hours on one graphics chip) to train, with only 37 billion of its 671 billion parameters active for each token, a word or piece of a word; the company noted the figure excludes “prior research and ablation experiments,” the test runs that come before the final one. In May 2025 Google DeepMind said its AlphaEvolve system “continuously recovers, on average, 0.7% of Google’s worldwide compute resources” and cut the training time of Google’s Gemini models by 1%. Berman described this as a 1% efficiency gain.
The argument over a self-improving loop turns on whether compute and researchers can stand in for each other. In the summary of Tom Davidson of Forethought, a research nonprofit, Epoch researchers argued from economists’ estimates that the two are complements: adding AI researchers without adding compute would soon hit a ceiling. Davidson answered in April 2025 that those estimates were shaky and put “a 10-40% chance” on a runaway loop happening despite compute limits, which he expected to bite late rather than early. A 2025 paper by Parker Whitfill, an MIT economics doctoral student who is now on METR’s staff and one of the Anthropic report’s four authors, and Yale’s Cheryl Wu found the answer flips with the method: one model of the AI companies’ data called compute and research labor substitutes, the other complements. Anthropic’s own explainer imagines a future in which AI fully develops itself; in that world, it says, “the pace of progress in AI development becomes determined entirely by the availability of compute.” METR’s report shows progress speeding up while training-compute growth did not. Whether compute becomes the ceiling later is what the report could not see.
What it means for a Mac Studio or a DGX Spark
In Profoundd’s view, the part of this story that carries over to a desk machine is that Anthropic’s speed-up ran on running AI, not on training it, and running AI is most of what a desk machine does. The limit there is usually memory. Each token a model writes requires reading its active weights from memory, so writing speed is capped at roughly memory bandwidth, how fast data moves out of memory, divided by the size of those weights. Reading a long prompt is limited by raw computing power instead.
NVIDIA’s DGX Spark has 128 GB of memory at 273 GB per second. It went on sale in October 2025 at $3,999; NVIDIA raised the price to $4,699 in February 2026, citing memory supply, and to $6,950 on Oct. 2, when it announced a 64 GB model at about $4,999, The Register reported. Apple’s Mac Studio with the M5 Ultra chip, on sale since Sept. 22, starts at $5,499 and offers up to 512 GB at 1.2 TB per second; the M5 Max version starts at $2,499 with up to 128 GB at 614 GB per second, according to Apple. A 70-billion-parameter model stored at 8 bits per weight is about 70 GB, which caps a Spark near 4 tokens per second; LMSYS, a research group that tests AI software, measured 2.7. On an M5 Ultra the same ceiling would be about 17.
Ways to get more work from the same machine, each with a source:
- Choose models where only part of the network works on each token. These “mixture-of-experts” models need memory for all their weights but read only a fraction for each token. OpenAI’s gpt-oss-20b has 21 billion parameters but uses 3.6 billion per token; LMSYS measured it on the Spark at 49.7 tokens per second. That test used 4-bit weights and different software from the 70-billion-parameter result, so it is not a like-for-like comparison, but the arithmetic is the point: about 2 GB read per token instead of 70.
- Store weights at 4 or 8 bits instead of 16. Halving the bits halves the memory each token must read, so the same machine holds a larger model or runs the same one faster. Check the answers on your own tasks, since lower precision can change a model’s output.
- Run many requests at once. Reading the weights once can serve many conversations. LMSYS measured Meta’s Llama 3.1 8B on the Spark at 20.5 tokens per second with one request and 368 with 32 running together. A batch of small research agents uses a desk machine far better than one chat window.
- Reuse the long part of the prompt. Agent work reads far more than it writes: in Anthropic’s internal benchmarks a typical task used about 2.5 million input tokens and 72,000 output tokens, METR reports. Local servers can keep a prompt’s processed form in memory, so a repeated set of instructions or a document is not reprocessed on every request.
- Let a small model draft and the big one check. In “speculative decoding,” a small, fast model guesses the next several tokens and the large model confirms them in one pass, spending some extra computing to save waiting time. LMSYS reported up to a 2x speed-up on the Spark with a method called EAGLE3.
- Supply facts instead of retraining. Fine-tuning, retraining a model on examples, made models invent more sources in a small test Profoundd ran on its own Spark in August 2026. Asked 15 questions with no documents supplied, one answer each, the untrained Qwen2.5-14B invented no web addresses; a version tuned on 1,916 examples invented 5, and one tuned on 7,840 invented 11. That test did not measure retrieval, which supplies documents to the model at question time, but retrieval is how Profoundd’s own system gives models facts, and it needs no training compute.
What the report could not show
METR’s project was not the “employee-like” access program Anthropic described in an Oct. 15 update; METR got most of its data through two Anthropic staff members, interviewed about 15 others, and says it could not publish enough of its method for independent review. Claude was “deeply involved in” the majority of the data and analysis METR received, in writing a significant share of the documents it received, and in the majority of METR’s own analysis. METR checked that work against human interviews, found no sign of systematic bias, and calls it “extremely unlikely Anthropic was actively trying to mislead us,” while adding that “greater access would allow us to increase the degree of adversarial robustness.”
METR also disclosed the pressures on it. Its work with unreleased models and its use of free tokens, the units of AI usage companies donate, “incentivize a cordial relationship with AI companies.” The pressure it ranked highest was the effect on its dealings with other companies and the “possibility of Anthropic retracting access.” It felt some of Anthropic’s feedback on the companion review was “overly forceful,” and Anthropic tightened its redaction guidance in the final days. One METR team member had been in job talks with Anthropic at the start, declined an offer and held equity in a competitor; that person’s work was checked by a colleague without those conflicts. No fees were paid in either direction. In his September essay, Amodei had named METR as a suitable outside evaluator, the AP reported.
Anthropic describes AI building AI in two registers. Its IPO filing presents “closing the loop” as “one of the most significant structural advantages in frontier AI,” and Amodei’s essay warned that AI building AI, “left unchecked, ... could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.” Profoundd did not contact Anthropic or METR for this story.
Profoundd's read
Berman’s video gets the headline right and reports most of METR’s caveats, including the 1.3-to-3 range, the 80% estimate, Anthropic’s veto over requests and the redactions. Three phrases go further than METR: “proof,” “none of that” and “no blocker.” METR held its bottleneck finding with low confidence and said its view changed several times. On compute, Berman and METR agree that a faster rise in training compute did not cause the speed-up. Anthropic’s filing supplies the other half: the speed-up came with rapidly growing compute spent running AI for its own staff, and the company says it is still short of compute. That is the lesson that carries to a Mac Studio or a DGX Spark. Anthropic’s gains came from running AI on more, cheaper experiments, and owners of one machine get more from it by choosing models that fit its memory, running work in batches, reusing what has already been computed, and supplying facts rather than retraining.
