Coinbase says a small AI model it trained on its own fraud records now catches fraud better than one of the most capable commercial models. In a three-part engineering series published October 6 to 8, the company reported that Qwen3.5-9B, a nine-billion-parameter model from Alibaba whose trained weights anyone can download, beat Anthropic's Claude Opus 4.5 on all four measures of its fraud test after training that cost under $100, and answered in less than half the time. The test is Coinbase's own, built from past transactions that it does not release. A research paper on the same test by two of the series' authors gives a fuller picture: Opus 4.5 was not the strongest commercial model on it, and against the best system built on a commercial model the small model roughly tied on the combined score rather than winning, while costing far less to run at high volume.
Chief executive Brian Armstrong drew a wider lesson in a post on X on October 11. "I think we'll see more of this pattern inside companies," he wrote. "The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves." He said Coinbase is "making it easy to fine tune models internally for our employees," with a ready-made command in its internal AI tools, "/fine-tune-and-serve-llm," that starts a training project, and added: "I imagine we'll have some backend services that could be replaced entirely by this new architecture."
The fraud agent, and where the 30% figure comes from
The model works inside Onramp, the Coinbase service that lets people buy cryptocurrency inside partner apps, including as guests without a Coinbase account. Because guests have little history, Coinbase's existing machine-learning model and rules screen each payment first. An AI agent then reviews selected transactions against the buyer's recent activity and returns only a risk level from 1 to 5. It can block a payment the first checks allowed, but it cannot reverse a block. Coinbase first ran that agent on Claude Opus. In a live test in which half of the traffic got the extra review, it reported 30% fewer fraudulent transactions and 22% less fraud value than the existing checks alone, with projected savings about three times the cost of running the model. Those figures appear in much of the coverage of the small model, but they come from the Opus-based agent, measured before the small model was trained. The series reports the small model's speed on live traffic, not its live fraud results.
The test, and how the small model was trained
Coinbase's test set holds 16,140 past transactions from 7,293 users over nine weeks, including 813 confirmed frauds. It scores four things: precision, the share of flagged transactions that really were fraud; recall, the share of fraud caught; F1, a single score that balances the two; and dollar-weighted recall, the share of fraud dollars caught. All the differences below are in percentage points, the plain gap between two percentages.
The research paper, SR-Fraud, was posted to arXiv in September by Xuwei Tan, who did the work as a Coinbase intern, Yao Ma of Coinbase, and Xueru Zhang of Ohio State University; it carries a note that it states personal opinions, not company policy. It also describes how the data was split, which the blog does not: "model training and operating-point selection for all systems use earlier, disjoint periods of the same stream that are not part of the evaluation cohort." No transaction appears in both training and testing; the same customers can, which the authors say is expected in a payment stream.
The training itself was reinforcement learning against known outcomes. The model rated about 4,000 past transactions, a fixed scoring program checked each rating against whether the transaction had turned out to be fraud, and the model was pushed toward ratings that scored well; no other AI model judged its answers. The scoring was weighted so that a model calling everything legitimate, or everything fraud, gained nothing. Only a small set of added parameters was trained, a method called LoRA, on one Nvidia RTX PRO 6000 graphics card, and Coinbase puts the total cost at under $100.
What the fuller numbers show
The blog reports only the gaps between models. The paper reports every system as a difference from a conventional fraud model, CatBoost, that the authors trained on the same earlier data, which puts all the systems on one scale. Working alone, one transaction at a time, Opus 4.5 scored 1.8 points below the conventional model on F1 and 15.3 points below it on dollar-weighted recall. Sonnet 4.6, another Anthropic model, scored 1.8 points above it on F1. The trained 9B model scored 7.8 points above it on F1 and 20.1 points above it on dollar-weighted recall; a version of the conventional model retrained every month scored 4.2 above on F1. The paper's best system on F1, Sonnet 4.6 plus a "reflection" layer that proposes new fraud rules from past mistakes once their outcomes are confirmed, and keeps only those a fixed checker accepts, scored 8.0 points above on F1 and 11.7 on dollar-weighted recall. The paper sums up the small model's result as "similar detection performance while reducing request latency and, at sufficient sustained volume, modeled serving cost." Of the six commercial models the paper tested in the single-agent role, Opus 4.5 ranked second on F1, behind Sonnet 4.6, and all six scored below the trained 9B model.
That list includes the newest releases, which Coinbase's second post said did worse than the versions they replaced. On its test, Opus 5, Sonnet 5 and GPT-5.6 each scored lower than Opus 4.5, Sonnet 4.6 and GPT-5.4 on recall, F1 and dollar-weighted recall. Sonnet's recall fell 22.2 points; GPT-5.6 flagged fraud more accurately, with precision up 11.5 points, but caught less of it, with recall down 20.7. Coinbase tested every model with the same instructions and decision rules from its existing system, and wrote that a model needing new instructions or thresholds should be tested that way separately; the series reports no such test. "Our benchmark shows a regression, not its cause," the company wrote, suggesting among other possibilities that "the same risk label may mean something different across versions."
Speed and cost
On live traffic, served on Nvidia L40S cards, the small model's median response took 0.683 seconds against 1.515 seconds for Opus 4.5, and 99 of every 100 responses came within 1.036 seconds, against 3.312 for Opus 4.5. Coinbase notes that speed and accuracy were measured in separate evaluations.
The blog does not compare running costs, but the paper does. At list prices, with about 6,000 tokens (the word-pieces models read) in and 11 out per call, an Opus call costs 3.03 cents and a Sonnet call 1.82 cents. One graphics card serving the 9B model costs $34 to $81 a day and can handle about 1.1 million calls a day. At 10,000 calls a day, the paper's table puts the small model at 11% to 27% of the Opus price, depending on the card and how it is rented; at 2,500 calls a day, the most expensive option costs slightly more than Opus. The cost advantage Armstrong described depends on volume.
What the test does not show
The data cannot be checked from outside. "Legal and compliance restrictions prevent release of transaction-level records," the paper says, though it publishes its method, prompts and settings. The test keeps every fraud case but only a sample of legitimate transactions, so its 5.04% fraud rate is not the real-world rate, and the authors say precision and F1 describe the test, not production. The model returns one number, so the result says nothing about tasks in which a model must write accurate text or answer questions outside its training.
The advantage also has to be maintained. "Its knowledge, however, is encoded in model weights: tracking new fraud patterns requires another training and deployment cycle," the paper says, while its reflection layer can add checked rules between retrainings. The blog adds that "fraud patterns change, so maintaining quality requires continued evaluation and retraining—not just a successful checkpoint." New versions of the base model bring the same chore. An August study of fine-tuned Qwen models, UpgradeBench, starts from the point that "each new base-model release forces a migration decision: retain existing specialists, port adapters, refresh from preserved behavior, or retrain."
The wider pattern
Coinbase is not the only company reporting this. Factiverse, a Norwegian fact-checking company, reported in June that a small model it fine-tuned to spot checkable claims outperformed large commercial models, including GPT-5.2 and Claude Opus 4.6, "particularly in low-resource and morphologically rich languages," at ten to a thousand times lower cost per item. It also listed where its models lost: the large models won "by large margins" at judging whether evidence supports a claim in some languages, including Amharic, Akan and Hindi, and at spotting claims in "highly idiomatic English (memes, satire)," where "the LLMs’ broader world knowledge helps."
Published fraud work rarely gives the numbers a business would need. A July survey of 49 sources on AI models in fraud detection and content moderation found that among its 18 fraud and investigation sources, "none report clean per-decision latency, per-decision dollar cost, or calibration evidence." Coinbase's paper, posted after that survey, reports both its speed and a modeled cost per call.
Sources: Coinbase, "Owning Intelligence" series, part 1, part 2 and part 3, October 6 to 8, 2026; Brian Armstrong on X, October 11, 2026; Tan, Ma and Zhang, "SR-Fraud", arXiv, September 2026; Amatya and Setty (Factiverse), "Multilingual Fact-Checking at Scale", arXiv, June 2026; Chen and Zhang, "UpgradeBench", arXiv, August 2026; Gabani, "Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows", arXiv, July 2026.
