What a local research-agent fleet actually delivers

Operational numbers from a fleet of small local LLM agents that search the web, read pages and write cited reports — running on one NVIDIA DGX Spark (GB10, 122 GiB unified memory), September 2026. No API calls, no external inference, no cost per token.

Plenty has been written about how to build agent swarms. Very little has been written about what they produce once you run one for a day. These are ours, including the parts that look bad.

Companion document: Tuning an NVIDIA DGX Spark (GB10) to serve many concurrent local LLM agents — the hardware and serving numbers behind this.

The shape of the system

One run = one research question. It fans out through five stages:

The parallelism lives in read, on the small model. That is the work that is cheap locally and expensive through an API.

Totals for one working day

Wall-clock

A depth-10 run — 10 sub-questions, up to 8 pages each, ~80 reader agents — completes in about 16 minutes end to end on this hardware, with four runs executing concurrently.

Runs that overlapped a machine outage that day recorded ~37 minutes; those are not clean measurements and are excluded from that figure.

The number nobody publishes: most reader agents produce nothing

Of 993 pages fetched and read by an agent, 604 produced no usable evidence at all — 61%.

The agent read the page, found nothing that answered its sub-question, and honestly returned an empty result. That is correct behaviour and it is also the single largest waste in the pipeline.

It is measurable before the spend. Scoring the sub-question's distinctive terms against the page's full text, on those same pages:

At 0.3 it is free: roughly a quarter of all reader calls removed and not one page lost that would have produced evidence. We shipped that gate and it has skipped 72 reads since.

If you are building something similar, this is the highest-leverage thing in the document. A cheap deterministic filter in front of the model beats a better model behind it.

Answer quality, honestly

Every sub-answer is labelled by the model that wrote it: CONFIDENT (independent sources agree), CONTESTED (sources disagree, and it says how), or UNDOCUMENTED (the evidence does not answer it). Across 166 labelled sub-answers:

About 48% usable. But the average is misleading, and the variance is the finding:

On the new-hardware questions the fleet searched hundreds of pages and returned UNDOCUMENTED for nearly every sub-question. It was not broken. The pages do not exist yet. It declined to invent benchmark numbers, which is the behaviour you want and does not feel like it at the time.

The practical rule: this is a good literature scan and a poor instrument. Use it for what other people have already written down. Measure your own hardware yourself — one benchmark run on the actual machine produced better numbers than several hundred agents searching for them.

What raised quality most

1. Give the agents your own source code. Runs that read a local repository alongside the web scored 8/8 and 7/8 usable sub-answers — the best of the day, roughly double the web-only average. There is no hunting: the material is right there to quote.
2. Never let a narrow category replace general web search. Routing a networking question to an IT-only engine set answered it from a container-registry listing and two browser-API reference pages.
3. Make the planner split rare qualifiers across sub-questions instead of repeating them in every one. Four sub-questions that were the same sentence with the product name swapped returned four copies of four product homepages.
4. Score search results before fetching them. Term overlap against the title, snippet and URL separates a benchmark article (1.00) from the project's own homepage (0.09) and a download page (0.00), with no model call.

Failure modes worth designing for

• Every task is a database row, not in-memory state. That machine lost power mid-run during this day's work. On restart, four runs resumed directly into their synthesis stage with their reading and reduction intact. Nothing restarted from the beginning.
• A thinking model under a JSON schema can return an empty string. It spends its entire token budget reasoning and emits nothing. The error surfaces several layers away as a parse failure.
• Truncated JSON is unparseable. When a generation stops on length rather than completion, retry with a larger budget instead of discarding it.
• A small model given a page containing "ignore previous instructions" will flag the attempt correctly and still copy the attacker's marker string into its output. Prompting a 7B out of that is not a control. Discard every fact from a page that tried to steer the reader — that is deterministic, and it happens in the runner where a model cannot argue with it.
• Citation numbering has to be global to a run. Numbered per sub-question, [1] in section two pointed at a different source than row 1 of the sources table. Silently, and confidently.

Caveats

One fleet, one machine, one day, two models (a 7B for reading and a 30B-A3B MoE for planning and synthesis). Search quality is capped by what a self-hosted metasearch instance can reach, which on ours is two general engines plus keyless academic and developer APIs. Every number here will move with different models, a different search backend, and different questions.

We are still measuring. This is a starting point, published because we could not find anyone else's.