What thousands of AI research agents contribute to a science project

Thousands of research agents can change which parts of a large experiment receive attention. They can investigate many candidate signals concurrently and hand researchers a broader set of explanations to inspect. For laboratories and research funders, the resulting question is how that additional analytical capacity changes the allocation of scarce expert time and experimental resources.

OpenAI’s October 21 account describes a collaboration with PerturbAI and Scripps Research analysing an existing mouse perturbation atlas. The partners report a 32-minute analysis and synthesis run. The comparison with months to a year of serial human analysis is an estimate, not a measured end-to-end race between research teams.

The most useful way to assess the project is to follow the work. The experimental resource existed before the agents interpreted it. Software prepared and exposed the data. Agents produced analytical material. Researchers still needed to decide what deserved attention and what should be tested next. Each stage has a different output, cost and definition of completion.

This analysis draws on the accessible announcements, underlying March atlas preprint, dataset documentation and released software. The October agent manuscript endpoint was inaccessible during this review; this article does not claim to have independently reviewed its full methods or reproduced its results. It examines the public workflow and the economic questions that workflow can support.

The division of labour is the central change

A large biological screen creates more possible questions than the initial research team can pursue equally. An expression change may be familiar, difficult to explain, confined to a small population or connected to a field outside the team’s immediate expertise. Distributed interpretation can give those signals separate analytical attention before a human team commits to deep follow-up.

The released Distributed Agents repository describes assigning agents to individual perturbations and combining expression information with literature. It provides narrative reports and structured records, along with tools for exploring relationships across the corpus. This is a division of an interpretation task into manageable units, followed by synthesis.

That design differs from merely asking a single assistant to summarise an enormous table. Separate assignments can preserve attention to local evidence. Synthesis can then compare results across assignments. The design is useful only if the local tasks have clear boundaries and their outputs can be brought back together without losing provenance.

The work-allocation map below distinguishes the reported project from the operational interpretation applied here. The cost column deliberately avoids invented totals. A public artifact makes a stage inspectable; it does not establish what that stage cost the participating organisations.

The handoffs matter as much as the individual stages. A report that cannot be traced to a specific comparison is hard to review. A statistically interesting comparison that does not suggest an informative experiment may have limited immediate value for a target-discovery team. A novel hypothesis that requires an unavailable assay may remain an exploratory lead.

The resulting unit of work is therefore more specific than an “AI discovery.” It is a progression from a measured response to an inspectable interpretation, then to a decision about further investigation. Agents can accelerate parts of that progression without completing all of it.

Coverage is a different benefit from substitution

A laboratory might use agents to replace some repetitive literature assembly. It might also use them to examine signals that would otherwise receive no substantial review. Those are different sources of value. The first can reduce effort for a fixed workload; the second expands the workload the team can consider.

A claim about saved time needs a counterfactual: what would the team actually have done without the agents? If the team would have investigated only its strongest signals, reviewing additional signals is an increase in coverage. Assigning a hypothetical human hour to every newly covered signal does not show that those hours were previously being spent.

This distinction permits a more accurate account of the opportunity. Some benefit may be faster completion of planned analysis. Another part may be a wider set of research options. The latter should be assessed through the usefulness of those options, rather than converted automatically into a labour saving.

A fast run and a cheap project are different results

Parallelism can reduce elapsed time because independent tasks overlap. Aggregate work is a different quantity. A thousand simultaneous tasks still consume resources even when the person watching them waits only for the slowest one to finish.

A simple hypothetical makes the distinction clear. If 120 independent tasks each require ten minutes of execution, serial execution takes 20 hours. Running all 120 at once could bring the execution portion close to ten minutes under ideal conditions. The aggregate task time remains 20 hours, before setup, waiting, retries or synthesis. These are illustrative arithmetic, not measurements of the biological project.

Real assignments are rarely equal. A poorly documented perturbation might require a longer search than a familiar one. A shared data service can become congested. Some tasks may fail or need another attempt. A synthesis step may begin before every local report finishes, or may have to wait for a required subset. These dependencies determine the actual critical path.

Adding workers helps the independently executable part. It does less for a stage that must reconcile the complete set of results or for a decision requiring one specialist. Once that stage dominates, additional generation can increase the amount waiting to be reviewed without shortening the final decision.

The benchmark question should therefore specify its stopping condition. “All agents returned a report” is different from “all assigned signals received an acceptable report,” and both differ from “the lab selected its next experiment.” A system can improve the first measure while leaving the last almost unchanged.

A useful timing record separates preparation, execution, reconciliation, review and decision. It also records whether the clock includes failed attempts. This does not require every experiment to use the same workflow. It requires each reported time to retain the boundary that gives it meaning.

More computation can be an acceptable trade

Higher aggregate computation can still be economically attractive. A lab may value an earlier decision enough to pay for concurrency. It may also value broader coverage or a reduction in repetitive specialist work. The relevant comparison is the cost of reaching the desired research decision under the alternatives available to that lab.

That decision should not be framed as a contest between free human time and expensive machines, or the reverse. Specialist time, compute, storage and laboratory capacity all have constraints. The question is whether the additional computational work releases a more valuable constraint elsewhere in the process.

Our analysis of an AI product’s full cost uses the same distinction between model expenditure and surrounding work. For research agents, the surrounding work includes data stewardship and scientific judgement. A short runtime is an important observation, but it is only one input to that ledger.

Shared data work makes the agents possible

The original experimental study is a substantial input rather than background scenery. The March mouse atlas preprint describes a large in vivo perturbation resource and RNA measurements, with limits including loss-of-function interventions, a single timepoint and restricted access to some populations. Later computational work inherits that scope; it does not recreate the experiment or extend its measurements by itself.

The dataset documentation exposes cell-level data, metadata and analysis artifacts. Information identifying samples, experimental sources and guide assignments helps an analyst decide what a comparison means. Providing a table without that context would leave more interpretive work to every downstream user.

This is where shared preparation can change the economics of a distributed system. If every agent independently interprets column names, locates the same reference files and resolves the same identifiers, a supposedly parallel workflow duplicates setup work. A common, verified representation lets agents spend more of their allocation on their assigned questions.

The released preparation script illustrates that shared work. It downloads a pinned source, checks its size and hash, prepares derived files and records provenance. It also verifies declared exclusions for holdout inputs. These are source-code observations; this publication did not download the full dataset or execute the preparation pipeline.

The script matters because data identity and transformation are scientific dependencies. If two agents receive differently prepared tables without that difference being recorded, a later disagreement may look biological when it comes from preprocessing. Pinning and provenance do not establish that a biological interpretation is correct, but they reduce ambiguity about which input produced it.

Retrieval and recomputation have different requirements

The repository distinguishes its hosted corpus from the local full-data workflow. The hosted service makes reports and linked records available, but lacks the raw dataset needed to recompute statistics. A researcher exploring a lead may begin with the corpus; a researcher challenging the underlying contrast may need the data and analytical environment.

Those are separate service levels with different costs. Providing fast access to a finding is cheaper in some respects than maintaining every dependency required for complete reanalysis. A system sold as a research assistant should make clear which level it offers, because the user may need to move from reading to checking when a lead becomes consequential.

The Apache Arrow reader documentation shows practical mechanisms for selective data access, including reading chosen columns and filtering rows. Such mechanisms can reduce unnecessary data handling in a suitable layout. They do not remove the need to define a scientifically appropriate subset.

The allocation principle is to share stable preparation while preserving the ability to inspect or recompute the relevant evidence. That balances repeated work against flexibility. A single fixed summary can make agents efficient while preventing them from asking a question the summary was not designed to answer.

Benchmark preparation is work, too

The released preparation code exposes a less visible part of the project: constructing the information boundaries for an evaluation. It checks that held-out targets are absent from the declared training counts, excludes them from a training derivative across cell groups and validates the resulting target sets. It also removes unused dictionary values and metadata that could retain excluded labels. These are inspectable safeguards in the code, not a claim that this publication independently verified the prepared files.

That work serves a different purpose from improving an agent’s biological reasoning. It defines what the agent is allowed to know for a particular test. A system can appear more predictive if its inputs accidentally include information that should have been withheld. Preparing the evaluation correctly is therefore part of the cost of producing a credible capability claim.

The distinction matters for replication budgets. Recreating a published result can involve obtaining pinned inputs, reconstructing the prescribed exclusions, checking the outputs and then running the model. A reproduction attempt that changes the information boundary may answer a useful question, but it is no longer a direct check of the same configuration.

For a lab evaluating a supplier, this creates a practical contractual distinction. Access to finished reports allows inspection of the supplier’s conclusions. Access to prepared inputs allows closer examination of what each model received. Access to transformation code allows the lab to trace how those inputs were made. Each additional level can support a stronger audit while requiring more preparation and technical expertise.

The buyer should choose the level needed for the decision. Exploratory browsing may require only a traceable report. A consequential claim that the system predicts unseen biology requires a clear account of what was unseen and how that boundary was enforced. One service description should not quietly cover both.

Failed assignments belong in the capacity plan

The number of requested assignments is also different from the number producing usable outputs. A task can time out, return incomplete evidence or complete technically while missing its scientific objective. Those outcomes create different recovery work. Restarting every assignment may waste resources; accepting every completed report may conceal missing coverage.

An operational record should distinguish attempted, completed and reviewable work. It should retain the reason for another attempt and the relationship between successive versions of a report. This is a proposed measurement practice rather than a reported failure rate for the project.

The resulting accounting can reveal where investment belongs. If most delay comes from repeated access to the same source, shared retrieval may help. If reports repeatedly omit an essential comparison, assignment design or input preparation may be the issue. If experts must reconstruct every inference, the output format may be the constraint. Increasing the agent count addresses none of those problems automatically.

The difficult output is a reviewable portfolio

An agent report is an intermediate product. A laboratory needs to determine which reports deserve attention, which contain related ideas and which would change an experimental decision. Producing many reports makes that selection problem larger unless the system also improves the organisation of the output.

Structured records can help by separating a finding from its supporting data and references. A reviewer can inspect the relevant evidence without reconstructing the entire search. Cross-report links can expose recurring patterns. But the interface should preserve the distinction between a statistical observation and an explanation offered for it.

PerturbAI reports that blinded experts preferred distributed-agent interpretations in 75% of comparisons with single-agent reports. This is a company-reported assessment of report quality. It does not measure experimentally confirmed mechanisms or establish an equivalent percentage reduction in review time.

The distinction is economically important. A more useful report may save time, but it may also invite a more detailed review because it raises a question worth pursuing. A report that an expert prefers can improve decision quality without reducing the number of minutes spent on that decision.

Review effort depends on the kind of claim. Checking whether a cited paper exists is one task. Checking whether the paper supports the specific assertion is another. Deciding whether that assertion applies to the assayed cell population can require specialist judgement. The time required by the third task is not captured by automating the first.

A review interface should therefore help experts localise the uncertainty. It should expose the original comparison, the literature connection and the additional inference. The expert can then accept, revise or reject the relevant step. A smooth narrative without those boundaries can shift work onto the reviewer, who must first disentangle what the agent actually did.

Duplicate ideas create real review work

Separate agents can reach closely related hypotheses through different perturbations or references. That overlap may be scientifically meaningful, but it should not automatically count as multiple independent opportunities. A portfolio needs to distinguish corroborating routes to one idea from distinct ideas requiring different tests.

The same principle applies to disagreement. Two reports may appear inconsistent because they concern different cell populations, readouts or conditions. Reconciling them can reveal a useful dependency. Flattening both into a generic mechanism can erase the feature that makes the comparison interesting.

The task is thus more than deduplication by wording. The system must preserve why two claims are related and what evidence they share. A research manager needs to know whether pursuing both would test two independent hypotheses or pay twice to investigate the same underlying question.

A useful measure is how much expert effort is required to turn the generated corpus into a small set of distinct, well-specified options. Another is how often the expert can inspect a disputed statement through its linked evidence. These are proposed operational measures, not results already demonstrated by the project.

The research budget has several unknown lines

The public materials inspected here do not establish a full project budget. They do not support a dollar-per-discovery figure. Any economic analysis should therefore keep the unknown lines visible rather than fill them with a model price multiplied by a guessed token count.

The released runtime code contains a parser for usage events that distinguishes input, cached input, output and reasoning-related counts. That shows how some computational activity can be recorded. It does not, by itself, establish the historical charge for the reported run or the cost of the surrounding services.

An invoice also answers a narrower question than a research budget. It may omit staff who prepared the data, maintained the tools, reviewed findings or designed the next experiment. The opposite mistake is to assign all the cost of a reusable dataset or platform to one analysis run when later users can share the same investment.

A sensible ledger separates reusable preparation, marginal analysis and downstream decisions. Reusable preparation includes work that can support another cohort of agents or another research question. Marginal analysis includes resources consumed by a particular run. Downstream work includes expert review and experimental activity triggered by its outputs.

The denominator should follow the purpose of the investment. Cost per generated report helps estimate production capacity. Cost per accepted interpretation helps assess analytical quality. Cost per experimental decision brings review and prioritisation into view. None is automatically cost per validated discovery.

A worked review budget without an invented saving

Suppose, hypothetically, a research team receives 120 reports. Reading every report for ten minutes would take 20 specialist hours. If structured screening reduces the set needing deep review to 30, and each receives 20 minutes, deep review takes ten hours. The remaining work includes screening, reconciliation and any corrections.

That arithmetic does not establish a ten-hour saving. The comparison still needs the time spent screening, the quality of the excluded reports and the workflow the team would have used otherwise. If important leads are discarded, a smaller review bill may conceal a worse scientific decision. If the team would never have reviewed all 120 manually, the larger number was not its actual baseline.

The example shows what must be measured. Record the review stages, the outputs reaching each stage and the expert time consumed. Inspect a sample of rejected outputs to understand what selection loses. The aim is to assess a complete decision process, rather than declare success whenever the number awaiting review falls.

Data stewardship is part of the productivity system

Agents become more useful when data can be found, interpreted and reused without repeated negotiation over its meaning. This is a longstanding research-infrastructure problem. It becomes more visible when many automated workers encounter the same missing definition or inaccessible artifact.

The FAIR Guiding Principles, published by Wilkinson and colleagues, explicitly address machine use of scientific data alongside human use. Their framework covers findability, accessibility, interoperability and reusability, including identifiers, provenance and usage conditions. It applies to workflows and tools as well as conventional datasets.

The economic implication follows from the work allocation. Clear identifiers can reduce time resolving whether two records describe the same entity. Provenance can reduce time tracing a result to its input. A machine-readable description can prevent many agents from repeating the same manual interpretation. These are mechanisms for reducing friction, not a quantified return attributed to the FAIR paper.

The NIH data-sharing overview likewise treats data management and sharing as activities that require planning and budgeting. Its policy discussion is relevant context for research infrastructure, not evidence that every part of this particular collaboration falls under the same funding terms.

For a research institution, the choice is therefore not simply whether to buy more model calls. It is whether its data environment allows those calls to produce work that can be inspected and reused. More analytical capacity can expose deficiencies in metadata, access and versioning that were previously handled informally by the original researchers.

A lab’s internal knowledge also belongs in that environment. A model may retrieve published literature while missing an unsuccessful local experiment, an assay limitation or an unavailable reagent. Making those constraints accessible can prevent the generation of attractive but impractical proposals. The result need not be more reports; it may be fewer reports that survive expert consideration.

The next experiment defines the commercial opportunity

The scarce downstream resource is the ability to resolve a meaningful uncertainty. A hypothesis earns follow-up when a feasible experiment could change the team’s understanding or priorities. An agent system that improves that selection can have value even when it does not reduce the number of experiments performed.

The opposite case is also possible. Generating more plausible leads can increase laboratory demand without improving the chance that a chosen test is informative. That is why a report count should not be treated as a productivity measure on its own. The task is to connect additional analysis to better allocation of experimental effort.

An original perturbation response and a later mechanistic hypothesis must remain distinct in that handoff. The response is measured under the original screen’s conditions. The hypothesis may propose an unmeasured intermediate or a relationship suggested by other studies. The next experiment needs to test that extension rather than merely repeat evidence already present in the atlas.

An informative test can also reject a proposed mechanism. That outcome may prevent an expensive programme from pursuing a weak explanation. A business metric that rewards only positive results can undervalue this contribution and encourage agents to produce claims that are easy to confirm superficially rather than useful to discriminate.

A buyer can make a limited adoption trial concrete by choosing a real decision backlog and recording the current workflow. Compare the time to a reviewed, distinct proposal, the expert corrections it needs and the additional resources it consumes. Preserve the original evidence and selection record so the team can later connect experimental outcomes to what was actually predicted.

Our guide to reading model evidence explains why capability claims belong to their tested configuration. Here, that configuration includes the experimental resource, agent assignment, tool access, review process and follow-up capacity. Replacing only the model may leave the dominant constraint untouched.

For labs evaluating distributed research agents, the concrete opportunity is a better connection between a large data resource and a limited experimental agenda. The next useful disclosure is a workflow result: which previously neglected signals reached expert review, what review cost, which distinct tests were selected and what those tests resolved. That is the evidence that turns analytical scale into research productivity.