Evidence-Driven Experiment Assignment

Parallel research workers can waste effort repeating one another; a stateless Advisor uses evidence across search directions to assign them concrete, distinct experiments.

Read the paper ↗

The short version

What it is

Advisor assignment An Advisor reviews findings across research directions and gives parallel Workers specific experiments to try.
Distinct parallel work The assignments are meant to keep Workers from spending their effort repeating the same attempt.

Why it matters

More Workers do not automatically mean broader exploration. Turning shared findings into separate tasks helps parallel effort cover distinct possibilities, while keeping coordination as an explicit part of the research process.

Before getting into the moving parts, here is the short version: an Advisor looks across what different search directions have already found and gives parallel Workers concrete experiments. That matters because adding Workers can add duplicated effort instead of new exploration.

Progress: In long research runs, having records of earlier attempts is not enough; the system still has to decide what to try next. The canonical example is a run where several Workers independently implement the same feature. That is parallel activity, but it has not expanded the search. More computation only helps if it keeps producing useful attempts.
Coordination: Each Worker needs a task that reflects what is already known across directions. Think of the Advisor as a research lead assigning separate tests after looking over everyone’s notes. The analogy stops at task-setting: Workers still carry out the experiments, and an assignment cannot ensure a successful result.

So the real question was never just how many Workers can run at once—it was how to make their work different. The Advisor uses evidence across search directions to make concrete assignments. In the paper’s description, the Advisor sees structured evidence across directions, while Workers get their assignments along with their retained candidate and local feedback. The mechanism is an evidence-to-task handoff, not a claim that a particular equation computes the best assignment. The paper gives this mechanism in prose, with no formula for the assignment rule.

A notebook analogy helps: shared findings are the notes the Advisor consults, and each Worker gets a different next experiment rather than copying the same page. But the notebook is not the algorithm; the important part is that evidence is turned into specific work. In the reported no-assignment comparison, parallel Workers repeatedly implemented the same feature. That is the concrete failure explicit assignments address.

What does the coordination buy, and what does it cost? The reported Advisor uses a small share of tokens and model cost compared with SwarmResearch’s Shepherd, though it still adds a coordination step. I find the design appealing because it targets wasted parallel effort directly; it does not make every experiment productive or guarantee that the search will improve. And the broader lesson is modest: if evidence does not change what Workers do next, having more of it is not enough.

Read the deep dive

Why People Wanted It

Long-running automated research has a problem beyond finding a good candidate: it must keep turning more computation into progress. A record of past attempts helps, but it does not by itself say what should happen next. Parallel Workers make the gap visible: without coordination, they can spend their effort repeating the same experiment. So the real question is not simply how to run more Workers, but how to use accumulated evidence to give each one a useful next task.

The Core Move

A stateless Advisor reads evidence spanning the search directions and turns it into concrete experiments for parallel Workers, so they explore distinct possibilities rather than duplicate effort. The paper states this mechanism in prose; it gives no formula for the assignment rule.

Why Evidence Must Become an Assignment

Here is the part that goes beyond the intuition: evidence is an input to a decision, not the decision itself. The Advisor has access to structured evidence across search directions, while a Worker receives its assignment, retained candidate, and local feedback. This division lets the Advisor compare directions and specify an experiment instead of handing every Worker the same undirected research problem.

The assignment matters because a list of findings does not determine what to try next. In the reported ablation, removing assignments led parallel Workers to repeatedly implement the same feature. That is the failure this mechanism is meant to prevent: duplicated work consumes parallel effort without broadening the search.

Why Parallelism Needs Distinct Experiments

Now the deeper coordination point: parallelism is useful only when the work is meaningfully differentiated. The Advisor’s role is to translate cross-direction evidence into concrete experiments; Workers then execute their respective assignments. The paper describes this as evidence-driven work assignment, rather than specifying a mathematical optimization rule or a fixed assignment algorithm.

This is a division of labor, not a guarantee that every experiment will succeed. It gives the search a way to spend parallel effort on different attempts, while the harness retains research state and constructs role-specific contexts. How much does that explicit coordination cost?

Quiz

What does the Advisor do with evidence from across search directions?

It guarantees that each assigned experiment succeeds.

It makes every Worker run the same experiment.

It assigns concrete experiments to Workers.

It carries its own conversation state into every later invocation.

Minimal Causal Code

The following is schematic pseudocode for the mechanism, not a reproduction of a paper implementation. It makes the assignment handoff visible; it does not prescribe an assignment formula.

# evidence_by_direction and worker_inputs are prepared by the harness.
assignments = advisor.assign_experiments(evidence_by_direction, worker_inputs)

results = []
for worker, assignment in zip(workers, assignments):
    result = worker.run(
        assignment=assignment,
        retained_candidate=worker_inputs[worker]["candidate"],
        local_feedback=worker_inputs[worker]["feedback"],
    )
    results.append(result)

What You Gain, and What You Lose

Evidence-driven assignments give parallel Workers distinct experiments instead of leaving them to repeat one another. Coordination is not free, but the reported Advisor consumes a small share of tokens and model cost compared with the Shepherd in SwarmResearch. The tradeoff is an added coordination step: the Advisor must interpret evidence and produce assignments, and those assignments still cannot guarantee a successful experiment. The benefit is a more deliberate use of parallel effort, not a promise that every run improves.

Quiz

Why use explicit assignments for parallel Workers?

To guarantee a better result from every Worker.

To direct Workers toward distinct experiments rather than duplicated work.

To replace the need to evaluate candidate solutions.

To ensure Workers never receive any feedback.

Core Takeaway

Parallel Workers help most when evidence gives each one a different experiment to try.