Recurrent Harness for Agentic Continual Learning

Forward-only memory can preserve misleading reflections; ReHarness uses later task outcomes to revise retrieved reflections while grounding changes in their source trajectories.

Read the paper ↗

The short version

What it is

Backward revision After a task, use its outcome to refine the earlier reflections that were retrieved for it, rather than only adding another note.
Source-grounded change A revised reflection can narrow or correct its advice, but new claims must come from the original task that produced that reflection.
Demonstrations for writing Use revised old reflections as examples when writing the current task’s new reflection, so the next memory entry can complement them.

Why it matters

Forward-only memory can keep retrieving an overgeneralized rule and repeat its mistake. ReHarness closes a feedback loop: later outcomes help refine retrieved advice, while source grounding limits what the revision may claim.

Before we get into the machinery, here is the idea: after an agent uses old reflections on a task, a revision agent looks at how those reflections worked and can narrow or correct them. The correction has to stay tied to the task that originally produced the reflection. Then the corrected reflections help shape the next note written into memory.

Forward memory can preserve a bad generalization: A harness writes reflections after tasks and retrieves them later. The canonical example is sorting: after correctly sorting numbers in descending order, a memory agent writes “put larger numbers first.” On an ascending task, that advice is misleading; a fresh reflection saying “put smaller numbers first” does not remove the old one, so it can mislead again. This is the failure of a notebook where you can only add pages, never amend one. The analogy stops short because retrieval selects relevant pages rather than reading the whole notebook.
Later outcomes provide a backward signal: The task outcome tells the system whether a retrieved reflection was useful in the new setting. In the sorting example, seeing that “larger numbers first” failed for an ascending request suggests the missing condition: use that rule only when sorting descending. Think of feedback as a note sent back to the advice that shaped the decision, not as a chance to redo the old task. The setting is strictly streaming: an instance is observed once, with no additional validation set, and the API agent’s parameters are unavailable for gradient updates. What keeps a correction from smuggling the new task into an old memory?
Source grounding limits the correction: The revision agent uses both the old reflection’s source trajectory and its effectiveness on the current task. The current outcome says the advice was a poor fit; the source trajectory says what the reflection may legitimately say. This is like editing a notebook page after learning it was overgeneralized, but checking the original notes before adding a new rule. The analogy’s limit is that the reviser is an agent making textual judgments, not a mechanical proof checker.

In words, the paper’s update has three parts: retrieve reflections for the current query, revise each retrieved reflection using its own source trajectory plus downstream feedback, and replace the retrieved versions while leaving unretrieved reflections in place. The supplied pages do not give a displayed formula for this process. The revised reflections then serve as examples for writing a new reflection from the current task; the prompt encourages the memory agent to add complementary, well-supported operational insight rather than repeat the examples. So the system doesn’t just add a fresh page—it also teaches the writer how the old page should have been scoped. The open question is whether that textual judgment stays useful as tasks shift and memories accumulate.

Read the deep dive

Why People Wanted It

A harness can help an agent learn across a stream of tasks by writing reflections after tasks and retrieving them later. But a reflection that worked once can be too broad for the next task: in the paper’s sorting example, “put larger numbers first” misleads an agent asked to sort ascending, and can keep misleading later instances. The real question is not only how to store experience. It is how later outcomes can correct the experience that was reused, when past tasks cannot be replayed and there is no separate validation set.

The Core Move

Use the outcome of a current task to revise the retrieved reflections that influenced it, while requiring revised content to remain grounded in each reflection’s original trajectory; then carry those revisions forward alongside a new reflection for the current task.

The paper gives no displayed formula for this update in the supplied pages. In words: read relevant reflections, act and observe the outcome, revise the retrieved reflections using both their source trajectories and the new outcome, then write a new reflection using the revised ones as demonstrations.

Why Backward Feedback Can Help a Forward Memory

Here is the part that goes beyond the basics: the loop is recurrent because a later trajectory sends feedback backward into earlier memory, rather than merely adding another note to the end. The task stream is strictly streaming: each instance is observed once, there is no additional validation set for choosing harness updates, and frontier API-based agents do not expose parameters for gradient updates. Textual feedback lets the harness adapt without those routes.

The paper’s concrete sorting case shows what this feedback means. An earlier reflection says “put larger numbers first”; the next task asks for ascending order and reveals that the reflection’s advice was too broad. The revision adds the condition “larger first only if descending.” The outcome is evidence about whether the retrieved advice helped, not permission to copy the current task’s answer into every old reflection. What other constraint must keep this correction honest?

Why Revisions Must Stay Grounded in Their Source

Now the version for readers who want more of the machinery: the revision agent examines each retrieved reflection independently, using its source trajectory and its effectiveness on the current trajectory. The current trajectory supplies downstream feedback for assessing the old reflection. Any new content in the revised reflection must remain grounded in the old reflection’s source trajectory, preventing information leakage.

This separates two roles that can otherwise get blurred. The current outcome can tell the reviser that an old rule failed in this setting; the source trajectory determines what that old reflection is actually allowed to claim. A revision can add evidence missed in the source, correct an unsupported hypothesis, or narrow the conditions under which the reflection applies. Once revised, it replaces the retrieved version in the intermediate harness state, while unretrieved reflections are retained. Why revise the old note and not just append a correction?

Why Revised Reflections Become Demonstrations

Here is the final deeper layer: revision also changes how the next reflection is written. The revised reflections serve as demonstrations for the memory agent as it reflects on the current trajectory. The prompt encourages that agent to retain operational evidence and well-supported interpretations or strategies, and to add a complementary perspective rather than simply repeat what the demonstrations already cover.

In the sorting example, the revised old reflection says when larger-first applies; the new reflection can express the corresponding condition for the current task, “smaller first only if ascending.” The next harness state therefore carries both the revised prior reflection and the new reflection. This closes the loop: retrieval affects a task, its outcome informs revision, and the revised text guides future writing. What does this buy, and what new dependence does it introduce?

Minimal Causal Code

# Pseudocode-like PyTorch-free loop: the state changes are the point.
harness = []

for query in task_stream:
    retrieved = retrieve(harness, query)
    answer, outcome, trajectory = run_task(query, retrieved)

    revised = []
    for reflection in retrieved:
        source = reflection.source_trajectory
        revised.append(revise(
            reflection=reflection,
            source_trajectory=source,
            downstream_trajectory=trajectory,
            outcome=outcome,
        ))

    # Replace retrieved entries; leave the rest of the harness intact.
    intermediate = replace_retrieved(harness, retrieved, revised)

    # Revised notes demonstrate how to write the current task's reflection.
    new_reflection = write_reflection(trajectory, demonstrations=revised)
    harness = intermediate + [new_reflection]

What You Gain, and What You Lose

You gain a backward path for correcting retrieved experience, rather than allowing an overgeneralized reflection to persist simply because forward writing only adds new notes. Grounding revised content in the source trajectory constrains what the revision can claim, while using revised reflections as demonstrations links correction to future writing.

The method still depends on textual judgments: the revision agent must assess downstream effectiveness, and the memory agent must produce useful, well-supported reflections. The harness revises retrieved reflections, so this is not a general guarantee that every stored reflection is corrected. Nor does textual feedback provide the counterfactual outcomes unavailable in a strictly streaming setting. ReHarness makes the memory loop recurrent; it does not make the outcome signal complete.

Quiz

What makes ReHarness recurrent rather than merely forward-writing memory?

It updates the agent’s parameters with gradients from a validation set.

It only appends a new reflection after each task.

It replays every past task to retrain the decision agent.

It revises retrieved reflections using the current task’s outcome, then writes a new reflection.

Quiz

What constrains new content in a revised reflection?

It may freely copy any details from the current task into the old reflection.

It is constrained only by the next task’s query.

It must be identical to the original reflection.

It must be grounded in the reflection’s original source trajectory.

Core Takeaway

A later outcome can correct an old reflection—but the old reflection’s source trajectory sets the bounds of that correction.