Curated Decision SFT

Why People Wanted It

A research agent does not make one isolated prediction: experiments unfold over hours, and each result can change what is worth trying next. A trajectory contains observations, reasoning, and actions, but copying the whole thing would not necessarily teach the decision that connects evidence to a useful next step. The real question is: can existing research trajectories provide reusable supervision for choosing what to do next, while keeping enough context to understand why?

The Core Move

Curated Decision SFT selects teacher responses that connect prior observations to meaningful research choices, preserves the context needed to interpret each response, and trains on the selected response alone.

For a selected example, let $h$ be the reconstructed history and $y$ the teacher's chosen response. Training minimizes the response loss conditioned on that history:

$\mathcal{L} = -\log p_\theta(y \mid h)$

The history informs the decision; the selected response is what contributes to the training loss.

Why the Decision, Not the Whole Trajectory, Is the Unit

Here is the part that gets technical: a trajectory is useful only when the retained example makes the link between preceding evidence and the next research choice available to the learner. The curation examines the observations before a decision, the teacher's reasoning, and the action taken next; it retains examples aimed at connecting evidence to a meaningful research decision. This makes the target a selected response within a research context, rather than an undifferentiated stream of all trajectory text.

The training setup separates context from target. If $h$ denotes the preceding history and $y$ the selected response, the model is conditioned on $h$, but only $y$ contributes to the loss. In practical terms, the history lets the learner interpret the choice; the target tells it which choice to imitate. This is a supervision design, not a guarantee that every selected choice is universally correct. What makes a decision meaningful enough to retain?

Why Long Histories Need Reconstruction

Now the version for readers who want more of the machinery: when the full history is longer than the training window, the example combines relevant earlier evidence with recent observations. It does not simply discard the past and train on the newest turn; nor does it require fitting the entire trajectory into one example. The reconstructed context is meant to preserve the evidence needed to interpret the selected response.

That creates a specific constraint: context selection must retain decision-relevant earlier information while keeping the example within the available training window. The recipe described here does not specify a universal rule for deciding relevance, so the important distinction is the structure of the example: selected earlier evidence plus recent observations as context, selected response as target. How much of the reasoning should be retained when the response is the supervised part?

Minimal Causal Code

This toy loop makes the context/target boundary visible. It assumes each curated record already contains reconstructed context tokens and selected response tokens; it does not implement curation or windowing.

for example in examples:
    context = example["context"]
    response = example["response"]

    logits = model(context + response[:-1])
    response_logits = logits[len(context) - 1:]

    # Only response tokens contribute to the objective.
    loss = cross_entropy(response_logits, response)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

What You Gain, and What You Lose

This approach turns selected teacher decisions into supervised targets while retaining the preceding context needed to interpret them. Its scope is deliberately narrower than imitation of every step in a trajectory: curation chooses which responses become training examples, and context reconstruction determines which history accompanies them.

The described dataset contains 400 training examples and approximately 2.19 million supervised tokens. Those figures characterize this dataset, not a general recipe or a guarantee of downstream performance. The method is also only one supervision route in the broader work; self-context distillation uses Nemotron's own guided attempts, which is a separate module. Curated Decision SFT offers a way to reuse teacher experience, but it depends on selecting useful decisions and preserving the context that makes them legible.