Multi-turn Bootstrap
A self-instruction given only once often fades over a long reasoning task; repeating it at every assistant turn helps Qwen3-32B sustain the requested constraint across turns.
Read the paper ↗What it is
- Multi-turn bootstrap A prompt asks the model to begin every reasoning turn with the same self-instruction, instead of giving that reminder only at the start.
- Instruction echo The model repeats the self-instruction at the start of a turn; checking this across turns shows whether the reminder keeps appearing.
Why it matters
A one-time prompt can lose influence as an agent continues through a task. Repeating the instruction helped sustain its echo across turns in the tested setup, though repeating words that must be avoided can create a conflict.
Before getting into the mechanics, here is the short version: if an agent is supposed to follow a self-instruction over a long task, telling it once may not be enough. A multi-turn bootstrap asks it to begin every reasoning turn with that instruction, so the reminder shows up again as the task continues.
-
Persistence: In the original setup, the self-instruction appears only at the start of the first assistant turn. Later turns often begin without it, and both the echo and per-turn controllability drop. The canonical example is Qwen3-32B on SWE-bench: compare the later turns under the original bootstrap with a prompt that explicitly repeats the instruction at every turn. This is not a claim that every task has the same failure; it is the concrete setting used to test whether repetition can keep the reminder present.
-
Local reminder: Think of the instruction as a note you put at the top of each new page, rather than a note written only on the first page. That can make the intended behavior easier to resume on each turn. The analogy has a limit: a model is not literally retrieving a note, and repeating text does not guarantee obedience. In the reported setup, the model echoed the self-instruction in 82–95% of turns with the multi-turn instruction, against 26–44% of later turns with the original one.
-
Conflict in the reminder: Repeating the note can repeat unwanted material too. For word suppression, the self-instruction names the forbidden words; the model sometimes loops that reminder, raising their counts while binary controllability stays roughly unchanged. This is the design tradeoff: the reminder must be easy to repeat without undermining the constraint it is meant to support.
So does more echo mean better control? In this Qwen3-32B SWE-bench comparison, binary controllability rises from 9.7% to 12.8%, while soft metrics improve for all constraints except suppression. I read that as evidence for a useful prompt pattern, not a universal fix: it helps with persistence, but the exact wording can become part of the failure mode. A related module covers broader self-instructed control of reasoning text; here the focus is specifically on repeating the instruction across turns.
Read the deep dive
Why People Wanted It
A bootstrap prompt can get a model to begin its reasoning with a requested self-instruction. But a multi-turn agent has a longer problem: the first turn may follow the instruction while later turns stop repeating it. If the constraint depends on that self-instruction, losing the reminder can mean losing control. So the real question is whether the instruction can be made present again at each turn, rather than trusted to persist from the beginning.
The Core Move
Put the same self-instruction at the start of every assistant reasoning turn, and show the model the pattern it should repeat. The paper gives this as a prompt instruction and example, not as a mathematical formula.
Why a One-Time Reminder Fades
Now the version for readers who want more of the machinery: the measured failure is turn-local, not simply a failure to echo at the beginning of a conversation. In the original setup, the self-instruction is prefilled or requested only at the start of the first assistant turn; later turns often begin without it. Per-turn controllability and bootstrap echo both drop sharply from the second turn onward. That distinction matters: a model can demonstrate the requested behavior once without maintaining it through a sequence of reasoning turns.
The relevant signal is therefore not just whether the model echoed the instruction somewhere. It is whether the echo remains present across turns. Figure 4 reports that, for Qwen3-32B on SWE-bench, the multi-turn instruction keeps the echo high; the text reports echoes in 82–95% of turns, compared with 26–44% of later turns under the original instruction. This tests persistence of the prompt pattern, not a general guarantee that every constraint will be obeyed.
Why Repeating the Prompt Changes the Tradeoff
Here is the part that gets technical: the multi-turn instruction tells the model to start every reasoning turn with the exact self-instruction, then gives an example of that pattern. Repetition makes the reminder available locally at each turn rather than relying on the model to carry it forward from turn one.
On Qwen3-32B’s SWE-bench evaluation, binary controllability rises from 9.7% with the original bootstrap to 12.8% with the multi-turn bootstrap. The soft metrics improve substantially for all constraints except suppression. Suppression is a revealing exception: the self-instruction itself names forbidden words, and the model often loops that instruction, increasing their counts while binary controllability stays roughly unchanged. Repeating a reminder can therefore strengthen the intended behavior while also repeating content that conflicts with a particular constraint.
Minimal Causal Code
This is the prompt-construction idea in plain Python; it does not implement a model-specific reasoning loop:
self_instruction = "{self-instruction}"
example = "REASONING INSTRUCTION: begin this turn with the instruction."
multi_turn_prefix = (
"Start every assistant reasoning turn with the exact following text: "
f"{self_instruction}\n"
f"Example: {example}"
)
The important choice is what the instruction asks the model to repeat, and that the bootstrap applies at every reasoning turn. The prompt alone does not ensure compliance.
What You Gain, and What You Lose
You gain a way to sustain the self-instruction echo across turns: in the tested Qwen3-32B SWE-bench setup, echo was higher with the multi-turn bootstrap, and binary controllability increased from 9.7% to 12.8%. You do not get uniform improvement: suppression metrics were the exception, because looping the reminder can reproduce the forbidden words. Nor does this result establish that repeating a bootstrap will work equally well for other models or tasks. It is a targeted prompt-level intervention with a concrete cost when the reminder's wording clashes with the constraint.
A reminder that must survive many turns may need to be repeated at every turn—but repeating the reminder can also repeat the very content you want suppressed.