Self-Instructed Chain-of-Thought Controllability
Reasoning constraints placed only in a user prompt are often ignored; this evaluation puts the constraint inside the reasoning as a self-instruction and measures whether the remaining reasoning follows it.
Read the paper ↗What it is
- Self-instruction A constraint is stated inside the model’s reasoning, as an instruction for the reasoning that follows.
- Reasoning prefill and bootstrap Prefill supplies that instruction as the start of the reasoning; bootstrap asks the model to start by repeating it.
- Compliance metrics Binary scoring counts full compliance, soft scoring captures partial compliance, and echo rate checks whether bootstrap repeated the requested instruction.
Why it matters
A model may ignore a request about its reasoning when it appears only in the user prompt. Putting the request inside the reasoning tests whether its location changes control, while separate scores distinguish repeating the instruction from following it.
Before we get into the evaluation, here is the short version: the method puts a constraint inside a model’s reasoning and checks whether the reasoning after it follows that constraint. The point is to distinguish “the model cannot do this” from “the model did not prioritize an instruction about its own reasoning.”
- Control: We often care about the shape of reasoning text, not just the answer. The paper gives examples such as a required structure or format, and constraints that suppress or add information.
- Oversight: There is a tension: if a model can shape what it says in its reasoning, that may make the reasoning harder to use as a monitor. That concern belongs to the motivation here; this module is about measuring controllability.
So why not just put the constraint in the prompt? That is the baseline: ask the model to solve the task while obeying a requirement about its chain of thought. Existing evaluations report very low controllability in that setup. The proposed test changes the location of the reminder, not the task: the model sees a self-instruction inside the reasoning and then continues. The canonical concrete example is word suppression: the user asks the model not to use a particular word, and the self-instruction restates that rule in the reasoning. The broader evaluation covers constraints that suppress information, add information, or modify style. What does the placement change?
The central intuition is like writing a reminder on the page you are working on rather than leaving it on a note beside the desk. That may make it harder to overlook—but it does not guarantee compliance. A self-instruction can be specific, naming the constraint, or generic, simply reminding the model to follow the user’s constraint. In the reported comparison, even the generic reminder improves controllability. So is the model writing that reminder itself, or is someone putting it there?
That is the difference between reasoning prefill and bootstrap. With prefill, the instruction is directly supplied as the beginning of the reasoning. With bootstrap, the prompt asks the model to start its reasoning by repeating a specified instruction and then carry on. In both cases, the important question is what happens after the reminder, not whether the reminder looks persuasive. The canonical example remains the forbidden word: one setup starts the reasoning with the rule already written; the other asks the model to write that rule first. How should we score the continuation without giving credit for the reminder itself?
The evaluation separates those behaviors. Binary controllability asks what percentage of reasoning continuations follow the constraint, excluding the prefilled or echoed self-instruction. But a binary pass/fail can make near-success look identical to total failure. Soft metrics therefore measure degree of compliance for most constraints—for example, how many letters use the required case or how many word gaps contain the required insertion. Bootstrap has one more separate measure: echo rate, the share of reasoning that begins with the requested instruction. Echoing and obeying are related, but they are not the same. Which one matters more depends on whether you want the model to repeat a rule, follow it, or do both.
Overall, I find the distinction useful: a weak result from a prompt-only test does not by itself tell you whether the model lacks control or simply deprioritized that request. The reported results show substantial gains from self-instruction across the tested open-weight models, while also making clear that partial compliance and echoing need their own measurements. The evaluation measures control over reasoning text; it does not, by itself, say whether that reasoning is correct.
Read the deep dive
Why People Wanted It
Can you control how a model reasons, not just what it answers? That matters in two directions: structured or stylistically constrained reasoning may be useful, while reasoning that can be deliberately shaped may be harder to monitor. This module focuses on the controllability evaluation, not the monitor-evasion use case.
Existing evaluations put a constraint in the user prompt and check whether the chain of thought (CoT) follows it. The reported controllability in that setting is very low. But that leaves a real question: did the model fail to follow the constraint, or did it simply deprioritize an instruction about its reasoning? The evaluation tests that distinction by placing an instruction inside the reasoning itself.
The Core Move
Put a self-instruction inside the CoT, then grade only the model’s continuation after that instruction for compliance with the constraint.
The evaluation compares a user-prompt-only baseline with methods that place a constraint-specific instruction in the reasoning, or ask the model to echo an instruction before continuing. The source gives no mathematical formula for controllability; it defines the principal score in words as the percentage of CoTs that follow the constraint.
Why the Instruction’s Location Matters
Now the version for readers who want more of the machinery: the key contrast is not a new constraint but where the model encounters it. In the baseline, the user prompt contains the task and the CoT constraint. In Self-Instructed CoT, the constraint is restated inside the CoT as the model’s own instruction to itself. The evaluation asks whether that placement changes the continuation.
A self-instruction can be constraint-specific, spelling out the requirement, or generic, reminding the model to follow the user’s constraint. The comparison separates the effect of putting a reminder in the reasoning from the effect of repeating the exact requirement. What remains open is whether a general reminder is enough for a particular constraint?
Why Prefill and Bootstrap Are Different
Here is the part that gets technical: reasoning prefill supplies the self-instruction directly as the beginning of the CoT; the model generates the continuation. Bootstrap instead puts an instruction in the user prompt asking the model to begin its reasoning with a specified self-instruction, then continue. Thus prefill provides the text directly, while bootstrap tests whether the model will produce that text itself.
The six elicitation methods distinguish these choices: user prompt only; constraint prefill; user prompt plus constraint prefill; user prompt plus generic prefill; constraint bootstrap; and generic bootstrap. The constraint-specific prefill can stand alone because it states the requirement; a generic prefill needs the constraint in the user prompt to refer to. The remaining question is whether a generated echo should count as compliance?
What the Metrics Actually Count
Now the measurement details: binary controllability grades only the continuation, excluding the prefilled or echoed instruction. A CoT counts as compliant if it follows the constraint; the score is the percentage that do. For SWE-bench, CoTs across turns are graded together as one CoT, and the reported score is a macro average over dataset–constraint pairs.
Binary scoring can treat near-compliance as failure, so the evaluation also uses soft metrics for all constraints except “ignore question.” These measure the relevant degree of compliance: for example, the share of letters in the required case, or the share of word gaps containing “meow.” For bootstrap, echo rate separately measures how often the CoT begins with the requested self-instruction, ignoring case and whitespace. Echo is not the same as following the constraint afterward. Which part of the behavior do you care about: starting the instruction, obeying it, or both?
Minimal Causal Code
This deliberately simple sketch makes the scoring boundary visible: the instruction is outside the text being graded. It is an illustration of the evaluation logic, not a claim about the paper’s exact grader implementation.
# `continuation` is the model's reasoning after any prefill or requested echo.
# `follows_constraint` stands for a constraint-specific grader.
compliant = follows_constraint(continuation, constraint)
score = 100.0 * sum(compliant_flags) / len(compliant_flags)
What You Gain, and What You Lose
Self-instruction tests a different capability from user-prompt-only control: whether the model follows a constraint once it appears inside its reasoning. In the reported comparison, self-instruction raises controllability substantially across the five open-weight models; a generic reminder also recovers much of the gain. The result supports treating prompt-only controllability as an incomplete picture, not as proof that a model cannot control its CoT.
The score is still task- and constraint-dependent, and binary compliance hides partial success; soft metrics address that limitation but do not collapse all constraint types into one directly comparable measure. Bootstrap also introduces a separate behavior—echoing the requested self-instruction—which echo rate tracks independently. This evaluation measures control of reasoning text, not whether the resulting reasoning is correct or desirable. A related multi-turn bootstrap setting is covered elsewhere.
A constraint inside the reasoning is a different test from the same constraint left in the user prompt.