Self-Conditioned HybridSFT
Hybrid latent training can fail when the model only sees target embeddings; HybridSFT feeds its own predicted latent inputs back during a second training pass.
Read the paper ↗What it is
- Hybrid format activation First teach the model where latent mode starts and ends, then align the component that turns hidden states into latent inputs.
- Self-conditioned training Run a second training pass that feeds the model its own predicted latent inputs instead of only the target embeddings.
Why it matters
At inference, latent predictions become the next inputs. Training on those predictions helps bridge the gap between learning from target embeddings and generating from the model’s own continuous outputs.
Before we get into the machinery, here is the short version: HybridSFT teaches a model to use its own latent predictions while it is still training. That matters because the model will consume those predictions at inference, not the clean target embeddings it sees in ordinary teacher-forced training.
- Format: The model first learns when to enter and leave latent mode by emitting two special anchor tokens. This is the control-flow part; without it, good latent vectors would not tell the model when to use them.
- Alignment: HybridLink is then trained to turn latent-position hidden states into inputs aligned with the target embeddings. The backbone is frozen during this alignment step.
- Feedback: A second training pass feeds HybridLink’s predicted latent inputs back to the model. That is the key change: the backbone is no longer trained only on idealized latent inputs.
The canonical example is a multi-step latent span. Imagine carrying a notebook from one town to the next: if you copy a clean reference page at every step, you never learn how to continue from your own imperfect notes. Here the “notes” are continuous vectors, not words. The analogy has a limit: vectors are model inputs, not a human-readable record.
In the first pass, the model sees the target hybrid sequence. HybridLink uses hidden states at latent positions to predict the next latent inputs. In the second pass, those predictions replace the target embeddings. The model is therefore asked to continue from what it produced, rather than from a clean input it would not receive at inference.
Why does that feedback matter more for latent mode than for text? In text mode, a generated output is sampled as a vocabulary token and re-embedded, so the next input remains a token representation. In latent mode, HybridLink’s output is continuous and fed back without that vocabulary projection. A deviation can therefore alter the next input itself and carry forward through the latent span. This is the train-inference mismatch the second pass targets.
The second pass supervises text positions with cross-entropy and latent positions with regression, while jointly training the backbone and HybridLink. The staged setup also keeps the format work explicit: learning to emit mode-entry and mode-exit anchors is not the same thing as learning to continue coherently inside the latent span.
So the real question is not just whether HybridLink can match target embeddings once. Can the backbone stay coherent when the next latent input is its own prediction? Self-conditioned training makes that situation part of supervised training.
Read the deep dive
Why People Wanted It
A latent span is not ordinary text. During inference, HybridLink produces a continuous vector, and the backbone consumes that vector at the next latent position. But ordinary teacher-forced training feeds the backbone the ground-truth embedding instead. The model is therefore trained on one kind of input and asked to continue from another. The real question is: how do you teach the backbone to keep reasoning when its own latent predictions are the inputs?
The Core Move
HybridSFT first activates the hybrid format and aligns HybridLink, then runs a second pass in which predicted latent inputs replace ground-truth latent embeddings.
The format stage teaches the backbone to enter and exit latent mode using two special anchor tokens. HybridLink is then trained to align its outputs at latent positions with the corresponding target input embeddings. No activation objective formula is specified here; the described objectives balance directional and magnitude alignment.
Why Format Activation and HybridLink Come First
Here is the part that gets technical: the first stage separates learning the control format from learning the latent-input mapping. Standard text SFT teaches when to emit the two anchor tokens that mark entry and exit. The backbone is then frozen while HybridLink learns to align last-layer hidden states at latent positions with the target embeddings. This gives the later self-conditioned pass a format-capable backbone and an initialized link, rather than asking both to discover the interface at once.
Why the Second Pass Closes the Feedback Gap
Now the version for readers who want more of the machinery: in pass one, the model is teacher-forced on the ground-truth hybrid sequence, and HybridLink maps last-layer hidden states one step ahead to produce latent inputs. In pass two, the same sequence is processed again, but those predicted inputs replace the ground-truth latent embeddings. Text positions receive cross-entropy supervision; latent positions receive regression supervision. The backbone and HybridLink are jointly trained in this pass.
The important constraint is where the replacement happens: it is at latent input positions, not a conversion of predictions back into vocabulary tokens. This trains the model on the continuous inputs it will consume while generating a latent span. Why should that matter? Because the next latent prediction depends on a backbone state produced from the previous predicted input.
Why Mode-Entry and Mode-Exit Supervision Still Matters
Here is a further technical detail: self-conditioning does not replace format supervision. The model still needs to learn where latent mode begins and ends, using the two special anchor tokens; the hybrid trajectory also retains text positions for interaction and auditing. Text cross-entropy in the second pass supervises those text outputs, while regression supervises latent inputs. The training procedure therefore covers both the mode boundaries and behavior inside the latent span.
Minimal Causal Code
This schematic loop makes the feedback visible. It omits the paper’s model-specific details; in particular, the cited pages do not give enough implementation detail to specify the exact tensor construction or losses in executable form.
# Pass 1: teacher-forced sequence; predict the next latent inputs.
first = model(ground_truth_sequence)
predicted_latents = hybrid_link(first.hidden_states_at_latent_positions)
# Pass 2: keep the sequence, but feed predicted vectors at latent inputs.
second_inputs = replace_latent_embeddings(
ground_truth_sequence, predicted_latents
)
second = model(second_inputs)
# Apply text CE at text positions and regression at latent positions.
loss = text_cross_entropy(second) + latent_regression(second, predicted_latents)
loss.backward()
What You Gain, and What You Lose
Self-conditioning exposes the backbone during training to its own continuous latent inputs, addressing the mismatch between teacher-forced inputs and free-running latent generation. The tradeoff is a two-pass procedure and joint training of the backbone and HybridLink in the second pass, rather than relying only on a single teacher-forced pass. It does not remove prediction error; it changes the inputs the model learns to handle. The broader HybridLink mode-switching mechanism is covered elsewhere.
Train the model on the latent inputs it will actually have to consume: its own.