LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

Riccardo De Santi, Federico Di Gennaro, Angelo Gnazzo, Sophia Tang, Pranam Chatterjee, Fanny Yang et al.

arXiv 2610.17246 45 pages Read the paper

The short version

The problem: Scientists may know what kind of molecule or peptide they want, but specialized generators do not understand ordinary-language goals. Language models can reason about such goals, yet their numerical ratings of scientific candidates may be unreliable, and a generator may almost never produce designs that meet a difficult goal at the outset.
The idea: The paper has a language model act as a coach for a specialist generator. It first sets an easier, scientifically relevant goal the generator can make progress on, compares candidates using simple better-or-worse judgments (and can abstain when uncertain), and uses those judgments to train the generator; as the generator improves, the coach makes the goal more demanding. It is like teaching a difficult skill through achievable steps rather than demanding expert performance immediately.
What it got: In tests on therapeutic peptides and small-molecule design, this approach beat reward-based methods on separate evaluation measures without using their task-specific scoring functions. For cell-penetrating peptides, the held-out evaluator score increased from 8.81% before adaptation to 69.06% with LFD.

Modules from this paper

Adaptive Natural-Language Curricula for Specialist Generators

A specialist generator may rarely produce designs relevant to a distant natural-language goal; LFD uses its current samples and observable properties to choose attainable scientific sub-goals, then tightens them toward that goal.

curriculum learninggenerative modelsLLM distillationscientific discovery

Distilling Categorical Judgments into Generator Updates

Numerical scores from a language-model judge can be poorly calibrated, so the generator learns from positive–negative preferences elicited on its own current samples and distilled through an online, reference-regularized DPO update.

preference learningDPOflow modelsdistillation