Paper Detail

LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators

Riccardo De Santi, Federico Di Gennaro, Angelo Gnazzo, Sophia Tang, Pranam Chatterjee, Fanny Yang, İsmail İlkan Ceylan, Michael M. Bronstein, Andreas Krause

huggingface Score 8.0

Published 2026-10-12 · First seen 2026-10-15

General AI

Abstract

Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. We study how to align such scientific generators to high-level natural-language discovery goals without requiring an explicit proxy reward, which is often unavailable or poorly captured by learned predictors in experimental sciences. We introduce LLM-to-Flow Distillation (LFD), where a frozen LLM acts as an adaptive teacher for a domain-specific generator. Given a goal such as 'generate peptides with promising anticancer activity', the teacher constructs a curriculum of intermediate sub-goals aware of the flow's current capabilities, and progressively directs them toward the original discovery goal. At each round, the LLM judges sampled candidates under the current objective. These judgments induce preferences that are iteratively distilled into the generator through DPO. This procedure provides dense learning signals even when the initial model places negligible probability on designs satisfying the final goal, enabling adaptation toward poorly covered, out-of-distribution regions. Experiments on therapeutic peptides and small-molecule structure design demonstrate that LFD enables a new form of generative control: aligning arbitrary domain-specific scientific generators to scientific discovery tasks expressed via high-level natural-language goals.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{santi2026llm,
  title = {LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators},
  author = {Riccardo De Santi and Federico Di Gennaro and Angelo Gnazzo and Sophia Tang and Pranam Chatterjee and Fanny Yang and İsmail İlkan Ceylan and Michael M. Bronstein and Andreas Krause},
  year = {2026},
  abstract = {Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries. We study how to align such scientific generators to high-level natural-language discovery goals },
  url = {https://huggingface.co/papers/2610.17246},
  keywords = {LLM-to-Flow Distillation, flow-based generators, diffusion-based generators, DPO, preference distillation, out-of-distribution adaptation, therapeutic peptides, small-molecule structure design, code available, huggingface daily},
  eprint = {2610.17246},
  archiveprefix = {arXiv},
}

Metadata

{}