Distillation: frontier behavior in a model you can afford to run.
Your prototype runs on the largest model money can rent. Distillation transfers its behavior on your task into a student model a tenth the size — private, fast and yours.
Knowledge distillation trains a small student model to imitate a large teacher. For enterprise tasks the teacher is usually a frontier API model with a carefully engineered prompt and retrieval pipeline; the student is a 1–14B open-weight model. The result typically matches the teacher on the target task while costing 5–30× less per request and running in a fraction of the latency.
The craft is in the data. A naive approach — collect teacher outputs and fine-tune — works, up to a point. We go further: generating diverse inputs that cover the real production distribution, filtering teacher outputs with verifiers and judges so the student learns from the teacher’s best work rather than its mistakes, adding on-policy distillation where the student’s own errors are corrected, and finishing with preference tuning on cases where the student and teacher disagree.
Signs you need model distillation.
The right technique, chosen by the eval.
- Sequence-level distillation. Student fine-tuned on filtered, high-quality teacher outputs.
- Logit / on-policy distillation. Student learns from teacher token distributions on its own generations, where licences allow.
- Reasoning distillation. Transfer step-by-step reasoning traces into compact reasoning models.
- Quantization & pruning. FP8, INT4 and structured pruning after distillation, each checked against the eval.
- Student model weights and serving build
- Teacher-vs-student parity report on held-out data
- Synthetic data pipeline you can rerun
- Cost and latency benchmarks on your hardware
All assets transfer to you at handover. No licence fees.
Questions buyers ask us
Is it allowed to distill from a commercial API model?
Terms differ by provider and change over time. We review the applicable terms with your counsel at kickoff and use permissively licensed open-weight teachers when restrictions apply.
How small can the student be?
Classification and extraction tasks often distill into 1–3B models; drafting and reasoning tasks usually need 7–14B. We size the student with an eval, not a guess.
Will the student be as good as the teacher?
On the distilled task it is commonly within a point or two, and after preference tuning on disagreement cases it frequently surpasses the teacher. It will not match the teacher on unrelated tasks — that is the point.
Bring us the task. We'll bring the model.
A 45-minute technical briefing with a senior research engineer. You leave with a feasibility read, a data plan and a fixed quote — whether or not you hire us.
Book a technical briefing →