Reference
The fine-tuning glossary.
30 post-training terms, defined in plain English by the people who use them daily.
- Adapter
- A small set of trainable parameters added to a frozen model (see LoRA). Many adapters can share one base model at serving time.
- Base model
- The pre-trained model you start from. Choosing it — family, size, licence — is one of the highest-leverage decisions in a project.
- Catastrophic forgetting
- Loss of general abilities after fine-tuning on a narrow task. Controlled with data mixing, lower learning rates and regression evals.
- Continued pre-training
- Further next-token training on a large unlabelled domain corpus before instruction tuning, to deepen domain knowledge and vocabulary.
- Contamination
- Overlap between training data and evaluation data, which inflates scores. Checked with n-gram and embedding similarity.
- Distillation
- Training a smaller student model to reproduce a larger teacher’s behavior on a task. See our distillation guide.
- DPO
- Direct Preference Optimization: training on chosen/rejected response pairs without a separate reward model or RL loop.
- Eval harness
- Code that runs a model over evaluation datasets and computes metrics, ideally in CI on every change.
- Few-shot prompting
- Including worked examples in the prompt. A useful baseline, and often the first thing fine-tuning replaces.
- Full fine-tuning
- Updating all of a model’s parameters during training, as opposed to parameter-efficient methods.
- GRPO
- Group Relative Policy Optimization: an RL method that scores a group of sampled responses and reinforces those above the group mean.
- Held-out set
- Evaluation data never used for training or tuning decisions. The only honest measure of generalization.
- Instruction tuning
- SFT on a broad set of instructions and responses so a base model follows directions.
- KL penalty
- A term that keeps an RL-trained model close to its reference model, limiting reward hacking and drift.
- KTO
- Kahneman–Tversky Optimization: preference tuning from binary good/bad labels instead of pairs.
- LLM-as-judge
- Using a language model to grade outputs against a rubric. Must be calibrated against human ratings to be trusted.
- LoRA
- Low-Rank Adaptation: trains small low-rank matrices alongside frozen weights, cutting memory and cost dramatically.
- Model card
- A document describing a model’s intended use, training data, evaluation results and limitations.
- Post-training
- All training after pre-training: SFT, preference optimization, RL, distillation and safety tuning.
- PPO
- Proximal Policy Optimization: the classic RL algorithm behind RLHF.
- QLoRA
- LoRA applied on top of a 4-bit quantized base model, making large-model fine-tuning possible on modest hardware.
- Quantization
- Storing weights in lower precision (FP8, INT8, INT4) to cut memory and serving cost, with small accuracy trade-offs.
- RAG
- Retrieval-augmented generation: retrieving documents and inserting them into the prompt. Compare in fine-tuning vs RAG.
- Reward hacking
- When a model exploits flaws in a reward signal to score highly without actually improving.
- Reward model
- A model trained on human preferences to score outputs; used as the objective in RLHF.
- RLHF
- Reinforcement learning from human feedback: optimizing a model against a reward model trained on human preferences.
- RLVR
- Reinforcement learning with verifiable rewards: rewards computed by programmatic checks such as tests or validators.
- SFT
- Supervised fine-tuning: training on input–output demonstrations with next-token prediction.
- Synthetic data
- Training examples generated by models, typically filtered by verifiers and judges before use.
- Win rate
- The share of side-by-side comparisons in which one model’s output is preferred over another’s.
Bring us the task. We'll bring the model.
A 45-minute technical briefing with a senior research engineer. You leave with a feasibility read, a data plan and a fixed quote — whether or not you hire us.
Book a technical briefing →