LLM post-training: the complete guide for enterprise teams

What post-training is#

Pre-training teaches a model language and world knowledge by predicting the next token across trillions of tokens. Post-training is everything after: the stages that teach it to follow instructions, use your formats and vocabulary, prefer the answers your experts prefer, and do all of that cheaply enough to run in production. For an enterprise, post-training is where the value specific to you is created, because it is the only stage trained on your data.

Four levers decide the outcome, roughly in order of impact: the data (which examples, labelled by whom, under which guidelines), the objective (imitation, preference, reward), the parameterization (which weights change: all of them, or a low-rank adapter) and the evaluation that decides between candidates. Most failed projects get the third right and the first and fourth wrong.

Stage 1: supervised fine-tuning#

SFT minimizes the negative log-likelihood of target outputs $y$ given inputs $x$, with the loss computed only on the response tokens:

It is cheap, stable and does most of the work for format, terminology and procedure. In our open experiment, SFT on 800 synthetic claim notes took a 0.5B model from 18% to 100% exact match on a five-field extraction task. What SFT cannot do is tell the model which of two acceptable answers is better, and it never shows the model its own mistakes.

Practical rules: compute loss on responses only; deduplicate aggressively; mix 5–20% general instruction data if you need to preserve broad capabilities; prefer two or three epochs on clean data over many epochs on noisy data. Details: supervised fine-tuning.

Stage 2: preference optimization#

Preference methods learn from comparisons rather than demonstrations, which suits judgement-heavy tasks where experts can say “this one is better” far faster than they can write the ideal answer. Direct Preference Optimization trains on chosen/rejected pairs with a classification-style loss derived from the KL-regularized RLHF objective; the full derivation is in DPO from first principles. Variants change the data they need: KTO works from thumbs-up/down labels, ORPO folds preference learning into SFT.

The finding from our DPO simulation that matters most in practice: with realistic amounts of preference data, the $\beta$ hyperparameter does not reliably limit how far the model moves from its starting point. Monitor the realized divergence on held-out prompts and treat it as a release gate.

Stage 3: reinforcement learning#

RL optimizes a reward by letting the model generate, scoring the outputs and reinforcing the better ones. Classic RLHF uses a learned reward model and PPO; the approach popularized for reasoning models uses verifiable rewards and GRPO, which replaces PPO’s learned value function with a group baseline (GRPO explained). RL is the only stage that can take a model clearly beyond its demonstrations, and it is the most expensive: each step generates several completions per prompt, so an RL stage often costs 3–10× an SFT stage (compute model).

Use it when the task has a checkable outcome — SQL that returns the right rows, tool calls that succeed, codes that match a reference — and the evaluation shows headroom after SFT and DPO.

Choosing methods: a decision table#

The last row matters: post-training teaches behaviour, not fresh facts. See fine-tuning vs RAG and fine-tuning vs prompt engineering.

How much data each stage needs#

The first few thousand examples deliver most of the gain on narrow tasks; extra data mostly buys robustness to unfamiliar inputs. In our experiment, a rank-16 adapter trained on far fewer examples already handled familiar templates; it was the unseen formats that improved with more data and capacity.

Full fine-tuning, LoRA or QLoRA?#

LoRA freezes the base weights and learns a low-rank update $\Delta W=\frac{\alpha }{r}BA$ for selected matrices. It trains well under 2% of the parameters and cuts training memory for a 7B model from about 113 GB of weight and optimizer state to about 15 GB (memory model). QLoRA additionally stores the frozen base in 4-bit precision, putting 70B-class models on one 80 GB GPU. Full fine-tuning updates every weight and can learn larger changes, at the cost of more memory and more forgetting. The full comparison, with numbers, is in LoRA vs QLoRA vs full fine-tuning.

Evaluation decides everything#

Every stage above is a choice between candidates, and the evaluation makes the choice. Three numbers from our research shape how we build evaluations:

• Detecting a 3-point improvement at 85% accuracy needs roughly 1,200–2,000 examples, depending on how much the models’ errors overlap (sample-size simulation).
• An LLM judge with 92% sensitivity and 70% specificity reports a real 5-point gain as about 3 points, and its naive confidence interval misses the truth over a quarter of the time (judge bias correction).
• In-distribution test sets can hide large generalization gaps: in our LoRA experiment, every configuration from rank 4 up scored the same on familiar templates and differed sharply on new ones.

Build the evaluation first, freeze it, and include a slice of inputs unlike the training data. The evaluation plan template is a starting point.

A reference recipe#

The starting configuration we use for a narrow enterprise task on a 7–14B base model, before the evaluation tells us to change anything. Every value is a default to be overturned by evidence, not a rule:

# stage 1 — SFT
base_model:        best of a 3–5 model bake-off on the frozen eval
method:            lora            # full FT only if eval shows a gap LoRA can't close
lora: {r: 16, alpha: 32, dropout: 0.05, targets: [q,k,v,o,gate,up,down]}
lr:                2.0e-4          # 1e-5 to 2e-5 for full fine-tuning
schedule:          linear warmup 3–10%, then cosine or linear decay
epochs:            2–3             # stop on held-out loss / task metric, not on train loss
batch:             32–128 sequences (via gradient accumulation)
loss_on:           response tokens only
data_mix:          task data + 5–20% general instructions if broad skills must survive
packing:           on, with attention masks per example

# stage 2 — preference tuning (if judgement matters)
method:            dpo             # kto if you have thumbs-up/down instead of pairs
beta:              0.1
pairs:             2k–10k, rejected = SFT model's own outputs, chosen = expert-corrected
epochs:            1–2
release_gate:      held-out KL budget + reward margin + task metric (see DPO research)

# stage 3 — RL (only with a verifier and measured headroom)
method:            grpo, G = 8–16, prompts filtered to 0 < pass rate < 1

The comments carry the important part. Two choices in particular come from our own measurements: picking LoRA rank on held-out formats rather than held-out examples (experiment), and gating DPO on measured divergence rather than trusting $\beta$ (simulation).

Team, timeline and where the hours go#

A production fine-tuning project for one task is typically a six-to-ten-week effort for a small team. The hours are not spread the way most plans assume:

Training itself — the part most people picture — is the smallest line. The single strongest predictor of a project finishing on time, in our experience, is whether the evaluation set is frozen by the end of week two.

Failure modes we see most#

Template overfitting#

The model learns the surface form of training inputs rather than the task. Symptom: excellent validation scores, failures on a new input channel. Defence: held-out formats, diverse sources, higher adapter rank or full fine-tuning when inputs vary.

Catastrophic forgetting#

Narrow training erodes general abilities the application still relies on, such as following an unusual instruction. Defence: data mixing, lower learning rates, LoRA, and a general-capability regression suite.

Reward hacking#

RL and preference stages find outputs that score well without being better: longer answers, confident hedging, exploited verifier gaps. Defence: held-out human evaluation, KL budgets, adversarial verifier tests.

Contamination#

Evaluation items leak into training data, often through synthetic data generated from the same documents. Defence: n-gram and embedding overlap checks on every data version.

What it costs#

Training compute is the smallest line in most budgets: a 60M-token full fine-tune of a 7B model is about 2.6 H100-hours. The real cost drivers are expert labelling, the number of experiments and evaluation. Break-even against a per-token API, in our base-case model, arrives around a million requests a month, and most sensitively depends on the API price you are replacing (break-even model). Worked budgets are in how much fine-tuning costs.

Regulated deployment#

In banking, insurance and healthcare, the model must also survive governance. The relevant rules have moved in 2026: US model-risk guidance was replaced in April, and the EU AI Act’s obligations for general-purpose models have applied since August 2025. Our regulated-industries hub covers what each means for a fine-tuned model, including the EU compute threshold that a typical enterprise fine-tune sits ten thousand times below.

References#

1. Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155.
2. Hu, E. J. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
3. Dettmers, T. et al. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. arXiv:2305.14314.
4. Rafailov, R. et al. (2023). Direct Preference Optimization. arXiv:2305.18290.
5. Shao, Z. et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300.
6. Hinton, G., Vinyals, O. and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531.