The short answer
A production-grade fine-tuned model for a single enterprise task typically costs $60,000–$250,000 all-in, of which GPU compute is usually 5–20%. The rest is data, evaluation, engineering time and deployment. A quick experiment on a public dataset can cost a few hundred dollars of compute; a model you would put in front of customers or regulators cannot.
The useful question is not “what does fine-tuning cost?” but “what does it cost compared with what we pay now?” For high-volume tasks, a fine-tuned small model often pays for itself within one to two quarters through lower inference spend alone.
1. Training compute
Training compute scales with model size, dataset size, sequence length, method and how many experiments you run. Rough figures for a single training run at current cloud GPU rates:
| Model size | LoRA / QLoRA run | Full fine-tune run | Typical project total |
|---|---|---|---|
| 1–3B | $5–$40 | $40–$300 | $500–$3,000 |
| 7–8B | $20–$150 | $150–$1,200 | $3,000–$15,000 |
| 13–14B | $50–$300 | $400–$2,500 | $6,000–$25,000 |
| 70B | $300–$2,000 | $3,000–$15,000 | $20,000–$90,000 |
| 400B+ | $3,000+ | $30,000+ | $150,000+ |
The “project total” column is what matters. A real project includes a base-model bake-off, hyperparameter sweeps, several data iterations and one to three preference or RL rounds, which is why it runs 20–60× the cost of a single run. Reinforcement learning is the expensive part: sampling many responses per prompt makes RL steps several times costlier than SFT steps.
Hosted fine-tuning APIs price per training token instead — currently roughly $0.50–$8 per million tokens depending on model size. They are convenient for SFT and DPO on supported models and often cheaper than self-managed GPUs for small jobs, but they limit the methods, models and deployment options available.
2. Data: the real budget
Data is usually the largest line. Costs depend on how much labelled data you already have and how specialized the labelling is.
- Mining existing data: historical tickets, decisions and corrections are often the best training source. Extraction, cleaning, deduplication and PII redaction typically take 1–3 engineering weeks.
- General annotation: $0.50–$3 per example for tasks a trained generalist can do.
- Expert annotation: $5–$40 per example when a clinician, lawyer, underwriter or engineer must label. Most enterprise tasks fall here.
- Preference data: comparisons are faster to produce than demonstrations — roughly 30–60% of the cost per item.
- Synthetic data: cheap to generate, not free to verify. Budget for filtering and spot-checking.
A typical program of 8,000 expert demonstrations plus 5,000 preference pairs costs $60,000–$150,000 in labelling alone if bought from scratch — which is why mining what you already have comes first.
3. Evaluation
A credible evaluation suite — an adjudicated golden set, calibrated automated judges and a human review protocol — costs $15,000–$50,000 to build. It is reused for every future model, prompt and vendor decision, so it is the best-value line in the budget. Skipping it is the single most expensive mistake we see: teams spend months iterating without knowing whether they are improving.
4. Serving and operations
Serving costs depend on traffic shape. A quantized 8B model on one data-center GPU can serve hundreds of requests per second for short tasks; reserved, that GPU costs roughly $1,200–$2,800 a month. A 70B model needs two to four high-memory GPUs. Add monitoring, on-call and quarterly retraining — typically 10–20% of the original project cost per year.
Three worked examples
A. Document extraction, 8B model
2M documents a month, 40 fields each. 6,000 expert-verified examples mined from historical data plus 3,000 new labels. SFT then one DPO round. Data $38k, engineering and evaluation $54k, compute $6k, deployment $12k. Total ≈ $110k. Replaced a frontier-API pipeline costing $41k a month; new run cost $3.2k a month. Payback: under three months.
B. Support agent with tool use, 14B model
SFT on 12,000 resolved conversations, then GRPO against a tool-call verifier and an escalation reward model. Data $46k, engineering $88k, compute $24k, deployment $18k. Total ≈ $176k. Value came from automation rate rather than inference savings: 58% of tier-1 tickets resolved without a human.
C. Clinical summarization, 70B model, on-prem
Continued pre-training on de-identified notes, SFT on 9,000 clinician-written summaries, DPO on 6,000 clinician preferences. Data $120k, engineering and evaluation $140k, compute $78k, on-prem deployment $45k. Total ≈ $383k. Justified by data-residency requirements that ruled out external APIs.
Calculating break-even
The break-even for replacing an API workflow with a fine-tuned model is:
months to payback = project cost ÷ (current monthly inference spend − new monthly run cost)
As a rough guide, workloads above $15,000 a month in API spend on a single, repetitive task usually pay back within a year on cost alone, before counting accuracy, latency or independence benefits. Try the savings estimator on our home page with your own numbers.
Every month spent iterating on prompts for a task that should be fine-tuned is a month of full-price inference plus engineering salary. Delay is a cost line too.
Costs teams forget to budget
The line items above cover the project. These are the ones that show up later and surprise finance:
- Expert time on your side. Even with outsourced annotation, your domain experts need to write guidelines, adjudicate disagreements and review outputs. Budget 2–4 hours a week from one or two senior people during the data phase.
- Security and legal review. Vendor assessments, DPAs and model-risk validation can take longer than the training. Start them in week one.
- Retraining. Production data drifts. Most models need refreshing quarterly; each cycle costs 10–25% of the original project, falling as the pipeline matures.
- Base-model migrations. Better open-weight models arrive several times a year. Re-running the pipeline on a new base is cheap if everything is versioned, expensive if it isn’t.
- Evaluation upkeep. Golden sets need new examples as the business changes; judges need recalibration when the judge model updates.
- GPU availability. Reserved capacity is cheaper per hour but costs money idle; on-demand is flexible but can be scarce for the largest GPUs. Plan which you need before the training phase.
A simple budget template
| Line | Share of a typical project | What drives it |
|---|---|---|
| Data (mining, labelling, preferences) | 30–45% | Label count, expertise required, existing data |
| Engineering & research | 25–35% | Number of training stages, task complexity |
| Evaluation | 10–20% | Golden-set size, human review depth |
| Compute | 5–20% | Model size, RL vs SFT/DPO, experiment count |
| Deployment & hardening | 8–15% | Target environment, latency targets, on-prem |
Seven ways to reduce fine-tuning cost
- Mine historical data before buying labels.
- Build the eval set first so no experiment is wasted.
- Start with the smallest base model that could plausibly work.
- Use LoRA for exploration; switch to full fine-tuning only if the eval says it is worth it.
- Label the examples the model gets wrong, not random ones (active learning).
- Prefer DPO over full RLHF unless you have a strong, verifiable reward.
- Quantize for serving and validate the accuracy impact.
See our pricing for fixed-price packages built around these principles.