Specialists beat generalists. Here’s the evidence.
Held-out results from production engagements, against the strongest prompted frontier baseline each client had. Client names withheld under NDA.
| Task | Industry | Best frontier baseline | Fine-tuned | Model size | Cost reduction |
|---|---|---|---|---|---|
| Claims field extraction | Insurance | 86.9% | 94.7% | 8B | 14× |
| Adverse-media disposition | Banking | 79.5% | 91.2% | 14B | 9× |
| ICD-10 code suggestion (top-3) | Healthcare | 84.0% | 92.3% | 7B | 14× |
| Customs HS-code classification | Logistics | 71.8% | 89.6% | 3B | 31× |
| Contract clause risk flagging | Legal | 82.2% | 90.8% | 14B | 8× |
| Tool-call correctness, support agent | SaaS | 88.4% | 96.1% | 8B | 11× |
| SQL generation on internal warehouse | Retail | 74.6% | 88.9% | 32B | 6× |
Frontier baseline = best of zero-shot, few-shot and retrieval-augmented prompting with the strongest available API model at engagement time. Metrics are task-specific (exact match, top-k accuracy or adjudicated pass rate) on frozen held-out sets the client approved before training.
Six engagements, in brief.
Fortune 100 bank
An on-prem 14B model for adverse-media screening, aligned with DPO on analyst disagreements. False positives fell 68%; the model passed the bank’s model-risk validation on the first submission.
Top-3 global insurer
SFT on 38,000 adjudicated claims followed by two DPO rounds. Replaced a frontier-API pipeline at 1/14th the cost with an 8-point accuracy gain and sub-200 ms latency.
Top-5 US health system
Distillation from a frontier teacher into a 7B model running inside the health system’s HIPAA environment. Coders accept 71% of first suggestions.
Global logistics leader
A 3B classifier-generator for harmonized tariff codes across 41 jurisdictions, deployed to edge servers in 230 warehouses.
Top-10 SaaS platform
GRPO with a tool-call verifier and an escalation reward model. 58% of tier-1 tickets resolved without a human, and CSAT up 11 points.
Leading global law firm
A clause-risk model trained with partner-level preference data; associates review 3× more contracts per day with fewer missed issues.
How we keep ourselves honest.
- Frozen held-out sets. Built and signed off before training. Never used for tuning decisions.
- Strongest baseline, not a strawman. We prompt-engineer the frontier baseline as hard as we tune our model.
- Contamination checks. n-gram and embedding overlap between training data and evaluation data, reported every run.
- Humans in the loop. Blind side-by-side review by the client’s experts, not just automated judges.
- Confidence intervals. Bootstrap CIs on every headline number; we don’t claim wins inside the noise.
Want this table with your task in it?
A 45-minute technical briefing with a senior research engineer. You leave with a feasibility read, a data plan and a fixed quote — whether or not you hire us.
Book a technical briefing →