Want to spend less on AI? Add another model.
It sounds contradictory, but in our test the decision model Jev made a decision in 0.32 seconds. The language model we tested took 2.44 seconds, and Jev was almost eighteen times cheaper per call. If Jev handles three quarters of the cases and only the uncertain ones go to the language model, costs drop by almost 70%. Whether that works without sacrificing quality is exactly what we are now investigating.
The reason is our new application. Some parts ask AI to make the same kind of assessment over and over. The cost of each assessment is small, but across thousands of requests, cost and waiting time quickly add up.
The idea is to divide the work. A specialised model handles the well-defined questions; a language model remains available for cases that need more interpretation. What that delivers depends on two things: how much work the first step completes on its own, and how well it does so. A cheap assessment is worth little if someone then has to fix the mistakes.
A model that decides instead of talking
The introduction of Jev by TypeSafe prompted us to test this. Jev is a decision model: you give it context and focused questions, and you get choices, scores and probabilities back.
It interprets language, but does not write any text itself. For software that only needs a category or a score, that is enough.
Jev is not the only one. There are now more candidates, including Clef by Cloudflare. So we compared several options, tested on our own examples.
Putting it to the test
We started with one part of the application. Jev, Clef Flash and Claude Haiku each assessed 48 practical examples, twice. Haiku received a variant of our existing prompt. That made 288 calls in total.
The trial ran completely separately from the existing workflow: we only collected the responses and applied nothing.
These were the measured response times and estimated API costs:
| Model | Median response time | Estimated API cost per 1,000 calls |
|---|---|---|
| Jev 1.13.0 | 0.32 seconds | $0.20 |
| Clef Flash | 0.68 seconds | $0.40 |
| Haiku 4.5 with our prompt variant | 2.44 seconds | $3.56 |
Measured on 3 October 2026. Costs are calculated from reported usage and the applicable rates. All responses are included, including those rejected by our validation.
Jev was the fastest and cheapest: more than seven times faster than Haiku and almost eighteen times cheaper per call. A fair caveat: the models did not do exactly the same thing. The decision models answered classification questions; Haiku wrote a more detailed proposal with text. So these are costs per call, not yet savings across the whole workflow.
The economics of combining models
It gets interesting when the models work together:
Jev assesses
Every case is assessed by Jev first.
The application checks
Is there enough information, do the answers agree, and is further assessment needed?
The LLM when needed
Only some cases then go on to the LLM.
Using our measured costs gives the following hypothetical scenarios:
| Workflow for 1,000 cases | Estimated API cost |
|---|---|
| Everything handled by the tested LLM variant | $3.56 |
| Everything through Jev first; 50% then go to the LLM | $1.98 |
| Everything through Jev first; 25% then go to the LLM | $1.09 |
In these scenarios, Jev pays for itself: each Jev call costs a fraction, and every case that does not need the LLM saves the most expensive step. But that only works if Jev takes enough work away. If everything still goes to the LLM, you actually pay more: about $3.76 instead of $3.56. And cases passed on get the waiting time of the first step on top.
An extra model lowers total costs as soon as it takes enough work off the more expensive model.
These are worked examples, not measured savings. We assume that a follow-up assessment by the LLM costs the same as in our trial. Human review, infrastructure and reassessments are not included, and different prompts, batching or reusing requests can change the balance.
What we are taking into the next trial
The first trial also provided practical lessons for the design of our application:
Fixed rules stay in the software.
A model sometimes recognised information correctly, but then did not apply the associated business rule properly.
Each part gets one task.
The decision model classifies, the software applies the rules, and an LLM or a person handles the unclear and complex cases.
Explanations can often use templates.
Established facts and assessments form a clear proposal. The user gives final approval.
Responses need to be usable.
Our validation rejected 30 of the 96 Haiku responses because of their structure. That does not establish 30 incorrect assessments, but it does give us a reason to improve the integration.
The model remains interchangeable.
With fixed questions and response structures, every new candidate goes through the same test. Switching does require further testing and calibration.
The most important question is still open: how much work can a decision model handle on its own? Confirmed context was missing in this trial, so all decision-model results still went past a person. There are also too few verified answers yet to name a winner on quality.
That is why the next trial is bigger: 300–1,000 human-reviewed examples, with a separate holdout set for the final evaluation. We will compare the complete workflow on quality, cost, speed and corrections needed. Then we will know how much of the speed and cost advantage we can actually realise.
To follow developments, see the community benchmark Decision Index on Hugging Face. The Rubik's Cube race by AI/ML API on X, which compares different types of models, is also a fun illustration of dividing the work. Jev made twenty moves in twenty seconds, but only Opus 5.5 solved the cube, in three moves: deciding fast is not the same as thinking ahead. Whether the combination works is ultimately something you test with your own practical examples.
Are you building an application with many recurring AI assessments?
We would be happy to help design a practical trial that compares speed, cost and quality together.
