Decision models, in plain words: AI that picks instead of writes

What's new

A new kind of AI model is getting a lot of attention this month. TypeSafe's Jev, which the company calls a "System One" model, takes a situation written in plain text plus a set of typed questions, and sends back answers with probabilities instead of prose. Open models built on the same idea are on Hugging Face as well: Cloudflare's clef, a 27-billion-parameter model built on Qwen, and Laya, at 421 million parameters small enough to run on a laptop. Both are released under the Apache 2.0 license.

How it works, in plain words

Think of a multiple-choice test. You write the question and the answer choices. The model doesn't write an answer; it marks how sure it is about each choice. Jev takes three kinds of question: a statement to judge true or false, a pick from a list of options you wrote, and a score on a scale you define. It can't come back with an option that isn't on your list.

If you've built a classifier, it will feel familiar: it's like asking for predicted probabilities, except you type in the classes at the moment you ask.

Why teams care

• Speed. TypeSafe says Jev answers in 70 to 500 milliseconds, fast enough to run inside a normal web request.
• Answers code can act on. A probability per option goes straight into an if-statement: route the ticket, flag the review, or hand the case to a person.
• No text to parse. The options are fixed, so the answer always arrives in the shape the code expects.

The catch: is the confidence honest?

A probability is only useful if it means what it says. That property is called calibration. Take all the cases where the model said about 90%. If it's well calibrated, about 9 in 10 of them are right. If only 7 in 10 are right, the model is overconfident, and a rule like "act automatically above 90%" lets through three times the mistakes you planned for.

Speed and tidy output say nothing about this. TypeSafe says it trains its models with calibration in mind, but no independent calibration numbers have been published yet. Until they are, the safe move is to measure it on your own data.

A five-step check before you rely on it

1. Label your own cases. 100 to 300 real examples with the right answer, in the mix you actually see, rare options included.
2. Run the model and keep every probability, not just its top pick.
3. Draw a reliability plot. Group the cases by the probability the model gave (0 to 10%, 10 to 20%, and so on) and compare each group's average confidence with how often it was right. Report the expected calibration error and the Brier score next to accuracy.
4. If it's off, fix it with one number. Temperature scaling stretches or shrinks the probabilities. Fit it on half of your labels and check it on the other half.
5. Pick the cut-off by cost. Above it, act automatically; below it, send the case to a person. Check again every month, because new kinds of cases drift in.

What it means

Decision models make it cheap to put a judgment call inside software. The open question isn't speed; it's whether the numbers can be trusted at the cut-off you choose. That's a measurement problem, and a solvable one: a few hundred labeled cases and an afternoon of checking will tell you. Calibration Check does steps 2 to 5 in your browser, with a demo on real reviews.