An experiment with Jev
← More posts
·7 min read

An experiment with Jev

I’d already built a small personal app based on the government service confusion I see in my everyday work: people describe their situation in their own words, and an LLM works out which service they need instead of making them pick blind from a list. If the description is ambiguous, it asks a clarifying question. Built quickly with AI tooling, running on Vercel with Claude Haiku 4.5 doing the routing. What I ended up with was really an intelligent state machine: the states and transitions are ordinary code, but picking the right transition means understanding what someone actually wrote.

Then TypeSafe AI released Jev, a model that doesn’t generate text at all. It answers bounded, typed questions and hands back probabilities. My first thought was: hey, all I’m doing here is making choices. Worth trying the swap.

This is about that switch, and playing with emerging tooling.

A note on the example, before anyone reaches for it. The services are made up. You could reasonably argue a service like this shouldn’t ask people to describe their situation at all, and a clear landing page or a good list would do the job without a model. You might well be right, but it isn’t what I’m testing. The services stay the same; what changes is what makes the routing decision - an open conversation with a generative model, or a bounded choice with a model built for choices.

What Jev actually is

Jev is TypeSafe AI’s first “System One” model. Given a piece of state (JSON) and a set of questions, it returns typed answers - never a sentence. There are three question types, all answered in parallel against the same state:

Primitive Answer shape Use
Choice one label + a probability per option pick from a fixed set
Score a score against levels you describe in order, plus how likely each level is grade something on a scale
Noul a single probability of “yes”, no separate confidence a yes/no judgement

Choice and Score also return a separate confidence value, but the thresholds below use the top option’s probability. It costs $0.042 per million input tokens and output is free. Extra questions in the same call run in parallel, so they’re nearly free too: two extra yes/no questions cost me about $0.000006 a request and no noticeable time.

In code, all three can go in one call:

const { answers } = await client.systemOne({
  state: "My order still hasn't arrived and I'm fed up",
  questions: {
    topic: choice("What is this about?", {
      delivery: "Late or missing orders",
      refund: "Getting money back",
    }),
    urgency: score("How urgent is this?", [
      "Can wait",          // 0
      "Reply today",       // 1
      "Reply right now",   // 2
    ]),
    upset: noul("Is the customer upset?"),
  },
});

// answers.topic   -> choice, probabilities, confidence
// answers.urgency -> score, legend, probabilities,
//                    confidence
// answers.upset   -> noul (the probability of yes)

Before: the model decided, from the prompt and service descriptions

Four services to choose between, in two easily confused pairs. Haiku, called through the app’s AI gateway, reads the description and either routes the user or asks a clarifying question, over a few turns if needed.

After: a probability per option, and the code decides

That same judgement becomes a single Choice question: the four services plus “none”, defined by the descriptions already written for the app.

What comes back is a probability for each option, and the code decides using two numbers I picked. If the leading option reaches 0.85, route them. Any option at 0.15 or above gets offered by name; anything below is noise. So two or more contenders get shown side by side, one gets offered with an invitation to say more, and none gives an honest “nothing here fits”.

const { answers } = await client.systemOne({
  state:
    "I need help finding somewhere to live, " +
    "my mobility isn't great",
  questions: {
    service: choice("Which service is this about?", {
      housing: "Applying for council housing...",
      "housing-benefit-disability":
        "Housing help because of a disability...",
      "council-tax": "Compulsory yearly property tax...",
      "garden-waste": "Optional paid bin collection...",
      none: "Only for needs outside all four services...",
    }),
  },
});

answers.service.probabilities;
// { "housing-benefit-disability": 0.88,
//   housing: 0.12, ... }

0.88 clears the 0.85 bar, so they go straight to Housing Benefit (disability): one call, no questions.

Before: the user’s description goes to an LLM call with the policy written in the prompt; the model decides whether to route or ask a question, and every answer loops back to another full call, capped at five rounds. After: one Jev call returns a probability per option; the code applies two thresholds, 0.85 to act and 0.15 to offer, and either routes, offers options by name, or says nothing fits. If the user says more after being offered options, a second call gives the final answer. There is no loop.

If the user taps an option, they’re routed. If they write more, a second call re-scores every option, not just the ones they were shown, so “actually it’s neither of those” stays reachable.

Then the actual work: making the decisions better

I put together a set of cases - obvious ones, easily confused pairs, and a couple that shouldn’t match anything. Out of the box, several were off: vague wording drifted towards “none of these” instead of the obvious service, and cases that should have gone straight through stopped to ask.

So I rewrote the option descriptions. No code changes.

  • Say what each option is not. Every description names the boundary with the option it gets mistaken for.
  • Write down what the model can’t know. Garden waste is a paid subscription; ordinary bin collection isn’t charged separately. So “pay for a collection” can only mean one of them.
  • Stop “none of these” being the safe answer. Being vague about which service you need isn’t the same as needing none of them, and it has to be told that.

After that, the clear cases route straight through and the ones that shouldn’t match anything say so, every time I re-ran them. My cases and my tuning, so it’s a development set rather than a proper evaluation.

The ambiguous cases taught me the most about Jev. A Choice tells you which option is most likely - not whether the person has said enough to choose one. “I want to pay the council” comes back confidently as council tax, and nothing in the distribution admits what’s missing. Rewording won’t fix that, but another question will. A yes/no in the same call - have they said which payment? - comes back a firm no, and just as firmly yes when they have. In testing, acting on that gave every vague payment request both options instead of a guess.

Looking back, nearly every improvement came from two things: sharper option descriptions and asking the right extra question. With Jev, content and question design turned out to be most of the logic.

Where the decision moved

Before, the model decided whether to route or ask, and the prompt had to talk it into concluding. Now the model says what’s likely and the code decides what to do about it. Jev made me decide up front what every possible answer was and what should happen with each, rather than leaving the model to improvise a question.

The numbers

From my setup, running locally: Jev through TypeSafe’s SDK, Haiku through Vercel’s AI gateway. Six journeys, four runs.

Six journeys, range across four runs Model calls Average per journey Cost per 1,000 journeys
Conversational LLM (Haiku 4.5) 10 2.1–3.8s $1.74–1.75
Jev, as the router actually runs 8–9 0.40–0.44s $0.047–0.053
Jev, forced to two calls every time 12 0.57–0.62s $0.068

List prices: Haiku 4.5 at $1 per million input tokens and $5 per million output; Jev at $0.042 per million input, output free.

Even forced to take a second call every time, Jev reached the same place as Haiku on all six journeys, every run, at about 5x the speed and a twenty-fifth of the cost. Run normally, it’s about 7x faster and a thirty-third to a thirty-seventh of the cost, though in one run it routed a borderline disability case without asking.

If you’ve got a step in a flow that’s really just a choice, this make complete sense. Jev and the System One primitives are documented at docs.typesafe.ai, and TypeSafe AI are at typesafe.ai.