Jev vs asking an LLM for JSON

Two different contracts

A chat model writes text. Even with JSON mode or strict structured outputs, it is still sampling tokens toward a schema — the shape is guaranteed, the content is generated. Jev works the other way around: the content is guaranteed (it can only return an option you declared) and what the model adds is the probability distribution over your declared options. Those are different contracts, and they break in different ways.

• Chat model with JSON mode: the shape comes out right, but it can surface a label you did not declare ("billingx" instead of "billing"), canonicalize inconsistently, or wrap the payload in prose. You write a parser, an allowlist check and a retry loop.
• Decision model: the answer space is the criteria you sent. A choice question with three options returns exactly one of those three plus a probability for each. There is nothing to parse, because nothing was generated.

Where the LLM genuinely wins

Generation tasks — write the email, summarize the ticket, draft the SQL — belong to the chat model. Nothing narrower replaces them. Structured outputs are also fine when what you need is a custom object (fields extracted from a document) rather than a branch decision. Keep that pipeline as it is.

The seam is the line of code that currently says: if (await classify(text) === "billing"). That line needs a bounded answer with a calibrated probability, not fluent text.

The confidence problem

Ask a chat model for a confidence number and you get one — but it is a self-report about its own fluency, not a calibrated probability over the options you declared. Two calls that both report "85% sure" can sit in completely different regimes. A decision model's probabilities are comparable across inputs, which is what lets you set one threshold ("route to a human below 0.8") and trust it across hundreds of thousands of calls.

A worked comparison

Take intent classification into three queues — billing, technical, sales.

• Chat model approach: one prompt with the labels, run in JSON mode. Cost scales with a system prompt and the full context you resend on every call. You defend against stray labels with an allowlist, and misroutes stay silent until a customer complains.
• Typed decision approach: a choice question with the three criteria descriptions. A short ticket weighs 300–500 input tokens — 1 credit minimum, $0.0001 on Starter. Misroutes surface as high-entropy probability distributions you can threshold, instead of code silently accepting a wrong label.

Using both in the same process

• Let the LLM generate or extract freely where creativity or comprehension is the point.
• Put Jev on the narrow questions before your code acts: which queue, which model tier, ship or block.
• Do not treat either output as a legal, medical or financial decision — set a threshold and keep a person on the high-stakes path.

Most production systems end up with both: Jev for the repeated branch, the chat model for what it is uniquely good at.

Failure modes, side by side

• Free-text label drift: the LLM returns "billing-related" where your switch wants "billing". The typed call indexes probabilities by the exact keys you declared, so drift is structurally impossible.
• Hallucinated fields: JSON mode can add keys you did not ask for. A decision model cannot — the response schema is fixed by the question types.
• Silent misroutes: the drift case that passes because the downstream code fell back to a default. In a typed decision you can log the full probability distribution per call, and review the bimodal ones.
• Confidence theater: parse-then-assert ("I am confident this is X") — imprecise, not calibrated. Jev's confidence accompanies every choice and score answer and drives your human-review path.

Migrating a parser-based classifier

The migration is mechanical: stop asking the chat model for a label, and — put the same taxonomy into a choice question's criteria with the same wording the prompt used. Resist the urge to re-describe the labels; your existing prompt wording already encodes the distinctions the team. Run two weeks asking both, log agreement — when disagreements are 2% or less, look at a handful of cases, always the same ending: either the criteria wording is ambiguous, or the case was genuinely ambiguous and the old system was silently misrouting anyway.

The same request, both ways

Here is what the two contracts look like over the same ticket:

system: Classify the support message into billing, technical
or sales. Reply with {"queue": "..."}. Do not invent labels.
user: Charged twice for order A-104, please refund the duplicate.

{
  "state": "Charged twice for order A-104, please refund the duplicate.",
  "questions": {
    "queue": {
      "type": "choice",
      "instructions": "Which team handles this?",
      "criteria": {
        "billing":  "Payments, payouts, invoicing, refunds",
        "technical": "Bugs, outages, integrations",
        "sales":    "Pricing, upgrades, new accounts"
      }
    }
  }
}

The first returns {"queue": "billing"} when it goes well, and anything at all when it goes badly. The second returns billing with a probability like 0.97, distributions over all three options, and a confidence value — plus the same failure contract every time (an error shape with error.code, never a graceful-sounding wrong answer). Design downstream code around whichever failure you would rather debug.