Use case
Model routing
Pick a cheap or expensive model from the task, not from a hardcoded list.
The problem
Sending every prompt to the largest model burns budget on lookups.
What your code does with the answer
Send `small` tasks to a fast model and escalate only `large` ones.
What the routing request looks like
State is the task itself — the ticket, the query, the prompt — and two questions: difficulty (choice across small, medium and large, with descriptions tied to what each tier handles well) and needs_tools (yes/no: does answering require code execution or retrieval?). Both answers come with probabilities, so borderline tasks show up as split confidence instead of a silently wrong route. The call is typically 1 credit; the saving is not paying the large tier's rate on lookups and rewrites.
Escalation is a second request, not a rebuild
Route on difficulty. Small-scored tasks go to the fast model directly. Everything above your confidence line — difficulty split across tiers, uncertain tool needs — escalates to the expensive path with the same state you already built, so no re-plumbing. Log route, confidence and latency per call and re-tune the tier descriptions as traffic shifts.
Pitfalls specific to routing
Tiers described by vibe rather than behavior ("cheap / pricier / priciest") invite inconsistent calls; describe what the task looks like instead. Do not hide the escalation path — measure where live traffic would land, and if most tasks score to the large tier, either your tiering is off or your criteria descriptions are too cautious. And route on the task, not on the user: a power user's typo deserves the small tier.
The request shape
This is the full request the playground sends for this preset — the same JSON your server sends to /v1/systemone with a Bearer key.
curl https://jevai.website/api/v1/systemone \
-H "Authorization: Bearer $JEV_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"state": "Write a SQL query that returns the top 5 customers by revenue in 2025 from tables orders(customer_id, total, created_at) and customers(id, name).",
"questions": {
"difficulty": {
"type": "choice",
"instructions": "Which model tier is enough to answer this well?",
"criteria": {
"small": "Simple lookup or rewrite a small model handles",
"medium": "Routine reasoning or code",
"large": "Multi-step reasoning, ambiguity or long context"
}
},
"needs_tools": {
"type": "noul",
"instructions": "Does answering require running code or fetching data?"
}
}
}'FAQ about this use case
- Should the router know my costs?
- Keep it behavioral: criteria describe what each tier handles. Pricing lives in your policy — a task scored small that returns low confidence is an escalation candidate, and the confidence number is what makes that rule work.
- How many tiers should I declare?
- Three is the sweet spot, written in behavioral terms: lookup and rewrite, routine reasoning or code, multi-step ambiguity or long context. Two tiers lose nuance; five start blurring.
- Does this work on non-English tasks?
- Yes — the state can carry the task in its own language. Keep the tier names in criteria as identifiers your code switches on.