Put a typed guardrail in front of an agent tool call

The failure mode guardrails exist for

Agents fail in a specific way: they emit a tool call that looks locally reasonable and is globally wrong. A SELECT on a production table that was meant for staging. A DROP TABLE where a DELETE was implied. Asking a chat model a second prompt — “is this safe?” — is slow, unstructured, and turns your safety policy into free text the next release can silently change.

A typed guardrail asks three questions instead: does this call match what the user asked for (noul), should a human confirm before it runs (noul), and how destructive is it if wrong (score). All three ride in one request, and every answer comes back inside the answer space you defined — your policy stays in code, not in prose.

Assembling the state

Send the user request and the proposed call as the state. An object state reads better than one long string here, because each fact keeps its name and the criteria can reference them.

{
  "state": {
    "user_request": "Clean up the old test data in the staging database.",
    "proposed_tool_call": {
      "tool": "sql.execute",
      "args": { "query": "DROP TABLE customers;" },
      "environment": "production"
    }
  },
  "questions": {
    "matches_intent": {
      "type": "noul",
      "instructions": "Does `proposed_tool_call` match what `user_request` asked for?"
    },
    "needs_confirmation": {
      "type": "noul",
      "instructions": "Should a human confirm this tool call before it runs?",
      "criteria": {
        "true": "Touches live data, is destructive, or the environment does not match the request",
        "false": "Read-only, staging-only, and exactly what was asked"
      }
    },
    "risk": {
      "type": "score",
      "instructions": "How destructive is this action if it is wrong?",
      "criteria": ["Harmless", "Recoverable", "Irreversible data loss"]
    }
  }
}

Keeping the policy in your code

Jev only judges; it never runs the tool. Your wrapper turns probabilities into actions:

const a = answers;
const must_confirm =
  a.matches_intent.noul < 0.8 ||
  a.needs_confirmation.noul > 0.7 ||
  a.risk.score >= 1.5; // past "Recoverable", toward the red end

if (a.risk.score >= 2.0 && a.matches_intent.noul < 0.5) {
  blockOrFlag(a.request_id); // never execute, log for review
} else if (must_confirm) {
  confirmWith(a.request_id); // ask the human
} else {
  execute(call);
}

Thresholds are yours to set. 0.8 is a reasonable starting confidence line for intent match; tighten it for irreversible tools (DROP, deploy, email-send) and loosen it for read-only ones. Run ten real examples from your own traffic — not toy cases — and move a threshold only where the model disagrees with what you would have decided anyway.

The test matrix before you trust it

• Benign call in the right environment — expect low risk, high intent match, no confirmation.
• Right intent, wrong environment (staging asked, production called) — expect intent match to stay high and needs_confirmation to rise. Read the mismatch honestly: the model can only flag what the state lets it see.
• Wrong intent, harmless call — expect needs_confirmation to stay low; harmless is harmless.
• Wrong intent, destructive call (the example above) — expect every signal to trip.
• Ambiguous user request — expect low confidence; route to a person rather than guessing.

If a case comes out wrong twice, do not tune a threshold to hide it — adjust the criteria wording, or put the missing fact (like the caller's environment) into the state.

Cost in practice

This three-question body weighs in around 350–500 tokens, comfortably inside the 1-credit minimum, so gating every tool call costs $0.0001 per call on Starter and proportionally less on the larger packs. Latency adds one round trip: 400–600 ms in typical measurements against the live gateway. That is acceptable in most agent loops. If your tool calls are sub-100 ms affairs and every millisecond shows, gate only the tools whose blast radius justifies it — filesystem writes, SQL, payments, email — and pass the rest through.

Where typed guardrails stop working

Be clear about the edges: a guardrail model cannot see what is not in the state. If you do not include the target environment, it cannot notice the environment mistake. It also will not replace deterministic controls — use allowlists, path checks and network scopes for the things they are good at, and let the typed decision layer handle the semantic question: does this plan match the request?