Use case
Content moderation
Decide allow, review or remove, and flag real threats separately.
The problem
Keyword filters over-block slang and miss doxxing dressed as a joke.
What your code does with the answer
Remove on high-confidence violations. Queue the rest for a moderator.
The two questions moderation actually needs
One choice — allow, review or remove — carries the policy posture; one noul asks separately whether the comment encourages harassment or violence against a person. Splitting them matters: the example comment is rude the way internet comments are rude, while the "show up at his house" clause makes it a removal with a threat flag. One question collapses that distinction; two keep it, with probabilities that let a moderator see contested cases instead of silently mislabeling them.
From probabilities to enforcement
Remove on a high-confidence removal; queue the rest for a moderator with the distribution attached — a 50/50 allow/review split is the item a moderator wants to see next. Keep the threat flag firing even when the action reads allow: harassment signals drive escalation and user-safety paths regardless of where the content lands. Fix thresholds that misfire by fixing the criteria wording, not the number.
Pitfalls specific to moderation
Moderation criteria drift with slang — review the allow/review/remove wording on a regular cycle. Watch for one-shot moderation on contested cases: a 45/55 allow-review split you label silently as "allow" is how review queues get bypassed exactly where they matter. Keep a person on the contested band and log the distribution, so patterns in later misjudgments stay visible.
The request shape
This is the full request the playground sends for this preset — the same JSON your server sends to /v1/systemone with a Bearer key.
curl https://jevai.website/api/v1/systemone \
-H "Authorization: Bearer $JEV_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"state": "lol this guy has no idea what he is talking about, someone should dox him and show up at his house",
"questions": {
"action": {
"type": "choice",
"instructions": "What should happen to this comment?",
"criteria": {
"allow": "Acceptable, even if rude",
"review": "Borderline, needs a moderator",
"remove": "Clear policy violation"
}
},
"threat": {
"type": "noul",
"instructions": "Does the comment encourage harassment or violence against a person?"
}
}
}'FAQ about this use case
- Does moderation need the full thread?
- Usually yes — sarcasm and context read differently with a couple of parent comments. States run up to 100,000 characters, so including context is cheap: it is one request either way.
- Can it detect coordinated harassment campaigns?
- No — this decides one item at a time. Campaign patterns need aggregation in your platform; typed decisions still feed that pipeline. Store the threat flag per item and analyze across accounts.
- What about images?
- Not in this call — Jev is text only. OCR or transcribe text into the state and the same decision shape works.