Classify Intent
Canonical intent vocabulary
The project validator (scripts/verify_output.py) enforces exactly five intent values. Anything else fails CI.
| Canonical |
What it covers |
Common synonyms (map these to the canonical) |
question |
how-to / what-is / docs lookup |
support, help, inquiry, general |
bug |
broken behavior, error, unexpected output |
technical, issue, incident, defect, problem |
feature |
new capability requested |
feature_request, enhancement, idea, wish |
billing |
invoices, refunds, plan changes, pricing |
payment, subscription, invoice, pricing |
other |
anything else |
account, unknown, misc, legal |
If your task spec uses domain names like technical or feature_request, map them to the canonical set before returning. Keep the mapping table in your code so the original term is recoverable for analytics — but the value that leaves your function must be one of the five canonical names.
Steps
- Read the message.
- If it asks a how-to or what-is question →
question.
- If it describes broken behavior, an error, or unexpected output →
bug.
- If it asks for a new capability →
feature.
- If it's about pricing, invoices, refunds, or plan changes →
billing.
- Otherwise →
other.
- Return
confidence honestly.
Confidence threshold — pick deliberately
The default low-confidence threshold is 0.5 (escalate below this). This is a tuning knob, not a constant.
| Lower the threshold (0.3–0.4) when… |
Raise the threshold (0.6–0.7) when… |
| Wrong answers are expensive (legal, financial, healthcare) |
KB coverage is broad and answers are mostly accurate |
| Escalations are cheap (you have 24/7 staffing) |
Human agents are scarce or expensive |
| Brand tolerance for "let me get someone" is high |
Users complain about being bounced to humans for trivial questions |
| Early days — you're still learning the failure modes |
Mature pipeline with proven recall |
Document your chosen threshold in code with the reason. Future-you will not remember why 0.45 won over 0.5.
Failure modes
- Treating "this is slow" as
question (it's a bug).
- Treating refund requests as
other to dodge billing escalation rules. Always check trigger words (lawyer, cancel, refund) before the classifier — they outrank intent.
- Stacking multiple intents into one ticket — split them.
- Hard-coding the threshold without documenting the trade-off (the next maintainer can't tune it safely).
- Shipping the task-domain vocabulary (
technical, feature_request) directly without mapping — breaks harnessforge verify.
1---2name: classify-intent3description: Classify an inbound support message into question / bug / feature / billing / other.4---56# Classify Intent78## Canonical intent vocabulary910The project validator (`scripts/verify_output.py`) enforces **exactly five** intent values. Anything else fails CI.1112| Canonical | What it covers | Common synonyms (map these to the canonical) |13|---|---|---|14| `question` | how-to / what-is / docs lookup | `support`, `help`, `inquiry`, `general` |15| `bug` | broken behavior, error, unexpected output | `technical`, `issue`, `incident`, `defect`, `problem` |16| `feature` | new capability requested | `feature_request`, `enhancement`, `idea`, `wish` |17| `billing` | invoices, refunds, plan changes, pricing | `payment`, `subscription`, `invoice`, `pricing` |18| `other` | anything else | `account`, `unknown`, `misc`, `legal` |1920**If your task spec uses domain names like `technical` or `feature_request`, map them to the canonical set before returning.** Keep the mapping table in your code so the original term is recoverable for analytics — but the value that leaves your function must be one of the five canonical names.2122## Steps23241. Read the message.252. If it asks a how-to or what-is question → `question`.263. If it describes broken behavior, an error, or unexpected output → `bug`.274. If it asks for a new capability → `feature`.285. If it's about pricing, invoices, refunds, or plan changes → `billing`.296. Otherwise → `other`.307. Return `confidence` honestly.3132## Confidence threshold — pick deliberately3334The default low-confidence threshold is **0.5** (escalate below this). **This is a tuning knob, not a constant.**3536| Lower the threshold (0.3–0.4) when… | Raise the threshold (0.6–0.7) when… |37|---|---|38| Wrong answers are expensive (legal, financial, healthcare) | KB coverage is broad and answers are mostly accurate |39| Escalations are cheap (you have 24/7 staffing) | Human agents are scarce or expensive |40| Brand tolerance for "let me get someone" is high | Users complain about being bounced to humans for trivial questions |41| Early days — you're still learning the failure modes | Mature pipeline with proven recall |4243**Document your chosen threshold in code with the reason.** Future-you will not remember why 0.45 won over 0.5.4445## Failure modes4647- Treating "this is slow" as `question` (it's a `bug`).48- Treating refund requests as `other` to dodge billing escalation rules. *Always* check trigger words (`lawyer`, `cancel`, `refund`) before the classifier — they outrank intent.49- Stacking multiple intents into one ticket — split them.50- Hard-coding the threshold without documenting the trade-off (the next maintainer can't tune it safely).51- Shipping the task-domain vocabulary (`technical`, `feature_request`) directly without mapping — breaks `harnessforge verify`.
jcaiagent7143-ui