aio-jev-hint — hint-injection hooks with an LLM classifier
The pattern
Two pieces, one job: surface the right hint at the right moment without the user asking. The catalog is a generic hint list — nothing ties it to any one kind of target. An entry can point at a skill, an MCP server, a project rule, a dashboard, or a plain reminder; the hook doesn't know or care which.
.jev-hint — a JSON array of {name, useWhen, context} entries. Global at ~/.claude/.jev-hint, optionally overridden per-project by a .jev-hint at the repo root (nearest-ancestor lookup, same resolution rule as .git). Project entries win name collisions.
- A
UserPromptSubmit hook — on every prompt, batches all catalog entries into one call to the OpenRouter Decisions API (POST https://openrouter.ai/api/alpha/decisions, model typesafe/jev-1.14) asking a yes/no (noul) "does this apply?" question per entry, then injects the context of every entry that clears a confidence threshold as additionalContext.
Full runnable template: ${CLAUDE_PLUGIN_ROOT}/skills/aio-jev-hint/reference/hook-template.mjs. Example catalog: ${CLAUDE_PLUGIN_ROOT}/skills/aio-jev-hint/reference/example.jev-hint.json.
Writing the .jev-hint catalog
| Field |
Required |
Purpose |
name |
yes |
Stable unique key. Used for dedup (project overrides global) and shows up in logs — pick something that identifies the target, whatever kind of target it is: a skill's slash command, an MCP server name, a rule label. |
useWhen |
yes |
This is what the LLM actually scores against the prompt. Get this wrong and every other field is moot. |
context |
yes |
Injected verbatim when matched — a slash command, an MCP server blurb, a dashboard URL, a project rule, a plain reminder sentence: whatever the target needs. Multiple matches get joined into one line, so keep it short. |
useWhen: before/after from real tuning
The first version of this catalog copied each entry's full SKILL.md description verbatim — second-person, keyword-dense, built for a different discovery mechanism:
"useWhen": "Use whenever you change anything that has a previous version —
code (functions, branches, dead blocks), database state (rows, schema
columns, indexes), configs (env vars, feature flags, secrets),
infrastructure... [continues for 900+ characters, second person, comma-spliced
trigger list]"
That failed in practice — worse match quality, and unreadable at a glance. Rewritten to third person, one sentence of intent, a couple of representative trigger phrases:
"useWhen": "User is replacing, removing, or cleaning up something with a
prior version — code, config, docs, infra, or the assistant's own response —
and the result should read as if born in its current form, with no leftover
trace of the old version."
Same rules apply whether the entry points at a skill, an MCP server, or a one-line project reminder — only what counts as "intent" changes, not the shape of the sentence. A reminder entry, for example:
"useWhen": "User is debugging a slow or timing-out request path and hasn't
mentioned checking metrics yet."
Rules that fell out of the rewrite:
- Third person ("User wants X" / "User is doing Y"), not second person ("Use when you...") — this is a classification prompt, not documentation.
- One sentence naming the underlying intent. Not a keyword dump — the LLM classifies meaning, it doesn't need every synonym enumerated.
- A couple of concrete trigger phrases (
— e.g. "deploy to cloudflare pages", "ship to cf pages") earn their place; a long comma-spliced list doesn't.
- Short and complete beat long and exhaustive. If
useWhen needs a paragraph to say what the entry is for, the entry is probably two entries.
Writing the hook
Checklist, in the order a request flows through hook-template.mjs:
- Event type:
UserPromptSubmit. Read user_input/prompt/message + cwd from the JSON stdin payload.
- Fail open, always. Any parse error, missing
OPENROUTER_API_KEY, empty catalog, non-2xx response, or timeout → return with no output, exit 0. This hook must never block or visibly break the user's prompt.
- Filter synthetic turns. Background-task completions arrive as a "user" turn whose entire text is a
<task-notification> block — not something the human typed. Classifying it produced measured noise in production (13 of 38 real hint emissions in one day fired on this pattern instead of an actual request). Skip any prompt starting with it.
- Resolve the catalog before calling out. Walk up from
cwd for the nearest project .jev-hint, merge with the global one, project wins name collisions. Skip the LLM call entirely if the merged catalog is empty.
- Batch, don't loop. One Decisions request with one
noul question per catalog entry — not N sequential calls. Measured with a 16-entry catalog: p50 0.56s, max 1.4s, ~$0.00006 per prompt. Put an AbortSignal.timeout on the request so a slow answer degrades to no hint, not a stalled prompt.
- Threshold and sort. Each answer is
{ type: "noul", noul: <probability of yes> }. Keep entries with noul > threshold (0.8 in production), sort descending. No matches → emit nothing.
- Output shape.
stdout gets exactly one JSON object: { hookSpecificOutput: { hookEventName: "UserPromptSubmit", additionalContext: "..." } }. Prefix the joined list with a plain label (Context Hint: ...) so it's recognizable in the transcript without implying the hint is always a skill.
- Log for yourself, not the user. Append timing (
stdin parse, catalog build, decision call), the serving model (the response's dated snapshot, e.g. typesafe/jev-1.14-20261014), and every entry's score to a local debug log on every run. This is the only way to tune the threshold or diagnose "why didn't it fire" later — there's no other visibility into a hook that fails open.
- Never hardcode secrets. Read the API key from an env var first; if you need a fallback file, keep the path out of anything that gets published (this catalog is user-local, but the hook code may end up in a shared repo).
Wiring it up
Register the hook once in settings.json:
{
"hooks": {
"UserPromptSubmit": [
{ "hooks": [{ "type": "command", "command": "bun /path/to/your/hook.mjs" }] }
]
}
}
Known unexplored lever
A noul question accepts an optional criteria: { true: "...", false: "..." } alongside instructions. Isolated testing suggested explicit criteria changes match behavior versus plain instructions alone, both standalone and batched — but the production hook above ships with plain instructions only; this was never folded in and re-validated at catalog scale. If a catalog's match quality is off and useWhen wording alone doesn't fix it, this is the next thing to try — not a confirmed win.
Common mistakes
| Mistake |
Fix |
useWhen = full SKILL.md description pasted in |
Rewrite as one third-person sentence of intent + 1-2 trigger phrases (see above) |
Catalog / output labels assume every entry is a skill ([skill-hint]:, "which skill applies?") |
Use target-agnostic wording — a generic label like Context Hint: and "does this apply?" — the catalog also carries MCP servers, project rules, plain reminders |
| Hook throws or writes to stderr on error |
Wrap every step in try/catch that returns silently; a hint hook that breaks the prompt is worse than no hint |
Classifying <task-notification> turns |
Check state.trimStart().startsWith('<task-notification>') before calling out |
| One Decisions request per catalog entry |
Batch all entries into one request — latency stays near one call instead of growing with the catalog |
Using typesafe/jev-router over chat/completions and reading booleans from its JSON |
Use the Decisions API. The router sends each request to a different underlying model; some return no logprobs, so there is no calibrated probability to threshold |
| Switching model version without re-checking the threshold |
Replay a sample of logged prompts against the new version and compare scores per entry before switching; the threshold belongs to one version |
context field repeats the whole useWhen text |
Keep context to what should actually appear inline — a slash command or one short reminder line |
| Threshold left unset or at 0.5 |
Start at 0.8; lower only after checking the debug log shows real misses, not noise |
1---2name: aio-jev-hint3description: |4 Author a UserPromptSubmit hook + `.jev-hint` catalog file — a hook that5 classifies every user prompt against a JSON catalog of6 {name, useWhen, context} entries via one batched yes/no decision call7 (OpenRouter Decisions API, model typesafe/jev-1.14), then8 injects the matched entries' `context` as additionalContext when9 confidence clears a threshold. The catalog is generic — entries can point10 at a relevant skill, an MCP server, a project rule, a dashboard to check,11 a standing reminder, or anything else worth surfacing on-topic; nothing12 about the pattern is skill-specific. Use when building a hook that13 surfaces contextual hints based on what the user's prompt is actually14 about, or when writing/tuning a `.jev-hint` catalog file: its fields, its15 `useWhen` wording, its confidence threshold. "context hint hook", "prompt16 classifier hook", ".jev-hint file", "UserPromptSubmit LLM classification",17 "surface a reminder on-topic", "viết hook gợi ý ngữ cảnh", "file .jev-hint".18when_to_use: jev-hint, .jev-hint, context hint hook, UserPromptSubmit hook, prompt classifier hook, OpenRouter Decisions API, typesafe/jev-1.14, noul question batch classification, hint catalog file, useWhen field, additionalContext injection, fail-open hook, confidence threshold context routing, viết hook gợi ý ngữ cảnh19argument-hint: "[what you're building: 'hook' or 'catalog entry']"20effort: medium21metadata:22 source: https://skillmd.com/skills/aiocean/aio-jev-hint23---2425# aio-jev-hint — hint-injection hooks with an LLM classifier2627## The pattern2829Two pieces, one job: surface the right hint at the right moment without the user asking. The catalog is a generic hint list — nothing ties it to any one kind of target. An entry can point at a skill, an MCP server, a project rule, a dashboard, or a plain reminder; the hook doesn't know or care which.30311. **`.jev-hint`** — a JSON array of `{name, useWhen, context}` entries. Global at `~/.claude/.jev-hint`, optionally overridden per-project by a `.jev-hint` at the repo root (nearest-ancestor lookup, same resolution rule as `.git`). Project entries win name collisions.322. **A `UserPromptSubmit` hook** — on every prompt, batches all catalog entries into **one** call to the OpenRouter Decisions API (`POST https://openrouter.ai/api/alpha/decisions`, model `typesafe/jev-1.14`) asking a yes/no (`noul`) "does this apply?" question per entry, then injects the `context` of every entry that clears a confidence threshold as `additionalContext`.3334Full runnable template: `${CLAUDE_PLUGIN_ROOT}/skills/aio-jev-hint/reference/hook-template.mjs`. Example catalog: `${CLAUDE_PLUGIN_ROOT}/skills/aio-jev-hint/reference/example.jev-hint.json`.3536## Writing the `.jev-hint` catalog3738| Field | Required | Purpose |39|---|---|---|40| `name` | yes | Stable unique key. Used for dedup (project overrides global) and shows up in logs — pick something that identifies the target, whatever kind of target it is: a skill's slash command, an MCP server name, a rule label. |41| `useWhen` | yes | **This is what the LLM actually scores against the prompt.** Get this wrong and every other field is moot. |42| `context` | yes | Injected **verbatim** when matched — a slash command, an MCP server blurb, a dashboard URL, a project rule, a plain reminder sentence: whatever the target needs. Multiple matches get joined into one line, so keep it short. |4344### `useWhen`: before/after from real tuning4546The first version of this catalog copied each entry's full `SKILL.md` description verbatim — second-person, keyword-dense, built for a different discovery mechanism:4748```49"useWhen": "Use whenever you change anything that has a previous version —50code (functions, branches, dead blocks), database state (rows, schema51columns, indexes), configs (env vars, feature flags, secrets),52infrastructure... [continues for 900+ characters, second person, comma-spliced53trigger list]"54```5556That failed in practice — worse match quality, and unreadable at a glance. Rewritten to third person, one sentence of *intent*, a couple of representative trigger phrases:5758```59"useWhen": "User is replacing, removing, or cleaning up something with a60prior version — code, config, docs, infra, or the assistant's own response —61and the result should read as if born in its current form, with no leftover62trace of the old version."63```6465Same rules apply whether the entry points at a skill, an MCP server, or a one-line project reminder — only what counts as "intent" changes, not the shape of the sentence. A reminder entry, for example:6667```68"useWhen": "User is debugging a slow or timing-out request path and hasn't69mentioned checking metrics yet."70```7172**Rules that fell out of the rewrite:**73- Third person ("User wants X" / "User is doing Y"), not second person ("Use when you...") — this is a classification prompt, not documentation.74- One sentence naming the underlying *intent*. Not a keyword dump — the LLM classifies meaning, it doesn't need every synonym enumerated.75- A couple of concrete trigger phrases (`— e.g. "deploy to cloudflare pages", "ship to cf pages"`) earn their place; a long comma-spliced list doesn't.76- Short and complete beat long and exhaustive. If `useWhen` needs a paragraph to say what the entry is for, the entry is probably two entries.7778## Writing the hook7980Checklist, in the order a request flows through `hook-template.mjs`:8182- **Event type: `UserPromptSubmit`.** Read `user_input`/`prompt`/`message` + `cwd` from the JSON stdin payload.83- **Fail open, always.** Any parse error, missing `OPENROUTER_API_KEY`, empty catalog, non-2xx response, or timeout → return with no output, exit 0. This hook must never block or visibly break the user's prompt.84- **Filter synthetic turns.** Background-task completions arrive as a "user" turn whose entire text is a `<task-notification>` block — not something the human typed. Classifying it produced measured noise in production (13 of 38 real hint emissions in one day fired on this pattern instead of an actual request). Skip any prompt starting with it.85- **Resolve the catalog before calling out.** Walk up from `cwd` for the nearest project `.jev-hint`, merge with the global one, project wins name collisions. Skip the LLM call entirely if the merged catalog is empty.86- **Batch, don't loop.** One Decisions request with one `noul` question per catalog entry — not N sequential calls. Measured with a 16-entry catalog: p50 0.56s, max 1.4s, ~$0.00006 per prompt. Put an `AbortSignal.timeout` on the request so a slow answer degrades to no hint, not a stalled prompt.87- **Threshold and sort.** Each answer is `{ type: "noul", noul: <probability of yes> }`. Keep entries with `noul > threshold` (0.8 in production), sort descending. No matches → emit nothing.88- **Output shape.** `stdout` gets exactly one JSON object: `{ hookSpecificOutput: { hookEventName: "UserPromptSubmit", additionalContext: "..." } }`. Prefix the joined list with a plain label (`Context Hint: ...`) so it's recognizable in the transcript without implying the hint is always a skill.89- **Log for yourself, not the user.** Append timing (`stdin parse`, `catalog build`, `decision call`), the serving model (the response's dated snapshot, e.g. `typesafe/jev-1.14-20261014`), and every entry's score to a local debug log on every run. This is the only way to tune the threshold or diagnose "why didn't it fire" later — there's no other visibility into a hook that fails open.90- **Never hardcode secrets.** Read the API key from an env var first; if you need a fallback file, keep the path out of anything that gets published (this catalog is user-local, but the hook *code* may end up in a shared repo).9192## Wiring it up9394Register the hook once in `settings.json`:9596```json97{98 "hooks": {99 "UserPromptSubmit": [100 { "hooks": [{ "type": "command", "command": "bun /path/to/your/hook.mjs" }] }101 ]102 }103}104```105106## Known unexplored lever107108A `noul` question accepts an optional `criteria: { true: "...", false: "..." }` alongside `instructions`. Isolated testing suggested explicit criteria changes match behavior versus plain `instructions` alone, both standalone and batched — but the production hook above ships with plain `instructions` only; this was never folded in and re-validated at catalog scale. If a catalog's match quality is off and `useWhen` wording alone doesn't fix it, this is the next thing to try — not a confirmed win.109110## Common mistakes111112| Mistake | Fix |113|---|---|114| `useWhen` = full `SKILL.md` description pasted in | Rewrite as one third-person sentence of intent + 1-2 trigger phrases (see above) |115| Catalog / output labels assume every entry is a skill (`[skill-hint]:`, "which skill applies?") | Use target-agnostic wording — a generic label like `Context Hint:` and "does this apply?" — the catalog also carries MCP servers, project rules, plain reminders |116| Hook throws or writes to stderr on error | Wrap every step in try/catch that returns silently; a hint hook that breaks the prompt is worse than no hint |117| Classifying `<task-notification>` turns | Check `state.trimStart().startsWith('<task-notification>')` before calling out |118| One Decisions request per catalog entry | Batch all entries into one request — latency stays near one call instead of growing with the catalog |119| Using `typesafe/jev-router` over `chat/completions` and reading booleans from its JSON | Use the Decisions API. The router sends each request to a different underlying model; some return no logprobs, so there is no calibrated probability to threshold |120| Switching model version without re-checking the threshold | Replay a sample of logged prompts against the new version and compare scores per entry before switching; the threshold belongs to one version |121| `context` field repeats the whole `useWhen` text | Keep `context` to what should actually appear inline — a slash command or one short reminder line |122| Threshold left unset or at 0.5 | Start at 0.8; lower only after checking the debug log shows real misses, not noise |