cawcut CawPlan Internal QA — Human Input Sample & Label
Bootstrap
cawplan skill check
Task
Fetch human-input rows via the CawPlan CLI, emit a fixture JSON aligned with production classify
context rules, then open the local label editor so a human can set expected_categories,
expected_topic, optionally expected_reason, and review_required per turn.
Workflow
1. Resolve scope (same disambiguation rules as humaninputs-test)
Member (optional): cawplan session members → exact --member git key. If ambiguous, ask
the user to pick — do not guess.
Product (optional): cawplan products list --search "<name>" → unique_id. If ambiguous,
ask the user to pick.
User within product (optional): needs PRM user_id for
product-human-input-logs --user-id, resolved via cawplan users query.
Date range (required): --from / --to, or --date for a single day. Default to last 2
days if the user gives no range.
Count: --limit N (default 20).
Session shape (pick one):
--one-session— auto-pick the session with the most rows in range (best for multi-turn prev chains, like the 20-turn Slack fixture).--session-id <id>— keep one session explicitly.- Neither — first N rows after sort (prev still computed per
session_id).
2. Build the fixture JSON
From this skill directory:
SKILL_DIR="/media/spx/work/github/flow-cawplan-skill/skills/cawplan-internal-qa-coding-humaninputs-sample"
node "$SKILL_DIR/scripts/build_sample.mjs" \
--from <YYYY-MM-DD> --to <YYYY-MM-DD> \
[--member "<exact_member>"] \
[--product "<product name>" | --product-id <id>] \
[--user-id <prm_user_id>] \
[--session-id <session_id> | --one-session] \
--limit <N> \
--output "<path>/human-input-sample.json" \
--copy-for-editor
Output shape (matches eval fixtures under /media/spx/work/2026/0830/):
| Field | Meaning |
|---|---|
items[].content |
Raw human input |
items[].assistant_message |
Raw paired assistant reply (full text from API) |
items[].prev_assistant_message |
Processed last paragraph of the previous turn's assistant in the same session (production prev slot). Resolved from all rows fetched in the date range, not only the selected N — so a recent slice can still get correct prev even when earlier turns of that session are not in the output |
items[].expected_categories |
Empty [] until labeled |
items[].expected_topic |
Empty until labeled |
items[].expected_reason |
Optional free-text rationale for reviewers (empty until filled) |
items[].review_required |
false by default; set true when category/topic labels need review |
items[].cloud_category / cloud_topic |
Optional hints from cloud enrichment |
Preprocessing mirrors uid.core-product ClassifyPrevAssistantTail /
ClassifyAssistantSnippet (see scripts/classify_context.mjs).
3. Open the label editor
After --copy-for-editor, start a static server and open the page:
cd "$SKILL_DIR/assets"
python3 -m http.server 8765
Browser: http://localhost:8765/label-editor.html
- Load latest sample — reads
assets/samples/latest.json(written by--copy-for-editor). - Load JSON — pick any fixture file.
- Download labeled JSON — exports
*-labeled.jsonwithexpected_categories/expected_topicfilled in;expected_reason/review_requiredincluded when set. - Draft auto-saves to browser
localStorage.
Tell the user the output path, editor URL, and that they should export the labeled JSON when
done (save under their eval folder, e.g. /media/spx/work/2026/0830/).
Output
Report to the user:
- Resolved filters (member / product / dates / limit / session).
- Path to the generated JSON and
assets/samples/latest.json. - Label editor URL (
http://localhost:8765/label-editor.html). - Reminder: export labeled JSON from the page when annotation is complete.
References
scripts/build_sample.mjs— fetch + preprocessscripts/classify_context.mjs— production-aligned paragraph rulesassets/label-editor.html— web UIreferences/CATEGORY_TAXONOMY.md— category label definitionsreferences/TOPIC_TAXONOMY.md— topic label definitions