How Am I Driving

How Am I Driving

A graded report card on the user's own Claude Code prompting — "how good were my prompts, measured?" Credibility comes from concrete measures and quoted evidence, not vibes. Voice: an encouraging coach with light driving-themed garnish; honest underlying numbers.

Scope: mechanical (deterministic) measures, the semantic / qualitative judged layer, the Markdown Full Report, the HTML render modes — the shareable Scorecard and an optional HTML Full Report — and the distribution surface: a Claude Code plugin, a curl … | bash installer, and manual clone. See CONTEXT.md and docs/adr/.

What counts as a prompt

A genuine, human-typed prompt — a type:"user" transcript entry that is plain typed text. Tool results, slash-command wrappers, injected skill instructions (isMeta), task-notification events, /compact continuation summaries, and interrupt markers are not prompts and are filtered out by analyze(). Rewinding or resuming a session re-logs earlier turns with the same uuid; analyze() de-duplicates those replays by uuid so a long, resumed session isn't counted several times over (genuinely repeated prompts like a second "Yes" carry distinct uuids and are kept). The report always states the actual prompt count analyzed, so a thin sample reads as a soft signal.

Measured vs judged

Two kinds of measure, and the report labels every number as one or the other:

• Measured — deterministic counts from analyze() (prompt length, prompts/session, context front-loading). Same corpus in → same numbers out.
• Judged — you, Claude, read the prompts and score them against the fixed rubric below. These are the semantic counts (rework, clarification, steering) and the five qualitative grades. A re-run can shift a judged score by a point; that's expected, and the labelling makes it read as such, not as a bug.

Never present a judged number as if it were measured, and never recompute the measured numbers yourself — the script is the single source of truth for those.

Flow

Mechanical analysis is deterministic; the judged layer is you scoring the extracted prompts in the middle. So a deep read is three steps: compute → judge → render. Work in a scratch dir (e.g. the session scratchpad).

Paths are relative to this skill's directory. The python3 scripts/… commands below assume your working directory is this skill's own folder (the base directory shown when the skill loads — e.g. ~/.claude/skills/how-am-i-driving/), not the user's project. cd into it first, or prefix the script path with that directory, so scripts/report.py resolves wherever the skill is installed.

1. Resolve depth and corpus

• Depth. --deep is the default and the right choice for ~10 sessions: full semantic judgment with evidence. Use --fast only when the user asks for it or the corpus is large (say ≳40 sessions), where a light semantic pass (headline rework only, no qualitative grades) keeps cost down.
• Corpus. Default = the most recent 10 sessions across all projects. A numeric argument overrides the count (e.g. /how-am-i-driving 50).

2. Compute once — emit the analysis seam

python3 scripts/report.py [count] --depth deep --emit-analysis > "$SCRATCH/analysis.json"

This is the single analysis pass. analysis.json holds the resolved corpus, the mechanical metrics, and prompts — the extracted genuine-prompt sequence you'll judge. Do not re-resolve or re-analyze later; phase 3 renders from this exact file ("compute once, render twice").

If metrics.prompt_count is 0, skip judging and just render — the report handles the empty corpus gracefully.

3. Judge the prompts → write judgment.json

Read analysis.json. Each prompts[i] has text, session_id, and index_in_session. Judge the whole sequence in order, per session, so you can see a redirect follow the attempt it corrects.

Score these and write a judgment.json matching the schema below.

Semantic counts (judged)

• Rework rate — the headline. Count prompts where the user re-prompts to correct or redirect Claude after a missed or partial attempt. Crucially, count implicit and polite rework, not just literal "no, I meant…":
  • "actually, can you also…", "hmm, that's not quite it", "let's try a different approach", "can you redo the X part", re-explaining the same ask in new words, or re-attaching context that should have landed the first time.
  • A fresh next step ("now add tests") is not rework. A genuine new task is not rework. Approvals ("looks good, ship it") are not rework.
  • Leave rate null — the renderer derives it from the prompt count.
• Clarification rate. Count prompts that exist because Claude asked a question back (the user is answering a clarifying question) — a signal the previous prompt was under-specified.
• Steering interventions. Count mid-task interrupts / course-corrections (e.g. "stop", "wait — not like that", redirecting while work is in flight).

Qualitative grades (judged, 1–5, with evidence)

Score each measure 1–5 against the fixed rubric, and back each score with a verbatim quoted prompt from the user's own transcripts (evidence_quote, plus its evidence_session_id). Pick a quote that genuinely typifies the score.

In --fast mode, skip the qualitative grades entirely and judge only the headline rework (optionally clarification). Leave qualitative an empty list and set depth to "fast".

judgment.json schema

{
  "depth": "deep",
  "rework":        { "count": 0, "rate": null, "note": "one-line summary (optional)" },
  "clarification": { "count": 0, "rate": null },
  "steering":      { "count": 0 },
  "qualitative": [
    {
      "measure": "specificity",
      "score": 4,
      "evidence_quote": "<verbatim prompt text from analysis.json>",
      "evidence_session_id": "<that prompt's session_id>",
      "rationale": "one line on why this score"
    }
    // ...one entry per measure: specificity, context_quality,
    //    decomposition, goal_clarity, correction_style
  ]
}

Quote the user's prompts verbatim — evidence_quote must be real text from analysis.json, never paraphrased or invented. (Evidence appears only in the private Full Report, never in the future shareable Scorecard.)

4. Render the Full Report

python3 scripts/report.py --render "$SCRATCH/analysis.json" --judgment "$SCRATCH/judgment.json"

Present the Markdown it prints unedited. It already carries the coach voice, the honest sample-size line, every measure's inline help text, the measured / judged labelling, and your quoted evidence. Do not re-summarize or recompute.

For a quick mechanical-only read (no judging), python3 scripts/report.py [count] still renders the measured section on its own.

5. Render modes — Scorecard and HTML (optional, same analysis pass)

Both HTML modes render from the same analysis.json + judgment.json (ADR-0003) — never re-resolve or re-judge to produce them.

• Scorecard (--scorecard) — a self-contained, shareable HTML card at a fixed 1200×630 social ratio: scores and grades only, no quoted prompts (the privacy guarantee). Render it when the user asks for something shareable:

  python3 scripts/report.py --render "$SCRATCH/analysis.json" \
      --judgment "$SCRATCH/judgment.json" --scorecard > "$SCRATCH/scorecard.html"
  

  Write it to a file and tell the user the path; it's designed to be opened and screenshotted. Because it carries no quotes, it's safe to share publicly.

• HTML Full Report (--html) — the same private Full Report as Markdown, including evidence quotes, as a styled standalone HTML page. Use when the user wants the detailed report in HTML rather than Markdown.

Both flags also work on the one-shot mechanical path (no --judgment).

Architecture (for maintainers)

Single deterministic seam, AnalysisResult, plus its judged sibling Judgment (ADR-0002, ADR-0005):

• scripts/analyze.py — analyze(corpus) -> AnalysisResult. Corpus resolution, genuine-prompt filtering, mechanical metrics. Deterministic.
• scripts/models.py — the AnalysisResult (measured) and Judgment (judged) data contracts. Judgment is plain data Claude emits as JSON.
• scripts/render_shared.py — content and formatters every renderer shares: the per-measure help text, the ordered qualitative-measure list, and the number/percentage/duration/grade-bar helpers. Neutral home so no renderer reaches into another's internals.
• scripts/render_full_report.py — render_full_report(result, judgment=None) -> str. Pure function, Markdown out, measured + judged sections.
• scripts/render_html.py — render_scorecard() and render_html_report(). Pure functions, self-contained HTML out, over the same AnalysisResult + Judgment (ADR-0003). The Scorecard carries scores only — no quoted prompts.
• scripts/report.py — CLI glue: --emit-analysis (phase 1), --render + --judgment (phase 3), --depth deep|fast, --scorecard / --html.
• tests/ — fixture-driven pytest. The deterministic analysis and the renderers are tested; the judgment itself is deliberately not unit-tested (it's non-deterministic by design — validated by example/eval). Run with python3 -m pytest.

Mechanical measures are deterministic Python (never regex-as-primary, never LLM — ADR-0002). The semantic/qualitative measures are Claude's judgment, anchored by the rubric above and labelled judged to keep the distinction honest.