Mk Observe

mk-observe — agent harness observability

Reads the native session stores each harness already writes, normalises them into one run model, scores them, keeps the processed runs in its own store (~/.mk-observe/store/) and shows the result in the terminal or a local web UI. Plain Node, no dependencies, no LLM call anywhere, nothing sent off the machine, harness stores are opened read-only.

node .claude/skills/observe/scripts/observe.cjs daemon install   # machine-level: copies the skill to ~/.mk-observe/app, always-on UI for ALL projects at http://127.0.0.1:3467
node .claude/skills/observe/scripts/observe.cjs daemon status    # installed? running? pid, app copy version, log path
node .claude/skills/observe/scripts/observe.cjs store status     # what the persistent store holds (runs, archived, size)
node .claude/skills/observe/scripts/observe.cjs serve            # one-off UI for the current project
node .claude/skills/observe/scripts/observe.cjs stats --days 14  # terminal twin of the UI
node .claude/skills/observe/scripts/observe.cjs runs             # one line per run
node .claude/skills/observe/scripts/observe.cjs run <id-prefix>  # event timeline of one run
node .claude/skills/observe/scripts/observe.cjs export > obs.json

Options: --cwd <project> (default: current dir), --all (every project any harness knows), --days N (default 30), --harness claude-code,opencode,kiro, --json, --port, --interval <s>, --no-open; daemon install --in-place (run from this checkout instead of the copy), --skill (symlink ~/.claude/skills/observe so /mk-observe exists in every project).

How it runs

Nothing is triggered by the agent's own session: no hook, no prompt, no LLM. The daemon only reads files, parses a transcript again only when its size or mtime changed (a 450 MB session parses in ~1.5 s; an idle refresh is ~0.1 s), keeps only file metadata in RAM for unchanged sessions (the runs themselves live in the store), and runs at nice 10, so it never competes with the harness you are working in.

Machine-level, not per project

The rest of the kit is installed per project (hooks, rules and agents are project-scoped). Observe is different: it reads every project's store, so it lives once per machine — the daemon copy under ~/.mk-observe/app, the store under ~/.mk-observe/store. A project only needs the skill folder if you want /mk-observe inside that project; daemon install --skill links it into ~/.claude/skills instead, once for all projects.

Persistent store

Harnesses clean up after themselves (Claude Code deletes transcripts after cleanupPeriodDays, 30 by default). Every pass — CLI, serve or the daemon — writes what it processed into the store:

Analysis is always built from the store; live collection only adds or rewrites runs whose source changed (size/mtime for transcripts, time_updated/updated_at for the sqlite stores). A run whose source vanished is flagged archived and keeps counting. store status shows size and age; store prune --days N deletes runs older than N days (nothing is pruned automatically). MK_OBSERVE_STORE=/path relocates it. A CLI run and the daemon can share the store: the index is merged on write, never clobbered.

Where the data comes from

Observe never turns tokens into money. It stores the raw counts per model, per run (and whatever number the harness itself reports — opencode's USD, Kiro's credits — as reported, clearly labelled). Cost is the reader's job: Σ over models (input × p_in + output × p_out + cacheRead × p_cache_read + cacheWrite × p_cache_write) with the price table valid for their contract at the time. This keeps the numbers true across harnesses, providers and price changes; a hard-coded price list would be wrong within a month.

Missing data is shown as n/a or a neutral score with the reason in the formula. It is never faked as 0.

How scoring works

Four dimensions, 0–100, equal weight, composite = plain mean. Every number carries its formula and the run ids behind it.

Grades are percentiles inside the fleet you actually have (harnesses, agents, models or runs): A ≥ p80, B ≥ p60, C ≥ p40, D ≥ p20. Fewer than 5 members → static bands (A ≥ 90, B ≥ 80, C ≥ 70, D ≥ 60) and the grade is marked uncalibrated (*). Fewer than 5 runs → low confidence.

Behaviour fingerprint per run: prompt length band (short ≤ 20 words, good ≤ 250, long), task size from input tokens, retry loops (3 consecutive errors on the same tool), subagents spawned, skills used (/mk-x in the prompt, SKILL.md reads), reports written under plans/, checklist completion.

Kit health, adoption, workflow, insights

• Kit health — hook runs, blocks, crashes and durations per hook (from hook-log.jsonl; entries carry sid + harness since kit v2.7.0 and are otherwise matched to runs by time window). Hook errors reported by the harness itself are attributed to the hook named in the error; a Cannot find module …/.claude/hooks/x.cjs is flagged as unresolved path — the session was started from a subdirectory, so nothing was protecting it.
• Adoption — a run "uses the kit" when it typed a kit slash command (/mk:… or any command matching a skill in the project's .claude/skills/), read a SKILL.md, called the Skill tool, or spawned a subagent. Observe reads the installed catalogue (.claude/metadata.json name + version, skills/*/SKILL.md, agents/*.md, hooks in settings.json) and reports: kit share of runs, commands/skills/agents/hooks by frequency, kit vs harness agents, never-used skills/agents/hooks, adoption per project with kit version, per week.
• Workflow & outcomes — prompt bands, steering per run, median duration and tool calls, marathons (> 4 h or > 250 tool calls), compactions, API errors and refusal fallbacks, retry loops, abandoned runs, model switches, subagent use, weekday × hour heatmap, hot files, tool errors by tool; outcomes: git commit / git push / PR links, test commands (npm/pnpm/yarn/bun test, pytest, go test, cargo test, jest, vitest, …) and how many failed, edit-heavy runs that ran tests, files edited, reports under plans/.
• Insights — threshold rules over those numbers, each with level (bad / warn / info / good), evidence, a suggestion and the run ids: kit adoption < 30 %, projects without the kit, older kit versions (per kit name), unused skills/agents, non-kit slash commands, short prompts > 40 %, steering avg > 5, compactions in > 25 % of runs, marathons, few subagents, tests in < 30 % of edit-heavy runs, test failure rate, tool error rate > 20 % on ≥ 20 calls, hook crashes / slow hooks, retry loops ≥ 10 %, abandoned > 20 %, API errors, one model ≥ 80 % of tokens, cache hit rate < 50 %, model switches, off-hours share, weakest dimension < 60. Rate rules wait for ≥ 5 runs. Rules live in scripts/lib/metrics.cjs (buildInsights) — thresholds are plain numbers, adjust them to the team.

When the user asks

• "How is the agent doing / which harness is better here" → run stats (or serve) and explain the numbers with their formulas. Quote the coverage limits above when relevant.
• "How much did this week cost" → stats (Usage by model) or export --json; give the token table per model and the formula above, and let the user apply their own price list. Do not invent prices.
• "Is the team using the kit / what should we improve" → stats --all (Adoption, Workflow & outcomes, Insights) or the Insights tab; read the findings top-down (bad → warn → info → good) and quote the evidence and run ids.
• "Why is autonomy low" → runs, then run <id> on the worst ones; interventions are listed as permission_ask events.
• "Are the hooks working" → the Kit health section; a hook with crashes > 0 or no runs at all is the finding.
• "Is the old data still there / the session got cleaned up" → store status; archived runs are listed like any other, with an archived pill in the UI.
• Do not edit the harness stores. Only ~/.mk-observe/store is written, and only by observe itself.