# Mk Observe

> Collect, score and browse agent activity across Claude Code, opencode and Kiro CLI — four honest dimensions (acceptance, success, autonomy, reliability) with formulas and provenance, token usage per model (raw counts, no price guessing), kit adoption (which commands/skills/agents/hooks a team really uses, per project), workflow and outcome facts (prompt bands, steering, compactions, commits, tests, PRs) and rule-based insights for team leads; CLI tables and a local web UI. Use when asked how the agent/harness is performing, how many tokens a week/model/project burned, whether the team uses the kit, which hooks fire or fail, or to compare harnesses/models/agents on the same project.

- Skill: `phuc-nt/mk-observe` (Agent Skill, multi-file: 16 files)
- Install (CLI): `npx skillmds@latest add phuc-nt/mk-observe`
- Raw SKILL.md: https://api.skillmd.com/api/skills/phuc-nt/mk-observe/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: phuc-nt (https://skillmd.com/u/phuc-nt)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/phuc-nt/mk-observe

---


# mk-observe — agent harness observability

Reads the native session stores each harness already writes, normalises them into one run model, scores them, **keeps the processed runs in its own store** (`~/.mk-observe/store/`) and shows the result in the terminal or a local web UI. Plain Node, no dependencies, no LLM call anywhere, nothing sent off the machine, harness stores are opened read-only.

```bash
node .claude/skills/observe/scripts/observe.cjs daemon install   # machine-level: copies the skill to ~/.mk-observe/app, always-on UI for ALL projects at http://127.0.0.1:3467
node .claude/skills/observe/scripts/observe.cjs daemon status    # installed? running? pid, app copy version, log path
node .claude/skills/observe/scripts/observe.cjs store status     # what the persistent store holds (runs, archived, size)
node .claude/skills/observe/scripts/observe.cjs serve            # one-off UI for the current project
node .claude/skills/observe/scripts/observe.cjs stats --days 14  # terminal twin of the UI
node .claude/skills/observe/scripts/observe.cjs runs             # one line per run
node .claude/skills/observe/scripts/observe.cjs run <id-prefix>  # event timeline of one run
node .claude/skills/observe/scripts/observe.cjs export > obs.json
```

Options: `--cwd <project>` (default: current dir), `--all` (every project any harness knows), `--days N` (default 30), `--harness claude-code,opencode,kiro`, `--json`, `--port`, `--interval <s>`, `--no-open`; `daemon install --in-place` (run from this checkout instead of the copy), `--skill` (symlink `~/.claude/skills/observe` so `/mk-observe` exists in every project).

## How it runs

| Mode | Started by | What happens |
|---|---|---|
| `stats` / `runs` / `run` / `export` | the developer (or the agent, when asked) | one pass over the stores, prints, exits |
| `serve` | the developer | UI for one project; collects when a request arrives, 15 s cache; stops with the terminal |
| `daemon install` | the developer, once per machine | copies this skill to `~/.mk-observe/app/observe` (so no project checkout has to stay put) and registers a user-level service (launchd on macOS, `systemd --user` on Linux) that runs `serve --all` from there; starts at login, restarts if it dies, re-collects every 60 s in the background so the UI answers instantly. Log: `~/.mk-observe/daemon.log`. Re-run after upgrading the kit (`daemon status` says when the copy is older). `daemon uninstall` removes service + copy, keeps the store. |

Nothing is triggered by the agent's own session: no hook, no prompt, no LLM. The daemon only reads files, parses a transcript again only when its size or mtime changed (a 450 MB session parses in ~1.5 s; an idle refresh is ~0.1 s), keeps only file metadata in RAM for unchanged sessions (the runs themselves live in the store), and runs at `nice 10`, so it never competes with the harness you are working in.

## Machine-level, not per project

The rest of the kit is installed per project (hooks, rules and agents are project-scoped). Observe is different: it reads *every* project's store, so it lives once per machine — the daemon copy under `~/.mk-observe/app`, the store under `~/.mk-observe/store`. A project only needs the skill folder if you want `/mk-observe` inside that project; `daemon install --skill` links it into `~/.claude/skills` instead, once for all projects.

## Persistent store

Harnesses clean up after themselves (Claude Code deletes transcripts after `cleanupPeriodDays`, 30 by default). Every pass — CLI, `serve` or the daemon — writes what it processed into the store:

| Path | Holds |
|---|---|
| `~/.mk-observe/store/index.json` | one summary row per run (scores, counts, hook stats) — what `stats` and the UI aggregate |
| `~/.mk-observe/store/runs/<id>.json` | the full run with its event timeline — what `run <id>` and the UI detail view show |

Analysis is always built **from the store**; live collection only adds or rewrites runs whose source changed (size/mtime for transcripts, `time_updated`/`updated_at` for the sqlite stores). A run whose source vanished is flagged **archived** and keeps counting. `store status` shows size and age; `store prune --days N` deletes runs older than N days (nothing is pruned automatically). `MK_OBSERVE_STORE=/path` relocates it. A CLI run and the daemon can share the store: the index is merged on write, never clobbered.

## Where the data comes from

| Harness | Store | Tokens | Harness-reported | Interventions | Kit hooks | Extras |
|---|---|---|---|---|---|---|
| Claude Code | `~/.claude/projects/<cwd-slug>/*.jsonl` | input / output / cache read / cache write **per model**, per API call | none | rejected tool calls, interrupts | `.claude/hooks/.logs/hook-log.jsonl` (+ hook errors the harness itself reports) | slash commands, compactions, API errors, refusal fallbacks, PR links, effort, thinking |
| opencode | `~/.local/share/opencode/opencode.db` | input / output / reasoning / cache per model | USD computed by opencode | interrupted tool calls only | none (kit ships no hooks for opencode) | lines added / removed |
| Kiro CLI | `~/Library/Application Support/kiro-cli/data.sqlite3` | schema has per-turn token fields but kiro-cli ≤ 2.21 leaves them null; read when present | credits | cancelled tool uses | `hook-log.jsonl` via the kit adapter | context %, time to first chunk |

**Observe never turns tokens into money.** It stores the raw counts per model, per run (and whatever number the harness itself reports — opencode's USD, Kiro's credits — as `reported`, clearly labelled). Cost is the reader's job: `Σ over models (input × p_in + output × p_out + cacheRead × p_cache_read + cacheWrite × p_cache_write)` with the price table valid for *their* contract at the time. This keeps the numbers true across harnesses, providers and price changes; a hard-coded price list would be wrong within a month.

Missing data is shown as `n/a` or a neutral score with the reason in the formula. It is never faked as 0.

## How scoring works

Four dimensions, 0–100, equal weight, composite = plain mean. Every number carries its formula and the run ids behind it.

| Dimension | Question | Formula |
|---|---|---|
| Acceptance | Is the code kept? | `100 × (1 − rejected edits / edits)` (rejected = failed or hook-blocked Write/Edit) |
| Success | Did the task finish? | `100 × done / ended runs` (done = no dangling tool call, checklist complete) |
| Autonomy | Self-driving or rescued? | `100 × (1 − interventions / user turns)` (permission ask, rejection, interrupt) |
| Reliability | Stable or flaky? | `100 × (1 − mean(tool error rate, hook block rate, abandoned rate))` |

Grades are percentiles inside the fleet you actually have (harnesses, agents, models or runs): A ≥ p80, B ≥ p60, C ≥ p40, D ≥ p20. Fewer than 5 members → static bands (A ≥ 90, B ≥ 80, C ≥ 70, D ≥ 60) and the grade is marked uncalibrated (`*`). Fewer than 5 runs → low confidence.

Behaviour fingerprint per run: prompt length band (short ≤ 20 words, good ≤ 250, long), task size from input tokens, retry loops (3 consecutive errors on the same tool), subagents spawned, skills used (`/mk-x` in the prompt, `SKILL.md` reads), reports written under `plans/`, checklist completion.

## Kit health, adoption, workflow, insights

- **Kit health** — hook runs, blocks, crashes and durations per hook (from `hook-log.jsonl`; entries carry `sid` + `harness` since kit v2.7.0 and are otherwise matched to runs by time window). Hook errors reported by the harness itself are attributed to the hook named in the error; a `Cannot find module …/.claude/hooks/x.cjs` is flagged as *unresolved path* — the session was started from a subdirectory, so nothing was protecting it.
- **Adoption** — a run "uses the kit" when it typed a kit slash command (`/mk:…` or any command matching a skill in the project's `.claude/skills/`), read a `SKILL.md`, called the Skill tool, or spawned a subagent. Observe reads the installed catalogue (`.claude/metadata.json` name + version, `skills/*/SKILL.md`, `agents/*.md`, hooks in `settings.json`) and reports: kit share of runs, commands/skills/agents/hooks by frequency, kit vs harness agents, **never-used** skills/agents/hooks, adoption per project with kit version, per week.
- **Workflow & outcomes** — prompt bands, steering per run, median duration and tool calls, marathons (> 4 h or > 250 tool calls), compactions, API errors and refusal fallbacks, retry loops, abandoned runs, model switches, subagent use, weekday × hour heatmap, hot files, tool errors by tool; outcomes: `git commit` / `git push` / PR links, test commands (npm/pnpm/yarn/bun test, pytest, go test, cargo test, jest, vitest, …) and how many failed, edit-heavy runs that ran tests, files edited, reports under `plans/`.
- **Insights** — threshold rules over those numbers, each with level (`bad` / `warn` / `info` / `good`), evidence, a suggestion and the run ids: kit adoption < 30 %, projects without the kit, older kit versions (per kit name), unused skills/agents, non-kit slash commands, short prompts > 40 %, steering avg > 5, compactions in > 25 % of runs, marathons, few subagents, tests in < 30 % of edit-heavy runs, test failure rate, tool error rate > 20 % on ≥ 20 calls, hook crashes / slow hooks, retry loops ≥ 10 %, abandoned > 20 %, API errors, one model ≥ 80 % of tokens, cache hit rate < 50 %, model switches, off-hours share, weakest dimension < 60. Rate rules wait for ≥ 5 runs. Rules live in `scripts/lib/metrics.cjs` (`buildInsights`) — thresholds are plain numbers, adjust them to the team.

## When the user asks

- "How is the agent doing / which harness is better here" → run `stats` (or `serve`) and explain the numbers **with their formulas**. Quote the coverage limits above when relevant.
- "How much did this week cost" → `stats` (Usage by model) or `export --json`; give the **token table per model** and the formula above, and let the user apply their own price list. Do not invent prices.
- "Is the team using the kit / what should we improve" → `stats --all` (Adoption, Workflow & outcomes, Insights) or the Insights tab; read the findings top-down (bad → warn → info → good) and quote the evidence and run ids.
- "Why is autonomy low" → `runs`, then `run <id>` on the worst ones; interventions are listed as `permission_ask` events.
- "Are the hooks working" → the Kit health section; a hook with crashes > 0 or no runs at all is the finding.
- "Is the old data still there / the session got cleaned up" → `store status`; archived runs are listed like any other, with an `archived` pill in the UI.
- Do not edit the harness stores. Only `~/.mk-observe/store` is written, and only by observe itself.