fr-audit
Announce at start: "I'm using fr-audit to audit ."
The numbers are not yours to compute. fr usage reads the harness store, classifies tool calls and splits the harness's dollar figure. Your job is choosing what to compare and saying what it means. Never add a figure by hand, never price tokens, never write 0 for —.
The engine
| Command |
Does |
fr usage collect --run <id> |
every session a run cursor records, normalized into $HOME/.cache/fr/usage/ |
fr usage collect --session <id> [--harness opencode|hermes] |
one session (default harness: claude-code) |
fr usage report --run <id> [--run <id>…] --format table |
per model, activity, sub-activity and step |
fr usage report --session <id>… --format html -o <file> |
the same as one self-contained page |
fr usage compare --before <date|timestamp|run> --after <date|timestamp|run> |
two sets of runs from the committed usage files, cursors and plan journals: phases, turns, main and subagent cache-read and output, cost (and how many runs were priced), review findings per phase, re-opened findings; per-run rows plus per-set medians and n; --steps adds the main session's per-step turns, tokens, cost and $/turn |
collect and report read a run cursor and the harness store and write only under
the cache; report reads any uncollected session live. compare reads committed
files and writes nothing. Pass --repo <path> when the cursor lives in another checkout.
Choosing what to compare
- One run:
--run <id>. The step table shows which pipeline step spent what.
- Before and after a process change:
fr usage compare --before <cutoff> --after <cutoff> (ISO date or timestamp, split on each run's started; or a run id). It
reads the committed files, prints — for what a run never recorded. Quote command
and output with n and the priced count; say "inconclusive" when the sets are thin
or confounded (model change, unpriced, pre-split runs).
- A sample: name the sessions and report them together; pooled shares weight each
session by its dollars.
A session another host ran comes back unavailable and renders —: an absence, not a
cheap session. Never drop it from the comparison silently or count it as zero.
Reading the split
- Dollars are the harness's. Each model's harness figure is divided over that
model's messages by price-weighted tokens with fixed ratios (input 1, cache write
1.25 for 5 min / 2 for 1 h, cache read 0.1, output 5). The split always sums back to
the harness total. What no message carries lands in
other / unattributed.
- Activities. paperwork is fr's own bookkeeping: specs, plans, journals, run
cursors, the acceptance matrix, their checks, and learning the CLI (
--help).
implementation is code read, code written and verification (tests, lint). other
is version control, orchestration, chores and narration (a message with no tool call).
- Context carry vs generation. Most of a long session's dollars are cache reads,
the whole context re-read on every turn. The context carried is the multiplier: a
turn late in a long session costs far more than the same turn early. Output is
the small part.
- Turns are the cost unit. An extra round trip re-reads everything. A
bookkeeping step that takes six turns costs six context re-reads, however little
it writes. When paperwork looks expensive, count its turns before blaming its size:
the report's
turns columns do (By activity, By step). A turn is one assistant
message, counted once for each activity its tool calls touched, so activity turns
can sum past the message count; step turns cannot.
- Coarse sessions. A
coarse attribution note means the harness keeps one token count
per message (Hermes): token-weighted split. State that caveat when comparing.
- source.
exact is the billed figure, estimated the harness's estimate, none no
figure. Carry it into any claim.
Pages
Four triage pages answer: "What do I do next?" (board), "Where do defects come from?" (origins), "What is the system?" (architecture), "How did we get here?" (history). Each holds hand-written analysis in fragments: renders rebuild sections, erasing hand edits.
Audit page (measured). fr usage report --session <id>… --format html -o $HOME/.cache/fr/usage/<name>.html. Report the pooled shares and the per-session
range, name the sessions, and state what was unavailable.
Architecture page (detailed). The engine builds it: fr triage architecture render --repo <owner/repo> [--checkout <clone>] [--now-ref <ref>] writes architecture.html into the
triage state directory (the usual fr triage scope options), and nothing else. Its sections
are generated, so you never type a figure into it: the summary and measured
sections (subsystem cards, size table),
then your fragments. You supply three inputs, all in the state directory:
subsystems.yaml: a subsystems: list, each with name, path (globs; * spans /),
then_ref (the git ref "then" is measured at) and themes (the judgement themes that place
an open issue on it); an issue whose theme maps nowhere lands under Other. Lines: every line
of every text file the globs match; binary files, symlinks and submodules are not counted, and
a moved file moves its lines. Each commit is named; a figure not taken is —.
architecture/<name>.html: authored fragments (diagrams, narrative, inline SVG), using the
page's tokens (var(--accent), var(--ink)). Close every tag; self-closing only on void
elements and SVG shapes. Refused, by name, with nothing written: <script>, <style>, <link>,
<iframe>, <object>, <embed>, <meta>, <base>, <form>, <html>, <head>, <body>,
a page-level <title> (<svg><title> is fine), on* attributes and javascript: or
data:text/html URLs. That is all the check does, so a fragment must carry no untrusted
text: HTML-escape any issue title or other outside text yourself.
architecture/manifest.yaml: an ordered list of generated section names
(summary, subsystems, size-table) and fragment file names. A listed file that does not exist is reported
and shown on the page; a generated section you do not name is appended, with a note on
the page, before the fragments; with no manifest every generated section shows. (The timeline, waves,
filings and origins now live on the history and origins pages respectively.)
Measured versus projected is the rule. The verb's sections are measurements. Everything
you author is the future state, design: for each step the spec changes, state the turns it removes
or adds and why, derive it from the measured figures, and label every such figure a projection
(in the fragment itself). A projection presented as a measurement is the defect this skill
exists to prevent. When the change ships, render again and put the measured figure beside the
projection; keep both.
Refresh the inputs first when you want a current page: fr triage render stores a snapshot, and
fr triage origins collect plus the fr-origins skill provide the origins data. Publishing
stays yours: fr writes a file in the scope's state directory (fr triage scope show names it);
copy or host it wherever the operator publishes, after redacting any third-party identity it carries.
Red flags
- A share quoted with no session list or no
source.
— turned into 0, or an unavailable session left out of the denominator's story.
- A dollar figure computed from a list price. fr knows only ratios.
- A before/after claim from runs of different shape or harness, unstated.
- Session ids, paths or model names pasted into a public artifact unredacted.
1---2name: fr-audit3description: >4 Use when asked what a run or session cost, where the money went, whether a5 process change made the pipeline cheaper, or to compare runs before and after6 a change; when asked for an audit page or an architecture page of the fr7 pipeline; or when a PR body needs its usage table.8metadata:9 source: https://skillmd.com/skills/derio-net/fr-audit10---1112# fr-audit1314**Announce at start:** "I'm using fr-audit to audit <runs or sessions>."1516The numbers are not yours to compute. `fr usage` reads the harness store, classifies tool calls and splits the harness's dollar figure. Your job is choosing what to compare and saying what it means. Never add a figure by hand, never price tokens, never write `0` for `—`.1718## The engine1920| Command | Does |21|---|---|22| `fr usage collect --run <id>` | every session a run cursor records, normalized into `$HOME/.cache/fr/usage/` |23| `fr usage collect --session <id> [--harness opencode\|hermes]` | one session (default harness: claude-code) |24| `fr usage report --run <id> [--run <id>…] --format table` | per model, activity, sub-activity and step |25| `fr usage report --session <id>… --format html -o <file>` | the same as one self-contained page |26| `fr usage compare --before <date\|timestamp\|run> --after <date\|timestamp\|run>` | two sets of runs from the committed usage files, cursors and plan journals: phases, turns, main and subagent cache-read and output, cost (and how many runs were priced), review findings per phase, re-opened findings; per-run rows plus per-set medians and `n`; `--steps` adds the main session's per-step turns, tokens, cost and $/turn |2728`collect` and `report` read a run cursor and the harness store and write only under29the cache; `report` reads any uncollected session live. `compare` reads committed30files and writes nothing. Pass `--repo <path>` when the cursor lives in another checkout.3132## Choosing what to compare3334- **One run:** `--run <id>`. The step table shows which pipeline step spent what.35- **Before and after a process change:** `fr usage compare --before <cutoff> --after36 <cutoff>` (ISO date or timestamp, split on each run's `started`; or a run id). It37 reads the committed files, prints `—` for what a run never recorded. Quote command38 and output with `n` and the priced count; say "inconclusive" when the sets are thin39 or confounded (model change, unpriced, pre-split runs).40- **A sample:** name the sessions and report them together; pooled shares weight each41 session by its dollars.4243A session another host ran comes back `unavailable` and renders `—`: an absence, not a44cheap session. Never drop it from the comparison silently or count it as zero.4546## Reading the split4748- **Dollars are the harness's.** Each model's harness figure is divided over that49 model's messages by price-weighted tokens with fixed ratios (input 1, cache write50 1.25 for 5 min / 2 for 1 h, cache read 0.1, output 5). The split always sums back to51 the harness total. What no message carries lands in `other / unattributed`.52- **Activities.** *paperwork* is fr's own bookkeeping: specs, plans, journals, run53 cursors, the acceptance matrix, their checks, and learning the CLI (`--help`).54 *implementation* is code read, code written and verification (tests, lint). *other*55 is version control, orchestration, chores and narration (a message with no tool call).56- **Context carry vs generation.** Most of a long session's dollars are cache reads,57 the whole context re-read on every turn. The context carried is the multiplier: a58 turn late in a long session costs far more than the same turn early. Output is59 the small part.60- **Turns are the cost unit.** An extra round trip re-reads everything. A61 bookkeeping step that takes six turns costs six context re-reads, however little62 it writes. When paperwork looks expensive, count its turns before blaming its size:63 the report's `turns` columns do (By activity, By step). A turn is one assistant64 message, counted once for each activity its tool calls touched, so activity turns65 can sum past the message count; step turns cannot.66- **Coarse sessions.** A `coarse attribution` note means the harness keeps one token count67 per message (Hermes): token-weighted split. State that caveat when comparing.68- **source.** `exact` is the billed figure, `estimated` the harness's estimate, `none` no69 figure. Carry it into any claim.7071## Pages7273Four triage pages answer: "What do I do next?" (board), "Where do defects come from?" (origins), "What is the system?" (architecture), "How did we get here?" (history). Each holds hand-written analysis in fragments: renders rebuild sections, erasing hand edits.74**Audit page (measured).** `fr usage report --session <id>… --format html -o75$HOME/.cache/fr/usage/<name>.html`. Report the pooled shares and the per-session76range, name the sessions, and state what was unavailable.7778**Architecture page (detailed).** The engine builds it: `fr triage architecture render --repo79<owner/repo> [--checkout <clone>] [--now-ref <ref>]` writes `architecture.html` into the80triage state directory (the usual `fr triage` scope options), and nothing else. Its sections81are generated, so you never type a figure into it: the summary and measured82sections (subsystem cards, size table),83then your fragments. You supply three inputs, all in the state directory:8485- `subsystems.yaml`: a `subsystems:` list, each with `name`, `path` (globs; `*` spans `/`),86 `then_ref` (the git ref "then" is measured at) and `themes` (the judgement themes that place87 an open issue on it); an issue whose theme maps nowhere lands under *Other*. Lines: every line88 of every text file the globs match; binary files, symlinks and submodules are not counted, and89 a moved file moves its lines. Each commit is named; a figure not taken is `—`.90- `architecture/<name>.html`: authored fragments (diagrams, narrative, inline SVG), using the91 page's tokens (`var(--accent)`, `var(--ink)`). Close every tag; self-closing only on void92 elements and SVG shapes. Refused, by name, with nothing written: `<script>`, `<style>`, `<link>`,93 `<iframe>`, `<object>`, `<embed>`, `<meta>`, `<base>`, `<form>`, `<html>`, `<head>`, `<body>`,94 a page-level `<title>` (`<svg><title>` is fine), `on*` attributes and `javascript:` or95 `data:text/html` URLs. That is all the check does, so a fragment must carry no untrusted96 text: HTML-escape any issue title or other outside text yourself.97- `architecture/manifest.yaml`: an ordered list of generated section names98 (`summary`, `subsystems`, `size-table`) and fragment file names. A listed file that does not exist is reported99 and shown on the page; a generated section you do not name is appended, with a note on100 the page, before the fragments; with no manifest every generated section shows. (The timeline, waves,101 filings and origins now live on the history and origins pages respectively.)102103**Measured versus projected is the rule.** The verb's sections are measurements. Everything104you author is the future state, design: for each step the spec changes, state the turns it removes105or adds and why, derive it from the measured figures, and label every such figure a projection106(in the fragment itself). A projection presented as a measurement is the defect this skill107exists to prevent. When the change ships, render again and put the measured figure beside the108projection; keep both.109110Refresh the inputs first when you want a current page: `fr triage render` stores a snapshot, and111`fr triage origins collect` plus the `fr-origins` skill provide the origins data. **Publishing112stays yours**: fr writes a file in the scope's state directory (`fr triage scope show` names it);113copy or host it wherever the operator publishes, after redacting any third-party identity it carries.114115## Red flags116117- A share quoted with no session list or no `source`.118- `—` turned into `0`, or an unavailable session left out of the denominator's story.119- A dollar figure computed from a list price. fr knows only ratios.120- A before/after claim from runs of different shape or harness, unstated.121- Session ids, paths or model names pasted into a public artifact unredacted.