ViderStar Experiment write-up
The value of a fixed skeleton is that it has sections for the things nobody volunteers. "What was not verified" and "Why this variant was rejected" get read more carefully than the list of successes, and they are what makes a result reusable six months later.
1. Gather the facts first, then write
Research notebooks run to 90k+ characters of code — do not read one whole into context.
Use ${CLAUDE_PLUGIN_ROOT}/skills/experiment-article/scripts/nb_extract.py:
| Command | What it gives | Where it goes |
|---|---|---|
nb_extract.py metrics <nb> |
cell outputs containing metrics | the results sections |
nb_extract.py timings <nb> |
seconds, RSS, VRAM per step | the header and the protocol |
nb_extract.py env <nb> |
env flags and their defaults | Reproduction |
nb_extract.py sources <nb> |
database tables the notebook reads | Data sources |
nb_extract.py outputs <nb> |
every non-empty output | when what you need is not above |
Plus the run log (the nb-run plugin writes one), which carries the date, the duration and
whatever never made it into a cell output. Plus the ticket text, for what was actually asked.
Every number in the write-up has to come from somewhere. If a number is not in the output, the log does not have it, and there is nothing to compute it from, then say so in "What was not verified" — do not substitute something plausible.
2. The skeleton
# <TICKET-ID>. <the conclusion, in one phrase>
**Notebook:** ... **Ticket:** ... **Request:** ...
**Run:** YYYY-MM-DD, <hardware>, ~N hours
<abstract: 3-6 paragraphs, verdicts in bold, the headline numbers right here>
## Data and method
| parameter | value | <- date windows, users and events, metric, starting checkpoint, peak RSS/VRAM
## Data sources
<tables>
## 1..N. <results>
<tables with slices: by user activity, by segment, by candidate configuration>
<metrics in pairs: ranking quality and the business metric, plus delta % vs baseline>
## Why <variant> was rejected
## What was not verified
## What to do next
## Side observations
## Answer to the original request
<plain language, for the person who asked, no metrics and no jargon>
## Reproduction
```bash
<the full env-flag line + the command that ran it>
| artefact | path | <- notebook, caches, checkpoints, log
## 3. Rules that are easy to break
- **The title is the conclusion, not the topic.** "the problem is real, the proposed fix is
not" beats "Experiment with sequence length". If the title does not tell you how it ended,
rewrite it.
- **The answer to the request is written for a non-technical reader.** They read that
paragraph and nothing else. No NDCG — what the user will see, and what you are doing.
- **Do not smooth anything over.** If the control turned out to be a copy of a checkpoint,
or a trial returned a cached value instead of computing, that goes in the write-up in
plain words. That admission is usually what stops a wrong conclusion from being used.
- **Delta % vs baseline is mandatory.** Absolute metrics on their own say nothing.
- Side findings do not get dropped; they go in "Side observations" marked as a separate
ticket.
## 4. Afterwards
The write-up lands as a `.md` next to the notebook. From there it goes to wherever the team
keeps its knowledge base — the `atlassian-kit` plugin publishes it to Confluence and links
it from the ticket, if that is your stack.