Review ML Experiment
Optional loop step after evaluate. Record-outcome stays with the
caller. This skill writes idea files and their Ideas rows. The
stem is whichever experiment was reviewed. Check results were
stored with the report, so the audit reads them.
Human-facing prose
Details: setup-workspace references/human_facing_prose.md.
Idea files describe this report and the follow-up — not skill
ids, cells run, or the wrapper CLI.
Procedure
- Run
python -m skore_skills status and
python -m skore_skills review consent --stem <stem>. Treat
JSON action as authoritative.
stop — no scratch/results/<stem>/report.html. Name that
file and stop. Do not audit. Do not record-outcome.
audit — report exists, digest does not. Load
audit-ml-pipeline when installed. That skill runs
materialize.py once and does not execute the audit file.
proceed — digest already on disk. Do not re-run
materialize.py unless the user asked to re-audit. A
re-audit loads audit-ml-pipeline; that skill runs
materialize.py once.
Otherwise refresh idea files from the existing digest.
- Missing audit skill → one-line skip. Return
n/a — audit not run and write no idea files. Do not open
the Project or call report.* here.
- Read the design note, the EDA summary, the last History row,
and the digest. One candidate per
Issues: / Tips: line.
A methodological gap the design note named and this run did
not test is another candidate. A user idea or a literature
query is not a candidate here: after this skill returns,
shape-user-idea or search-ml-literature writes that file
and its Ideas row when the user asks. Load research-ml-practice only if
status.skills.research-ml-practice is true and an audit or
design candidate needs sources; otherwise one-line skip. Do
not invent papers, metrics, or a winner.
- Write one file per candidate at
journal/ideas/<stem>-<slug>.md with Experiment, Source
(audit:<stem>:checks.<code> or design:<stem>), Triage
open, Question, Why now, What changes, Open gaps. No
acceptance criteria. For a check candidate, read the
documentation URL on that line and apply that page's
recommendation in What changes. On a refresh, keep an existing file's
Triage value and the matching Ideas status. A new candidate
is open.
- Upsert one
## Ideas row per file in journal/JOURNAL.md.
If that table is missing, insert it between History and
Backlog. Columns: Question, Status, Experiment, Source.
Question is the file's Question as plain text, not a link.
Status is open, discarded, or aside, matching Triage.
A promoted file has no Ideas row. Experiment is this run's
stem. Source is copied verbatim. Edit only that table.
- Return the digest, JSON
finding from
python -m skore_skills audit finding --stem <stem>, the
locator from python -m skore_skills loop locator --stem <stem>,
and the idea paths.
Stop conditions
- On
proceed, do not re-run materialize.py unless the user
asked to re-audit.
- Do not write History, Backlog, Status, or a design note.
The Ideas table is the only
JOURNAL.md edit.
- Do not call
skore.evaluate or project.put.
- Do not pick a winning idea.
- Do not invent a missing child's procedure.
1---2name: review-ml-experiment3description: >4 Read the stored report, then write one idea file and one Ideas row per5 candidate. Trigger after a successful evaluate, on "review6 this stem", or when review consent is audit or proceed. Do not write7 History, Backlog, or a design note.8metadata:9 source: https://skillmd.com/skills/probabl-ai/review-ml-experiment10 modelTier: medium11---1213# Review ML Experiment1415Optional loop step after evaluate. Record-outcome stays with the16caller. This skill writes idea files and their Ideas rows. The17stem is whichever experiment was reviewed. Check results were18stored with the report, so the audit reads them.1920## Human-facing prose2122Details: `setup-workspace` `references/human_facing_prose.md`.23Idea files describe this report and the follow-up — not skill24ids, `cells run`, or the wrapper CLI.2526## Procedure27281. Run `python -m skore_skills status` and29 `python -m skore_skills review consent --stem <stem>`. Treat30 JSON `action` as authoritative.31 - `stop` — no `scratch/results/<stem>/report.html`. Name that32 file and stop. Do not audit. Do not record-outcome.33 - `audit` — report exists, digest does not. Load34 `audit-ml-pipeline` when installed. That skill runs35 `materialize.py` once and does not execute the audit file.36 - `proceed` — digest already on disk. Do not re-run37 `materialize.py` unless the user asked to re-audit. A38 re-audit loads `audit-ml-pipeline`; that skill runs39 `materialize.py` once.40 Otherwise refresh idea files from the existing digest.412. Missing audit skill → one-line skip. Return42 `n/a — audit not run` and write no idea files. Do not open43 the Project or call `report.*` here.443. Read the design note, the EDA summary, the last History row,45 and the digest. One candidate per `Issues:` / `Tips:` line.46 A methodological gap the design note named and this run did47 not test is another candidate. A user idea or a literature48 query is not a candidate here: after this skill returns,49 `shape-user-idea` or `search-ml-literature` writes that file50 and its Ideas row when the user asks. Load `research-ml-practice` only if51 `status.skills.research-ml-practice` is true and an audit or52 design candidate needs sources; otherwise one-line skip. Do53 not invent papers, metrics, or a winner.544. Write one file per candidate at55 `journal/ideas/<stem>-<slug>.md` with Experiment, Source56 (`audit:<stem>:checks.<code>` or `design:<stem>`), Triage57 `open`, Question, Why now, What changes, Open gaps. No58 acceptance criteria. For a check candidate, read the59 documentation URL on that line and apply that page's60 recommendation in What changes. On a refresh, keep an existing file's61 `Triage` value and the matching Ideas status. A new candidate62 is `open`.635. Upsert one `## Ideas` row per file in `journal/JOURNAL.md`.64 If that table is missing, insert it between History and65 Backlog. Columns: Question, Status, Experiment, Source.66 Question is the file's Question as plain text, not a link.67 Status is `open`, `discarded`, or `aside`, matching `Triage`.68 A `promoted` file has no Ideas row. Experiment is this run's69 stem. Source is copied verbatim. Edit only that table.706. Return the digest, JSON `finding` from71 `python -m skore_skills audit finding --stem <stem>`, the72 locator from `python -m skore_skills loop locator --stem <stem>`,73 and the idea paths.7475## Stop conditions7677- On `proceed`, do not re-run `materialize.py` unless the user78 asked to re-audit.79- Do not write History, Backlog, Status, or a design note.80 The Ideas table is the only `JOURNAL.md` edit.81- Do not call `skore.evaluate` or `project.put`.82- Do not pick a winning idea.83- Do not invent a missing child's procedure.