This skill should be used when the user asks to "analyze experimental results", "run strict statistical analysis", "compare model performance", "generate scientific figures", "check significance", "do ablation analysis", or mentions interpreting experiment data with rigorous statistics and visualization. It focuses on strict analysis bundles, not Results-section prose.
Run strict, evidence-first experimental analysis for ML/AI research.
Use this skill to produce a strict analysis bundle:
analysis-report.md
stats-appendix.md
figure-catalog.md
figures/
When the user asks for review, audit, no-write, dry-run, or when inputs are incomplete, use read-only audit mode instead of producing files or figures. In that mode, output only valid/invalid statistics, blockers, claim candidates, and what evidence is missing. If invoked by /analyze-results, the command layer may write a blocker summary, but this skill should not create figures, reports, or polished conclusions from incomplete evidence.
Do not use this skill to draft a paper Results section or a full experiment wrap-up report. Those belong to ml-paper-writing or results-report.
Core contract
This skill is responsible for
validating experiment artifacts and comparison units,
running rigorous descriptive and inferential statistics,
generating real scientific figures when data/logs are available,
writing figure purposes, caption requirements, and interpretation checklists,
surfacing limits, blockers, and missing evidence explicitly.
This skill is not responsible for
paper-ready Results prose,
manuscript narrative polishing,
paper-ready figure/table packaging with pubfig / pubtab,
project-level experiment retrospectives.
If the user wants the complete post-experiment summary report, hand off to results-report after this bundle is ready. If the user wants publication-grade figures/tables, export parameters, publication QA, or figure/table redesign, hand off to publication-chart-skill.
Non-negotiable quality bar
Prefer real figures over figure specs.
If the data can be read, generate real figures. Do not stop at “recommended visualization”.
Exception: in read-only audit mode, do not generate figures; describe what figure would be valid after evidence is complete.
Never fabricate statistics.
If sample size, seeds, or raw metrics are missing, state the blocker clearly.
Report complete statistics.
Do not report only best scores or only p-values.
Interpret every main figure.
Every major figure must have purpose, caption requirements, and post-figure interpretation notes.
Separate evidence from prose.
This skill produces analysis artifacts; it does not write manuscript sections.
Standard workflow
1. Inventory and validate artifacts
Start by identifying:
metric tables (csv, json, tsv, logs),
training curves and checkpoints,
seeds / repeated runs,
baselines, ablations, and comparison families,
evaluation protocol metadata.
Validate:
metric direction (higher/lower is better),
unit of analysis (run, subject, fold, dataset, seed),
number of runs / seeds,
missing values or silent failures,
comparability across methods.
If the comparison is not statistically valid, say so before continuing. Do not treat repeated subject × task rows, folds, windows, trials, or seeds as independent units unless the design justifies it.
Common blocker: a subject × task summary table is usually a repeated-measure summary, not an independent subject-level sample. If subjects have multiple task rows or missing task cells, state that before any significance or winner claim.
2. Lock the comparison questions
Before running statistics, define the exact comparison questions:
Which method is compared to which baseline?
What is the primary metric?
What is the repeated-measure unit?
Which ablation or robustness questions matter?
Which findings are decision-changing?
Do not mix unrelated comparisons into one undifferentiated table.
3. Run strict statistics
Always produce:
descriptive statistics: mean ± std when appropriate,
95% CI or another clearly justified interval,
run/seed counts,
significance tests with assumptions stated,
effect sizes,
multiple-comparison handling when several contrasts are reported.
Default expectation:
check parametric assumptions first,
use non-parametric fallback when assumptions fail,
state exactly what was tested and on what samples.
See:
references/statistical-methods.md
references/statistical-reporting.md
4. Generate real scientific figures
Produce actual figures whenever artifacts are available.
Minimum expectation for a non-trivial analysis bundle:
For every major figure, answer all three questions:
Why does this figure exist?
What exactly should the reader notice?
What does that observation change in our belief or next decision?
If a figure cannot answer question 3, it is probably decorative rather than scientific.
Read-only audit mode
Use this mode when:
the user asks to audit or review existing artifacts,
the environment is read-only,
the user forbids file writes or figure generation,
core evidence is missing.
Return:
analysis questions,
valid statistics,
invalid or unsafe statistics,
claim candidates with allowed and forbidden wording,
blockers before report/figure generation.
Do not create analysis-output/, figures, or reports in this mode.
Quarantine any statistics file whose interpretation contradicts its own p-value, test method, unit of analysis, or comparison family. Do not reuse that file for claim wording until provenance is checked.
Failure mode policy
When inputs are incomplete, say so explicitly.
Examples:
no seed-level data -> descriptive summary only; inferential claims blocked,
no comparable baseline outputs -> no significance claim,
no readable logs -> cannot generate dynamics figure,
too few runs -> effect size may be unstable; report this limitation.
unclear unit of analysis -> no winner claim or significance claim,
analysis file with contradictory interpretation -> quarantine it until provenance is checked.
Never replace missing evidence with confident prose.
Reference files
Load only what is needed:
references/statistical-methods.md - test selection and assumptions
references/statistical-reporting.md - minimum reporting standard
references/figure-interpretation.md - how to explain figures with evidence
references/analysis-depth.md - move from observation to mechanism and decision
references/common-pitfalls.md - common analysis and reporting failures
../research-ideation/references/research-contract.md - shared claim candidate and claim strength contract
Example files
examples/example-analysis-report.md
examples/example-stats-appendix.md
examples/example-figure-catalog.md
1---
2name: results-analysis
3description: This skill should be used when the user asks to "analyze experimental results", "run strict statistical analysis", "compare model performance", "generate scientific figures", "check significance", "do ablation analysis", or mentions interpreting experiment data with rigorous statistics and visualization. It focuses on strict analysis bundles, not Results-section prose.
4tags: [Research, Analysis, Statistics, Visualization, Scientific Reporting]
5version: 0.2.0
6metadata:
7 source: https://skillmd.com/skills/galaxy-dawn/results-analysis
8---
910# Results Analysis
1112Run **strict, evidence-first experimental analysis** for ML/AI research.
1314Use this skill to produce a **strict analysis bundle**:
15- `analysis-report.md`
16- `stats-appendix.md`
17- `figure-catalog.md`
18- `figures/`
1920When the user asks for review, audit, no-write, dry-run, or when inputs are incomplete, use **read-only audit mode** instead of producing files or figures. In that mode, output only valid/invalid statistics, blockers, claim candidates, and what evidence is missing. If invoked by `/analyze-results`, the command layer may write a blocker summary, but this skill should not create figures, reports, or polished conclusions from incomplete evidence.
2122Do **not** use this skill to draft a paper `Results` section or a full experiment wrap-up report. Those belong to `ml-paper-writing` or `results-report`.
2324## Core contract
2526### This skill is responsible for
27- validating experiment artifacts and comparison units,
28- running rigorous descriptive and inferential statistics,
29- generating **real scientific figures** when data/logs are available,
30- writing figure purposes, caption requirements, and interpretation checklists,
31- surfacing limits, blockers, and missing evidence explicitly.
3233### This skill is not responsible for
34- paper-ready `Results` prose,
35- manuscript narrative polishing,
36- paper-ready figure/table packaging with `pubfig` / `pubtab`,
37- project-level experiment retrospectives.
3839If the user wants the complete post-experiment summary report, hand off to `results-report` after this bundle is ready. If the user wants publication-grade figures/tables, export parameters, publication QA, or figure/table redesign, hand off to `publication-chart-skill`.
4041## Non-negotiable quality bar
42431. **Prefer real figures over figure specs.**
44 If the data can be read, generate real figures. Do not stop at “recommended visualization”.
45 Exception: in read-only audit mode, do not generate figures; describe what figure would be valid after evidence is complete.
462. **Never fabricate statistics.**
47 If sample size, seeds, or raw metrics are missing, state the blocker clearly.
483. **Report complete statistics.**
49 Do not report only best scores or only p-values.
504. **Interpret every main figure.**
51 Every major figure must have purpose, caption requirements, and post-figure interpretation notes.
525. **Separate evidence from prose.**
53 This skill produces analysis artifacts; it does not write manuscript sections.
5455## Standard workflow
5657### 1. Inventory and validate artifacts
5859Start by identifying:
60- metric tables (`csv`, `json`, `tsv`, logs),
61- training curves and checkpoints,
62- seeds / repeated runs,
63- baselines, ablations, and comparison families,
64- evaluation protocol metadata.
6566Validate:
67- metric direction (higher/lower is better),
68- unit of analysis (run, subject, fold, dataset, seed),
69- number of runs / seeds,
70- missing values or silent failures,
71- comparability across methods.
7273If the comparison is not statistically valid, say so before continuing. Do not treat repeated `subject × task` rows, folds, windows, trials, or seeds as independent units unless the design justifies it.
74Common blocker: a `subject × task` summary table is usually a repeated-measure summary, not an independent subject-level sample. If subjects have multiple task rows or missing task cells, state that before any significance or winner claim.
7576### 2. Lock the comparison questions
7778Before running statistics, define the exact comparison questions:
79- Which method is compared to which baseline?
80- What is the primary metric?
81- What is the repeated-measure unit?
82- Which ablation or robustness questions matter?
83- Which findings are decision-changing?
8485Do not mix unrelated comparisons into one undifferentiated table.
8687### 3. Run strict statistics
8889Always produce:
90- descriptive statistics: `mean ± std` when appropriate,
91- `95% CI` or another clearly justified interval,
92- run/seed counts,
93- significance tests with assumptions stated,
94- effect sizes,
95- multiple-comparison handling when several contrasts are reported.
9697Default expectation:
98- check parametric assumptions first,
99- use non-parametric fallback when assumptions fail,
100- state exactly what was tested and on what samples.
101102See:
103- `references/statistical-methods.md`
104- `references/statistical-reporting.md`
105106### 4. Generate real scientific figures
107108Produce actual figures whenever artifacts are available.
109110Minimum expectation for a non-trivial analysis bundle:
111- **one main comparison figure**,
112- **one supporting figure** (training dynamics / ablation / breakdown / error analysis),
113- **one exact numeric summary table** in markdown.
114115Every main figure must define:
116- figure purpose,
117- plotted variables,
118- error bar meaning,
119- caption requirements,
120- interpretation checklist.
121122See:
123- `references/visualization-best-practices.md`
124- `references/figure-interpretation.md`
125126### 5. Write analysis artifacts
127128#### `analysis-report.md`
129Summarize:
130- the analysis question,
131- key findings,
132- strongest supported comparisons,
133- main caveats,
134- what changed in the experimental understanding,
135- claim candidates that may later be used in reports or manuscript writing.
136137Each claim candidate should use this shape:
138139```md
140## Claim Candidates
141142- Claim:
143 - Source evidence:
144 - Allowed wording:
145 - Forbidden stronger wording:
146 - Uncertainty:
147 - Next check:
148 - Decision: keep | weaken | revise | discard
149```
150151#### `stats-appendix.md`
152Record:
153- descriptive statistics,
154- test choices,
155- assumptions checked,
156- effect sizes,
157- confidence intervals,
158- multiple comparison corrections,
159- explicit blockers and limitations.
160161#### `figure-catalog.md`
162For each figure, record:
163- filename,
164- purpose,
165- data source,
166- caption draft requirements,
167- key observation,
168- interpretation checklist,
169- known caveats.
170171### 6. Final QA gate
172173Do not finish until all are true:
174- [ ] the primary comparison question is explicit,
175- [ ] sample size / seed count is stated,
176- [ ] inferential tests are justified,
177- [ ] effect sizes are reported for major contrasts,
178- [ ] real figures exist when data exists,
179- [ ] each figure has an interpretation note,
180- [ ] limitations and blockers are explicit,
181- [ ] each supported or strong claim candidate has evidence, uncertainty, and allowed wording,
182- [ ] over-strong manuscript wording is explicitly blocked when evidence is insufficient,
183- [ ] no manuscript-style `Results` draft is included.
184185## Output structure
186187```text
188analysis-output/
189├── analysis-report.md
190├── stats-appendix.md
191├── figure-catalog.md
192└── figures/
193 ├── figure-01-main-comparison.pdf
194 ├── figure-02-ablation.pdf
195 └── ...
196```
197198## Figure interpretation rule
199200For every major figure, answer all three questions:
2011. **Why does this figure exist?**
2022. **What exactly should the reader notice?**
2033. **What does that observation change in our belief or next decision?**
204205If a figure cannot answer question 3, it is probably decorative rather than scientific.
206207## Read-only audit mode
208209Use this mode when:
210- the user asks to audit or review existing artifacts,
211- the environment is read-only,
212- the user forbids file writes or figure generation,
213- core evidence is missing.
214215Return:
216- analysis questions,
217- valid statistics,
218- invalid or unsafe statistics,
219- claim candidates with allowed and forbidden wording,
220- blockers before report/figure generation.
221222Do not create `analysis-output/`, figures, or reports in this mode.
223Quarantine any statistics file whose interpretation contradicts its own p-value, test method, unit of analysis, or comparison family. Do not reuse that file for claim wording until provenance is checked.
224225## Failure mode policy
226227When inputs are incomplete, say so explicitly.
228229Examples:
230- no seed-level data -> descriptive summary only; inferential claims blocked,
231- no comparable baseline outputs -> no significance claim,
232- no readable logs -> cannot generate dynamics figure,
233- too few runs -> effect size may be unstable; report this limitation.
234- unclear unit of analysis -> no winner claim or significance claim,
235- analysis file with contradictory interpretation -> quarantine it until provenance is checked.
236237Never replace missing evidence with confident prose.
238239## Reference files
240241Load only what is needed:
242- `references/statistical-methods.md` - test selection and assumptions
243- `references/statistical-reporting.md` - minimum reporting standard
244- `references/visualization-best-practices.md` - publication-quality figure rules
245- `references/figure-interpretation.md` - how to explain figures with evidence
246- `references/analysis-depth.md` - move from observation to mechanism and decision
247- `references/common-pitfalls.md` - common analysis and reporting failures
248- `../research-ideation/references/research-contract.md` - shared claim candidate and claim strength contract
249250## Example files
251252- `examples/example-analysis-report.md`
253- `examples/example-stats-appendix.md`
254- `examples/example-figure-catalog.md`
Run npx get-skill add galaxy-dawn/results-analysis in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
This skill should be used when the user asks to "analyze experimental results", "run strict statistical analysis", "compare model performance", "generate scientific figures", "check significance", "do ablation analysis", or mentions interpreting experiment data with rigorous statistics and visualization. It focuses on strict analysis bundles, not Results-section prose. It is listed under AI & ML on SkillMD.
This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
Galaxy-Dawn (@galaxy-dawn) published this skill. Their other Agent Skills are listed on their SkillMD profile.