Replicative Reproducibility Visualization Lens
Philosophical Mode: Replicative
Primary Question: "Can the figures be reproduced from the data and code?"
Focus: Data availability (public / restricted / embargoed), preprocessing parameter
disclosure (bin widths, smoothing windows, normalization), plotting library and
version pinning, random seed documentation, per-figure code reference (script or
notebook cell)
Arguments
/autoskillit:vis-lens-reproducibility [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- Auditing figure reproducibility before public release
- Checking whether preprocessing parameters are fully disclosed
- Verifying that random seeds are documented for stochastic plots
- Linking each figure to the script or notebook cell that generates it
- User invokes
/autoskillit:vis-lens-reproducibility
Critical Constraints
NEVER:
- Modify any source code files
- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves
- Create files outside
{{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/
- Treat "code available on request" as equivalent to public availability
- Omit random seed documentation for any figure derived from stochastic processes
ALWAYS:
Check data availability status for every figure (public/restricted/embargoed)
Document bin widths for histograms, smoothing windows for time-series, normalization parameters for heatmaps
Pin plotting library name and version (matplotlib 3.8.2, seaborn 0.13.0, etc.)
Record the random seed(s) used for any stochastic component (sampling, bootstrapping, noise injection)
Provide a per-figure code reference: script path or notebook cell identifier
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool — this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/vis_spec_reproducibility_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/vis_spec_reproducibility_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1–4; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Data Availability Inventory
For each figure:
- Identify the data source (file path, dataset name, external URL)
- Classify availability: PUBLIC (DOI / URL), RESTRICTED (license required), EMBARGOED (not yet released)
- FLAG FAIL if data is restricted/embargoed with no access plan stated
Step 2: Preprocessing Parameter Audit
For each figure, identify all preprocessing steps that affect visual output:
- Histograms: bin width or bin count; normalization (density vs count vs probability)
- Time-series / learning curves: smoothing window type (rolling mean, EMA) and window size
- Heatmaps: normalization method (min-max, z-score, none); colormap clipping range
- Scatter/line with aggregation: aggregation function (mean, median) and grouping
- FLAG WARNING for any preprocessing parameter not documented
Step 3: Library and Version Audit
Scan the codebase for plotting imports:
- Record:
import matplotlib, import seaborn, import plotly, etc.
- Check
pyproject.toml or requirements.txt for pinned versions
- FLAG WARNING if plotting library version is not pinned
Step 4: Random Seed Audit
For each figure involving a stochastic component:
- Identify source of randomness: bootstrapping, subsampling, t-SNE/UMAP, noise injection
- Verify
random_state, seed, np.random.seed, or equivalent is documented per figure
- FLAG FAIL if any stochastic figure has no documented seed
Step 5: Per-Figure Code Reference
For each figure:
- Identify the script or notebook cell that generates it
- Record: file path + function or cell ID
- FLAG WARNING if a figure has no traceable code reference
Step 6: Emit yaml:figure-spec Blocks
For each figure, emit one yaml:figure-spec fenced block with data_source and
annotations fields capturing reproducibility metadata. Then LOAD /autoskillit:mermaid
and create a diagram showing: data availability → preprocessing → library version →
seed documentation → code reference → verdict.
Output Template
# Replicative Reproducibility Spec: {System / Experiment Name}
**Lens:** Replicative Reproducibility (Replicative)
**Question:** Can the figures be reproduced from the data and code?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}
## Reproducibility Audit Summary
| Figure | Data Available | Preprocessing Documented | Library Pinned | Seed Documented | Code Reference | Status |
|--------|---------------|--------------------------|----------------|-----------------|----------------|--------|
| fig-01 | PUBLIC | PASS | PASS | N/A | scripts/plot_main.py | OK |
| fig-02 | RESTRICTED | WARNING | FAIL | PASS | notebooks/ablation.ipynb#cell-7 | FAIL |
## Figure Specs
```yaml
# yaml:figure-spec — canonical schema (spec_version: "1.0")
figure_id: "fig-01-main-result"
figure_title: "Model A achieves state-of-the-art on all benchmarks"
spec_version: "1.0"
chart_type: "bar"
chart_type_fallback: "table"
perceptual_justification: "Bars communicate exact values; error bars show CI95 over 5 seeds."
data_source: "results/main.csv (DOI: 10.xxxx/xxxxx)"
data_mapping:
x: "benchmark"
y: "score"
color: "model"
size: ""
facet: ""
layout:
width_inches: 6.0
height_inches: 4.0
dpi: 300
stat_overlay:
type: "error_bar"
measure: "CI95"
n_seeds: 5
annotations: ["data: public (DOI); preprocessing: none; library: matplotlib==3.8.2; seeds: 0,1,2,3,4; code: scripts/plot_main.py:plot_main_result()"]
anti_patterns: []
palette: "okabe-ito"
format: "pdf"
target_dpi: 300
library: "matplotlib==3.8.2"
report_section: "Section 4 Results"
priority: "P0"
placement_tier: "main"
conflicts: []
metadata:
created_by: "vis-lens-reproducibility"
reviewed_by: ""
last_updated: "{YYYY-MM-DD}"
Reproducibility Diagram
%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
%% CLASS DEFINITIONS %%
classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
subgraph Data ["DATA AVAILABILITY"]
D1["source: {path / DOI}<br/>━━━━━━━━━━<br/>public / restricted / embargoed"]
end
subgraph Preproc ["PREPROCESSING"]
P1["bin width / window: {N}<br/>normalization: {method}<br/>━━━━━━━━━━<br/>documented: PASS / FAIL"]
end
subgraph Library ["LIBRARY VERSION"]
L1["matplotlib=={version}<br/>━━━━━━━━━━<br/>pinned: PASS / WARNING"]
end
subgraph Seeds ["RANDOM SEEDS"]
S1["seeds: {list}<br/>━━━━━━━━━━<br/>documented: PASS / FAIL / N/A"]
end
subgraph CodeRef ["CODE REFERENCE"]
C1["script: {path}:{function}<br/>━━━━━━━━━━<br/>traceable: PASS / WARNING"]
end
subgraph Verdict ["VERDICT"]
V1["{OK / WARNING / FAIL}<br/>━━━━━━━━━━<br/>{reason}"]
end
D1 --> P1
P1 --> L1
L1 --> S1
S1 --> C1
C1 --> V1
class D1 stateNode;
class P1 handler;
class L1 cli;
class S1 output;
class C1 phase;
class V1 detector;
Color Legend:
| Color |
Category |
Description |
| Dark Teal |
Data |
Data availability status |
| Orange |
Preprocessing |
Parameter documentation check |
| Dark Blue |
Library |
Plotting library version pin |
| Teal |
Seeds |
Random seed documentation |
| Purple |
Code Ref |
Per-figure code traceability |
| Red |
Verdict |
OK / WARNING / FAIL assessment |
---
## Pre-Diagram Checklist
Before creating the diagram, verify:
- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table
- [ ] Every restricted/embargoed dataset is flagged as FAIL or WARNING
- [ ] Every histogram bin width and time-series smoothing window is audited
- [ ] Every stochastic figure has its seeds documented or flagged
1---2name: vis-lens-reproducibility3categories: [vis-lens]4activate_deps: [mermaid]5description: "Create Replicative Reproducibility visualization planning spec showing data availability, preprocessing parameter disclosure (bin widths, smoothing windows), plotting library/version, random seeds, and code reference per figure. Replicative lens answering \"Can the figures be reproduced from the data and code?\""6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Reproducibility Lens - Auditing figure reproducibility...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/vis-lens-reproducibility15---1617# Replicative Reproducibility Visualization Lens1819**Philosophical Mode:** Replicative20**Primary Question:** "Can the figures be reproduced from the data and code?"21**Focus:** Data availability (public / restricted / embargoed), preprocessing parameter22 disclosure (bin widths, smoothing windows, normalization), plotting library and23 version pinning, random seed documentation, per-figure code reference (script or24 notebook cell)2526## Arguments2728`/autoskillit:vis-lens-reproducibility [context_path] [experiment_plan_path]`2930- **context_path** (optional positional arg 1) — Absolute path to a lens context file31 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.32 If provided, read this file before beginning analysis to obtain structured context.33 If omitted, discover context by exploring the CWD.34- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full35 experiment plan. If provided, read for complete experimental methodology and design.36 If omitted, locate the experiment plan by exploring the CWD.3738## When to Use3940- Auditing figure reproducibility before public release41- Checking whether preprocessing parameters are fully disclosed42- Verifying that random seeds are documented for stochastic plots43- Linking each figure to the script or notebook cell that generates it44- User invokes `/autoskillit:vis-lens-reproducibility`4546## Critical Constraints4748**NEVER:**49- Modify any source code files50- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves51- Create files outside `{{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/`52- Treat "code available on request" as equivalent to public availability53- Omit random seed documentation for any figure derived from stochastic processes5455**ALWAYS:**56- Check data availability status for every figure (public/restricted/embargoed)57- Document bin widths for histograms, smoothing windows for time-series, normalization parameters for heatmaps58- Pin plotting library name and version (matplotlib 3.8.2, seaborn 0.13.0, etc.)59- Record the random seed(s) used for any stochastic component (sampling, bootstrapping, noise injection)60- Provide a per-figure code reference: script path or notebook cell identifier61- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool — this is MANDATORY62- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.63- Write output to `{{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/vis_spec_reproducibility_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)64- After writing the file, emit the structured output token as **literal plain text** with no65 markdown formatting on the token name (the adjudicator performs a regex match):6667 ```68 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/vis-lens-reproducibility/vis_spec_reproducibility_{...}.md69 ```7071---7273## Analysis Workflow7475### Step 0: Parse optional arguments7677If positional arg 1 (context_path) is provided and the file exists, read it to obtain78IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional79arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full80methodology. Use this structured context as the foundation for Steps 1–4; skip the CWD81exploration for these fields if the context file supplies them.8283### Step 1: Data Availability Inventory8485For each figure:86- Identify the data source (file path, dataset name, external URL)87- Classify availability: PUBLIC (DOI / URL), RESTRICTED (license required), EMBARGOED (not yet released)88- FLAG FAIL if data is restricted/embargoed with no access plan stated8990### Step 2: Preprocessing Parameter Audit9192For each figure, identify all preprocessing steps that affect visual output:93- **Histograms**: bin width or bin count; normalization (density vs count vs probability)94- **Time-series / learning curves**: smoothing window type (rolling mean, EMA) and window size95- **Heatmaps**: normalization method (min-max, z-score, none); colormap clipping range96- **Scatter/line with aggregation**: aggregation function (mean, median) and grouping97- FLAG WARNING for any preprocessing parameter not documented9899### Step 3: Library and Version Audit100101Scan the codebase for plotting imports:102- Record: `import matplotlib`, `import seaborn`, `import plotly`, etc.103- Check `pyproject.toml` or `requirements.txt` for pinned versions104- FLAG WARNING if plotting library version is not pinned105106### Step 4: Random Seed Audit107108For each figure involving a stochastic component:109- Identify source of randomness: bootstrapping, subsampling, t-SNE/UMAP, noise injection110- Verify `random_state`, `seed`, `np.random.seed`, or equivalent is documented per figure111- FLAG FAIL if any stochastic figure has no documented seed112113### Step 5: Per-Figure Code Reference114115For each figure:116- Identify the script or notebook cell that generates it117- Record: file path + function or cell ID118- FLAG WARNING if a figure has no traceable code reference119120### Step 6: Emit yaml:figure-spec Blocks121122For each figure, emit one `yaml:figure-spec` fenced block with `data_source` and123`annotations` fields capturing reproducibility metadata. Then LOAD `/autoskillit:mermaid`124and create a diagram showing: data availability → preprocessing → library version →125seed documentation → code reference → verdict.126127---128129## Output Template130131```markdown132# Replicative Reproducibility Spec: {System / Experiment Name}133134**Lens:** Replicative Reproducibility (Replicative)135**Question:** Can the figures be reproduced from the data and code?136**Date:** {YYYY-MM-DD}137**Scope:** {What was analyzed}138139## Reproducibility Audit Summary140141| Figure | Data Available | Preprocessing Documented | Library Pinned | Seed Documented | Code Reference | Status |142|--------|---------------|--------------------------|----------------|-----------------|----------------|--------|143| fig-01 | PUBLIC | PASS | PASS | N/A | scripts/plot_main.py | OK |144| fig-02 | RESTRICTED | WARNING | FAIL | PASS | notebooks/ablation.ipynb#cell-7 | FAIL |145146## Figure Specs147148```yaml149# yaml:figure-spec — canonical schema (spec_version: "1.0")150figure_id: "fig-01-main-result"151figure_title: "Model A achieves state-of-the-art on all benchmarks"152spec_version: "1.0"153chart_type: "bar"154chart_type_fallback: "table"155perceptual_justification: "Bars communicate exact values; error bars show CI95 over 5 seeds."156data_source: "results/main.csv (DOI: 10.xxxx/xxxxx)"157data_mapping:158 x: "benchmark"159 y: "score"160 color: "model"161 size: ""162 facet: ""163layout:164 width_inches: 6.0165 height_inches: 4.0166 dpi: 300167stat_overlay:168 type: "error_bar"169 measure: "CI95"170 n_seeds: 5171annotations: ["data: public (DOI); preprocessing: none; library: matplotlib==3.8.2; seeds: 0,1,2,3,4; code: scripts/plot_main.py:plot_main_result()"]172anti_patterns: []173palette: "okabe-ito"174format: "pdf"175target_dpi: 300176library: "matplotlib==3.8.2"177report_section: "Section 4 Results"178priority: "P0"179placement_tier: "main"180conflicts: []181metadata:182 created_by: "vis-lens-reproducibility"183 reviewed_by: ""184 last_updated: "{YYYY-MM-DD}"185```186187## Reproducibility Diagram188189```mermaid190%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%191flowchart TB192 %% CLASS DEFINITIONS %%193 classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;194 classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;195 classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;196 classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;197 classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;198 classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;199200 subgraph Data ["DATA AVAILABILITY"]201 D1["source: {path / DOI}<br/>━━━━━━━━━━<br/>public / restricted / embargoed"]202 end203204 subgraph Preproc ["PREPROCESSING"]205 P1["bin width / window: {N}<br/>normalization: {method}<br/>━━━━━━━━━━<br/>documented: PASS / FAIL"]206 end207208 subgraph Library ["LIBRARY VERSION"]209 L1["matplotlib=={version}<br/>━━━━━━━━━━<br/>pinned: PASS / WARNING"]210 end211212 subgraph Seeds ["RANDOM SEEDS"]213 S1["seeds: {list}<br/>━━━━━━━━━━<br/>documented: PASS / FAIL / N/A"]214 end215216 subgraph CodeRef ["CODE REFERENCE"]217 C1["script: {path}:{function}<br/>━━━━━━━━━━<br/>traceable: PASS / WARNING"]218 end219220 subgraph Verdict ["VERDICT"]221 V1["{OK / WARNING / FAIL}<br/>━━━━━━━━━━<br/>{reason}"]222 end223224 D1 --> P1225 P1 --> L1226 L1 --> S1227 S1 --> C1228 C1 --> V1229230 class D1 stateNode;231 class P1 handler;232 class L1 cli;233 class S1 output;234 class C1 phase;235 class V1 detector;236```237238**Color Legend:**239| Color | Category | Description |240|-------|----------|-------------|241| Dark Teal | Data | Data availability status |242| Orange | Preprocessing | Parameter documentation check |243| Dark Blue | Library | Plotting library version pin |244| Teal | Seeds | Random seed documentation |245| Purple | Code Ref | Per-figure code traceability |246| Red | Verdict | OK / WARNING / FAIL assessment |247```248249---250251## Pre-Diagram Checklist252253Before creating the diagram, verify:254255- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool256- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)257- [ ] Diagram will include a color legend table258- [ ] Every restricted/embargoed dataset is flagged as FAIL or WARNING259- [ ] Every histogram bin width and time-series smoothing window is audited260- [ ] Every stochastic figure has its seeds documented or flagged