Uncertainty Representation Visualization Lens
Philosophical Mode: Statistical
Primary Question: "How is uncertainty honestly represented?"
Focus: Error Bar Definitions, Distribution-Aware Alternatives, Multi-Seed Variance Protocols
Arguments
/autoskillit:vis-lens-uncertainty [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- Reviewing figures that show means, scores, or aggregate metrics without uncertainty
- Checking whether error bars are correctly defined (SD vs SE vs CI is not interchangeable)
- Evaluating whether n_seeds is sufficient to quantify variance
- Planning uncertainty overlays for distribution-heavy results (RL reward curves, ablations)
- User invokes
/autoskillit:vis-lens-uncertainty
Critical Constraints
NEVER:
- Modify any source code files
- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves
- Create files outside
{{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/
- Conflate SD, SE, CI95, and PI95 — they have fundamentally different interpretations
- Omit the CRITICAL flag when n_seeds == 1; single-seed variance is unquantifiable
ALWAYS:
CRITICAL: if n_seeds == 1, flag the figure as CRITICAL — single-seed variance is unquantifiable and must be reported
If n_seeds >= 10, prefer violin/box/strip over error bars to show the full distribution shape
Label every uncertainty overlay with its exact measure (SD, SE, CI95, PI95) and n in the figure caption
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool - this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/vis_spec_uncertainty_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/vis_spec_uncertainty_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1–4; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Inventory Figures and Seeds
Scan experiment plan, context file, and codebase for:
Figures with Error-Bearing Quantities
- Find all figures that aggregate over multiple runs, seeds, or samples
- Look for:
mean, average, std, stderr, ci, error_bar, errorbar, fill_between
Seed Count
- Find the number of random seeds or independent runs used
- Look for:
n_seeds, num_seeds, seeds, SEEDS, random_state, seed_list, runs
Existing Uncertainty Representation
- Find whether any uncertainty overlay exists at all
- Look for:
plt.fill_between, ax.errorbar, capsize, yerr, xerr, ci=, sd=, ci_band
Claims About Variance
- Find claims that assert robustness, stability, or statistical significance
- Look for:
significant, robust, stable, p-value, consistent, reproducible
Step 2: Determine Correct Uncertainty Measure Per Figure
For each figure that shows an aggregated quantity, determine the correct measure:
SD — Standard Deviation
- Definition: spread of the data distribution (population variability), not inference error
- When to use: when the claim is about spread of the distribution, not precision of the mean
- Formula:
σ = sqrt(1/(n-1) × Σ(xᵢ - x̄)²)
SE — Standard Error of the Mean
- Definition: SD / √n — sampling error of the estimated mean
- When to use: when the claim is about how precisely the mean is estimated
- Formula:
SE = SD / √n
CI 95% — Confidence Interval
- Definition: mean ± t(0.975, n−1) × SE — inferential, sample-size dependent
- When to use: when making statistical inference claims; shrinks with larger n
- Formula:
CI = mean ± t(0.975, n−1) × (SD / √n)
PI 95% — Prediction Interval
- Definition: interval for a future single observation (wider than CI)
- When to use: when the claim is about where a new individual result will fall
- Formula:
PI = mean ± t(0.975, n−1) × SD × sqrt(1 + 1/n)
Distribution-Aware Selection:
- n_seeds == 1 → CRITICAL: variance cannot be quantified; flag this immediately
- n_seeds 2–4 → CI95 or SE; warn that the interval is unreliable at low n
- n_seeds 5–9 → error_bar with CI95 is acceptable; consider box plot
- n_seeds ≥ 10 → prefer violin/box/strip over error bars to show full distribution shape
Step 3: Flag Critical Cases
For every figure where n_seeds == 1:
- Mark with severity: CRITICAL
- Reason: single-seed variance is unquantifiable; the result may not replicate
- Remediation: report results across ≥ 3 seeds minimum (5+ recommended)
For figures where the wrong measure is used (e.g., SD labeled as CI, or CI claimed without stating n):
- Mark with severity: WARNING
- Document the specific mislabeling and the correct interpretation
Step 4: Emit yaml:figure-spec Blocks
For each figure, emit one yaml:figure-spec fenced block (schema defined in vis-lens-chart-select)
with stat_overlay filled in. Then LOAD /autoskillit:mermaid and create the mermaid diagram.
Output Template
# Uncertainty Representation Spec: {System / Experiment Name}
**Lens:** Uncertainty Representation (Statistical)
**Question:** How is uncertainty honestly represented?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}
**n_seeds detected:** {N}
## Uncertainty Measure Summary
| Figure | n_seeds | Recommended Measure | Current Measure | Status |
|--------|---------|---------------------|-----------------|--------|
| {fig-01} | 1 | N/A — CRITICAL | none | CRITICAL |
| {fig-02} | 5 | CI95 | SE | WARNING — mislabeled |
| {fig-03} | 10 | violin | error_bar | WARNING — prefer distribution plot |
| {fig-04} | 5 | CI95 | CI95 | OK |
## Figure Specs
```yaml
# yaml:figure-spec — canonical schema (spec_version: "1.0")
figure_id: "fig-02-ablation-accuracy"
figure_title: "Ablation Study: Component Contribution"
spec_version: "1.0"
chart_type: "grouped-bar"
chart_type_fallback: "dot-plot"
perceptual_justification: "Position encoding for nominal × quantitative comparison."
data_source: "results/ablation.csv"
data_mapping:
x: "variant"
y: "accuracy"
color: "component"
size: ""
facet: ""
layout:
width_inches: 5.0
height_inches: 3.5
dpi: 300
stat_overlay:
type: "error_bar"
measure: "CI95"
n_seeds: 5
annotations: ["n=5 seeds; CI95 shown"]
anti_patterns: ["ap-bar-no-error"]
palette: "okabe-ito"
format: "pdf"
target_dpi: 300
library: "matplotlib"
report_section: "Section 5 Ablation"
priority: "P1"
placement_tier: "main"
conflicts: []
metadata:
created_by: "vis-lens-uncertainty"
reviewed_by: ""
last_updated: "{YYYY-MM-DD}"
Uncertainty Representation Diagram
%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
%% CLASS DEFINITIONS %%
classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;
subgraph Seeds ["SEED COUNT"]
N1["n_seeds = {N}<br/>━━━━━━━━━━<br/>{adequacy assessment}"]
end
subgraph Measure ["UNCERTAINTY MEASURE"]
M1["SD / SE / CI95 / PI95<br/>━━━━━━━━━━<br/>{chosen measure + formula}"]
end
subgraph Overlay ["OVERLAY TYPE"]
O1["{error_bar / violin / box}<br/>━━━━━━━━━━<br/>{distribution-aware choice}"]
end
subgraph Verdict ["VERDICT"]
V1["{OK / WARNING / CRITICAL}<br/>━━━━━━━━━━<br/>{reason}"]
end
N1 --> M1
M1 --> O1
O1 --> V1
class N1 stateNode;
class M1 cli;
class O1 output;
class V1 detector;
Color Legend:
| Color |
Category |
Description |
| Dark Teal |
Seed Count |
Number of independent runs |
| Dark Blue |
Measure |
Chosen uncertainty measure |
| Teal |
Overlay |
Distribution-aware display type |
| Red |
Verdict |
OK / WARNING / CRITICAL assessment |
---
## Pre-Diagram Checklist
Before creating the diagram, verify:
- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table
- [ ] Every CRITICAL (n_seeds == 1) figure is flagged
- [ ] Every stat_overlay has both `measure` and `n_seeds` filled in
1---2name: vis-lens-uncertainty3categories: [vis-lens]4activate_deps: [mermaid]5description: "Create Uncertainty Representation visualization planning spec showing error bar definitions, distribution-aware alternatives, and multi-seed variance protocols. Statistical lens answering \"How is uncertainty honestly represented?\""6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Uncertainty Lens - Analyzing uncertainty representation...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/vis-lens-uncertainty15---1617# Uncertainty Representation Visualization Lens1819**Philosophical Mode:** Statistical20**Primary Question:** "How is uncertainty honestly represented?"21**Focus:** Error Bar Definitions, Distribution-Aware Alternatives, Multi-Seed Variance Protocols2223## Arguments2425`/autoskillit:vis-lens-uncertainty [context_path] [experiment_plan_path]`2627- **context_path** (optional positional arg 1) — Absolute path to a lens context file28 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.29 If provided, read this file before beginning analysis to obtain structured context.30 If omitted, discover context by exploring the CWD.31- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full32 experiment plan. If provided, read for complete experimental methodology and design.33 If omitted, locate the experiment plan by exploring the CWD.3435## When to Use3637- Reviewing figures that show means, scores, or aggregate metrics without uncertainty38- Checking whether error bars are correctly defined (SD vs SE vs CI is not interchangeable)39- Evaluating whether n_seeds is sufficient to quantify variance40- Planning uncertainty overlays for distribution-heavy results (RL reward curves, ablations)41- User invokes `/autoskillit:vis-lens-uncertainty`4243## Critical Constraints4445**NEVER:**46- Modify any source code files47- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves48- Create files outside `{{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/`49- Conflate SD, SE, CI95, and PI95 — they have fundamentally different interpretations50- Omit the CRITICAL flag when n_seeds == 1; single-seed variance is unquantifiable5152**ALWAYS:**53- CRITICAL: if `n_seeds == 1`, flag the figure as **CRITICAL** — single-seed variance is unquantifiable and must be reported54- If `n_seeds >= 10`, prefer violin/box/strip over error bars to show the full distribution shape55- Label every uncertainty overlay with its exact measure (SD, SE, CI95, PI95) and n in the figure caption56- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY57- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.58- Write output to `{{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/vis_spec_uncertainty_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)59- After writing the file, emit the structured output token as **literal plain text** with no60 markdown formatting on the token name (the adjudicator performs a regex match):6162 ```63 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/vis-lens-uncertainty/vis_spec_uncertainty_{...}.md64 ```6566---6768## Analysis Workflow6970### Step 0: Parse optional arguments7172If positional arg 1 (context_path) is provided and the file exists, read it to obtain73IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional74arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full75methodology. Use this structured context as the foundation for Steps 1–4; skip the CWD76exploration for these fields if the context file supplies them.7778### Step 1: Inventory Figures and Seeds7980Scan experiment plan, context file, and codebase for:8182**Figures with Error-Bearing Quantities**83- Find all figures that aggregate over multiple runs, seeds, or samples84- Look for: `mean`, `average`, `std`, `stderr`, `ci`, `error_bar`, `errorbar`, `fill_between`8586**Seed Count**87- Find the number of random seeds or independent runs used88- Look for: `n_seeds`, `num_seeds`, `seeds`, `SEEDS`, `random_state`, `seed_list`, `runs`8990**Existing Uncertainty Representation**91- Find whether any uncertainty overlay exists at all92- Look for: `plt.fill_between`, `ax.errorbar`, `capsize`, `yerr`, `xerr`, `ci=`, `sd=`, `ci_band`9394**Claims About Variance**95- Find claims that assert robustness, stability, or statistical significance96- Look for: `significant`, `robust`, `stable`, `p-value`, `consistent`, `reproducible`9798### Step 2: Determine Correct Uncertainty Measure Per Figure99100For each figure that shows an aggregated quantity, determine the correct measure:101102**SD — Standard Deviation**103- Definition: spread of the data distribution (population variability), not inference error104- When to use: when the claim is about spread of the distribution, not precision of the mean105- Formula: `σ = sqrt(1/(n-1) × Σ(xᵢ - x̄)²)`106107**SE — Standard Error of the Mean**108- Definition: SD / √n — sampling error of the estimated mean109- When to use: when the claim is about how precisely the mean is estimated110- Formula: `SE = SD / √n`111112**CI 95% — Confidence Interval**113- Definition: mean ± t(0.975, n−1) × SE — inferential, sample-size dependent114- When to use: when making statistical inference claims; shrinks with larger n115- Formula: `CI = mean ± t(0.975, n−1) × (SD / √n)`116117**PI 95% — Prediction Interval**118- Definition: interval for a future single observation (wider than CI)119- When to use: when the claim is about where a new individual result will fall120- Formula: `PI = mean ± t(0.975, n−1) × SD × sqrt(1 + 1/n)`121122**Distribution-Aware Selection:**123- n_seeds == 1 → **CRITICAL**: variance cannot be quantified; flag this immediately124- n_seeds 2–4 → CI95 or SE; warn that the interval is unreliable at low n125- n_seeds 5–9 → error_bar with CI95 is acceptable; consider box plot126- n_seeds ≥ 10 → prefer violin/box/strip over error bars to show full distribution shape127128### Step 3: Flag Critical Cases129130For every figure where `n_seeds == 1`:131- Mark with severity: **CRITICAL**132- Reason: single-seed variance is unquantifiable; the result may not replicate133- Remediation: report results across ≥ 3 seeds minimum (5+ recommended)134135For figures where the wrong measure is used (e.g., SD labeled as CI, or CI claimed without stating n):136- Mark with severity: **WARNING**137- Document the specific mislabeling and the correct interpretation138139### Step 4: Emit yaml:figure-spec Blocks140141For each figure, emit one `yaml:figure-spec` fenced block (schema defined in vis-lens-chart-select)142with `stat_overlay` filled in. Then LOAD `/autoskillit:mermaid` and create the mermaid diagram.143144---145146## Output Template147148```markdown149# Uncertainty Representation Spec: {System / Experiment Name}150151**Lens:** Uncertainty Representation (Statistical)152**Question:** How is uncertainty honestly represented?153**Date:** {YYYY-MM-DD}154**Scope:** {What was analyzed}155**n_seeds detected:** {N}156157## Uncertainty Measure Summary158159| Figure | n_seeds | Recommended Measure | Current Measure | Status |160|--------|---------|---------------------|-----------------|--------|161| {fig-01} | 1 | N/A — CRITICAL | none | CRITICAL |162| {fig-02} | 5 | CI95 | SE | WARNING — mislabeled |163| {fig-03} | 10 | violin | error_bar | WARNING — prefer distribution plot |164| {fig-04} | 5 | CI95 | CI95 | OK |165166## Figure Specs167168```yaml169# yaml:figure-spec — canonical schema (spec_version: "1.0")170figure_id: "fig-02-ablation-accuracy"171figure_title: "Ablation Study: Component Contribution"172spec_version: "1.0"173chart_type: "grouped-bar"174chart_type_fallback: "dot-plot"175perceptual_justification: "Position encoding for nominal × quantitative comparison."176data_source: "results/ablation.csv"177data_mapping:178 x: "variant"179 y: "accuracy"180 color: "component"181 size: ""182 facet: ""183layout:184 width_inches: 5.0185 height_inches: 3.5186 dpi: 300187stat_overlay:188 type: "error_bar"189 measure: "CI95"190 n_seeds: 5191annotations: ["n=5 seeds; CI95 shown"]192anti_patterns: ["ap-bar-no-error"]193palette: "okabe-ito"194format: "pdf"195target_dpi: 300196library: "matplotlib"197report_section: "Section 5 Ablation"198priority: "P1"199placement_tier: "main"200conflicts: []201metadata:202 created_by: "vis-lens-uncertainty"203 reviewed_by: ""204 last_updated: "{YYYY-MM-DD}"205```206207## Uncertainty Representation Diagram208209```mermaid210%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%211flowchart TB212 %% CLASS DEFINITIONS %%213 classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;214 classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;215 classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;216 classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;217 classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;218 classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;219220 subgraph Seeds ["SEED COUNT"]221 N1["n_seeds = {N}<br/>━━━━━━━━━━<br/>{adequacy assessment}"]222 end223224 subgraph Measure ["UNCERTAINTY MEASURE"]225 M1["SD / SE / CI95 / PI95<br/>━━━━━━━━━━<br/>{chosen measure + formula}"]226 end227228 subgraph Overlay ["OVERLAY TYPE"]229 O1["{error_bar / violin / box}<br/>━━━━━━━━━━<br/>{distribution-aware choice}"]230 end231232 subgraph Verdict ["VERDICT"]233 V1["{OK / WARNING / CRITICAL}<br/>━━━━━━━━━━<br/>{reason}"]234 end235236 N1 --> M1237 M1 --> O1238 O1 --> V1239240 class N1 stateNode;241 class M1 cli;242 class O1 output;243 class V1 detector;244```245246**Color Legend:**247| Color | Category | Description |248|-------|----------|-------------|249| Dark Teal | Seed Count | Number of independent runs |250| Dark Blue | Measure | Chosen uncertainty measure |251| Teal | Overlay | Distribution-aware display type |252| Red | Verdict | OK / WARNING / CRITICAL assessment |253```254255---256257## Pre-Diagram Checklist258259Before creating the diagram, verify:260261- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool262- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)263- [ ] Diagram will include a color legend table264- [ ] Every CRITICAL (n_seeds == 1) figure is flagged265- [ ] Every stat_overlay has both `measure` and `n_seeds` filled in