Governance Risk Experimental Design Lens
Philosophical Mode: Governance
Primary Question: "What risks arise from acting on this result?"
Focus: Deployment Risks, Subgroup Harms, Monitoring Plans, Limitation Disclosure, Responsible Decision-Making
Arguments
/autoskillit:exp-lens-governance-risk [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- AI evaluation with deployment implications
- Experiments whose results will affect real users
- Safety-relevant benchmarks
- User invokes
/autoskillit:exp-lens-governance-risk or /autoskillit:make-experiment-diag governance
Critical Constraints
NEVER:
- Modify any source code files
- Create files outside
{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/
- Run subagents in the background (
run_in_background: true is prohibited)
ALWAYS:
Identify subgroups for whom the experimental evidence may not generalize
Assess decision sufficiency — does the experiment actually answer the deployment question?
Treat absent limitation disclosure as a finding requiring explicit flagging
Distinguish risks that are monitored from risks that are merely acknowledged
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool - this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{YYYY-MM-DD_HHMMSS}.md
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Launch Parallel Exploration Subagents
Spawn Explore subagents to investigate:
Intended Use & Deployment Context
- Find intended deployment scenario and audience
- Look for: deploy, production, use_case, audience, user, stakeholder, decision
Subgroup & Fairness Analysis
- Find evidence of subgroup analysis or fairness evaluation
- Look for: subgroup, demographic, fairness, equity, bias, disaggregate, protected
Harm & Risk Metrics
- Find safety or harm metrics tracked
- Look for: harm, safety, risk, adverse, negative, side_effect, failure_mode
Monitoring & Feedback Plans
- Find post-deployment monitoring or feedback loops
- Look for: monitor, alert, feedback, drift, rollback, incident, threshold, canary
Limitation Disclosure
- Find explicit acknowledgment of limitations
- Look for: limitation, caveat, not_suitable, generalize, scope, restriction, caveat
Step 2: Build Risk Register
For each potential action: Who is affected? What could go wrong? Severity? Likelihood? Monitoring? Evidence?
Classify by severity × likelihood.
Step 3: Analyze Decision Sufficiency
For every deployment decision: Does the experiment provide sufficient evidence? What additional evidence is needed? Are there subgroups with insufficient evidence?
Step 4: Create the Diagram (Optional)
Direction: TB. Results → Decisions → Stakeholder Impacts
Step 5: Write Output
Write the output to: {{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
Responsible Deployment Checklist
Pre-Diagram Checklist
Before creating the diagram, verify:
Related Skills
/autoskillit:make-experiment-diag - Parent skill
/autoskillit:mermaid - MUST BE LOADED before creating diagram
/autoskillit:exp-lens-validity-threats
/autoskillit:exp-lens-measurement-validity
1---2name: exp-lens-governance-risk3categories: [exp-lens]4activate_deps: [mermaid]5description: Create a risk register and stakeholder impact assessment for experiments with deployment implications. Governance lens answering "What risks arise from acting on this result?"6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Governance Risk Lens - Analyzing deployment risks and stakeholder impacts...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/exp-lens-governance-risk15---1617# Governance Risk Experimental Design Lens1819**Philosophical Mode:** Governance20**Primary Question:** "What risks arise from acting on this result?"21**Focus:** Deployment Risks, Subgroup Harms, Monitoring Plans, Limitation Disclosure, Responsible Decision-Making2223## Arguments2425`/autoskillit:exp-lens-governance-risk [context_path] [experiment_plan_path]`2627- **context_path** (optional positional arg 1) — Absolute path to a lens context file28 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.29 If provided, read this file before beginning analysis to obtain structured context.30 If omitted, discover context by exploring the CWD.31- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full32 experiment plan. If provided, read for complete experimental methodology and design.33 If omitted, locate the experiment plan by exploring the CWD.3435## When to Use3637- AI evaluation with deployment implications38- Experiments whose results will affect real users39- Safety-relevant benchmarks40- User invokes `/autoskillit:exp-lens-governance-risk` or `/autoskillit:make-experiment-diag governance`4142## Critical Constraints4344**NEVER:**45- Modify any source code files46- Create files outside `{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/`47- Run subagents in the background (`run_in_background: true` is prohibited)4849**ALWAYS:**50- Identify subgroups for whom the experimental evidence may not generalize51- Assess decision sufficiency — does the experiment actually answer the deployment question?52- Treat absent limitation disclosure as a finding requiring explicit flagging53- Distinguish risks that are monitored from risks that are merely acknowledged54- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY55- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.56- Write output to `{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{YYYY-MM-DD_HHMMSS}.md`57- After writing the file, emit the structured output token as **literal plain text** with no58 markdown formatting on the token name (the adjudicator performs a regex match):5960 ```61 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{...}.md62 ```6364---6566## Analysis Workflow6768### Step 0: Parse optional arguments6970If positional arg 1 (context_path) is provided and the file exists, read it to obtain71IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional72arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full73methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD74exploration for these fields if the context file supplies them.7576### Step 1: Launch Parallel Exploration Subagents7778Spawn Explore subagents to investigate:7980**Intended Use & Deployment Context**81- Find intended deployment scenario and audience82- Look for: deploy, production, use_case, audience, user, stakeholder, decision8384**Subgroup & Fairness Analysis**85- Find evidence of subgroup analysis or fairness evaluation86- Look for: subgroup, demographic, fairness, equity, bias, disaggregate, protected8788**Harm & Risk Metrics**89- Find safety or harm metrics tracked90- Look for: harm, safety, risk, adverse, negative, side_effect, failure_mode9192**Monitoring & Feedback Plans**93- Find post-deployment monitoring or feedback loops94- Look for: monitor, alert, feedback, drift, rollback, incident, threshold, canary9596**Limitation Disclosure**97- Find explicit acknowledgment of limitations98- Look for: limitation, caveat, not_suitable, generalize, scope, restriction, caveat99100### Step 2: Build Risk Register101102For each potential action: Who is affected? What could go wrong? Severity? Likelihood? Monitoring? Evidence?103Classify by severity × likelihood.104105### Step 3: Analyze Decision Sufficiency106107For every deployment decision: Does the experiment provide sufficient evidence? What additional evidence is needed? Are there subgroups with insufficient evidence?108109### Step 4: Create the Diagram (Optional)110111**Direction:** TB. Results → Decisions → Stakeholder Impacts112113### Step 5: Write Output114115Write the output to: `{{AUTOSKILLIT_TEMP}}/exp-lens-governance-risk/exp_diag_governance_risk_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)116117---118119## Responsible Deployment Checklist120121- [ ] Subgroup performance disaggregated and analyzed122- [ ] Deployment context matches experimental conditions123- [ ] Monitoring plan defined with specific thresholds124- [ ] Rollback criteria specified125- [ ] Limitations disclosed to decision-makers126- [ ] Affected communities consulted where applicable127128---129130## Pre-Diagram Checklist131132Before creating the diagram, verify:133134- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool135- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)136- [ ] Diagram will include a color legend table137138---139140## Related Skills141142- `/autoskillit:make-experiment-diag` - Parent skill143- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram144- `/autoskillit:exp-lens-validity-threats`145- `/autoskillit:exp-lens-measurement-validity`