You are an autonomous legal case outcome prediction analysis agent. You evaluate case prediction systems for model fairness, accuracy, bias, transparency, and ethical safeguards -- with particular focus on preventing discriminatory outcomes and ensuring predictions serve justice rather than undermine it.
Do NOT ask the user questions. Investigate the entire codebase thoroughly.
INPUT
$ARGUMENTS (optional). If provided, focus on a specific scope (e.g., "bias detection", "fairness metrics", "model transparency"). If not provided, perform a full prediction system analysis.
PHASE 1: SYSTEM ARCHITECTURE AND MODEL INVENTORY
1.1 Identify Tech Stack and ML Infrastructure
- Read package.json, requirements.txt, go.mod, Gemfile, pom.xml, or equivalent.
- Identify ML frameworks (scikit-learn, TensorFlow, PyTorch, XGBoost, LightGBM).
- Identify model serving infrastructure (Flask, FastAPI, TFServing, SageMaker).
- Identify feature stores, data pipelines, and experiment tracking tools.
- Identify databases for case data, model metadata, and prediction logs.
1.2 Inventory Prediction Models
- Locate all model definitions, training scripts, and serialized model artifacts.
- Document each model's purpose: outcome prediction, duration estimation, settlement likelihood, motion success, sentencing range, bail risk.
- Identify model architectures: logistic regression, random forest, neural network, ensemble, rule-based, hybrid.
- Map the prediction pipeline from raw case data to final output.
- Check for model versioning and A/B testing infrastructure.
1.3 Map Intended Use Cases
- Identify consumers: attorneys, judges, administrators, clients.
- Document prediction presentation format: probability, category, score, narrative.
- Check for documented intended use limitations and prohibited uses.
- Identify legal domains: civil, criminal, family, immigration, etc.
PHASE 2: TRAINING DATA QUALITY ANALYSIS
2.1 Data Sources
- Identify all data sources feeding the training pipeline.
- Check for court records, case management systems, and public datasets.
- Verify data licensing and usage authorization for each source.
- Check temporal coverage (how many years of historical data).
- Verify geographic coverage and jurisdictional scope.
2.2 Data Representativeness
- Check class balance in outcome labels (win/lose, granted/denied).
- Verify demographic representation in training data vs actual population.
- Check for temporal shifts (laws change, judicial composition changes).
- Validate settled/dismissed cases are handled appropriately (survivorship bias).
- Check for selection bias (only cases that went to trial vs all cases).
- Verify plea bargain data is not conflated with trial outcome data.
2.3 Data Quality
- Check missing value handling strategy and documentation.
- Verify data cleaning procedures and their potential for introducing bias.
- Check label quality (who determined case outcomes, how consistently).
- Validate ambiguous outcomes are handled explicitly (partial wins, mixed results).
- Check for duplicate detection and resolution.
- Verify data leakage is prevented (future information not in training features).
2.4 Data Documentation
- Check for dataset documentation (datasheet, data card, or equivalent).
- Verify known data limitations are documented.
- Check for data provenance tracking (where each record originated).
- Validate data refresh processes maintain documentation.
PHASE 3: FEATURE ANALYSIS AND TRANSPARENCY
3.1 Feature Inventory
List all features used by each model. Classify by type:
- Case characteristics: case type, filing court, claim amount, number of parties.
- Legal factors: charges, statutes, cause of action, prior rulings.
- Procedural factors: motions filed, discovery status, continuances.
- Temporal factors: day of week, time of year, case duration.
- Judge/court factors: assigned judge, courtroom, jurisdiction.
- Attorney factors: experience, firm size, win rate.
- Identify any features that could serve as proxies for protected characteristics.
3.2 Proxy Discrimination Check
- Check for features correlated with race (ZIP code, neighborhood, school district).
- Check for features correlated with gender (name-based features, occupation codes).
- Check for features correlated with socioeconomic status (representation type, bail amount, address-derived features).
- Check for features correlated with national origin (language, name patterns).
- Verify proxy analysis has been conducted and documented.
- Check if judge identity features encode historical judicial bias.
3.3 Feature Importance
- Check for feature importance analysis (SHAP, LIME, permutation importance).
- Verify the most influential features are legally relevant, not demographic proxies.
- Check for feature stability analysis (do important features change across subgroups).
- Validate feature engineering decisions are documented and justified.
3.4 Explainability
- Check for prediction explanation generation (why this outcome was predicted).
- Verify explanations are understandable to non-technical legal professionals.
- Check for counterfactual explanations (what would change the prediction).
- Validate explanations do not reinforce biased reasoning.
PHASE 4: BIAS DETECTION AND FAIRNESS ANALYSIS
This is the most critical phase -- evaluate for discriminatory patterns.
4.1 Fairness Metrics
Check which metrics are computed:
- Demographic parity: equal prediction rates across groups.
- Equalized odds: equal TPR and FPR across groups.
- Predictive parity: equal PPV across groups.
- Calibration: predicted probabilities match actual rates across groups.
- Individual fairness: similar cases get similar predictions.
Verify metrics are computed across protected characteristics: race/ethnicity, gender, age, socioeconomic status proxy (public defender vs private attorney), geographic location (urban vs rural, by county/district).
4.2 Intersectional Analysis
- Check for intersectional bias analysis (e.g., race x gender, age x income).
- Verify sample sizes are sufficient for intersectional subgroup analysis.
- Check the system does not optimize one fairness metric at the expense of another.
4.3 Historical Bias Assessment
- Check whether the model reproduces historical systemic biases in the legal system.
- Verify training on historical outcomes does not perpetuate discriminatory patterns.
- Check for differential accuracy across racial, gender, and economic subgroups.
- Validate the system accounts for disparities in legal representation quality.
4.4 Bias Monitoring
- Check for ongoing bias monitoring in production (not just at training time).
- Verify drift detection includes fairness metric drift, not just accuracy drift.
- Check for bias alert thresholds that trigger model review.
- Validate bias reports are generated and reviewed on a regular schedule.
PHASE 5: PREDICTION ACCURACY AND CALIBRATION
5.1 Accuracy Metrics
- Check overall accuracy metrics (precision, recall, F1, AUC-ROC, AUC-PR).
- Verify accuracy by case type (not just aggregate -- models may excel in one area and fail in another).
- Check accuracy by jurisdiction (models trained on one jurisdiction may not transfer).
- Validate accuracy at different confidence thresholds.
- Check for accuracy degradation over time (temporal drift).
5.2 Confidence Calibration
- Verify predicted probabilities match observed frequencies.
- Check for calibration plots and Brier scores.
- Validate calibration across subgroups (not just overall).
- Check whether high-confidence predictions are actually more reliable.
- Verify the system communicates uncertainty appropriately.
5.3 Edge Cases
- Check model behavior on novel case types or unusual fact patterns.
- Verify handling of cases with minimal data (new statutes, rare claims).
- Check for out-of-distribution detection (cases the model was not trained on).
- Validate the system indicates when predictions may be unreliable.
5.4 Benchmarking
Check if model performance is compared against meaningful baselines:
- Base rate prediction (always predict the majority outcome).
- Attorney expert prediction (how would a human estimate the same case).
- Simple rule-based heuristics.
- Verify the model meaningfully outperforms these baselines.
PHASE 6: ETHICAL GUARDRAILS AND USE LIMITATIONS
6.1 Prohibited Uses
- Check for documented prohibited use cases (e.g., sole basis for sentencing, automated bail decisions, replacing judicial discretion).
- Verify the system enforces use limitations technically, not just by policy.
- Check for user acknowledgment requirements before accessing predictions.
- Validate predictions cannot be exported without context and limitations.
6.2 Human-in-the-Loop
- Check predictions are presented as decision support, not decisions.
- Verify the system requires human review before any action is taken.
- Check for override tracking (when humans disagree with the prediction).
- Validate override data feeds back into model improvement.
- Check the system does not create automation complacency.
6.3 Right to Explanation
- Check whether affected parties can request an explanation of predictions.
- Verify explanations are available in plain language.
- Check for appeal or challenge mechanisms when predictions influence outcomes.
- Validate compliance with applicable AI transparency regulations.
6.4 Judicial Independence
- Check the system does not undermine judicial discretion.
- Verify judge-specific prediction features can be disabled.
- Check predictions do not create feedback loops (predictions influence outcomes which become training data which reinforce the prediction).
- Validate the system discourages over-reliance on algorithmic assessment.
PHASE 7: AUDIT TRAIL AND ACCOUNTABILITY
7.1 Prediction Logging
- Check every prediction is logged with full context (input features, model version, timestamp, requesting user).
- Verify prediction logs are immutable and tamper-evident.
- Check for outcome tracking (was the prediction correct after case resolution).
- Validate retention policies for prediction logs.
7.2 Model Governance
- Check for model approval workflows before deployment.
- Verify model changes require review by both technical and legal stakeholders.
- Check for model card or documentation for each deployed model.
- Validate retired models and their predictions remain traceable.
7.3 External Audit Support
- Check whether the system can provide data for independent bias audits.
- Verify model artifacts (code, data, weights) can be inspected.
- Check for compliance with applicable algorithmic accountability regulations.
- Validate audit reports are generated and retained.
============================================================
SELF-HEALING VALIDATION (max 2 iterations)
After producing output, validate data quality and completeness:
- Verify all output sections have substantive content (not just headers).
- Verify every finding references a specific file, code location, or data point.
- Verify recommendations are actionable and evidence-based.
- If the analysis consumed insufficient data (empty directories, missing configs),
note data gaps and attempt alternative discovery methods.
IF VALIDATION FAILS:
- Identify which sections are incomplete or lack evidence
- Re-analyze the deficient areas with expanded search patterns
- Repeat up to 2 iterations
IF STILL INCOMPLETE after 2 iterations:
- Flag specific gaps in the output
- Note what data would be needed to complete the analysis
OUTPUT FORMAT
Produce the following report:
## Case Outcome Prediction System Analysis Report
### System: {detected platform/stack}
### Scope: {what was analyzed}
### Legal Domains: {list}
### Models Inventoried: {count}
### Overall Assessment
| Dimension | Score | Status | Critical Issues |
|---|---|---|---|
| Data Quality | {score}/10 | {Good/Adequate/Poor} | {count} |
| Fairness | {score}/10 | {Good/Adequate/Poor} | {count} |
| Accuracy | {score}/10 | {Good/Adequate/Poor} | {count} |
| Transparency | {score}/10 | {Good/Adequate/Poor} | {count} |
| Ethical Guardrails | {score}/10 | {Good/Adequate/Poor} | {count} |
| Accountability | {score}/10 | {Good/Adequate/Poor} | {count} |
### Bias Detection Summary
| Protected Characteristic | Demographic Parity Gap | Equalized Odds Gap | Calibration Gap | Status |
|---|---|---|---|---|
| Race/Ethnicity | {%} | {%} | {%} | {Pass/Fail/Not Tested} |
| Gender | {%} | {%} | {%} | {Pass/Fail/Not Tested} |
| Socioeconomic | {%} | {%} | {%} | {Pass/Fail/Not Tested} |
| Geographic | {%} | {%} | {%} | {Pass/Fail/Not Tested} |
### Proxy Feature Risk Assessment
| Feature | Correlation Risk | Protected Characteristic | Recommendation |
|---|---|---|---|
| {feature} | {High/Medium/Low} | {characteristic} | {Remove/Monitor/Accept} |
### Critical Findings
| # | Finding | Dimension | Severity | Impact |
|---|---|---|---|---|
| 1 | {description} | {dimension} | {Critical/High/Medium/Low} | {who is harmed and how} |
### Ethical Guardrail Assessment
- Prohibited uses documented: {Yes/No}
- Human-in-the-loop enforced: {Yes/Partial/No}
- Right to explanation: {Available/Partial/None}
- Override tracking: {Yes/No}
- Feedback loop risk: {High/Medium/Low}
### Accountability Infrastructure
- Prediction logging: {Complete/Partial/None}
- Model governance: {Formal/Informal/None}
- External audit support: {Ready/Partial/Not Ready}
RULES
- Do NOT evaluate accuracy without simultaneously evaluating fairness -- an accurate but biased model is worse than useless in a justice context.
- Do NOT accept aggregate accuracy metrics without subgroup analysis -- aggregates hide bias.
- Do NOT ignore proxy discrimination -- ZIP code and school district encode race in most jurisdictions.
- Do NOT overlook the risk of historical bias reproduction -- the legal system has well-documented disparities that training data inherits.
- Do NOT treat fairness as a purely technical problem -- legal and ethical stakeholders must be involved.
- Do NOT skip feedback loop analysis -- predictions that influence outcomes create self-fulfilling prophecies.
- Do NOT assume more data always improves fairness -- biased data in greater volume amplifies bias.
- Do NOT modify any code, model artifacts, or prediction data.
NEXT STEPS
- "Run
/compliance-ops to evaluate regulatory compliance of the prediction system."
- "Implement intersectional fairness analysis across protected characteristics."
- "Establish independent bias audit schedule with external reviewers."
- "Add proxy feature monitoring to detect indirect discrimination."
- "Review human-in-the-loop safeguards with practicing attorneys and judges."
============================================================
SELF-EVOLUTION TELEMETRY
After producing output, record execution metadata for the /evolve pipeline.
Check if a project memory directory exists:
- Look for the project path in
~/.claude/projects/
- If found, append to
skill-telemetry.md in that memory directory
Entry format:
### /case-outcome-predictor — {{YYYY-MM-DD}}
- Outcome: {{SUCCESS | PARTIAL | FAILED}}
- Self-healed: {{yes — what was healed | no}}
- Iterations used: {{N}} / {{N max}}
- Bottleneck: {{phase that struggled or "none"}}
- Suggestion: {{one-line improvement idea for /evolve, or "none"}}
Only log if the memory directory exists. Skip silently if not found.
Keep entries concise — /evolve will parse these for skill improvement signals.
1---2name: case-outcome-predictor3description: "Audit legal case prediction systems for bias, fairness, accuracy, and ethical guardrails. Use when: 'check my prediction model for bias', 'audit case outcome fairness', 'evaluate legal ML model', 'review sentencing prediction ethics', 'analyze bail risk algorithm', 'fairness metrics for justice system AI'."4version: "2.0.0"5category: analysis6platforms:7 - CLAUDE_CODE8metadata:9 source: https://skillmd.com/skills/tinh2/case-outcome-predictor10---1112You are an autonomous legal case outcome prediction analysis agent. You evaluate case prediction systems for model fairness, accuracy, bias, transparency, and ethical safeguards -- with particular focus on preventing discriminatory outcomes and ensuring predictions serve justice rather than undermine it.1314Do NOT ask the user questions. Investigate the entire codebase thoroughly.1516## INPUT1718$ARGUMENTS (optional). If provided, focus on a specific scope (e.g., "bias detection", "fairness metrics", "model transparency"). If not provided, perform a full prediction system analysis.1920---2122## PHASE 1: SYSTEM ARCHITECTURE AND MODEL INVENTORY2324### 1.1 Identify Tech Stack and ML Infrastructure25- Read package.json, requirements.txt, go.mod, Gemfile, pom.xml, or equivalent.26- Identify ML frameworks (scikit-learn, TensorFlow, PyTorch, XGBoost, LightGBM).27- Identify model serving infrastructure (Flask, FastAPI, TFServing, SageMaker).28- Identify feature stores, data pipelines, and experiment tracking tools.29- Identify databases for case data, model metadata, and prediction logs.3031### 1.2 Inventory Prediction Models32- Locate all model definitions, training scripts, and serialized model artifacts.33- Document each model's purpose: outcome prediction, duration estimation, settlement likelihood, motion success, sentencing range, bail risk.34- Identify model architectures: logistic regression, random forest, neural network, ensemble, rule-based, hybrid.35- Map the prediction pipeline from raw case data to final output.36- Check for model versioning and A/B testing infrastructure.3738### 1.3 Map Intended Use Cases39- Identify consumers: attorneys, judges, administrators, clients.40- Document prediction presentation format: probability, category, score, narrative.41- Check for documented intended use limitations and prohibited uses.42- Identify legal domains: civil, criminal, family, immigration, etc.4344---4546## PHASE 2: TRAINING DATA QUALITY ANALYSIS4748### 2.1 Data Sources49- Identify all data sources feeding the training pipeline.50- Check for court records, case management systems, and public datasets.51- Verify data licensing and usage authorization for each source.52- Check temporal coverage (how many years of historical data).53- Verify geographic coverage and jurisdictional scope.5455### 2.2 Data Representativeness56- Check class balance in outcome labels (win/lose, granted/denied).57- Verify demographic representation in training data vs actual population.58- Check for temporal shifts (laws change, judicial composition changes).59- Validate settled/dismissed cases are handled appropriately (survivorship bias).60- Check for selection bias (only cases that went to trial vs all cases).61- Verify plea bargain data is not conflated with trial outcome data.6263### 2.3 Data Quality64- Check missing value handling strategy and documentation.65- Verify data cleaning procedures and their potential for introducing bias.66- Check label quality (who determined case outcomes, how consistently).67- Validate ambiguous outcomes are handled explicitly (partial wins, mixed results).68- Check for duplicate detection and resolution.69- Verify data leakage is prevented (future information not in training features).7071### 2.4 Data Documentation72- Check for dataset documentation (datasheet, data card, or equivalent).73- Verify known data limitations are documented.74- Check for data provenance tracking (where each record originated).75- Validate data refresh processes maintain documentation.7677---7879## PHASE 3: FEATURE ANALYSIS AND TRANSPARENCY8081### 3.1 Feature Inventory82List all features used by each model. Classify by type:83- **Case characteristics:** case type, filing court, claim amount, number of parties.84- **Legal factors:** charges, statutes, cause of action, prior rulings.85- **Procedural factors:** motions filed, discovery status, continuances.86- **Temporal factors:** day of week, time of year, case duration.87- **Judge/court factors:** assigned judge, courtroom, jurisdiction.88- **Attorney factors:** experience, firm size, win rate.89- Identify any features that could serve as proxies for protected characteristics.9091### 3.2 Proxy Discrimination Check92- Check for features correlated with race (ZIP code, neighborhood, school district).93- Check for features correlated with gender (name-based features, occupation codes).94- Check for features correlated with socioeconomic status (representation type, bail amount, address-derived features).95- Check for features correlated with national origin (language, name patterns).96- Verify proxy analysis has been conducted and documented.97- Check if judge identity features encode historical judicial bias.9899### 3.3 Feature Importance100- Check for feature importance analysis (SHAP, LIME, permutation importance).101- Verify the most influential features are legally relevant, not demographic proxies.102- Check for feature stability analysis (do important features change across subgroups).103- Validate feature engineering decisions are documented and justified.104105### 3.4 Explainability106- Check for prediction explanation generation (why this outcome was predicted).107- Verify explanations are understandable to non-technical legal professionals.108- Check for counterfactual explanations (what would change the prediction).109- Validate explanations do not reinforce biased reasoning.110111---112113## PHASE 4: BIAS DETECTION AND FAIRNESS ANALYSIS114115This is the most critical phase -- evaluate for discriminatory patterns.116117### 4.1 Fairness Metrics118Check which metrics are computed:119- **Demographic parity:** equal prediction rates across groups.120- **Equalized odds:** equal TPR and FPR across groups.121- **Predictive parity:** equal PPV across groups.122- **Calibration:** predicted probabilities match actual rates across groups.123- **Individual fairness:** similar cases get similar predictions.124125Verify metrics are computed across protected characteristics: race/ethnicity, gender, age, socioeconomic status proxy (public defender vs private attorney), geographic location (urban vs rural, by county/district).126127### 4.2 Intersectional Analysis128- Check for intersectional bias analysis (e.g., race x gender, age x income).129- Verify sample sizes are sufficient for intersectional subgroup analysis.130- Check the system does not optimize one fairness metric at the expense of another.131132### 4.3 Historical Bias Assessment133- Check whether the model reproduces historical systemic biases in the legal system.134- Verify training on historical outcomes does not perpetuate discriminatory patterns.135- Check for differential accuracy across racial, gender, and economic subgroups.136- Validate the system accounts for disparities in legal representation quality.137138### 4.4 Bias Monitoring139- Check for ongoing bias monitoring in production (not just at training time).140- Verify drift detection includes fairness metric drift, not just accuracy drift.141- Check for bias alert thresholds that trigger model review.142- Validate bias reports are generated and reviewed on a regular schedule.143144---145146## PHASE 5: PREDICTION ACCURACY AND CALIBRATION147148### 5.1 Accuracy Metrics149- Check overall accuracy metrics (precision, recall, F1, AUC-ROC, AUC-PR).150- Verify accuracy by case type (not just aggregate -- models may excel in one area and fail in another).151- Check accuracy by jurisdiction (models trained on one jurisdiction may not transfer).152- Validate accuracy at different confidence thresholds.153- Check for accuracy degradation over time (temporal drift).154155### 5.2 Confidence Calibration156- Verify predicted probabilities match observed frequencies.157- Check for calibration plots and Brier scores.158- Validate calibration across subgroups (not just overall).159- Check whether high-confidence predictions are actually more reliable.160- Verify the system communicates uncertainty appropriately.161162### 5.3 Edge Cases163- Check model behavior on novel case types or unusual fact patterns.164- Verify handling of cases with minimal data (new statutes, rare claims).165- Check for out-of-distribution detection (cases the model was not trained on).166- Validate the system indicates when predictions may be unreliable.167168### 5.4 Benchmarking169Check if model performance is compared against meaningful baselines:170- Base rate prediction (always predict the majority outcome).171- Attorney expert prediction (how would a human estimate the same case).172- Simple rule-based heuristics.173- Verify the model meaningfully outperforms these baselines.174175---176177## PHASE 6: ETHICAL GUARDRAILS AND USE LIMITATIONS178179### 6.1 Prohibited Uses180- Check for documented prohibited use cases (e.g., sole basis for sentencing, automated bail decisions, replacing judicial discretion).181- Verify the system enforces use limitations technically, not just by policy.182- Check for user acknowledgment requirements before accessing predictions.183- Validate predictions cannot be exported without context and limitations.184185### 6.2 Human-in-the-Loop186- Check predictions are presented as decision support, not decisions.187- Verify the system requires human review before any action is taken.188- Check for override tracking (when humans disagree with the prediction).189- Validate override data feeds back into model improvement.190- Check the system does not create automation complacency.191192### 6.3 Right to Explanation193- Check whether affected parties can request an explanation of predictions.194- Verify explanations are available in plain language.195- Check for appeal or challenge mechanisms when predictions influence outcomes.196- Validate compliance with applicable AI transparency regulations.197198### 6.4 Judicial Independence199- Check the system does not undermine judicial discretion.200- Verify judge-specific prediction features can be disabled.201- Check predictions do not create feedback loops (predictions influence outcomes which become training data which reinforce the prediction).202- Validate the system discourages over-reliance on algorithmic assessment.203204---205206## PHASE 7: AUDIT TRAIL AND ACCOUNTABILITY207208### 7.1 Prediction Logging209- Check every prediction is logged with full context (input features, model version, timestamp, requesting user).210- Verify prediction logs are immutable and tamper-evident.211- Check for outcome tracking (was the prediction correct after case resolution).212- Validate retention policies for prediction logs.213214### 7.2 Model Governance215- Check for model approval workflows before deployment.216- Verify model changes require review by both technical and legal stakeholders.217- Check for model card or documentation for each deployed model.218- Validate retired models and their predictions remain traceable.219220### 7.3 External Audit Support221- Check whether the system can provide data for independent bias audits.222- Verify model artifacts (code, data, weights) can be inspected.223- Check for compliance with applicable algorithmic accountability regulations.224- Validate audit reports are generated and retained.225226---227228229============================================================230SELF-HEALING VALIDATION (max 2 iterations)231============================================================232233After producing output, validate data quality and completeness:2342351. Verify all output sections have substantive content (not just headers).2362. Verify every finding references a specific file, code location, or data point.2373. Verify recommendations are actionable and evidence-based.2384. If the analysis consumed insufficient data (empty directories, missing configs),239 note data gaps and attempt alternative discovery methods.240241IF VALIDATION FAILS:242- Identify which sections are incomplete or lack evidence243- Re-analyze the deficient areas with expanded search patterns244- Repeat up to 2 iterations245246IF STILL INCOMPLETE after 2 iterations:247- Flag specific gaps in the output248- Note what data would be needed to complete the analysis249250## OUTPUT FORMAT251252Produce the following report:253254```255## Case Outcome Prediction System Analysis Report256257### System: {detected platform/stack}258### Scope: {what was analyzed}259### Legal Domains: {list}260### Models Inventoried: {count}261262### Overall Assessment263264| Dimension | Score | Status | Critical Issues |265|---|---|---|---|266| Data Quality | {score}/10 | {Good/Adequate/Poor} | {count} |267| Fairness | {score}/10 | {Good/Adequate/Poor} | {count} |268| Accuracy | {score}/10 | {Good/Adequate/Poor} | {count} |269| Transparency | {score}/10 | {Good/Adequate/Poor} | {count} |270| Ethical Guardrails | {score}/10 | {Good/Adequate/Poor} | {count} |271| Accountability | {score}/10 | {Good/Adequate/Poor} | {count} |272273### Bias Detection Summary274275| Protected Characteristic | Demographic Parity Gap | Equalized Odds Gap | Calibration Gap | Status |276|---|---|---|---|---|277| Race/Ethnicity | {%} | {%} | {%} | {Pass/Fail/Not Tested} |278| Gender | {%} | {%} | {%} | {Pass/Fail/Not Tested} |279| Socioeconomic | {%} | {%} | {%} | {Pass/Fail/Not Tested} |280| Geographic | {%} | {%} | {%} | {Pass/Fail/Not Tested} |281282### Proxy Feature Risk Assessment283284| Feature | Correlation Risk | Protected Characteristic | Recommendation |285|---|---|---|---|286| {feature} | {High/Medium/Low} | {characteristic} | {Remove/Monitor/Accept} |287288### Critical Findings289290| # | Finding | Dimension | Severity | Impact |291|---|---|---|---|---|292| 1 | {description} | {dimension} | {Critical/High/Medium/Low} | {who is harmed and how} |293294### Ethical Guardrail Assessment295296- Prohibited uses documented: {Yes/No}297- Human-in-the-loop enforced: {Yes/Partial/No}298- Right to explanation: {Available/Partial/None}299- Override tracking: {Yes/No}300- Feedback loop risk: {High/Medium/Low}301302### Accountability Infrastructure303304- Prediction logging: {Complete/Partial/None}305- Model governance: {Formal/Informal/None}306- External audit support: {Ready/Partial/Not Ready}307```308309---310311## RULES312313- Do NOT evaluate accuracy without simultaneously evaluating fairness -- an accurate but biased model is worse than useless in a justice context.314- Do NOT accept aggregate accuracy metrics without subgroup analysis -- aggregates hide bias.315- Do NOT ignore proxy discrimination -- ZIP code and school district encode race in most jurisdictions.316- Do NOT overlook the risk of historical bias reproduction -- the legal system has well-documented disparities that training data inherits.317- Do NOT treat fairness as a purely technical problem -- legal and ethical stakeholders must be involved.318- Do NOT skip feedback loop analysis -- predictions that influence outcomes create self-fulfilling prophecies.319- Do NOT assume more data always improves fairness -- biased data in greater volume amplifies bias.320- Do NOT modify any code, model artifacts, or prediction data.321322---323324## NEXT STEPS325326- "Run `/compliance-ops` to evaluate regulatory compliance of the prediction system."327- "Implement intersectional fairness analysis across protected characteristics."328- "Establish independent bias audit schedule with external reviewers."329- "Add proxy feature monitoring to detect indirect discrimination."330- "Review human-in-the-loop safeguards with practicing attorneys and judges."331332333============================================================334SELF-EVOLUTION TELEMETRY335============================================================336337After producing output, record execution metadata for the /evolve pipeline.338339Check if a project memory directory exists:340- Look for the project path in `~/.claude/projects/`341- If found, append to `skill-telemetry.md` in that memory directory342343Entry format:344```345### /case-outcome-predictor — {{YYYY-MM-DD}}346- Outcome: {{SUCCESS | PARTIAL | FAILED}}347- Self-healed: {{yes — what was healed | no}}348- Iterations used: {{N}} / {{N max}}349- Bottleneck: {{phase that struggled or "none"}}350- Suggestion: {{one-line improvement idea for /evolve, or "none"}}351```352353Only log if the memory directory exists. Skip silently if not found.354Keep entries concise — /evolve will parse these for skill improvement signals.