AI Ethics Validator
Overview
Validate AI/ML models and datasets for bias, fairness, and ethical compliance using quantitative fairness metrics and structured audit workflows.
Prerequisites
- Python 3.9+ with Fairlearn >= 0.9 (
pip install fairlearn)
- IBM AI Fairness 360 toolkit (
pip install aif360) for comprehensive bias analysis
- pandas, NumPy, and scikit-learn for data manipulation and model evaluation
- Model predictions (probabilities or binary labels) and corresponding ground truth labels
- Demographic attribute columns (age, gender, race, etc.) accessible under appropriate data governance
- Optional: Google What-If Tool for interactive fairness exploration on TensorFlow models
Instructions
- Load the model predictions and ground truth dataset using the Read tool; verify schema includes sensitive attribute columns
- Define the protected attributes and privileged/unprivileged group definitions for the fairness analysis
- Compute representation statistics: group counts, class label distributions, and feature coverage per demographic segment
- Calculate core fairness metrics using Fairlearn or AIF360:
- Demographic parity ratio (selection rate parity across groups)
- Equalized odds difference (TPR and FPR parity)
- Equal opportunity difference (TPR parity only)
- Predictive parity (precision parity across groups)
- Calibration scores per group (predicted probability vs observed outcome)
- Apply four-fifths rule: flag any metric where the ratio falls below 0.80 as potential adverse impact
- Classify each finding by severity: low (ratio 0.90-1.0), medium (0.80-0.90), high (0.70-0.80), critical (below 0.70)
- Identify proxy variables by computing correlation between non-protected features and sensitive attributes
- Generate mitigation recommendations: resampling, reweighting, threshold adjustment, or in-processing constraints (e.g.,
ExponentiatedGradient from Fairlearn)
- Produce a compliance assessment mapping findings to IEEE Ethically Aligned Design, EU Ethics Guidelines for Trustworthy AI, and ACM Code of Ethics
- Document all ethical decisions, trade-offs, and residual risks in a structured audit report
Output
- Fairness metric dashboard: per-group values for demographic parity, equalized odds, equal opportunity, predictive parity, and calibration
- Severity-classified findings table: metric name, affected groups, ratio value, severity level, recommended action
- Representation analysis: group sizes, class distributions, feature coverage gaps
- Proxy variable report: features correlated with protected attributes above threshold (r > 0.3)
- Mitigation plan: ranked strategies with expected fairness improvement and accuracy trade-off estimates
- Compliance matrix: pass/fail against IEEE, EU, and ACM ethical guidelines with evidence citations
Error Handling
| Error |
Cause |
Solution |
| Insufficient group sample size |
Fewer than 30 observations in a demographic group |
Aggregate related subgroups; use bootstrap confidence intervals; flag metric as unreliable |
| Missing sensitive attributes |
Protected attribute columns absent from dataset |
Apply proxy detection via correlated features; request attribute access under data governance approval |
| Conflicting fairness criteria |
Demographic parity and equalized odds contradict |
Document the impossibility theorem trade-off; prioritize the metric most aligned with the deployment context |
| Data quality failures |
Inconsistent encoding or null values in attribute columns |
Standardize categorical encodings; impute or exclude nulls; validate with schema checks before analysis |
| Model output format mismatch |
Predictions not in expected probability or binary format |
Convert logits to probabilities via sigmoid; binarize at the decision threshold before metric computation |
Examples
Scenario 1: Hiring Model Audit -- Validate a resume-screening classifier for gender and age bias. Compute demographic parity across male/female groups and age buckets (18-30, 31-50, 51+). Apply the four-fifths rule. Finding: female selection rate at 0.72 of male rate (critical severity). Recommend reweighting training samples and adjusting the decision threshold.
Scenario 2: Credit Scoring Fairness -- Assess a credit approval model for racial disparate impact. Calculate equalized odds (TPR and FPR) across racial groups. Finding: FPR for Group A is 2.1x Group B (high severity). Recommend in-processing constraint using ExponentiatedGradient with FalsePositiveRateParity.
Scenario 3: Healthcare Risk Prediction -- Evaluate a patient risk model for age and socioeconomic bias. Compute calibration curves per group. Finding: model overestimates risk for low-income patients by 15%. Recommend recalibration using Platt scaling per subgroup with post-deployment monitoring for fairness drift.
Resources
1---
2name: validating-ai-ethics-and-fairness-2
3description: |
4 Validate AI/ML models and datasets for bias, fairness, and ethical concerns.
5 Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection.
6 Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics".
7
8allowed-tools: Read, Write, Edit, Grep, Glob, Bash(python:*)
9version: 1.0.0
10author: Jeremy Longshore <jeremy@intentsolutions.io>
11license: MIT
12compatible-with: claude-code, codex, openclaw
13tags: [ai, compliance, audit]
14
15---
16
17# AI Ethics Validator
18
19## Overview
20
21Validate AI/ML models and datasets for bias, fairness, and ethical compliance using quantitative fairness metrics and structured audit workflows.
22
23## Prerequisites
24
25- Python 3.9+ with Fairlearn >= 0.9 (`pip install fairlearn`)
26- IBM AI Fairness 360 toolkit (`pip install aif360`) for comprehensive bias analysis
27- pandas, NumPy, and scikit-learn for data manipulation and model evaluation
28- Model predictions (probabilities or binary labels) and corresponding ground truth labels
29- Demographic attribute columns (age, gender, race, etc.) accessible under appropriate data governance
30- Optional: Google What-If Tool for interactive fairness exploration on TensorFlow models
31
32## Instructions
33
341. Load the model predictions and ground truth dataset using the Read tool; verify schema includes sensitive attribute columns
352. Define the protected attributes and privileged/unprivileged group definitions for the fairness analysis
363. Compute representation statistics: group counts, class label distributions, and feature coverage per demographic segment
374. Calculate core fairness metrics using Fairlearn or AIF360:
38 - Demographic parity ratio (selection rate parity across groups)
39 - Equalized odds difference (TPR and FPR parity)
40 - Equal opportunity difference (TPR parity only)
41 - Predictive parity (precision parity across groups)
42 - Calibration scores per group (predicted probability vs observed outcome)
435. Apply four-fifths rule: flag any metric where the ratio falls below 0.80 as potential adverse impact
446. Classify each finding by severity: low (ratio 0.90-1.0), medium (0.80-0.90), high (0.70-0.80), critical (below 0.70)
457. Identify proxy variables by computing correlation between non-protected features and sensitive attributes
468. Generate mitigation recommendations: resampling, reweighting, threshold adjustment, or in-processing constraints (e.g., `ExponentiatedGradient` from Fairlearn)
479. Produce a compliance assessment mapping findings to IEEE Ethically Aligned Design, EU Ethics Guidelines for Trustworthy AI, and ACM Code of Ethics
4810. Document all ethical decisions, trade-offs, and residual risks in a structured audit report
49
50## Output
51
52- Fairness metric dashboard: per-group values for demographic parity, equalized odds, equal opportunity, predictive parity, and calibration
53- Severity-classified findings table: metric name, affected groups, ratio value, severity level, recommended action
54- Representation analysis: group sizes, class distributions, feature coverage gaps
55- Proxy variable report: features correlated with protected attributes above threshold (r > 0.3)
56- Mitigation plan: ranked strategies with expected fairness improvement and accuracy trade-off estimates
57- Compliance matrix: pass/fail against IEEE, EU, and ACM ethical guidelines with evidence citations
58
59## Error Handling
60
61| Error | Cause | Solution |
62|-------|-------|----------|
63| Insufficient group sample size | Fewer than 30 observations in a demographic group | Aggregate related subgroups; use bootstrap confidence intervals; flag metric as unreliable |
64| Missing sensitive attributes | Protected attribute columns absent from dataset | Apply proxy detection via correlated features; request attribute access under data governance approval |
65| Conflicting fairness criteria | Demographic parity and equalized odds contradict | Document the impossibility theorem trade-off; prioritize the metric most aligned with the deployment context |
66| Data quality failures | Inconsistent encoding or null values in attribute columns | Standardize categorical encodings; impute or exclude nulls; validate with schema checks before analysis |
67| Model output format mismatch | Predictions not in expected probability or binary format | Convert logits to probabilities via sigmoid; binarize at the decision threshold before metric computation |
68
69## Examples
70
71**Scenario 1: Hiring Model Audit** -- Validate a resume-screening classifier for gender and age bias. Compute demographic parity across male/female groups and age buckets (18-30, 31-50, 51+). Apply the four-fifths rule. Finding: female selection rate at 0.72 of male rate (critical severity). Recommend reweighting training samples and adjusting the decision threshold.
72
73**Scenario 2: Credit Scoring Fairness** -- Assess a credit approval model for racial disparate impact. Calculate equalized odds (TPR and FPR) across racial groups. Finding: FPR for Group A is 2.1x Group B (high severity). Recommend in-processing constraint using `ExponentiatedGradient` with `FalsePositiveRateParity`.
74
75**Scenario 3: Healthcare Risk Prediction** -- Evaluate a patient risk model for age and socioeconomic bias. Compute calibration curves per group. Finding: model overestimates risk for low-income patients by 15%. Recommend recalibration using Platt scaling per subgroup with post-deployment monitoring for fairness drift.
76
77## Resources
78
79- [Fairlearn Documentation](https://fairlearn.org/) -- bias detection, mitigation algorithms, MetricFrame API
80- [AI Fairness 360 (AIF360)](https://aif360.mybluemix.net/) -- comprehensive fairness toolkit with 70+ metrics
81- [Google What-If Tool](https://pair-code.github.io/what-if-tool/) -- interactive fairness exploration
82- [EU Ethics Guidelines for Trustworthy AI](https://ec.europa.eu/digital-strategy/en/policies/expert-group-ai) -- regulatory framework
83- [IEEE Ethically Aligned Design](https://ethicsinaction.ieee.org/) -- technical ethics standards
84- Impossibility theorem reference: Chouldechova (2017) on incompatibility of fairness criteria
jeremylongshore