# Ds Model Fairness > "Evaluates and mitigates fairness issues including bias detection, fairness metrics, and debiasing strategies in machine learning" - Skill: `paulpas/ds-model-fairness` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-model-fairness` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-model-fairness/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-model-fairness --- # Model Fairness Comprehensive guide to model fairness in machine learning and data science workflows. ## When to Use This Skill - Solving real-world reproducibility & responsible ai problems - Building machine learning pipelines with model fairness - Implementing best practices for model fairness - Optimizing model performance using model fairness techniques - Learning industry-standard approaches to model fairness ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require model fairness rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Model Fairness is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Model Fairness ```python import pandas as pd import numpy as np from sklearn.datasets import make_classification from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score # Generate synthetic dataset with a sensitive attribute X, y = make_classification(n_samples=1000, n_features=5, n_informative=3, random_state=42) sensitive_attr = np.random.randint(0, 2, size=1000) df = pd.DataFrame(X, columns=[f'feat_{i}' for i in range(5)]) df['target'] = y df['sensitive'] = sensitive_attr # Split data train_df, test_df = train_test_split(df, test_size=0.2, random_state=42) # Train baseline model model = LogisticRegression(random_state=42) model.fit(train_df.drop(columns=['target', 'sensitive']), train_df['target']) # Predictions y_pred = model.predict(test_df.drop(columns=['target', 'sensitive'])) # Calculate Demographic Parity Difference def calc_demographic_parity_diff(y_true, y_pred, sensitive): groups = np.unique(sensitive) rates = [np.mean(y_pred[sensitive == g]) for g in groups] return max(rates) - min(rates) dp_diff = calc_demographic_parity_diff(test_df['target'], y_pred, test_df['sensitive']) print(f"Demographic Parity Difference: {dp_diff:.4f}") ``` ### Pattern 2: Production-Ready Model Fairness ```python import logging import pandas as pd import numpy as np from typing import Dict, Any, List from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.datasets import make_classification logger = logging.getLogger(__name__) class ModelFairnessAuditor: """Production-grade fairness auditing and debiasing class""" def __init__(self, sensitive_features: str, target_col: str, threshold: float = 0.5): self.sensitive_features = sensitive_features self.target_col = target_col self.threshold = threshold self.model = None self.results: Dict[str, Any] = {} def _validate_input(self, df: pd.DataFrame) -> None: if df is None or df.empty: raise ValueError("Input DataFrame cannot be None or empty") required_cols = [self.sensitive_features, self.target_col] missing = [col for col in required_cols if col not in df.columns] if missing: raise ValueError(f"Missing required columns: {missing}") def _calculate_metrics(self, y_true: np.ndarray, y_pred: np.ndarray, sensitive: np.ndarray) -> Dict[str, float]: groups = np.unique(sensitive) if len(groups) < 2: return {"error": "Need at least two groups for fairness metrics"} group_rates = {g: np.mean(y_pred[sensitive == g]) for g in groups} dp_diff = max(group_rates.values()) - min(group_rates.values()) tp_rates = {g: np.mean(y_pred[sensitive == g] & y_true == 1) / max(np.sum(y_true == 1), 1) for g in groups} eod = max(tp_rates.values()) - min(tp_rates.values()) return {"demographic_parity_diff": dp_diff, "equal_opportunity_diff": eod} def execute(self, data: pd.DataFrame) -> Dict[str, Any]: self._validate_input(data) try: X = data.drop(columns=[self.target_col, self.sensitive_features]) y = data[self.target_col] sensitive = data[self.sensitive_features] X_train, X_test, y_train, y_test, s_train, s_test = train_test_split( X, y, sensitive, test_size=0.2, random_state=42 ) self.model = RandomForestClassifier(n_estimators=50, random_state=42) self.model.fit(X_train, y_train) y_pred = self.model.predict(X_test) metrics = self._calculate_metrics(y_test.values, y_pred, s_test.values) self.results = { "status": "success" "metrics": metrics "predictions": y_pred.tolist() "model_type": "RandomForestClassifier" } logger.info(f"Fairness audit complete. DP Diff: {metrics['demographic_parity_diff']:.4f}") return self.results except Exception as e: logger.error(f"Audit failed: {str(e)}") return {"status": "error", "message": str(e)} ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Validate all data preprocessing steps are fit-only on training data, never on validation or test sets - Implement reproducible pipelines with fixed random seeds and deterministic operations where possible - Report model performance with confidence intervals via bootstrapping or cross-validation across multiple runs - Log all experiments with parameters, metrics, and artifacts using MLflow or equivalent tracking system ### MUST NOT DO - Do not evaluate a model on the same data used for training — always hold out a proper test set - Avoid overfitting to the validation set by limiting hyperparameter search iterations - Never use features that can only be computed at inference time (look-ahead bias) - Do not report single-run accuracy without statistical significance testing or error bars ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Fairlearn Documentation](https://fairlearn.org/) - [Fairness in Machine Learning — Wikipedia](https://en.wikipedia.org/wiki/Fairness_(machine_learning)) - [AI Fairness 360 (IBM)](https://aif360.res.ibm.com/) - [Google What-If Tool](https://pair.withgoogle.com/tool/) - [Explainable AI for ML — Microsoft Azure](https://learn.microsoft.com/en-us/azure/machine-learning/concept-explanation-of-models)