Model Evaluation

Model Evaluation — The Judge

A model that scores well on the wrong metric, validated with a leaky pipeline, and deployed without statistical rigor is worse than no model at all — it ships confident wrong answers. This skill covers the full evaluation lifecycle: picking the right metrics, validating honestly, comparing models with statistical discipline, checking calibration, and auditing for fairness before anything reaches production.

1. Metric Selection by Task

No single metric captures model quality. The right choice depends on task type, class balance, and what errors cost the business.

Key decision rules:

• Imbalanced classification: default to AUC-PR, MCC, or F1 on the minority class. Never report accuracy alone.
• When probabilities drive decisions (setting thresholds, expected value calculations): use log loss and calibration metrics.
• Regression with outliers: prefer MAE or median absolute error over MSE.
• Ranking with graded relevance: NDCG. Binary relevance: MAP.

See references/metrics-catalog.md for formulas, ranges, and detailed trade-off analysis.

2. Validation Strategies

The validation strategy must match the data structure. Using random k-fold on time-series data or grouped observations produces optimistic, misleading estimates.

Expanding vs sliding window for time series: Expanding window grows the training set over time (mirrors production where you retrain on all history). Sliding window uses a fixed-size recent window (better when the data distribution drifts and old data hurts).

Nested CV protocol:

1. Outer loop: k-fold split producing train/test pairs.
2. Inner loop: for each outer fold, run k-fold CV on the training portion to select hyperparameters.
3. Retrain with selected hyperparameters on the full outer training fold, evaluate on the outer test fold.
4. Report: mean and standard deviation across outer folds.

This gives an unbiased estimate of generalization performance for the tuned model.

3. Statistical Model Comparison

Reporting "Model A: 0.85, Model B: 0.83" without a significance test is not a comparison — it is anecdote. Performance differences must survive statistical scrutiny.

Practical protocol:

1. Run both models under identical CV splits (same random seed, same folds).
2. Record per-fold performance differences.
3. Apply the Nadeau-Bengio corrected t-test (or Wilcoxon if differences are non-normal).
4. For multiple model comparison across multiple datasets, use the Friedman test followed by Nemenyi post-hoc.
5. Report effect size alongside p-values — a statistically significant but tiny improvement may not justify deployment complexity.

4. Calibration

A model that outputs 0.7 probability for an event should be right about 70% of the time. Many models (gradient-boosted trees, SVMs, neural networks) produce scores that rank well but are not calibrated probabilities.

Why calibration matters: Any decision that uses predicted probabilities directly — expected cost calculations, threshold selection, risk stratification, uncertainty-aware downstream systems — requires calibration. A well-discriminating but poorly calibrated model will set thresholds incorrectly and misestimate expected outcomes.

Diagnostic tools:

• Reliability diagram: Bin predictions by predicted probability, plot mean predicted vs mean observed. A perfectly calibrated model follows the diagonal.
• Expected Calibration Error (ECE): Weighted average of per-bin |predicted - observed|. Lower is better. Typical target: ECE < 0.05.

Calibration methods:

• Platt scaling: Fit a logistic regression on the model's raw outputs using a held-out calibration set. Works well for sigmoid-shaped distortions. Two parameters.
• Isotonic regression: Non-parametric monotonic fit. More flexible than Platt scaling but needs more calibration data (risk of overfitting with <1,000 samples).
• Temperature scaling: Single parameter (temperature T) dividing logits before softmax. Popular for neural networks, especially in multi-class settings.

Protocol: Always calibrate on a held-out set, never on training data. After calibrating, re-check the reliability diagram and ECE to confirm improvement.

5. Fairness Auditing

Model quality means nothing if the model systematically harms protected groups. Fairness auditing is not optional — it is a deployment requirement.

Core fairness criteria:

The impossibility theorem: Except in trivial cases (equal base rates across groups or perfect prediction), you cannot simultaneously satisfy demographic parity, equalized odds, and predictive parity. You must choose which criterion aligns with your application's values and legal context.

Practical audit workflow:

1. Identify protected attributes (even if not used as features — proxy variables can encode them).
2. Compute all four criteria above, disaggregated by group.
3. Choose the criterion most appropriate to the domain (e.g., equal opportunity for lending, demographic parity for hiring screens).
4. Quantify disparity: ratios (e.g., adverse impact ratio > 0.8 under the 4/5 rule) or absolute differences.
5. If disparities exceed thresholds, intervene: re-sample training data, apply in-processing constraints, or post-process predictions.
6. Document decisions and trade-offs. Fairness is a policy choice, not a purely technical one.

6. Beyond Single Metrics

Single-number metrics compress too much information. Supplement them with deeper analysis.

• Confusion matrix deep dives: Go beyond aggregate rates. Segment by subpopulation, feature range, or data source. A model with 95% overall accuracy can have 40% accuracy on a critical subgroup.
• Error analysis — systematic vs random: Cluster misclassifications. If errors concentrate on specific patterns (short texts, low-resolution images, a demographic), the model has a systematic blind spot that more data or architectural changes can fix. Random errors suggest you are near the irreducible error floor.
• Learning curves: Plot training and validation performance vs training set size. Diverging curves (high training, low validation) signal high variance — get more data or regularize. Converging low curves signal high bias — increase model capacity.
• Prediction interval coverage: For regression, check that your 90% prediction intervals actually contain 90% of observations. Under-coverage means your uncertainty estimates are overconfident.

Common Mistakes

1. Reporting accuracy on imbalanced data. A 95/5 class split gives 95% accuracy by predicting the majority class every time. Use AUC-PR, F1, or MCC instead.

2. Data leakage in cross-validation. Fitting a scaler, selecting features, or oversampling (SMOTE) on the full dataset before splitting. All preprocessing that uses statistics from data must happen inside each fold.

3. Optimizing a proxy metric that diverges from business value. Improving log loss by 0.01 means nothing if the business cares about precision at a specific operating threshold. Map ML metrics to business KPIs explicitly.

4. Comparing models without statistical tests. A 2-point difference on one random split is noise. Use corrected resampled t-tests or Bayesian comparison on multiple folds.

5. Ignoring calibration before deploying probability-based decisions. Ranking ability (AUC) does not guarantee calibrated probabilities. Platt scaling or isotonic regression is cheap insurance.

6. Treating fairness as a post-hoc checkbox. Auditing for fairness after the model is built limits your options. Build fairness considerations into the problem formulation, data collection, and metric selection from the start.

Implementation Libraries

Recommended starting stack (Python): sklearn.metrics + sklearn.model_selection for core evaluation, optuna for tuning, yellowbrick for visual diagnostics, fairlearn for fairness audits.

When This Applies

Reference this skill when you are:

• Choosing which metric(s) to report for a modeling project
• Designing a cross-validation strategy for a new dataset
• Deciding between two or more candidate models
• Preparing a model for deployment and need calibrated probabilities
• Conducting a fairness or bias review before launch
• Performing error analysis to diagnose why a model underperforms
• Writing a model card or evaluation report for stakeholders

For detailed metric formulas, ranges, and trade-off analysis, see references/metrics-catalog.md.

Cross-Domain Connections

• Investing/adaptive-monitoring/performance-attribution: Model evaluation metrics map to investment performance metrics — calibration (predicted probabilities = true outcomes) parallels risk model calibration (predicted VaR = realized losses). Cross-validation strategies (expanding window, sliding window) ARE walk-forward backtesting of trading strategies.
• Investing/risk-architecture/tail-risk: Evaluating model performance in tails is critical for financial risk — standard metrics (RMSE, accuracy) fail when the cost of errors is asymmetric. Tail-risk evaluation requires metrics that weight extreme outcomes appropriately.

Related Skills

• quality-assessment — Both formalize "how good is the output" at different levels: model-evaluation for ML model performance against ground truth; quality-assessment for tasting/sensory evaluation. Shared concept (rubric-grounded scoring), different domains.