Loom Model Evaluation

Model Evaluation

Overview

Choose metrics that match the business cost and data distribution, prevent leakage, validate with the right CV scheme, calibrate and threshold deliberately, and monitor for drift in production. This skill is the decision layer above sklearn/eval tooling.

Metric selection (the highest-leverage decision)

Wrong metric = confidently shipping a bad model. Pick from the cost structure and class balance, not habit.

• ROC-AUC vs PR-AUC: ROC-AUC is invariant to class balance (baseline 0.5 always) — misleadingly optimistic when positives are rare. PR-AUC's baseline = positive prevalence, so it exposes the hard problem. Report PR-AUC for imbalanced detection.
• average= trap (sklearn): weighted/micro hide minority-class failure; use macro (or per-class) when minority classes matter. Default binary assumes label 1 is positive.
• MCC (Matthews correlation) is the most robust single scalar for imbalanced binary — high only when all four confusion cells are good.
• Always report a baseline (majority-class, random-stratified, or last-value for time series). A metric without a baseline is uninterpretable.

Data leakage (the most common silent invalidator)

Leakage makes offline metrics great and production terrible. Hunt for it explicitly:

⚠ The tell: near-perfect validation scores, or a feature with implausibly high importance. Investigate before celebrating.

Cross-validation — match the scheme to the data

from sklearn.model_selection import StratifiedKFold, GroupKFold, TimeSeriesSplit
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

# Preprocessing INSIDE the pipeline → refit per fold, no leakage
pipe = make_pipeline(StandardScaler(), estimator)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)  # classification default

• Classification: StratifiedKFold (preserves class ratio per fold) — plain KFold can leave a rare class absent from a fold.
• Grouped data: StratifiedGroupKFold — no entity in two folds.
• Time series: TimeSeriesSplit (expanding/rolling window, forward-chaining) — never shuffle; test is always after train. Add an embargo/gap between train and test if features use look-back windows.
• Small data / model selection: nested CV (outer loop = performance estimate, inner loop = hyperparameter search). Tuning and estimating on the same CV inflates the score.
• Report mean ± std across folds; a large std means the estimate is unstable (small data / leakage / non-stationarity), not that the mean is trustworthy.
• Never touch the test set until final reporting. Repeatedly tuning against the validation set turns it into training data (validation overfitting).

Calibration (probabilities must mean what they say)

A model can rank well (high AUC) yet be badly calibrated — critical when the score feeds a threshold, expected-value decision, or is shown to users.

• Diagnose: reliability curve (sklearn.calibration.calibration_curve) + Brier score + ECE (expected calibration error). Perfect calibration = predicted 0.7 is correct 70% of the time.
• Fix: CalibratedClassifierCV — Platt/sigmoid (parametric, good for small data or SVM-shaped distortions) or isotonic (non-parametric, needs more data, can overfit small sets). Fit calibration on a held-out set, not training data.
• ⚠ Tree ensembles and SVMs are often miscalibrated (over/under-confident); neural nets with modern training tend to be over-confident. Don't assume predict_proba is a real probability.

Threshold selection (separate from training)

The 0.5 default is almost never optimal for imbalanced or asymmetric-cost problems. Choose the threshold on validation data, driven by cost:

• Maximize expected utility: threshold = argmax(TP·v_tp − FP·c_fp − FN·c_fn …) over the PR/ROC curve.
• Or pick recall@precision-floor (SLA style), or Youden's J on ROC, or F-beta max.
• Tune the threshold on validation, then report the fixed threshold's metrics on test. Choosing it on test is leakage.

Overfitting / underfitting diagnosis

• Learning curve (score vs training-set size): gap that closes with more data ⇒ overfitting curable by data; both-low-and-flat ⇒ bias, need a better model.
• Validation curve (score vs one hyperparameter): locate the bias/variance sweet spot.
• Sanity check: overfit a single tiny batch first — if the model can't reach ~0 loss there, the training loop/loss is broken.

Hyperparameter tuning

• Grid (small spaces) → random (better in high dimensions; most params don't matter) → Bayesian/Optuna (sample-efficient) → Hyperband/ASHA (early-stop bad trials). Learning rate is usually the highest-impact knob.
• Search log-scale for LR/regularization; select on validation; final metric on the untouched test set.

LLM / generative evaluation

• Reference-based (BLEU/ROUGE/METEOR/exact-match): cheap but weak — penalize valid paraphrases; BERTScore is better on semantics. Use for regression testing, not quality ceilings.
• LLM-as-judge biases (must mitigate — judges are not neutral):
  • Position bias: favors the first (or a fixed side) option → run both orders, average, or count only if consistent.
  • Verbosity bias: prefers longer answers → constrain a rubric on correctness, control for length.
  • Self-preference bias: a model rates its own family higher → use a different-family judge for cross-model comparisons.
  • Sycophancy / leniency and scale compression (clusters at 4–5/5) → prefer pairwise comparisons over absolute 1–5 scores; use a concrete rubric.
• Practice: judge at temperature 0; give an explicit rubric with anchors; require reasoning-before-score (CoT); anchor to human labels on a sample and measure judge↔human agreement (Cohen's κ) before trusting it at scale.
• Task-level: pass@k for code, factuality/groundedness for RAG (does the answer cite retrieved context?), safety/toxicity/refusal-rate as guardrails.

A/B testing for model comparison

• Peeking is the #1 error: checking a fixed-horizon test repeatedly and stopping at significance inflates false-positive rate far above α. Fix: pre-compute sample size + fixed horizon, or use sequential/always-valid methods (mSPRT, group-sequential, Bayesian) designed for continuous monitoring.
• Sample Ratio Mismatch (SRM): observed split ≠ intended (e.g. 48/52 vs 50/50) → chi-square test; a failing SRM invalidates the experiment (assignment/logging bug).
• Multiple comparisons: testing many metrics/segments inflates false positives → Bonferroni/BH correction; pre-register the primary metric.
• Guardrails: latency, error rate, and fairness must not regress even if the primary metric wins.
• Novelty/primacy effects and seasonality → run ≥1–2 full business cycles; don't decide on day one.
• Report effect size + CI, not just p<0.05. Statistical significance ≠ practical significance.

Fairness

• Group metrics: selection rate, TPR, FPR, precision per protected group.
• Definitions (mutually incompatible — pick per context): demographic parity (equal selection rate), equalized odds (equal TPR & FPR), equal opportunity (equal TPR), calibration within groups.
• ⚠ You generally cannot satisfy all simultaneously (impossibility results) except in degenerate cases — choose based on harm model and document the tradeoff.

Production monitoring / drift

• Data drift (input distribution shifts, label unchanged): per-feature PSI (>0.1 moderate, >0.2 significant), KS test, KL divergence. Covariate shift.
• Concept drift (P(y|x) changes — the relationship itself): detect via prediction-distribution shift + delayed ground-truth performance tracking; needs retraining, not just re-weighting.
• Prediction drift: score distribution moves even before labels arrive — an early warning when labels are delayed.
• Performance monitoring: track the live metric vs baseline; alert on threshold; set meaningful thresholds to avoid alert fatigue. Watch training/serving skew (different feature code paths).
• Deploy via shadow mode → canary/gradual rollout → automated rollback on regression.

Reference examples

Classification metrics + confusion analysis:

from sklearn.metrics import (classification_report, roc_auc_score,
                             average_precision_score, matthews_corrcoef, confusion_matrix)

report = classification_report(y_true, y_pred, digits=4)            # per-class + macro/weighted
pr_auc = average_precision_score(y_true, y_prob)                    # PR-AUC — use for imbalance
roc_auc = roc_auc_score(y_true, y_prob)
mcc = matthews_corrcoef(y_true, y_pred)                             # robust single scalar
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
specificity, sensitivity = tn / (tn + fp), tp / (tp + fn)

Leak-free CV with per-fold preprocessing and overfit gap:

from sklearn.model_selection import cross_validate
res = cross_validate(pipe, X, y, cv=cv,
                     scoring=["f1_macro", "average_precision"],
                     return_train_score=True, n_jobs=-1)
gap = res["train_f1_macro"].mean() - res["test_f1_macro"].mean()   # large gap ⇒ overfitting

Data drift (KS + PSI):

from scipy.stats import ks_2samp
import numpy as np

def psi(expected, actual, bins=10):
    q = np.quantile(expected, np.linspace(0, 1, bins + 1))
    q[0], q[-1] = -np.inf, np.inf
    e = np.histogram(expected, q)[0] / len(expected) + 1e-6
    a = np.histogram(actual, q)[0] / len(actual) + 1e-6
    return np.sum((a - e) * np.log(a / e))          # >0.2 ⇒ significant shift

ks_stat, p = ks_2samp(baseline_feature, current_feature)  # p < 0.05 ⇒ drift

A/B analysis (report effect size + CI, not just p):

from scipy import stats
t, p = stats.ttest_ind(treatment, control)            # fixed-horizon; DON'T peek
lift = (treatment.mean() - control.mean()) / control.mean() * 100
pooled = np.sqrt((control.var(ddof=1) + treatment.var(ddof=1)) / 2)
cohens_d = (treatment.mean() - control.mean()) / pooled

Common pitfalls

1. Accuracy on imbalanced data; ROC-AUC where PR-AUC is needed.
2. Any leakage (preprocessing on full data, temporal shuffle, group straddling, target encoding across folds).
3. Tuning on the test set / peeking at A/B tests / validation overfitting.
4. Reporting point estimates without CIs or a baseline.
5. Trusting predict_proba without checking calibration; using the 0.5 threshold blindly.
6. LLM-as-judge without debiasing (position/verbosity/self-preference) or human anchoring.
7. Training/serving skew and un-monitored drift after deployment.

Checklist — before declaring a model evaluated

• Metric matches class balance + business cost (PR-AUC/F-beta/MCC for imbalance, not accuracy)
• Baseline reported (majority/random/last-value)
• No leakage: preprocessing fit inside folds; temporal/group/duplicate splits correct
• CV scheme fits the data (stratified / grouped / time-series); mean ± std reported
• Test set untouched until final; hyperparameters tuned on validation only (nested CV if small)
• Calibration checked if probabilities are used downstream; threshold chosen on validation by cost
• Learning/validation curves inspected for over/underfitting
• Fairness assessed across protected groups with a documented definition
• (LLM) judge debiased + human-anchored; (A/B) sample size pre-set, SRM checked, no peeking
• Drift + performance monitoring wired for production

Resources

Metrics: scikit-learn, HF evaluate. Fairness: Fairlearn, AIF360. LLM eval: HELM, lm-evaluation-harness, RAGAS. A/B: sequential-testing / experimentation platforms. Monitoring: Evidently, WhyLabs, NannyML.