Ema Auditing Eval

ema-auditing-eval

EMA: Auditing Data Removal from Trained Models — Huang et al. (2021) (arXiv:2109.03675, 2021)

What this evaluates

This evaluation probes a trained model's data memorization by auditing whether specific query images were included in its training set. It measures the ability to correctly distinguish memorized data from non-memorized data under varying calibration set qualities and query dataset constraints.

Datasets

• MNIST — total 60000; splits: train (10000), calibration (1000), query (-1); HF mnist
• SVHN — total 73257; splits: train (-1), query (-1); HF svhn
• Chest X-ray — total ?; splits: query (-1)

Metrics

• auditing score (ρ_EMA / ρ_KS) (primary) — range: other
  • Computes per-sample memorization indicators from model predictions, then aggregates them across a query dataset using a statistical test (e.g., t-test) to produce a single audit score. Scores are evaluated by counting true/false positives and negatives to assess memorization detection accuracy.

Input / output format

Input: Model predictions (e.g., confidence scores, negative entropy) for each image in a query dataset.

Output: A single aggregated auditing score per query dataset, used to classify the dataset as memorized or not memorized.

Scoring recipe

indicators = [compute_memorization_indicator(pred) for pred in predictions]
audit_score = t_test_aggregate(indicators)
predicted_label = 1 if audit_score > threshold else 0
return accuracy(predicted_label, gold)

Common pitfalls

• Calibration set quality heavily impacts baseline KS scores; varying noise/rotation percentages (k%) is necessary to simulate realistic low-quality calibration.
• Query datasets must be carefully constructed to include both memorized and non-memorized samples to properly measure false positive/negative rates.
• The choice of statistical aggregation test affects the final audit score and should be ablated, as noted in the appendix.

Evidence (verbatim from paper)

Table 2: Auditing scores of both methods on benchmark datasets. Each column corresponds to a query dataset, and each row corresponds to a calibration set with quality controlled by k. False positive results are in red, while false negative results are in blue.

Citation

@misc{huang2021ema,
  title={EMA: Auditing Data Removal from Trained Models},
  author={Huang et al. (2021)},
  year={2021},
  note={arXiv:2109.03675}
}

• arXiv: 2109.03675