Data Poisoning Eval

data-poisoning-eval

Detecting and Preventing Data Poisoning Attacks on AI Models — Kure et al. (2025) (arXiv:2503.09302, 2025)

What this evaluates

This evaluation probes the robustness of machine learning models against data poisoning attacks by measuring classification accuracy degradation and recovery under label flipping and image replacement attacks. It assesses how well statistical anomaly detection, adversarial training, and ensemble learning defenses mitigate performance drops and false prediction rates.

Datasets

• CIFAR-10 — total 60000; splits: train (-1), test (-1); HF cifar10
• Insurance Claims — total 1000000; splits: train (-1), test (-1)

Metrics

• classification accuracy (primary) — range: percent
  • Number of correctly classified instances divided by the total number of instances.
• false positive/negative rate — range: percent
  • Proportion of incorrectly predicted positive or negative instances relative to actual conditions.

Input / output format

Input: Image tensors (CIFAR-10) or tabular feature vectors (insurance claims) representing training or test samples.

Output: Predicted class labels (e.g., 10 object categories or fraud/legitimate).

Scoring recipe

def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return correct / len(gold)

Common pitfalls

• Poisoning is applied selectively to specific classes ('cat' and 'dog') rather than uniformly, which skews class balance and affects per-class accuracy differently.
• The provided excerpt reports aggregate accuracy drops and defense recovery rates but lacks explicit per-split sizes and exact threshold settings for anomaly detection, making direct replication of exact numbers difficult.

Evidence (verbatim from paper)

As a result, models trained on this poisoned data exhibit reduced accuracy and struggle with correct class identification. This demonstrates the adverse impact that even minor data poisoning can have on model performance.

Citation

@misc{kure2025detecting,
  title={Detecting and Preventing Data Poisoning Attacks on AI Models},
  author={Kure et al. (2025)},
  year={2025},
  note={arXiv:2503.09302}
}

• arXiv: 2503.09302