Vid Ad Eval

vid-ad-eval

VID-AD: A Dataset for Image-Level Logical Anomaly Detection under Vision-Induced Distraction — Nakata et al. (2026) (arXiv:2603.13964, 2026)

What this evaluates

Probes image-level logical anomaly detection under vision-induced distractions such as background changes, blur, and low light. It tests whether models can identify violations of logical constraints (e.g., quantity, length, type, placement) by reasoning over textual descriptions rather than relying on brittle low-level visual features.

Datasets

• VID-AD — total ?; splits: train (-1), test (-1); repo https://github.com/nkthiroto/VID-AD

Metrics

• AUROC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic curve. Image-level scores are computed by taking the maximum spatial response for pixel-level methods, or aggregating reconstruction error over pixels for reconstruction-based methods.

Input / output format

Input: Images from the VID-AD dataset. For the proposed method, images are processed by a VLM preprocessor to generate scenario-specific textual descriptions. For baselines, raw images are processed according to each method's default preprocessing protocol.

Output: Image-level anomaly scores (continuous values). For pixel-level baselines, scores are derived by taking the maximum response over spatial locations; for reconstruction-based baselines, scores are aggregated reconstruction errors.

Scoring recipe

def compute_auroc(predictions, labels):
    # predictions: array of image-level anomaly scores
    # labels: binary ground truth (0=normal, 1=anomaly)
    fpr, tpr, _ = roc_curve(labels, predictions)
    return auc(fpr, tpr)

Common pitfalls

• Pixel-level anomaly maps from vision-based baselines must be converted to image-level scores by taking the maximum spatial response before computing AUROC.
• The dataset uses a one-class learning setting, meaning models are trained exclusively on normal images, which constrains how baselines are implemented and evaluated.
• Robustness is explicitly measured via scenario-wise sensitivity (standard deviation of AUROC across capture conditions), not just mean performance.

Evidence (verbatim from paper)

Image-level scoring. We report image-level anomaly scores for all methods to ensure a consistent evaluation protocol, with performance measured by AUROC (Area Under the Receiver Operating Characteristic curve). For methods producing pixel-level anomaly maps (PaDiM, PatchCore, EfficientAD, and CSAD), we convert them to an image-level score by taking the maximum response over spatial locations. For reconstruction-based methods (AnoGAN and VAE), we use the reconstruction error aggregated over pixels as the image-level score.

Citation

@misc{nakata2026vidad,
  title={VID-AD: A Dataset for Image-Level Logical Anomaly Detection under Vision-Induced Distraction},
  author={Nakata et al. (2026)},
  year={2026},
  note={arXiv:2603.13964}
}

• arXiv: 2603.13964