Laft Ad Eval

laft-ad-eval

Language-Assisted Feature Transformation for Anomaly Detection — Yun et al. (2025) (arXiv:2503.01184, 2025)

What this evaluates

Evaluates a language-assisted feature transformation framework for anomaly detection. It probes the model's ability to use textual prompts to define normality boundaries and selectively suppress or emphasize specific image attributes without retraining, across both semantic and industrial anomaly detection benchmarks.

Datasets

• Colored MNIST — total ?; splits: train (-1), test (-1)
• Waterbirds — total ?; splits: train (-1), test (-1)
• CelebA — total ?; splits: train (-1), test (-1)
• MVTec AD — total ?; splits: train (-1), test (-1)
• VisA — total ?; splits: train (-1), test (-1)

Metrics

• AUROC (primary) — range: percent
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.
• AUPRC — range: percent
  • Area under the Precision-Recall Curve, summarizing the precision-recall trade-off across thresholds, particularly useful for imbalanced datasets.
• FPR95 — range: percent
  • False Positive Rate when the True Positive Rate is fixed at 95%.

Input / output format

Input: CLIP image features extracted from input images, paired with textual prompts defining normal and anomalous attribute values.

Output: Anomaly scores for each test image, converted to binary anomaly predictions based on a threshold.

Scoring recipe

def compute_metrics(scores, labels):
    fpr, tpr, _ = roc_curve(labels, scores)
    auroc = auc(fpr, tpr) * 100
    precision, recall, _ = precision_recall_curve(labels, scores)
    auprc = auc(recall, precision) * 100
    fpr95_idx = np.argmin(np.abs(tpr - 0.95))
    fpr95 = fpr[fpr95_idx] * 100
    return auroc, auprc, fpr95

Common pitfalls

• Methods relying on image-text similarity are highly sensitive to inaccurate prompts for anomalous images, leading to degraded performance.
• Ignoring irrelevant attributes during feature transformation is a unique capability but generally poses a more challenging evaluation scenario.
• Multi-scale feature extraction does not necessarily benefit semantic anomaly detection tasks compared to single-scale features.

Evidence (verbatim from paper)

Table 3: Anomaly detection AUROC (%) on MVTec AD and VisA datasets in few-shot settings. We use five different sets of reference samples from the training set. K denotes the number of reference samples. The best values are shown in bold, and the second-best values are underlined.

Citation

@misc{yun2025laft,
  title={Language-Assisted Feature Transformation for Anomaly Detection},
  author={Yun et al. (2025)},
  year={2025},
  note={arXiv:2503.01184}
}

• arXiv: 2503.01184