qhjqhj00 cmte-xray-detection-eval
Superpowering Open-Vocabulary Object Detectors for X-ray Vision — Garcia-Fernandez et al. (2025) (arXiv:2503.17071, 2025)
What this evaluates
Evaluates the zero-shot cross-modality transfer capability of open-vocabulary object detectors from RGB to X-ray imaging. It measures how well pre-trained RGB detectors can localize and classify objects in X-ray images without any fine-tuning or labeled X-ray data.
Datasets
- DET-COMPASS — total ?; splits: test (-1)
- PIXray — total ?; splits: test (-1)
- PIDray — total ?; splits: test (-1)
- CLCXray — total ?; splits: test (-1)
- DvXray — total ?; splits: test (-1)
- HiXray — total ?; splits: test (-1)
Metrics
AP(primary) — range: percent- Average Precision computed over IoU thresholds from 0.50 to 0.95 in steps of 0.05 and across three object area sizes (small, medium, large), averaged uniformly. Standard MS COCO metric.
AP50— range: percent- Average Precision computed at a single IoU threshold of 0.50.
AP75— range: percent- Average Precision computed at a single IoU threshold of 0.75.
Input / output format
Input: X-ray images and a fixed vocabulary of object class names. The model processes the image without any gradient updates or fine-tuning on the target modality.
Output: Bounding box coordinates, predicted class labels, and confidence scores for each detected object per image.
Scoring recipe
def compute_coco_ap(predictions, ground_truths):
# predictions: list of dicts with 'bbox', 'category_id', 'score'
# ground_truths: list of dicts with 'bbox', 'category_id', 'iscrowd'
# Uses standard COCO API to match predictions to GTs across IoU thresholds [0.50:0.05:0.95]
# Computes precision-recall curves per category and area size
# Returns mean AP across all categories, IoUs, and areas
return coco_eval.evaluate(predictions, ground_truths)['AP']
Common pitfalls
- Assuming the protocol allows fine-tuning on target X-ray data (CMTE is strictly training-free).
- Using raw web-retrieved X-ray images for descriptor construction degrades performance; filtering and material transfer are required.
- Confusing Cross-Modality Transfer Evaluation (CMTE) with standard Cross-Dataset Transfer Evaluation (CDTE), which assumes the same imaging modality.
Evidence (verbatim from paper)
Under CMTE, an OvOD model is trained on a source RGB dataset and subsequently evaluated on target X-ray datasets without any additional training or fine-tuning. For performance assessment, we employ the standard MS COCO metrics: AP, AP50, and AP75.
Citation
@misc{garciafernandez2025superpowering,
title={Superpowering Open-Vocabulary Object Detectors for X-ray Vision},
author={Garcia-Fernandez et al. (2025)},
year={2025},
note={arXiv:2503.17071}
}
- arXiv: 2503.17071