Titan Sfdaod Eval

titan-sfdaod-eval

TITAN: Query-Token based Domain Adaptive Adversarial Learning — Ashraf et al. (2025) (arXiv:2506.21484, 2025)

What this evaluates

Evaluates source-free domain adaptive object detection (SF-DAOD) and unsupervised domain adaptation (UDA) across natural and medical imaging domains. Probes a model's ability to align features and reduce pseudo-label noise when adapting to a target domain without access to target labels during training.

Datasets

• Cityscapes — total 3475; splits: train (2975), val (500)
• Foggy Cityscapes — total ?; splits: (unstated)
• KITTI — total 7481; splits: train (7481)
• SIM10k — total 10000; splits: train (10000)
• BDD100k — total 41986; splits: train (36728), val (5258)
• RSNA-BSD1K — total 1000; splits: (unstated)
• INBreast — total 410; splits: (unstated)
• DDSM — total 2620; splits: (unstated)

Metrics

• mAP (primary) — range: percent
  • Mean Average Precision across all object classes. AP is computed per class by integrating the precision-recall curve. Consistent with standard object detection benchmarks.
• FROC/F1/AUC — range: [0, 1]
  • FROC plots sensitivity (recall) against false positives per image (FPI). F1-score and AUC are computed for classification results. A prediction is a true positive if the center of the predicted bounding box falls within the ground truth box.

Input / output format

Input: RGB images (natural scenes or mammograms) with bounding box annotations for the source domain; target domain images provided without annotations during training.

Output: Predicted bounding boxes with class labels and confidence scores for each image.

Scoring recipe

def is_tp(pred_box, gt_box):
    return center_in_box(pred_box.center, gt_box)

aps = []
for class_id in classes:
    preds = filter_by_class(predictions, class_id)
    gts = filter_by_class(ground_truth, class_id)
    aps.append(calculate_ap(preds, gts, tp_fn=is_tp))
mAP = mean(aps)

fpi_values = [0.05, 0.3, 0.5, 1.0]
sensitivities = []
for fpi in fpi_values:
    threshold = find_threshold_for_fpi(fpi, predictions)
    sensitivities.append(calculate_sensitivity(predictions, ground_truth, threshold, tp_fn=is_tp))

Common pitfalls

• Source-free domain adaptation requires training without target labels; using target annotations during adaptation invalidates the SF-DAOD setting.
• Medical evaluation uses FROC/FPI rather than standard IoU-based AP; TP is defined by center-in-box overlap, not bounding box intersection.
• Datasets are paired in specific source-to-target configurations (e.g., C2F, R2In); results are not directly comparable across different adaptation directions.

Evidence (verbatim from paper)

For natural image datasets, we report the average precision (AP) for each individual class along with the mean average precision (mAP) score, consistent with previous studies. For medical image datasets, we utilize the Free-Response Receiver Operating Characteristic (FROC) curves to evaluate detection performance, alongside F1-score and AUC for classification results. The FROC curves visually represent the trade-off between sensitivity/recall and false positives per image (FPI). We consider a prediction to be a true positive if the center of the predicted bounding box falls within the ground truth box

Citation

@misc{ashraf2025titan,
  title={TITAN: Query-Token based Domain Adaptive Adversarial Learning},
  author={Ashraf et al. (2025)},
  year={2025},
  note={arXiv:2506.21484}
}

• arXiv: 2506.21484