Tsbow Eval

tsbow-eval

TSBOW: Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions — Huynh et al. (2026) (arXiv:2602.05414, 2026)

What this evaluates

Evaluates object detection models on traffic surveillance footage under diverse weather conditions and varying degrees of vehicle occlusion. It probes robustness to environmental degradation, scale variation, and dense urban traffic scenarios.

Datasets

• TSBOW — total 29621; splits: train (-1), val (-1), test (-1); repo https://github.com/SKKUAutoLab/TSBOW

Metrics

• mAP50 (primary) — range: [0, 1]
  • Mean Average Precision at an IoU threshold of 0.5. Computed as the mean of AP across all object classes, where AP is the area under the precision-recall curve.
• mAP50-95 — range: [0, 1]
  • Mean Average Precision averaged over IoU thresholds from 0.50 to 0.95 in steps of 0.05. Reflects localization accuracy across varying strictness.
• Precision — range: [0, 1]
  • Ratio of true positive detections to the total number of detections (true positives + false positives) at a confidence threshold of 0.5.
• Recall — range: [0, 1]
  • Ratio of true positive detections to the total number of ground truth instances.

Input / output format

Input: RGB frames extracted from traffic surveillance videos, resized to 1280 pixels resolution.

Output: Bounding box coordinates, class labels (e.g., car, bus, pedestrian, micromobility, truck), and confidence scores.

Scoring recipe

def compute_metrics(preds, gts, iou_thresh=0.5, conf_thresh=0.5):
    tp, fp, fn = 0, 0, 0
    for pred in preds:
        if pred.conf < conf_thresh: continue
        best_iou = max(box_iou(pred.box, gt.box) for gt in gts)
        if best_iou >= iou_thresh: tp += 1
        else: fp += 1
    fn = len(gts) - tp
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    ap = compute_ap_curve(precision, recall)
    return precision, recall, ap
mAP = mean(ap_per_class)

Common pitfalls

• Temporal video splitting (first 5 min test, next 2 min val, final 3 min train) differs from standard random frame sampling, risking temporal leakage if not handled carefully.
• Heavy occlusion and extreme weather (snow, rain) cause detectors to merge multiple vehicles into single boxes, inflating precision but deflating recall and mAP.
• Inference uses an IoU threshold of 0.6 for parameter tuning, but standard mAP50/50-95 uses 0.5/0.5-0.95; ensure consistency when reproducing.

Evidence (verbatim from paper)

Evaluation metrics include average precision (AP), mean average precision (mAP), intersection over union (IoU), precision, and recall. ... Inference parameters include an IoU threshold of 0.6, an image size of 1280 pixels, and a confidence score of 0.5. Tab. 6 illustrates the precision, recall, mAP50, and mAP50-95 scores of the YOLOv8x, YOLO11x, YOLOv12x, and RT-DETR-x models after training for 100 epochs.

Citation

@misc{huynh2026tsbow,
  title={TSBOW: Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions},
  author={Huynh et al. (2026)},
  year={2026},
  note={arXiv:2602.05414}
}

• arXiv: 2602.05414