Polypsegtrack Eval

polypsegtrack-eval

PolypSegTrack: Unified Foundation Model for Colonoscopy Video Analysis — Choudhuri et al. (2025) (arXiv:2503.24108, 2025)

What this evaluates

Evaluates a unified foundation model's ability to perform joint polyp detection, segmentation, classification, and unsupervised tracking on colonoscopy video frames. It tests generalization to unseen clinical datasets and consistency of object association across frames without task-specific fine-tuning.

Datasets

• Kvasir-SEG — total ?; splits: train (900), test (-1)
• CVC-ClinicDB — total ?; splits: train (550), test (-1)
• CVC-ColonDB — total ?; splits: test (-1)
• ETIS — total ?; splits: test (-1)
• CVC-300 — total ?; splits: test (-1)
• KUMC — total ?; splits: train (28000), val (-1)
• REAL-Colon — total 3000; splits: test (3000)

Metrics

• Dice (primary) — range: [0, 1]
  • Dice coefficient = 2 * |A ∩ B| / (|A| + |B|), measuring overlap between predicted and ground-truth segmentation masks.
• IoU — range: [0, 1]
  • Intersection over Union = |A ∩ B| / |A ∪ B|, standard segmentation overlap metric.
• Precision — range: [0, 1]
  • Ratio of true positive detections to all predicted detections.
• Recall — range: [0, 1]
  • Ratio of true positive detections to all ground-truth detections.
• F1 Score — range: [0, 1]
  • Harmonic mean of precision and recall, used for detection and classification performance.
• HOTA — range: [0, 1]
  • Higher Order Tracking Accuracy, a composite metric balancing detection and association accuracy over multiple thresholds.
• MOTA — range: [0, 1]
  • Multi-Object Tracking Accuracy = 1 - (FN + FP + ID switches) / GT count.
• IDF1 — range: [0, 1]
  • ID F1 score, measuring the harmonic mean of ID precision and recall across tracks.

Input / output format

Input: Colonoscopy video frames or static endoscopic images.

Output: Bounding boxes, pixel-level segmentation masks, polyp classification labels (AD/HP), and temporal track IDs.

Scoring recipe

def compute_metrics(preds, gold):
    # Segmentation
    dice = 2 * (preds.mask & gold.mask).sum() / (preds.mask.sum() + gold.mask.sum())
    iou = (preds.mask & gold.mask).sum() / (preds.mask | gold.mask).sum()
    # Detection
    tp = count_true_positives(preds.bboxes, gold.bboxes, iou_thresh=0.5)
    precision = tp / max(len(preds.bboxes), 1)
    recall = tp / max(len(gold.bboxes), 1)
    # Classification
    f1 = 2 * (precision * recall) / (precision + recall + 1e-8)
    # Tracking (standard MOT evaluation)
    mota = 1 - (fn + fp + id_switches) / len(gold.tracks)
    hota = compute_hota(preds.tracks, gold.tracks)
    return {'dice': dice, 'iou': iou, 'precision': precision, 'recall': recall, 'f1': f1, 'mota': mota, 'hota': hota}

Common pitfalls

• Training splits are highly restricted (only 900/550 images for Kvasir-SEG/CVC-ClinicDB), making direct comparison with models trained on full datasets unfair.
• Tracking is evaluated unsupervised on a small subset (3 videos, 1000 frames) without fine-tuning, which may not reflect real-world tracking performance.
• KUMC dataset only provides bounding box annotations, so segmentation metrics cannot be computed for it.
• No single dataset covers all four tasks (detection, segmentation, classification, tracking) simultaneously; evaluation is split across multiple benchmarks.

Evidence (verbatim from paper)

For joint detection and segmentation, we use the precision and recall metrics to evaluate the detection performance and the dice and IoU scores to measure the segmentation accuracy to be consistent with prior works. For detection and classification on KUMC dataset, we report the F1 score following prior work. To evaluate tacking, we report the object tracking metrics of DetA (detection accuracy), AssA (association accuracy), HOTA, MOTA (multi-object tracking accuracy) and IDF1 following prior works on multi-object tracking.

Citation

@misc{choudhuri2025polypsegtrack,
  title={PolypSegTrack: Unified Foundation Model for Colonoscopy Video Analysis},
  author={Choudhuri et al. (2025)},
  year={2025},
  note={arXiv:2503.24108}
}

• arXiv: 2503.24108