Sportmot Eval

sportmot-eval

SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes — Cui et al. (2023) (arXiv:2304.05170, 2023)

What this evaluates

Evaluates multi-object tracking performance in sports scenes, specifically probing a model's ability to maintain track identities under fast, variable-speed motion and highly similar player appearances.

Datasets

• SportsMOT — total ?; splits: train (-1), test (-1)

Metrics

• HOTA (primary) — range: [0, 100]
  • Higher Order Tracking Accuracy combines detection accuracy (DetA) and association accuracy (AssA) by averaging their F1-scores across multiple IoU thresholds.
• MOTA — range: percent
  • Multi-Object Tracking Accuracy measures tracking performance by penalizing false positives, false negatives, and identity switches relative to the total number of ground truth frames.

Input / output format

Input: Video sequences of sports scenes (basketball, volleyball, football) with ground truth bounding boxes and track IDs per frame.

Output: Per-frame bounding boxes with assigned track IDs for each detected object.

Scoring recipe

def compute_mot_metrics(pred_tracks, gt_tracks):
    # 1. Compute IoU between predicted and ground truth boxes per frame
    # 2. Match tracks using greedy/Hungarian matching across frames
    # 3. Calculate DetA and AssA F1-scores across IoU thresholds [0.5, 0.95]
    # 4. HOTA = geometric mean of DetA and AssA averaged over thresholds
    # 5. MOTA = 1 - (FN + FP + IDS) / GT_frames
    return hota_score, mota_score

Common pitfalls

• Assuming linear motion models (e.g., standard Kalman filters) work well; sports motion is highly non-linear and variable-speed.
• Focusing only on detection accuracy while neglecting appearance-based association, which is critical due to similar player uniforms.
• Evaluating on private detection protocols without standardizing the detector, leading to unfair comparisons.

Evidence (verbatim from paper)

The wide range of HOTA and MOTA denotes SportsMOT is more distinguishable among different kinds of algorithms.

Citation

@misc{cui2023sportmot,
  title={SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes},
  author={Cui et al. (2023)},
  year={2023},
  note={arXiv:2304.05170}
}

• arXiv: 2304.05170