Multisports Eval

multisports-eval

MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions — Li et al. (2021) (arXiv:2105.07404, 2021)

What this evaluates

Evaluates multi-person spatio-temporal action detection in sports videos, probing the model's ability to localize fine-grained actions across multiple concurrent persons, handle occlusion, and model long-range temporal context.

Datasets

• MultiSports — total ?; splits: train (18422), val (6577), test (1071); repo https://github.com/MCG-NJU/MultiSports

Metrics

• frame-mAP@0.5 (primary) — range: [0, 1]
  • Mean Average Precision computed per frame. A predicted bounding box is matched to a ground-truth box if the 2D IoU is ≥ 0.5. AP is averaged across the 60 filtered action classes.
• video-mAP@0.2 — range: [0, 1]
  • Mean Average Precision computed over temporal tracks. A predicted track matches a ground-truth track if the 3D IoU ≥ 0.2. 3D IoU = temporal domain IoU × average frame-level IoU over overlapped frames.
• video-mAP@0.5 — range: [0, 1]
  • Same as video-mAP@0.2 but with a stricter 3D IoU threshold of 0.5.

Input / output format

Input: Video clips resized to 720P containing multiple persons performing sports actions, with frame-wise bounding boxes and action class labels.

Output: Per-frame bounding boxes and temporal action tracks (start frame, end frame, action class, bounding box coordinates).

Scoring recipe

def compute_frame_mAP(preds, gts, iou_thresh=0.5):
    matches = []
    for frame in preds:
        for pred_box in frame:
            gt_box = max(gts[frame], key=lambda g: iou(pred_box, g))
            if iou(pred_box, gt_box) >= iou_thresh:
                matches.append((pred_box, gt_box))
    return compute_ap(matches)

def compute_video_mAP(pred_tracks, gt_tracks, iou_thresh=0.2):
    matches = []
    for pred in pred_tracks:
        gt = max(gt_tracks, key=lambda g: three_d_iou(pred, g))
        if three_d_iou(pred, gt) >= iou_thresh:
            matches.append((pred, gt))
    return compute_ap(matches)
# 3D IoU = temporal_iou * mean_frame_iou

Common pitfalls

• Long-tailed class distribution requires filtering to only 60 classes with ≥25 instances in val/test splits.
• Temporal localization is critical; evaluating on untrimmed videos significantly penalizes models with poor action boundary prediction.
• Multi-person concurrency and occlusion require robust track association, not just frame-level detection.

Evidence (verbatim from paper)

Following the standard practice, we utilize frame-mAP and video-mAP to evaluate action detection performance. For video-mAP, we use the 3D IoU, which is defined as the temporal domain IoU of two tracks, multiplied by the average of the IoU between the overlapped frames. The threshold is 0.5 for frame-mAP, 0.2 and 0.5 for video-mAP.

Citation

@misc{li2021multisports,
  title={MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions},
  author={Li et al. (2021)},
  year={2021},
  note={arXiv:2105.07404}
}

• arXiv: 2105.07404