Mot15 Tracking Eval

mot15-tracking-eval

PhiNets: a scalable backbone for low-power AI at the edge — Paissan et al. (2021) (arXiv:2110.00337, 2021)

What this evaluates

Evaluates multi-object tracking accuracy in a tracking-by-detection pipeline on edge hardware, measuring how well backbones support SORT-based tracking under strict computational and power constraints.

Datasets

• MOT15 — total ?; splits: test (-1)

Metrics

• MOTA (primary) — range: percent
  • Multi Object Tracking Accuracy, combining false positives, false negatives, and ID switches relative to the number of ground truth tracks.

Input / output format

Input: Video frames processed through a detection backbone followed by the SORT tracker, with detector training augmented by 360 epochs on the benchmark data.

Output: Tracked object IDs and trajectories over time for each video sequence.

Scoring recipe

def compute_MOTA(tracks, gts):
    fp = count_false_positives(tracks, gts)
    fn = count_false_negatives(tracks, gts)
    ids = count_id_switches(tracks, gts)
    total_gt = len(gts)
    return 1 - (fp + fn + ids) / total_gt

Common pitfalls

• Tracking performance is heavily dependent on detection quality; poor detector IoU directly lowers tracking scores, making it hard to isolate backbone efficiency from detection head performance.
• Detectors are trained with 360 epochs of augmentation directly on the benchmark data, which risks data leakage or overfitting to the test set.

Evidence (verbatim from paper)

This choice of hardware and software allowed for a state of the art power consumption of under 1.3mJ for the 1.2MMACC PhiNet (53.7 / 60.3 mAP on a subset of the COCO/VOC2012 datasets) and 11.8mJ for the 9.8MMACC PhiNet (64.1 / 73.9 mAP on COCO/VOC2012, 60.8 MOTA on MOT15).

Citation

@misc{paissan2021phinet,
  title={PhiNets: a scalable backbone for low-power AI at the edge},
  author={Paissan et al. (2021)},
  year={2021},
  note={arXiv:2110.00337}
}

• arXiv: 2110.00337