Videocube Eval

videocube-eval

Global Instance Tracking: Locating Target More Like Humans — Hu et al. (2022) (arXiv:2202.13073, 2022)

What this evaluates

Evaluates a model's ability to track arbitrary visual instances across complex, unstructured real-world videos without assuming motion continuity or fixed categories. It measures both local search accuracy and global robustness against challenges like occlusion, fast motion, and scene transitions.

Datasets

• VideoCube — total ?; splits: test (-1)

Metrics

• PRE (primary) — range: percent
  • Calculates the Euclidean center distance between the predicted bounding box and the ground-truth bounding box in pixels. A precision plot is generated by counting the percentage of frames where this distance falls below a given threshold.
• Robustness score — range: count
  • Quantifies the number of times an algorithm is restarted after detecting tracking failure, reflecting global-search ability under the R-OPE mechanism.

Input / output format

Input: Video sequence frames with an initial target bounding box provided in the first frame.

Output: Predicted bounding box coordinates for the target in each frame.

Scoring recipe

def compute_PRE(pred_boxes, gt_boxes):
    distances = [center_dist(p, g) for p, g in zip(pred_boxes, gt_boxes)]
    thresholds = range(0, 51)
    scores = [sum(1 for d in distances if d <= t) / len(distances) for t in thresholds]
    return scores

Common pitfalls

• Standard pixel-based PRE ignores target size and video resolution, making fixed thresholds (e.g., 20px) overly strict or misleading for high-resolution videos.
• OPE evaluation assumes continuous motion, causing trackers to fail on scene transitions or target jumps, whereas R-OPE explicitly handles these by restarting the tracker.

Evidence (verbatim from paper)

PRE measures the center distance between the predicted result and the ground-truth in pixels, but ignores the impact of target size and video resolution (for detailed analysis, please refer to the methods chapter). This makes the ranking threshold with 20 pixels unreasonable.

Citation

@misc{hu2022global,
  title={Global Instance Tracking: Locating Target More Like Humans},
  author={Hu et al. (2022)},
  year={2022},
  note={arXiv:2202.13073}
}

• arXiv: 2202.13073