Scopeflow Eval

scopeflow-eval

ScopeFlow: Dynamic Scene Scoping for Optical Flow — Bar-Haim et al. (2020) (arXiv:2002.10770, 2020)

What this evaluates

Evaluates dense optical flow estimation accuracy and occlusion detection on standard video benchmarks. Probes the model's ability to predict pixel-wise motion vectors and identify occluded regions under varying motion magnitudes and scene complexities.

Datasets

• Sintel — total ?; splits: clean (-1), final (-1)
• KITTI — total ?; splits: 2012 (-1), 2015 (-1)

Metrics

• End Point Error (EPE) (primary) — range: other
  • Average Euclidean distance between predicted and ground truth flow vectors across all pixels: EPE = (1/N) * Σ ||v_pred - v_gt||_2.
• F1 score — range: [0, 1]
  • Harmonic mean of precision and recall for binary occlusion mask prediction.
• Outlier percentage — range: percent
  • Percentage of pixels where the End Point Error exceeds 3 pixels.

Input / output format

Input: Paired consecutive frames (images) from a video sequence.

Output: Dense optical flow field (2D displacement vectors per pixel) and/or occlusion mask.

Scoring recipe

def compute_epe(pred_flow, gt_flow):
    diff = pred_flow - gt_flow
    epe_per_pixel = np.sqrt(diff[..., 0]**2 + diff[..., 1]**2)
    return np.mean(epe_per_pixel)

def compute_outlier(pred_flow, gt_flow, threshold=3.0):
    epe = compute_epe(pred_flow, gt_flow)
    outlier_mask = epe > threshold
    return np.mean(outlier_mask) * 100

Common pitfalls

• EPE is computed over all pixels by default, but the paper reports subset metrics (matched/unmatched, fast/slow pixels) that must be tracked separately.
• KITTI outlier threshold is fixed at 3 pixels, unlike the standard 1px/3px/5px reporting in other flow benchmarks.
• Sintel clean and final passes must be evaluated separately as they use different ground truth generation pipelines.

Evidence (verbatim from paper)

All of our experiments employ the common End Point Error metric for flow evaluation, and F1 for occlusion evaluation. KITTI experiments also present outlier percentage.

Citation

@misc{barhaim2020scopeflow,
  title={ScopeFlow: Dynamic Scene Scoping for Optical Flow},
  author={Bar-Haim et al. (2020)},
  year={2020},
  note={arXiv:2002.10770}
}

• arXiv: 2002.10770