Hmaflow Eval

hmaflow-eval

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment — Ma et al. (2024) (arXiv:2409.05531, 2024)

What this evaluates

Evaluates the accuracy of optical flow estimation models on synthetic and real-world video sequences. It specifically probes the model's ability to capture fine object contours, handle small or fast-moving targets, and maintain robustness under downscaling and occlusion.

Datasets

• Sintel — total ?; splits: test (-1)
• KITTI-2015 — total ?; splits: test (-1)

Metrics

• EPE (primary) — range: other
  • Average end-point error across all pixels, computed as the mean Euclidean distance between predicted and ground truth flow vectors.
• Fl-all (%) — range: percent
  • Percentage of outliers (pixels where the flow error exceeds 3 pixels or 5% of the ground truth flow magnitude), averaged over all ground truth pixels.

Input / output format

Input: Paired consecutive frames (source and target images) from a video sequence.

Output: Per-pixel 2D flow vectors (u, v) representing the displacement from the source frame to the target frame.

Scoring recipe

def compute_epe(pred_flow, gt_flow):
    return np.mean(np.sqrt(np.sum((pred_flow - gt_flow)**2, axis=-1)))

def compute_fl_all(pred_flow, gt_flow):
    errors = np.sqrt(np.sum((pred_flow - gt_flow)**2, axis=-1))
    gt_mag = np.sqrt(np.sum(gt_flow**2, axis=-1))
    outlier_mask = (errors > 3.0) | (errors > 0.05 * gt_mag)
    return np.mean(outlier_mask) * 100

Common pitfalls

• Sintel inference requires a warm-start strategy (using previous frame's flow as initialization) to match reported EPE scores; skipping it yields significantly worse results.
• Sintel results must be reported separately for 'clean' and 'final' passes, as they differ in noise and compression levels.
• Fl-all uses a dual outlier threshold (3 pixels OR 5% of GT magnitude); using only one condition produces incorrect outlier percentages.

Evidence (verbatim from paper)

The Sintel benchmark uses the average end-point error (EPE) as evaluation metric, which measures the average flow error across all pixels. Similarly, for the KITTI 2015 benchmark, we report the average end-point error (EPE) across all pixels, along with the Fl-all (%) metric, which represents the percentage of outliers (pixels where the flow error exceeds 3 pixels or 5% of the ground truth flow), averaged over all ground truth pixels.

Citation

@misc{ma2024hmaflow,
  title={HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment},
  author={Ma et al. (2024)},
  year={2024},
  note={arXiv:2409.05531}
}

• arXiv: 2409.05531