Flow360 Eval

flow360-eval

Optical Flow Estimation in 360$^\circ$ Videos: Dataset, Model and Application — Bin Duan et al. (2023) (arXiv:2301.11880, 2023)

What this evaluates

Evaluates the accuracy of predicted optical flow fields on spherical 360° video frames, measuring both endpoint displacement and angular deviation. It also assesses egocentric activity recognition performance using these flow features to test rotation-invariant representation learning.

Datasets

• FLOW360 — total ?; splits: test (-1)
• EGOK360 — total ?; splits: train (-1), test (-1)

Metrics

• EPE (primary) — range: other (pixels)
  • End Point Error: the mean Euclidean distance between predicted and ground truth flow vectors across N pixels. Formula: EPE = (1/N) * sum(||f_pred - f_gt||_2). Lower is better.
• AE — range: other (degrees)
  • Angular Error: the mean angle between predicted and ground truth flow vectors. Formula: AE = acos((u_eu_r + v_ev_r + 1) / (sqrt(u_r^2+v_r^2+1) * sqrt(u_e^2+v_e^2+1))). Lower is better.
• EPEd — range: other (pixels)
  • Distortion-aware EPE: penalizes flow error in distorted regions using a distortion density map d. Formula: EPEd = (1/N) * sum(||f_pred - f_gt||_2 / (1 - d)). Lower is better.
• AEd — range: other (degrees)
  • Distortion-aware AE: penalizes angular error in distorted regions. Formula: AEd = (1/N) * sum(AE / (1 - d)). Lower is better.
• Top-1 accuracy — range: percent
  • Percentage of correctly classified egocentric activity labels. Computed as mean(pred_label == gt_label) over the test set.

Input / output format

Input: Paired 360° video frames (or sequences of 10 frames) with optional rotational augmentations (pitch, roll, yaw) applied during training; raw frames during evaluation.

Output: Per-pixel optical flow vectors (u, v) for flow estimation; class labels for activity recognition.

Scoring recipe

def compute_metrics(f_pred, f_gt, d, labels_pred, labels_gt):
    u_e, v_e = f_pred
    u_r, v_r = f_gt
    epe = mean(sqrt((u_e - u_r)**2 + (v_e - v_r)**2))
    ae = mean(acos((u_e*u_r + v_e*v_r + 1) / (sqrt(u_r**2+v_r**2+1) * sqrt(u_e**2+v_e**2+1))))
    eped = mean(sqrt((u_e - u_r)**2 + (v_e - v_r)**2) / (1 - d))
    aed = mean(ae / (1 - d))
    # Speed-based sub-metrics: average over pixels where s=sqrt(u^2+v^2) in [0,5), [5,10), [10,20), [20,inf)
    accuracy = mean(labels_pred == labels_gt)
    return epe, ae, eped, aed, accuracy

Common pitfalls

• Distortion-aware metrics (EPEd/AEd) require the distortion density map d, which the authors remap to [0.5, 1.0) to prevent metric inflation and maintain scale.
• Results are frequently reported as averages over specific speed ranges (s<5, s<10, s<20, s>=20) rather than a single global average, which can mask performance in high-motion regions.
• Normal EPE can appear comparable between baselines, but distortion-aware metrics reveal significant failures in distorted areas (e.g., RAFT+KTN predicts shallow flow fields).

Evidence (verbatim from paper)

We evaluate our method based on 2D-raw flow. Besides, using EPE (End Point Error in Eq. (10)), i.e., Euclidean distance between the predicted flow and ground truth flow, as a single evaluation metric, we incorporate AE (Angular Error) as shown in Eq. (9) as the second measure. To explain the error in the omnidirectional setting, we introduce a distortion-aware measure called EPEd as in Eq. (11). This metric penalizes the error in the distorted area based on the distortion density map.

Citation

@misc{duan2023flow360,
  title={Optical Flow Estimation in 360$^\circ$ Videos: Dataset, Model and Application},
  author={Bin Duan et al. (2023)},
  year={2023},
  note={arXiv:2301.11880}
}

• arXiv: 2301.11880