Kitti Depth Flow Pose Eval

kitti-depth-flow-pose-eval

Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video — Li et al. (2021) (arXiv:2105.14520, 2021)

What this evaluates

Evaluates a model's ability to jointly estimate monocular depth, optical flow, and camera ego-motion from consecutive video frames in driving scenes. It probes geometric consistency, motion handling, and self-supervised learning robustness on standard autonomous driving benchmarks.

Datasets

• KITTI Raw — total 44686; splits: train (40864), val (3822)
• KITTI Flow 2012 — total ?; splits: test (-1); repo http://www.cvlibs.net/datasets/kitti/eval_stereo_flow.php?benchmark=flow
• KITTI Flow 2015 — total ?; splits: test (-1); repo http://www.cvlibs.net/datasets/kitti/eval_scene_flow.php?benchmark=flow
• KITTI Odometry — total ?; splits: test_seq09 (-1), test_seq10 (-1)
• KITTI Eigen Split — total ?; splits: test (-1)

Metrics

• EPE (primary) — range: other
  • Average End-Point Error computed over non-occluded (noc), occluded (occ), and all regions.
• odometry error — range: other
  • Average translation and rotation error per trajectory segment, following the SfMLearner protocol.
• depth error — range: other
  • Standard monocular depth metrics (e.g., absolute relative error, RMSE) evaluated on the KITTI Eigen split.

Input / output format

Input: Consecutive pairs of RGB images from KITTI driving videos, down-sampled to 832x256 resolution.

Output: Predicted monocular depth map, optical flow field between frames, and 6-DoF camera ego-motion (translation and rotation).

Scoring recipe

def compute_epe(flow_pred, flow_gt, region_mask):
    # region_mask: boolean array for noc, occ, or all pixels
    diff = flow_pred[region_mask] - flow_gt[region_mask]
    epe = np.sqrt(np.sum(diff**2, axis=1))
    return np.mean(epe)

# For odometry: compute per-segment translation/rotation error per SfMLearner protocol
# For depth: compute standard metrics (AbsRel, RMSE, etc.) on KITTI Eigen split

Common pitfalls

• Failing to mask dynamic/moving objects during loss computation, which biases depth and flow estimates.
• Using incorrect evaluation splits (e.g., not using KITTI Eigen split for depth or seq 09/10 for pose).
• Ignoring the 3-stage progressive training schedule with specific loss weights, which significantly impacts final convergence and metric scores.

Evidence (verbatim from paper)

We report the performance using the average end-point error (EPE) over non-occluded regions (noc), occluded regions(occ) and overall regions (all).

Citation

@misc{li2021unsupervisedjoint,
  title={Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video},
  author={Li et al. (2021)},
  year={2021},
  note={arXiv:2105.14520}
}

• arXiv: 2105.14520