Kitti Speed Estimation Eval

kitti-speed-estimation-eval

Speed estimation evaluation on the KITTI benchmark based on motion and monocular depth information — Rill (2019) (arXiv:1907.06989, 2019)

What this evaluates

Ego-vehicle longitudinal speed estimation from monocular video sequences. It probes a model's ability to infer real-world velocity by combining optical flow magnitude and monocular depth/disparity cues over time.

Datasets

• KITTI — total ?; splits: test (15)

Metrics

• RMSE (primary) — range: other
  • Root Mean Square Error between predicted ego-speed (in m/s) and ground truth longitudinal velocity from the KITTI localization system. Computed as sqrt(mean((v_pred - v_gt)^2)).

Input / output format

Input: Rectified left-camera color images (1242×375) from KITTI video sequences, optionally cropped to predefined bounding boxes. Images are resized to specific resolutions per model (Table 1) before processing.

Output: Predicted ego-speed scalar per frame (in m/s), temporally smoothed and scaled to real-world units.

Scoring recipe

# 1. Filter valid pixels: OF > 0.2, DISP > 0.01
# 2. Compute raw speed = mean(OF) / mean(DISP) per frame
# 3. Concatenate over frames and apply 1D temporal smoothing (kernel=25, uniform weights)
# 4. Compute scaling factor s = mean(gold) / mean(smoothed_pred)
# 5. Final prediction = smoothed_pred * s
# 6. Metric = RMSE = sqrt(mean((final_prediction - gold)^2))

Common pitfalls

• Including stationary drives (zero speed) artificially inflates accuracy and was explicitly excluded.
• The scaling factor is computed per video/recordings, not globally, which affects cross-video comparison.
• Valid pixel thresholds (OF > 0.2, DISP > 0.01) must be applied before computing the OF/DISP quotient.

Evidence (verbatim from paper)

To evaluate our speed estimation pipeline we manually selected 15 recordings of rectified images from the left input color camera of the KITTI dataset. These are reresentative videos where the car is moving almost always. We did not include recordings in which the car is stationary from start to finish, as zero speed might highly increase the accuracy of our method. The aggregated vectors are temporally smoothed using a 1D convolution of size 25 with equal weights. Finally, the resulting smoothed lists are taken for multiple video recordings and a scaling factor is approximated that minimizes the ratio between the ground truth and predicted speed. The approach achieves an RMSE < 1 m/s by exploiting spatial depth cues and motion patterns in single-view sequences.

Citation

@misc{rill2019speed,
  title={Speed estimation evaluation on the KITTI benchmark based on motion and monocular depth information},
  author={Rill (2019)},
  year={2019},
  note={arXiv:1907.06989}
}

• arXiv: 1907.06989