Depth Anything Ac Eval

depth-anything-ac-eval

Depth Anything at Any Condition — Sun et al. (2025) (arXiv:2507.01634, 2025)

What this evaluates

Evaluates zero-shot monocular relative depth estimation robustness under complex environmental conditions such as low light, adverse weather (rain, fog, snow), and synthetic noise. It probes the model's ability to recover fine-grained spatial relationships and object boundaries from degraded inputs without fine-tuning.

Datasets

• DA-2K (multi-condition) — total ?; splits: test (-1)
• NuScenes-night — total ?; splits: test (-1)
• Robotcar-night — total ?; splits: test (-1)
• Driving-Stereo — total ?; splits: test (-1)
• KITTI-C — total ?; splits: test (-1)
• KITTI — total ?; splits: test (-1)
• NYU-D — total ?; splits: test (-1)
• Sintel — total ?; splits: test (-1)
• ETH3D — total ?; splits: test (-1)
• DIODE — total ?; splits: test (-1)

Metrics

• AbsRel (primary) — range: [0, 1]
  • Mean absolute relative error: (1/N) Σ |d_pred - d_gt| / d_gt. Lower is better.
• δ1 (primary) — range: [0, 1]
  • Percentage of pixels where max(d_pred/d_gt, d_gt/d_pred) < 1.25. Higher is better.
• Acc — range: [0, 1]
  • Accuracy of pairwise relative depth ordering on the DA-2K benchmark. Higher is better.

Input / output format

Input: Single RGB image (often subjected to synthetic perturbations like darkness, fog, snow, motion blur, or Gaussian noise for robustness evaluation).

Output: Predicted relative depth map (affine-invariant scale).

Scoring recipe

def compute_metrics(pred, gt):
    # pred, gt: 2D arrays of predicted and ground-truth relative depth
    # Ensure scale alignment if necessary (e.g., median ratio)
    abs_rel = np.mean(np.abs(pred - gt) / gt)
    ratio = np.maximum(pred / gt, gt / pred)
    delta1 = np.mean(ratio < 1.25)
    return {'AbsRel': abs_rel, 'δ1': delta1}

Common pitfalls

• Ground truth from LiDAR/depth cameras is often sparse and noisy, which limits fine-grained evaluation and can artificially cap metric scores.
• Models predict affine-invariant/relative depth, so absolute scale cannot be directly compared without proper alignment or scaling.
• DA-2K uses pairwise relative depth accuracy, which evaluates ordering rather than pixel-wise regression, making it incomparable to AbsRel/δ1.

Evidence (verbatim from paper)

For instance, compared to Depth Anything V2, our DepthAnything-AC achieves 0.037 improvements of δ1 metric on the Robotcar-night benchmark.

Citation

@misc{sun2025depthanythingac,
  title={Depth Anything at Any Condition},
  author={Sun et al. (2025)},
  year={2025},
  note={arXiv:2507.01634}
}

• arXiv: 2507.01634