Habitat Gs Eval

habitat-gs-eval

Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting — Xia et al. (2026) (arXiv:2604.12626, 2026)

What this evaluates

Evaluates embodied agents' navigation capabilities in photorealistic 3D Gaussian Splatting environments versus traditional mesh-based simulators. It probes cross-domain generalization, visual robustness, and human-aware collision avoidance in dynamic scenes.

Datasets

• InteriorGS + Real-world GS — total 120; splits: train (100), test (20)
• Habitat-Matterport 3D (HM3D) — total 120; splits: train (100), test (20)
• AnimatableGaussians — total 6; splits: train (3), test (3)

Metrics

• SR (primary) — range: percent
  • Fraction of episodes where the agent reaches the goal within a predefined distance threshold.
• SPL (primary) — range: percent
  • SR × (ℓ*/ℓ), where ℓ* is the geodesic shortest-path length and ℓ is the agent's actual path length.
• DTG — range: other
  • Euclidean distance between the agent and the goal at episode termination.
• CR — range: percent
  • Fraction of steps in an episode where the agent collides with an obstacle or avatar.
• PSI — range: other
  • Average degree to which the agent enters the 1.0 m personal-space radius around each avatar.

Input / output format

Input: 256×256 RGB and depth observations per timestep.

Output: Sequential navigation actions (e.g., forward, turn left/right) to reach a point-goal.

Scoring recipe

def compute_metrics(trajectory, goal_pos, shortest_path_len, collision_steps, personal_space_intrusions):
    sr = 1.0 if distance(trajectory[-1], goal_pos) <= threshold else 0.0
    spl = sr * (shortest_path_len / len(trajectory)) if len(trajectory) > 0 else 0.0
    dtg = distance(trajectory[-1], goal_pos)
    cr = collision_steps / len(trajectory)
    psi = sum(personal_space_intrusions) / len(personal_space_intrusions)
    return {'SR': sr, 'SPL': spl, 'DTG': dtg, 'CR': cr, 'PSI': psi}

Common pitfalls

• Test sets for mesh and GS scenes are drawn from disjoint physical spaces, so results measure cross-domain generalization rather than representation fidelity on identical geometry.
• VLM quality assessment uses blind, randomized batches of 5 GS and 5 mesh images to prevent source bias.
• Fixed training budget (5×10^7 steps) means GS-only agents may not fully converge, affecting direct performance comparisons with mesh-only baselines.

Evidence (verbatim from paper)

We adopt standard embodied navigation metrics. Success Rate (SR) measures the fraction of episodes in which the agent reaches the goal within a distance threshold. Success weighted by Path Length (SPL) jointly captures success and path efficiency: SPL = SR × (ℓ*/ℓ), where ℓ* is the geodesic shortest-path length and ℓ the agent’s actual path length. Distance to Goal (DTG) records the Euclidean distance between the agent and the goal at episode termination.

Citation

@misc{xia2026habitatgs,
  title={Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting},
  author={Xia et al. (2026)},
  year={2026},
  note={arXiv:2604.12626}
}

• arXiv: 2604.12626