Minivla Nav V1 Eval

minivla-nav-v1-eval

MiniVLA-Nav v1: A Multi-Scene Simulation Dataset for Language-Conditioned Robot Navigation — Al-Bustami et al. (2026) (arXiv:2605.00397, 2026)

What this evaluates

Language-conditioned robot navigation in continuous differential-drive settings. It probes a model's ability to process multi-modal observations (RGB, depth) and language instructions to output continuous control actions to reach a target object within a specified distance.

Datasets

• MiniVLA-Nav v1 — total 1174; splits: test (-1)

Metrics

• success (primary) — range: [0, 1]
  • Fraction of episodes where the robot's final position is within 1.0 m of the target object's centroid and remains stationary for ≥5 consecutive steps.

Input / output format

Input: Natural-language instruction and a time-series stream of front-facing RGB and depth observations at 60 Hz.

Output: Sequence of continuous control actions (linear velocity v ∈ [0,1] m/s, angular velocity ω ∈ [-1.5,1.5] rad/s), or quantized 7×7 discrete tokens for VLA models.

Scoring recipe

def compute_success(predictions, gold):
    successes = 0
    for pred, gold_ep in zip(predictions, gold):
        final_pos = pred[-1].position
        target_pos = gold_ep.target_position
        if distance(final_pos, target_pos) <= 1.0:
            if is_stationary_for_n_steps(pred, n=5):
                successes += 1
    return successes / len(predictions)

Common pitfalls

• The success criterion requires the robot to be stationary for ≥5 consecutive steps after reaching the target, not just reaching the 1.0 m radius.
• Episodes are terminated early on collision (stall detection) or timeout (1000 steps), so models must handle failure conditions gracefully rather than just optimizing for final position.
• Action space is quantized to 49 discrete tokens for VLA models, but the ground truth and evaluation assume continuous control; mismatched discretization can cause evaluation errors.

Evidence (verbatim from paper)

Given a natural-language instruction $\ell$ and a stream of front-facing observations $o_{t}=(\mathbf{I}{t}^{\text{RGB}},\mathbf{D}{t})$, the robot must output a sequence of actions $a_{t}=(v_{t},\omega_{t})$ such that $|p_{T}-p_{g}|\leq r_{\text{success}}=1.0,\text{m}$... Success: robot within $r_{\text{success}}$ and stationary for $\geq 5$ consecutive steps (stopped-hold criterion).

Citation

@misc{al-bustami2026minivlanav,
  title={MiniVLA-Nav v1: A Multi-Scene Simulation Dataset for Language-Conditioned Robot Navigation},
  author={Al-Bustami et al. (2026)},
  year={2026},
  note={arXiv:2605.00397}
}

• arXiv: 2605.00397