Libero Liberoplus Eval

libero-liberoplus-eval

Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System — Wei et al. (2026) (arXiv:2604.24921, 2026)

What this evaluates

Evaluates a Vision-Language-Action model's ability to perform precise robotic manipulation and maintain robustness under environmental perturbations. It probes spatial understanding, object manipulation, instruction following, and long-horizon task execution in both standard and perturbed simulation environments.

Datasets

• LIBERO — total ?; splits: test (-1)
• LIBERO-Plus — total ?; splits: test (-1)

Metrics

• success_rate (primary) — range: percent
  • Calculated as the percentage of successful rollouts out of the total number of attempts (50 per task). The average success rate is computed across all four task suites or across the seven perturbation dimensions.
• latency_ms — range: other
  • Inference time measured in milliseconds per action step on an Nvidia RTX 4090 GPU.

Input / output format

Input: RGB visual observations from a camera, natural language task instructions, and robot state/history context.

Output: Hybrid action space: a discrete coarse action token (macro-intent) followed by a continuous fine-grained action vector (micro-pose refinement).

Scoring recipe

def compute_success_rate(success_flags, num_trials=50):
    return sum(success_flags) / num_trials * 100.0

def compute_avg_success_rate(suite_results):
    return sum(suite_results) / len(suite_results)

Common pitfalls

• Confusing the Zero-Shot Transfer setting (models trained on LIBERO tested on LIBERO-Plus) with the Supervised Fine-Tuning setting (models trained directly on LIBERO-Plus data).
• Misinterpreting the inverted-U performance curve for coarse bin size N as a monotonic relationship; optimal performance occurs at a specific equilibrium point (N=10), not at maximum or minimum granularity.
• Overlooking that LIBERO-Plus evaluates robustness across 7 specific perturbation dimensions (camera, lighting, background, layout, robot state, etc.) rather than standard task success.

Evidence (verbatim from paper)

Following standard evaluation protocols in prior works, each task is evaluated 50 times independently, 500 rollouts total for the task suites. We report the success rates on the four task suites, as well as the average success rate across all four tasks.

Citation

@misc{wei2026libravla,
  title={Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System},
  author={Wei et al. (2026)},
  year={2026},
  note={arXiv:2604.24921}
}

• arXiv: 2604.24921