# Androidlh Eval

> Evaluates a GUI agent's ability to perform long-horizon, multi-app tasks in a mobile environment. It probes the agent's planning and skill-retrieval capabilities across complex, real-world application scenarios. Use when the user wants to benchmark on AndroidLH, or asks about evaluating this task. Reports task success rate.

- Skill: `qhjqhj00/androidlh-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/androidlh-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/androidlh-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/androidlh-eval

---


# androidlh-eval

> Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills — Xie et al. (2025) (arXiv:2506.10387, 2025)

## What this evaluates

Evaluates a GUI agent's ability to perform long-horizon, multi-app tasks in a mobile environment. It probes the agent's planning and skill-retrieval capabilities across complex, real-world application scenarios.

## Datasets

- **AndroidLH** — total 30; splits: test (30)

## Metrics

- `task success rate` **(primary)** — range: percent
  - Percentage of long-horizon tasks completed successfully.
- `completion rate` — range: percent
  - Proportion of sub-goals or steps completed successfully across tasks.

## Input / output format

**Input**: Screen screenshots of the Android emulator environment for multi-app, long-horizon tasks.

**Output**: Action selection to navigate and interact across multiple applications.

## Scoring recipe

```python
success = 1 if task goal is met according to system state-based verification, else 0.
SR = mean(success) * 100.
CR = mean(completion proportion) * 100.
```

## Common pitfalls

- Tasks are synthetically generated by GPT-4o, which may not perfectly reflect real-world complexity or edge cases.
- System state-based verification requires accurate state tracking across app boundaries, which can be brittle if the emulator state diverges from the agent's perception.

## Evidence (verbatim from paper)

> Specifically, Mirage-1 exhibits significant performance gains with mean improvements of 53.3% in completion rate and 56.1% in success rate compared to the baseline agents.

## Citation

```bibtex
@misc{xie2025mirage1,
  title={Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills},
  author={Xie et al. (2025)},
  year={2025},
  note={arXiv:2506.10387}
}
```

- arXiv: 2506.10387