# Android Vision

> Visual fallback observation — when the UI tree is missing, stale, or blocked, derive what is on screen from captured frames (plus any OCR/vision pass) so automation continues without coordinates.

- Skill: `samraddhashrivastavatech/android-vision` (Agent Skill)
- Install (CLI): `npx skillmds@latest add samraddhashrivastavatech/android-vision`
- Raw SKILL.md: https://api.skillmd.com/api/skills/samraddhashrivastavatech/android-vision/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: SAMRADDHASHRIVASTAVATECH (https://skillmd.com/u/samraddhashrivastavatech)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/samraddhashrivastavatech/android-vision

---


# Android vision fallback

Observe the device visually when the accessibility tree cannot be trusted, then
let the higher skills act on the visual understanding.

## When to use

- UI tree missing/stale (games, embedded browsers, canvas/Skia surfaces, webviews).
- The tree exists but does not match what is visibly on screen.
- Screen shows content that must be read (labels, errors, QR, media) as text.

## Tools used

- `take_screenshot` — the frame to understand.
- `screen_info` — resolution/density so visual regions map to UI tools.
- `ui_read` / `ui_find` — try the tree first; vision is the fallback, not the
  default.

## Procedure

1. Prefer `ui_tree`/`ui_find`. Only if the tree is unusable, go visual.
2. `take_screenshot` (task-scoped region when possible) + `screen_info`.
3. Interpret the frame (labels, element regions, states).
4. Map understood regions back to `android-ui`/`android-input` actions, or
   report the reading to the user.
5. Delete on-screen records that are no longer needed.

## Security

- Reading the screen may read private content: only capture what the task needs.
- Vision is an observation tool; it never authorises an action the gates refuse.

## Related skills

`android-ui` · `android-screen` · `android-input` · `android-control`