API-backed vision-language Q&A. Zero GPU: every provider is a remote
endpoint. Images are gap-native uint8 [H, W, 3] numpy arrays, PNG-encoded
on the wire.
Providers
Selected by GAP_VLM_PROVIDER (default openrouter); each tool also accepts
a per-call provider= override.
Provider
Backend
Config (env)
openrouter
OpenRouter's OpenAI-compatible chat-completions API
OPENROUTER_API_KEY (or GAP_VLM_API_KEY); GAP_VLM_MODEL (default gemini-3.1-flash-lite-preview, see DEFAULT_MODEL in tools.py); set GAP_VLM_BASE_URL to point at another OpenAI-compatible server (e.g. a local vLLM)
vertex
Vertex AI via google-genai (Gemini models)
GAP_VLM_MODEL, GAP_VLM_PROJECT_ID, GAP_VLM_REGION
The vertex provider lazy-imports google-genai — install the engine's vertex
extra first: pip install "graph-as-policy[vertex]".
When to use
Semantic scene checks and checkpoint verification (vlm.query_yes_no).
Free-form scene descriptions or attribute queries (vlm.query).
Prefer gemini-er.detect when you need pixel-space bounding boxes, and
molmo.point_prompt when you need a single click point.
Notes
vlm.query_yes_no coerces with the source-verbatim rule: answer is true
iff "yes" appears in the lowercased reply.
Requests carry no system prompt and no temperature knob (mirrors the
original vlm.v1 proto); both providers pin temperature: 0.0.
1---2name: vlm3description: vlm4---56# vlm78API-backed vision-language Q&A. Zero GPU: every provider is a remote9endpoint. Images are gap-native `uint8 [H, W, 3]` numpy arrays, PNG-encoded10on the wire.1112## Providers1314Selected by `GAP_VLM_PROVIDER` (default `openrouter`); each tool also accepts15a per-call `provider=` override.1617| Provider | Backend | Config (env) |18|--------------|-----------------------------------------------------|-----------------------------------------------------------|19| `openrouter` | OpenRouter's OpenAI-compatible chat-completions API | `OPENROUTER_API_KEY` (or `GAP_VLM_API_KEY`); `GAP_VLM_MODEL` (default `gemini-3.1-flash-lite-preview`, see `DEFAULT_MODEL` in `tools.py`); set `GAP_VLM_BASE_URL` to point at another OpenAI-compatible server (e.g. a local vLLM) |20| `vertex` | Vertex AI via google-genai (Gemini models) | `GAP_VLM_MODEL`, `GAP_VLM_PROJECT_ID`, `GAP_VLM_REGION` |2122The `vertex` provider lazy-imports google-genai — install the engine's vertex23extra first: `pip install "graph-as-policy[vertex]"`.2425## When to use2627- Semantic scene checks and checkpoint verification (`vlm.query_yes_no`).28- Free-form scene descriptions or attribute queries (`vlm.query`).29- Prefer `gemini-er.detect` when you need pixel-space bounding boxes, and30 `molmo.point_prompt` when you need a single click point.3132## Notes3334- `vlm.query_yes_no` coerces with the source-verbatim rule: answer is true35 iff `"yes"` appears in the lowercased reply.36- Requests carry no system prompt and no temperature knob (mirrors the37 original `vlm.v1` proto); both providers pin `temperature: 0.0`.
Run npx skillmds@latest add graph-robots/vlm in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
vlm It is listed under Coding & Dev Tools on SkillMD.
This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
graph-robots (@graph-robots) published this skill. Their other Agent Skills are listed on their SkillMD profile.
graph-robots