# Vlm

> vlm

- Skill: `graph-robots/vlm` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add graph-robots/vlm`
- Raw SKILL.md: https://api.skillmd.com/api/skills/graph-robots/vlm/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: graph-robots (https://skillmd.com/u/graph-robots)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/graph-robots/vlm

---


# vlm

API-backed vision-language Q&A. Zero GPU: every provider is a remote
endpoint. Images are gap-native `uint8 [H, W, 3]` numpy arrays, PNG-encoded
on the wire.

## Providers

Selected by `GAP_VLM_PROVIDER` (default `openrouter`); each tool also accepts
a per-call `provider=` override.

| Provider     | Backend                                             | Config (env)                                              |
|--------------|-----------------------------------------------------|-----------------------------------------------------------|
| `openrouter` | OpenRouter's OpenAI-compatible chat-completions API | `OPENROUTER_API_KEY` (or `GAP_VLM_API_KEY`); `GAP_VLM_MODEL` (default `gemini-3.1-flash-lite-preview`, see `DEFAULT_MODEL` in `tools.py`); set `GAP_VLM_BASE_URL` to point at another OpenAI-compatible server (e.g. a local vLLM) |
| `vertex`     | Vertex AI via google-genai (Gemini models)          | `GAP_VLM_MODEL`, `GAP_VLM_PROJECT_ID`, `GAP_VLM_REGION`   |

The `vertex` provider lazy-imports google-genai — install the engine's vertex
extra first: `pip install "graph-as-policy[vertex]"`.

## When to use

- Semantic scene checks and checkpoint verification (`vlm.query_yes_no`).
- Free-form scene descriptions or attribute queries (`vlm.query`).
- Prefer `gemini-er.detect` when you need pixel-space bounding boxes, and
  `molmo.point_prompt` when you need a single click point.

## Notes

- `vlm.query_yes_no` coerces with the source-verbatim rule: answer is true
  iff `"yes"` appears in the lowercased reply.
- Requests carry no system prompt and no temperature knob (mirrors the
  original `vlm.v1` proto); both providers pin `temperature: 0.0`.