# Hermes Gateway Image Routing

> Diagnose and fix Hermes Telegram gateway image routing — Path B model override, supports_vision decision chain, and common failure modes when images don't reach the LLM.

- Skill: `ariffazil/hermes-gateway-image-routing` (Agent Skill, multi-file: 12 files)
- Install (CLI): `npx skillmds@latest add ariffazil/hermes-gateway-image-routing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ariffazil/hermes-gateway-image-routing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: ariffazil (https://skillmd.com/u/ariffazil)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/ariffazil/hermes-gateway-image-routing

---


# Hermes Gateway Image Routing

Diagnose and fix how the Hermes Telegram gateway routes images to LLM models.

## Architecture

When a user sends an image via Telegram, the gateway decides routing in `gateway/run.py`:

### Architectural Classification: PRMT (Pre-Routing Modality Translation)

Hermes' image routing does NOT fit into the standard multimodal router taxonomy found in the literature:

| Pattern | Description | Example Systems |
|---|---|---|
| **Pattern A: Model Swap** | Detect modality → swap primary model to a VLM | NVIDIA LLM Router v2, RouteLLM, MMR-Bench |
| **Pattern B: Signal-Decision Fabric** | Extract signals (CLIP embed, PII, intent) → policy engine → route/block/redact decision | vLLM Semantic Router (VSR) |
| **PRMT (Hermes)** | Modality → text translation at gateway → same text-only primary reasoner | Hermes Agent — no equivalent in published literature |

PRMT key properties:
- **Fault isolation**: Vision failure → graceful "sorry, can't see" — no cascade crash, no model swap
- **No 413 risk**: Image bytes never enter reasoning context — transcript text only (zero risk of oversized payload)
- **Provider-agnostic fallback**: Every fallback model can process text transcripts — none need vision capability
- **Auditability**: Vision transcript can be inspected before reasoning begins
- **Tradeoff**: Translation errors are unrecoverable — if Qwen-VL misses detail, DeepSeek cannot recover (it never saw original pixels)

The standard research (vLLM Semantic Router, RouteLLM, MMR-Bench, NVIDIA LLM Router v2) assumes the reasoning model must handle all modalities natively. PRMT reverses this: the reasoning model never needs to know an image existed. Vision processing finishes at the gateway.

See `references/prmt-architecture.md` for full analysis, taxonomy comparison, and related research.

### Path: NATIVE
The image bytes are attached as `image_url` content parts directly into the user message. The active model sees pixels natively.
- **Condition**: The active model supports vision (Claude, GPT-4o, Qwen-VL, Gemini, MiniMax-M3)
- **How to verify**: `_decide_image_input_mode()` returns `"native"`

### Path: TEXT + [IMAGE TRANSCRIPT] (current, preferred)
When the primary model is text-only (e.g. DeepSeek V4 Flash), the gateway calls `_enrich_message_with_vision()` which:
1. Runs `vision_analyze_tool` against the configured `auxiliary.vision` provider
2. Currently: **Qwen-VL-Max via MuleRouter** (`qwen-vl-max`)
3. Produces a structured `[IMAGE TRANSCRIPT]` with SCENE/OCR/DATA/**IDENTITY** sections
4. Prepends it to the user message as plain text
5. The primary model receives TEXT ONLY — zero image bytes in context

**Configuration** (`~/.hermes/config.yaml`):
```yaml
auxiliary:
  vision:
    provider: mulerouter
    model: qwen-vl-max  # same provider + key as primary chat model
    timeout: 120
```
Key sourced from `MULEROUTER_API_KEY` in kunci-mas.env — same key as the chat provider. This collapses vision and chat into one auth domain: if the primary provider works, vision works. No split failure domain.

**WHY MuleRouter instead of OpenRouter:** Previously vision enrichment used OpenRouter (separate key, separate billing). When OpenRouter hit a payment error, enrichment was skipped, raw image bytes hit the text-only primary model, and the entire fallback chain crashed with 413. With MuleRouter, vision shares the same key and provider as chat — a single failure domain means vision doesn't fail independently of the primary model.

**Condition**: `_decide_image_input_mode()` returns `"text"`
**How to verify**: Message contains `[IMAGE TRANSCRIPT — generated by vision model, not seen by agent directly]`
**Log**: Gateway log shows `"Image routing: enrich via vision_analyze. N image(s) will be analyzed."`

**Why this is preferred over model swap:**
- Vision call is isolated — if the auxiliary model fails, agent still gets the message with a fallback note
- Output is TEXT — every fallback chain model can process it (no 413 cascade from image bytes)
- Primary model stays DeepSeek Flash (proven, reliable) — no model swap at runtime
- No single point of failure that poisons all fallbacks

### Model Selection: qwen-vl-max via MuleRouter is the default

PRMT only needs good scene/OCR description — the sweet spot is **qwen-vl-max** on MuleRouter (same key as chat, no split failure domain):

| Model | Provider | Notes |
|---|---|---|
| `qwen-vl-max` | MuleRouter | **Active** — best quality, one key with chat |
| `qwen3-vl-plus` | MuleRouter | Balanced — faster, good for PRMT |
| `qwen3-vl-flash` | MuleRouter | Fastest — adequate for basic PRMT |
| `qwen/qwen2.5-vl-72b-instruct` | OpenRouter | Legacy — separate key, separate billing |
| `qwen/qwen-vl-plus` | OpenRouter | Legacy — separate key |

**Design rationale:** Sharing the same provider and key for vision and chat eliminates the split failure domain. If OpenRouter had a payment error, vision enrichment failed while chat still worked — causing raw image bytes to hit the text-only primary → 413 cascade. With MuleRouter, if the primary provider works, vision works.

Switch command when you need a different tier:
```bash
hermes config set auxiliary.vision.model qwen3-vl-plus
hermes gateway restart
```

### MuleRouter Integration

See `references/mulerouter-integration.md` for full details.

**Status:** ✅ **Active auxiliary.vision.provider** since 2026-07-30. Key registered (`MULEROUTER_API_KEY` in kunci-mas.env), vision tested and working.

**MuleRouter vs OpenRouter for vision:**

| Capability | OpenRouter | MuleRouter |
|---|---|---|
| Base64 image (data: URI) | ✅ Works | ✅ Works (qwen-vl-max supports it) |
| Image URL | ✅ Works | ✅ Works |
| Chat endpoint | `/v1/chat/completions` | `/vendors/openai/v1/chat/completions` |
| Key management | Separate key | **Single key for all modalities** |
| Failure domain | Independent — vision fails separately from chat | **Shared — vision fails only if chat fails** |

**Why MuleRouter won over OpenRouter for PRMT:**
- **Single failure domain**: vision and chat share one provider + one key. If the key works for chat, it works for vision. No split-brain where chat responds but vision enrichment silently fails.
- **Single bill**: one provider bill instead of tracking OpenRouter credits + MuleRouter credits.
- **Qwen-VL-Max quality**: proven sufficient for PRMT — accurate SCENE/OCR/DATA/IDENTITY extraction on Telegram images.

**MuleRouter vision test results (2026-07-30):**
- `qwen-vl-max` model via `/vendors/openai/v1/chat/completions` — text ✅, base64 image ✅
- `qwen3-vl-plus` — text ✅, base64 image ✅ (faster, lower cost)
- Production use: successfully enriches Telegram images with 4-section [IMAGE TRANSCRIPT] format

**Pre-requisites for switching to MuleRouter:**
```bash
hermes config set auxiliary.vision.provider mulerouter
hermes config set auxiliary.vision.model qwen-vl-max
hermes config set auxiliary.vision.base_url ""
hermes config set auxiliary.vision.api_key ""
# Then regenerate flat.env and restart gateway
# ⚠️  Run this from a DIFFERENT shell, not inside the gateway session:
systemctl restart hermes-asi-gateway
```

### ⛔ DEPRECATED: Path B model swap (2026-07-29 to 2026-07-30)
Previously used a model-override approach: set `_pending_vision_model_overrides[session_key]` to swap the primary model to a vision-native one (qwen-vl / minimax-m3) for the image turn, then restore afterwards. **Reverted 2026-07-30** due to cascade failure: when the override provider failed (auth/network), image bytes were still in context, and all text-only fallback models crashed with 413 "request too large."

## Decision Chain

```
decide_image_input_mode(provider, model, cfg)
  ↓
1. Check cfg["agent"]["image_input_mode"]
   "native" → return "native"
   "text"   → return "text"
   "auto"   → continue
  ↓
2. _lookup_supports_vision(provider, model, cfg)
   ↓
   a. cfg["model"]["supports_vision"]  ← TOP-LEVEL OVERRIDE
      If present, returns IMMEDIATELY — skips actual model capability lookup
      Setting `true` on a text-only model (e.g. DeepSeek V4 Flash) POISONS
      all downstream logic — Path B and enrich-with-vision never activate
   b. Per-provider per-model override (cfg["providers"][provider]["models"][model]["supports_vision"])
   c. models.dev capability lookup
   d. (optional) Ollama vision probe
  ↓
3. If supports is True → return "native" (attach pixels directly)
4. If supports is False/None → return "text"
   → Gateway calls _enrich_message_with_vision() for [IMAGE TRANSCRIPT] path
```

## Common Failure Modes

### `model.supports_vision: true` on a text-only model
**Symptom**: `_enrich_message_with_vision` never activates. Images are dropped or fail silently.
**Root cause**: The top-level `model.supports_vision` override fires FIRST in `_lookup_supports_vision`, returning `True` before the text-only model is even checked. `decide_image_input_mode` returns `"native"`, which attaches image bytes directly to the text-only model's API call — producing cryptic "unknown variant `image_url`" errors.
**Fix**: Set `model.supports_vision: false` or remove the field entirely.
```bash
hermes config set model.supports_vision false
```

### ❌ MINIMAX_BASE_URL has sops-encrypted value (DEPRECATED — use OpenRouter instead)
**Symptom**: All vision calls fail with `"Invalid IPv6 URL"`. Gateway logs show `urlparse` errors. `vision_analyze_tool` crashes inside `resolve_provider_client` → `base_url_host_matches` → `urlparse`.
**Root cause**: The `MINIMAX_BASE_URL` env var contains `ENC[AES256_GCM,...]` ciphertext from sops-encrypted vault files. This is a GENERAL pattern: ANY env var in kunci-mas.env that stores a URL and carries sops ciphertext will crash `urlparse`.

**Diagnostic rule**: When you see `Invalid IPv6 URL` in Hermes gateway logs, the FIRST thing to check is whether any URL-type env var (`*_BASE_URL`, `*_HOST`, `*_ENDPOINT`, `*_API_URL`) in kunci-mas.env contains sops ciphertext (`ENC[AES256_GCM,`). Debug in this order:
1. `grep -l 'ENC\\[AES256_GCM' /root/.secrets/kunci-mas.env` — scan for ciphertext
2. `grep -iE 'base_url|host|endpoint|api_url' /root/.secrets/kunci-mas.env` — find URL vars
3. Cross-reference: which of those contain ciphertext?
4. Check `hermes config get auxiliary.vision.provider` and confirm which provider is in use

**Current fix (recommended)**: Switch to OpenRouter vision instead of fixing the sops-encrypted MiniMax var. OpenRouter's `OPENROUTER_API_KEY` in kunci-mas.env is stored as a raw key (not sops-encrypted), so urlparse never sees ciphertext:
```bash
hermes config set auxiliary.vision.provider openrouter
hermes config set auxiliary.vision.model "qwen/qwen3-vl-30b-a3b-instruct"
hermes config set auxiliary.vision.base_url ""
```
Then regenerate kunci-mas.flat.env and restart gateway.

**Alternate fix** (if you must keep MiniMax): Replace the encrypted value with the real URL in both `kunci-mas.env` and `kunci-mas.flat.env`:
```bash
MINIMAX_BASE_URL="https://api.minimax.io"
```
And set explicit override:
```bash
hermes config set auxiliary.vision.base_url "https://api.minimax.io/v1"
```

### OpenRouter provider not configured for vision model
**Symptom**: Vision model swap triggers but the API call fails (401/timeout).
**Root cause**: OpenRouter provider config missing `key_env: OPENROUTER_API_KEY` or the env var isn't set.
**Fix**: Verify under `providers.openrouter`:
```yaml
providers:
  openrouter:
    api: https://openrouter.ai/api/v1
    key_env: OPENROUTER_API_KEY
```

### OpenRouter vision model does not exist / renamed
**Symptom**: Gateway log shows `"No endpoints found for qwen/qwen-vl-plus" — 404`. Images fail to enrich with no fallback.
**Fix**: Query available Qwen VL models via OpenRouter API, then update config:
```python
import requests
resp = requests.get("https://openrouter.ai/api/v1/models",
    headers={"Authorization": f"Bearer $OPENROUTER_API_KEY"})
models = [m['id'] for m in resp.json()['data']
          if 'qwen' in m['id'] and ('vl' in m['id'] or 'vision' in m['id'])]
```
Known working models (2026-07-30):
| Model | Size | Cost | Notes |
|---|---|---|---|
| `qwen/qwen-vl-plus` | distilled | ~$0.000014/call | **Default** — best value for PRMT |
| `qwen/qwen2.5-vl-72b-instruct` | 72B | ~$0.35/M | Reliable, proven, overkill |
| `qwen/qwen3-vl-32b-instruct` | 32B | Medium | Good balance |
| `qwen/qwen3-vl-8b-instruct` | 8B | Cheap | Fast, basic |
| `qwen/qwen3-vl-235b-a22b-instruct` | 235B MoE | Expensive | Most powerful |

Also see MuleRouter's `qwen3-vl-plus` (URL vision only) — documented in `references/mulerouter-integration.md`.

```bash
hermes config set auxiliary.vision.model "qwen/qwen2.5-vl-72b-instruct"
```

### MiniMax MCP server crash blocks vision enrichment
**Symptom**: Gateway log shows `"MCP server 'minimax' failed initial connection after 3 attempts, parking until a reconnect is requested"`. Enrichment falls through to next provider or main model.
**Root cause**: The `minimax-mcp` binary/process exits immediately on launch. Common with uv-installed packages that have dependency changes.
**Fix**: Either fix the MCP server (reinstall minimax-mcp) or remove it from MCP config. If OpenRouter is the primary auxiliary provider, the MiniMax MCP failure is noise — the enrichment still tries OpenRouter next.

### OpenRouter unhealthy flag blocks auxiliary vision
**Symptom**: Gateway log shows `"Auxiliary: marking openrouter unhealthy for 600s (payment / credit error)"`. All vision calls are skipped for 10 minutes. Main model fallback crashes if it doesn't support image_url.
**Root cause**: OpenRouter returns a 402/403 payment error. The auxiliary client circuit-breaks OpenRouter for 600s. Subsequent calls use fallback models only.
**Fix**:
1. Check OpenRouter credit balance: visit openrouter.ai or call their API
2. Top up credits if needed
3. Verify the same API key works with a test call:
```python
requests.post("https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": "Bearer $OPENROUTER_API_KEY"},
    json={"model": "qwen/qwen2.5-vl-72b-instruct", "messages": [{
        "role": "user",
        "content": [{"type": "text", "text": "Test"},
                     {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="}}]
    }]})
```
4. **CRITICAL: Gateway restart must come from OUTSIDE the running gateway**. If you `systemctl restart` from inside a gateway session (during a conversation response), the SIGTERM propagates to the command's child processes, killing the restart command itself and leaving the gateway in `deactivating` state. **Always run the restart from a separate SSH session or terminal**:
```bash
# From a DIFFERENT shell, NOT inside the gateway conversation:
systemctl restart hermes-asi-gateway
```
If the gateway gets stuck in `deactivating` state, force-kill the lingering process:
```bash
ps aux | grep 'hermes.*gateway' | grep -v grep | awk '{print $2}' | head -1 | xargs kill -9
systemctl restart hermes-asi-gateway
```

### `OPENAI_BASE_URL` env var poisons auxiliary client routing (PROVEN 2026-07-30)

**Symptom**: Vision enrichment fails with `models/deepseek-v4-flash is not found for API version v1main` (404). Gateway log shows the vision tool calling the WRONG endpoint — Aliyun Token Plan instead of MuleRouter/OpenRouter. Followed by 413 cascade as raw image bytes reach text-only models.

**Root cause**: `OPENAI_BASE_URL` env var is set to an existing Aliyun Token Plan endpoint. The Hermes `auxiliary_client` automatically uses this env var to override ALL API base URLs, including the auxiliary vision call. Every vision enrichment call routes through Aliyun instead of the configured provider.

**Diagnosis**:
```bash
echo "$OPENAI_BASE_URL"
grep 'OPENAI_BASE_URL' /root/.secrets/kunci-mas.env
```
If non-empty (especially set to a non-Hermes provider like Aliyun), it poisons all OpenAI-compatible routing.

**Fix**: Unset `OPENAI_BASE_URL` in the gateway's environment before launching Hermes:

```bash
# Add to /usr/local/bin/hermes-gateway-secure.sh BEFORE the exec line:
unset OPENAI_BASE_URL
```

This keeps the env var available for other scripts (godel_enforcement.py, tovana_compiler.py) that genuinely need it, while preventing it from corrupting Hermes' provider routing.

**Why this happens**: The env var `OPENAI_BASE_URL` is a well-known convention — many OpenAI SDK clients check for it automatically. When it's set to ANY value, the Hermes OpenAI-compatible client uses it as the default base URL instead of the provider-specific URL from config.yaml. This is not a Hermes bug — it's a feature of the OpenAI SDK that becomes a footgun when the env var points to a different API.

**Prevention**: Gateway startup script MUST unset `OPENAI_BASE_URL` before `exec hermes gateway run`. Any script that needs it (godel, tovana) explicitly sources kunci-mas.env in its own process — it doesn't need the inherited env var.

### `image_input_mode` + provider mismatch causes 413 cascade (PROVEN 2026-07-30)

**Symptom**: Changing `model.provider` (e.g. from `opencode-go` to `mulerouter`) causes ALL images to fail with 413 cascade through every fallback provider.

**Root cause**: When `model.provider` is changed, `auxiliary.vision.provider` was NOT changed to match. The primary provider (MuleRouter) has no balance issues, but the auxiliary vision provider (OpenRouter) has $0 credit → 402 → vision enrichment fails → raw image bytes forwarded to text-only primary → 413 → cascade through all fallbacks.

**The mechanism:**
1. Primary model sees `supports_vision: false` → `image_input_mode: text` → enrichment needed
2. `auxiliary.vision.provider` = OpenRouter → fails (402 payment error)
3. No fallback enrichment → raw base64 image bytes embedded in text context
4. Text-only primary model (DeepSeek V4 Flash) gets 413 from oversized payload
5. All fallback providers ALSO get 413 — same payload

**Fix**: Always change `auxiliary.vision.provider` to match the new primary provider's family:

```bash
hermes config set auxiliary.vision.provider <NEW_PRIMARY_PROVIDER>
hermes config set auxiliary.vision.model <MODEL_WITH_VISION>
```

**Check**: `grep -A2 "vision:" /root/.hermes/config.yaml | head -4` — verify provider matches primary.

**Prevention**: When changing `model.provider`, the `auxiliary.vision.provider` MUST be updated in the same change. The two settings share a failure domain — if they diverge, one can fail independently of the other, causing raw image bytes to reach the text-only model.
When an image doesn't reach the agent, check journalctl for ALL of:
```bash
journalctl -u hermes-asi-gateway --since "5 min ago" --no-pager | grep -i "image\\|vision\\|enrich\\|unhealthy\\|auxiliary\\|image_url"
```

The failure chain is usually:
1. MiniMax MCP server dead ❌
2. OpenRouter unhealthy (payment error, 600s) ❌
3. Main model fallback doesn't support image_url ❌
4. Image raw path dumped to agent

Fix at least one link in the chain to restore enrichment.

## Verification

### Live test
Send an image via Telegram. Check gateway logs:
```bash
journalctl -u hermes-asi-gateway -f --since "1 min ago" | grep -i "image routing\|vision_analyze\|enrich message\|native"
```

Expected enrich-with-vision log line:
```
Image routing: enrich via vision_analyze. N image(s) will be analyzed.
```

Check the response message contains:
```
[IMAGE TRANSCRIPT — generated by vision model, not seen by agent directly]
```

### Code-level verification
The text-mode path calls `_enrich_message_with_vision` at `gateway/run.py`:
```python
message_text = await self._enrich_message_with_vision(
    message_text, list(image_paths),
)
```

## Enrichment Prompt Format (4 sections)

The `_enrich_message_with_vision()` method at `gateway/run.py:15237` sends this structured prompt to the vision model:

```
Analyse this image and output FOUR sections:
1) SCENE: factual description (objects, layout, colours, people, setting, expressions, estimated age range)
2) OCR: ALL visible text transcribed VERBATIM
3) DATA: tables, charts, lists, structured info
4) IDENTITY: known individuals, brands, logos, products, team affiliations
```

**IDENTITY section (added 2026-07-30):** Allows the vision model to name known public figures. Before this section existed, the model was told "Do NOT speculate" and would describe Chris Bumstead as "a shirtless man" instead of "Chris Bumstead — 5× Mr. Olympia Classic Physique". The IDENTITY section explicitly permits recognition of public figures while SCENE/OCR/DATA remain factual-only.

**How to verify the 4-section format is active:** Send an image of a recognizable person. The transcript should include an `IDENTITY:` section with named identification.

## Response Pattern: Relay, Don't Pretend

When you receive an `[IMAGE TRANSCRIPT]` block (you are a text-only model — you never see pixels), follow these rules when talking to Arif:

1. **Never say "I see" or "I can confirm"** — you didn't see anything. The vision model (Qwen-VL-Max via MuleRouter) saw the image and wrote a text description for you.
2. **Attribute the description explicitly**: Prepend every relayed description with `[Qwen-VL description -- agent does not see images]` so Arif knows the provenance.
3. **Pass through raw, don't summarize** — Arif wants the raw Qwen-VL output verbatim (SCENE/OCR/DATA/IDENTITY). Do NOT paraphrase or summarize what the vision model said. If Arif asks "send raw image description", show the exact transcript.
4. **Never assume whose picture it is** — if the image shows a person, don't assume it's Arif. The IDENTITY section may name the person; if it doesn't, leave it unidentified. Arif corrected this when I assumed his picture was him — it was actually Syed (Abang Sado).
5. **Use the IDENTITY section for named reference** — if the transcript includes `IDENTITY: Chris Bumstead`, use that name directly. If it says `[unidentifiable]`, don't guess.
6. **When vision pipeline fails** and you need to manually call it, do NOT say "I can't see it" passively — diagnose aggressively. Check the failure chain in order: MiniMax MCP dead, OpenRouter/MuleRouter unhealthy, main model fallback, OPENAI_BASE_URL env poisoning.

**Example — correct (with IDENTITY):**
```
[Qwen-VL description -- agent does not see images]
SCENE: Young Malay male, dark spiky hair, sawo matang skin...
OCR: Yellow text on grey shirt -- "ALPHA-ZEN"
DATA: [none]
IDENTITY: Syed (Abang Sado) — known contact
```

**Example — WRONG (two distinct mistakes):**
```
"Confirm gambar kau tu gambar diri kau dalam kereta -- lelaki muda, baju kelabu with yellow text"
```
Wrong 1: Sounds like YOU saw the image. You didn't. Wrong 2: Assumed it was Arif. It wasn't.

## Manual Fallback: Direct MiniMax Vision API Call

**⚠️ This path is superseded by Qwen-VL via OpenRouter (see `references/openrouter-qwen-vl-config.md`).** Keep this section only for historical/alternative reference.

When `vision_analyze()` fails because the primary model is text-only (e.g. DeepSeek V4 Flash doesn't accept `image_url` content), and the image arrived outside the gateway pipeline (e.g. a local file path, not through Telegram), call a vision model directly via Python `requests`.

### Primary path: OpenRouter + Qwen2.5-VL-72B (preferred)

| Item | Value |
|---|---|
| Endpoint | `https://openrouter.ai/api/v1/chat/completions` |
| Model | `qwen/qwen2.5-vl-72b-instruct` |
| Auth header | `Authorization: Bearer $OPENROUTER_API_KEY` |
| Env source | `/root/.secrets/kunci-mas.env` |
| Format | OpenAI-compatible chat completions |
| Cost | ~$0.000014/call (negligible) |

```python
import base64, os, requests, subprocess

# 1. Encode image
with open('/path/to/image.jpg', 'rb') as f:
    b64 = base64.b64encode(f.read()).decode('utf-8')

# 2. Get API key from kunci-mas.env
result = subprocess.run(
    ['bash', '-c', 'source /root/.secrets/kunci-mas.env && echo "$OPENROUTER_API_KEY"'],
    capture_output=True, text=True, timeout=10
)
api_key = result.stdout.strip()

# 3. Call Qwen2.5-VL-72B on OpenRouter
resp = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Content-Type": "application/json", "Authorization": f"Bearer {api_key}"},
    json={
        "model": "qwen/qwen2.5-vl-72b-instruct",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in detail."},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}
            ]
        }],
        "max_tokens": 1024
    },
    timeout=60
)
print(resp.json()['choices'][0]['message']['content'])
```

### Legacy path: MiniMax M3 (deprecated, MiniMax-MCP server often dead)

| Item | Value |
|---|---|
| Endpoint | `https://api.minimax.io/v1/chat/completions` |
| Model | `minimax-m3` |
| Auth header | `Authorization: Bearer $MINIMAX_API_KEY` |

**Pitfalls**:
- MiniMax MCP server (`minimax-mcp`) frequently crashes on launch — parked after 3 attempts
- MINIMAX_BASE_URL was historically sops-encrypted causing "Invalid IPv6 URL" — see `references/minimax-base-url-sops-bug.md`
- Use OpenRouter path above instead

### Python template (general — works with any OpenAI-compatible endpoint)

```python
import base64, os, requests, subprocess

with open('/path/to/image.jpg', 'rb') as f:
    b64 = base64.b64encode(f.read()).decode('utf-8')

result = subprocess.run(
    ['bash', '-c', 'source /root/.secrets/kunci-mas.env && echo "$VISION_API_KEY"'],
    capture_output=True, text=True, timeout=10
)
api_key = result.stdout.strip()

resp = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Content-Type": "application/json", "Authorization": f"Bearer {api_key}"},
    json={
        "model": "qwen/qwen2.5-vl-72b-instruct",
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in detail."},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}
            ]
        }],
        "max_tokens": 1024
    },
    timeout=60
)
description = resp.json()['choices'][0]['message']['content']
print(description)
```

### Pitfalls

- **"Argument list too long"** — do NOT inline base64 in `curl -d`. A 119KB JPEG → ~159KB base64 string exceeds shell arg limits. Use Python `requests` instead.
- **Model name must exist on provider** — always verify with `GET /api/v1/models` first. `qwen/qwen-vl-plus` was previously deprecated on OpenRouter but was working as of 2026-07-30.
- **Key sourcing** — both `OPENROUTER_API_KEY` and `MINIMAX_API_KEY` are in kunci-mas.env. Use `source /root/.secrets/kunci-mas.env` or `subprocess.run(['bash', '-c', 'source ...'])` from Python.

## Config File Locations

| File | Role |
|---|---|
| `~/.hermes/config.yaml` | Runtime config (the one load_config() reads) |
| `~/.hermes/profiles/<name>/config.yaml` | Profile-specific config |
| `/root/HERMES/config.yaml` | May be a hardlink to `~/.hermes/config.yaml` (same inode) |

Use `hermes config set` to modify, not direct file editing.

## Reference Files

This skill's directory includes:

| File | Content |
|------|---------|
| `references/compression-tuning-413-cascade.md` | **Compression tuning for 413 cascade from large image payloads in high-volume groups** |
| `references/prmt-architecture.md` | PRMT architecture, taxonomy comparison, related research |
| `references/mulerouter-integration.md` | MuleRouter as secondary vision provider (URL-only) |
| `references/minimax-base-url-sops-bug.md` | "Invalid IPv6 URL" — sops-encrypted URL env var |
| `references/2026-07-30-supports-vision-preempts-pathb.md` | `supports_vision: true` on text-only model |
| `references/openrouter-qwen-vl-config.md` | Qwen-VL via OpenRouter config |
| `references/mas-framework.md` | MAS framework mapping |
| `references/direct-minimax-vision-fallback.md` | Direct MiniMax M3 vision API |
| `references/2026-07-30-openrouter-vision-enrichment-chain.md` | OpenRouter vision enrichment chain |