ionclaw-org Vision
Analyze images using the vision tool. When a user uploads an image, you receive an [image attached: path] annotation — use that path as the path parameter.
Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
path |
string | one of path/url/base64 | Path to a local image file (relative to project root, or absolute for temp files) |
url |
string | one of path/url/base64 | URL of a remote image to fetch and analyze |
base64 |
string | one of path/url/base64 | Base64-encoded image data (with or without data URI prefix) |
question |
string | no | Specific question about the image. If omitted, provides a general description |
mime_type |
string | no | Override MIME type (auto-detected from file extension or URL) |
How It Works
The vision tool loads an image from any source, resizes it to an LLM-friendly preview (max 1024px wide, JPEG), and passes it to the model as a visual content block. The model then "sees" the image and can describe, analyze, or answer questions about it — no external vision API required.
Common Workflows
Describe a local image
vision path="public/media/photo.jpg"
Analyze a screenshot
vision path="public/screenshots/page.png" question="What error message is shown?"
Analyze an image from a URL
vision url="https://example.com/chart.png" question="What trends does this chart show?"
Read text from an image (OCR)
vision path="public/documents/scan.png" question="Extract all text from this image"
Analyze a browser screenshot (with output_path)
browser action="screenshot" output_path="public/screenshots/page.png"
vision path="public/screenshots/page.png" question="What is displayed on this page?"
Use with base64 data
vision base64="data:image/png;base64,iVBOR..." question="What is this?"
Supported Formats
JPEG, PNG, GIF, WebP, SVG, BMP, ICO, TIFF, AVIF.
Max image size: 20MB.
Notes
- Provide exactly one image source:
path,url, orbase64 - MIME type is auto-detected from file extension or Content-Type header
- Use
questionto focus the analysis on specific aspects - The model's vision capabilities depend on the underlying LLM (Claude, GPT-4V, Gemini all support vision)