Qwen-MM-Plugins Core
You have qwen-mm-plugins-core MCP tools available. Use them to read and visualize supported local files, inspect media metadata, crop/annotate images, and save document pages or video frames. Prefer these MCP tools over manual scripting.
In the default native mode, these tools return text and images for the agent to inspect without a model API call. With QWEN_MM_NATIVE_MODE=0, the shared caption fallback sends image results to the configured VL endpoint and requires DASHSCOPE_API_KEY.
Check the qwen-mm-plugins-core tools in your tool list for full schemas and parameters.
When to Use Which Tool
Reading and inspection:
- Inspect metadata FIRST for any video/audio (duration, resolution, fps, codecs, bitrate, audio/video/subtitle tracks, rotation, chapters) →
media_info. Run it before read_video and before any clip/edit — see Metadata first below.
- See a file (PDF, Office, CSV, code, notebook, 3D, ...) →
visualize
- Read an image with dynamic resolution →
read_image
- Read a video (extract frames) →
read_video
- Save specific frame(s) of a video to file →
save_view (pass times=[...])
- Save document page(s) as images →
save_view (pass pages="...")
Producing / annotating (writes an image file):
- Crop a rectangular region from an image →
crop
- Photo coordinates:
read_image, crop, and draw_bbox apply EXIF orientation. Use 0–1000 coordinates in the displayed image, including boxes returned by grounding.
- Draw bounding boxes on an image →
draw_bbox
Visualize — Supported Formats
| Category |
Extensions |
Notes |
| Documents |
.pdf, .svg |
Built-in (pypdfium2 + resvg) |
| Office |
.docx, .pptx, .vsdx |
Needs libreoffice |
| Data |
.csv, .xlsx |
Text table + chart image |
| Code |
.js, .ts, .py, .go, .rs, .md, ... |
Returns text (markdown code block) |
| Plain text |
.txt, .text, .log |
Returns text (fenced block) |
| Web pages |
.html, .htm, .mhtml |
Screenshot; needs playwright |
| Diagrams |
.drawio |
XML → SVG rendering |
| Subtitles |
.srt, .vtt |
Returns text |
| 3D Models |
.obj, .stl, .glb, .gltf, .fbx, .ply, .step, .stp |
Built-in; blender for best quality |
| Medical volumes |
.nii, .nii.gz |
Local/read-only (nibabel); 3 center slices; 4D pages selects volumes (default 1) |
| GIS/Geo |
.geojson, .kml, .shp |
Built-in |
| Notebooks |
.ipynb |
Text cells + embedded images |
| LaTeX |
.tex |
Compiles to PDF; falls back to source on failure |
| Images/Videos |
.jpg, .mp4, ... |
Delegates to read_image/read_video |
The table lists supported formats; files with unknown extensions return an unsupported-type error.
Use pages for page ranges, budget for resolution, max_pages to cap output.
NIfTI uses closest-canonical voxel axes without resampling and is intended for inspection, not
clinical diagnosis.
Metadata First
Always run media_info on a video/audio file before read_video and before any clip/trim/concat/mux — it reads only the header, so it is fast even on huge files, and it tells you how to treat the source. read_video also samples smarter once you know the fps and duration.
When juggling heterogeneous assets (action-cam clips, VFX/stock footage, voiceover, SFX), media_info is how you catch the traps that silently corrupt an edit — check for:
- VFR (variable frame rate — common on phones/action cams):
media_info flags it as possibly VFR. Frame-accurate cuts need a constant-fps conform first (a video-edit step) — never assume a fixed fps on VFR sources.
- Rotation (
rotation 90°/180°): the display orientation differs from stored pixels; bake it in before cropping or overlaying.
- Mismatched fps / resolution / DAR across clips → pick one project timebase and conform the outliers.
- Audio sample rate & channels (voiceover vs music vs SFX often differ, e.g. 44.1k vs 48k, mono vs stereo) → resample to one project rate before mixing.
- No audio track / odd container start_time →
media_info calls these out; both shift or break naive timestamp math.
Tips
Resolution budgets: small for preview, normal (~1024) default, large for fine detail.
Video strategy: fps=0 auto-selects. Skim with fps=1, budget="normal" in 5-min chunks, then detail with fps=2, budget="large" on interesting segments. Use start_time/end_time for windowing.
Relationship to Other Capabilities
Core handles file reading, rendering, and basic image operations. Install these capabilities for dedicated model inference or search:
- Understand media with a model →
qwen-mm-plugins-api, grouped by model family: VL (vision_chat, ocr, grounding), Omni A/V (timestamped captioning, multi-speaker ASR, temporal grounding, event counting), plus transcribe_audio and segmentation. Annotate its grounding output with draw_bbox here.
- Confirm a fact / identify an entity (reverse image + web) →
qwen-mm-plugins-search. Grab the frame with save_view here first.
1---2name: qwen-mm-plugins-core3description: Read and visualize any file — images, video, documents, code, data, 3D models, NIfTI volumes, and more — with MCP tools. Use when the agent needs to inspect file contents or media metadata, crop an image, draw bounding boxes, or save document pages and video frames.4---56# Qwen-MM-Plugins Core78You have `qwen-mm-plugins-core` MCP tools available. Use them to read and visualize supported local files, inspect media metadata, crop/annotate images, and save document pages or video frames. Prefer these MCP tools over manual scripting.910In the default native mode, these tools return text and images for the agent to inspect without a model API call. With `QWEN_MM_NATIVE_MODE=0`, the shared caption fallback sends image results to the configured VL endpoint and requires `DASHSCOPE_API_KEY`.1112Check the `qwen-mm-plugins-core` tools in your tool list for full schemas and parameters.1314## When to Use Which Tool1516Reading and inspection:17- **Inspect metadata FIRST** for any video/audio (duration, resolution, fps, codecs, bitrate, audio/video/subtitle tracks, rotation, chapters) → `media_info`. Run it before `read_video` and before any clip/edit — see *Metadata first* below.18- **See a file** (PDF, Office, CSV, code, notebook, 3D, ...) → `visualize`19- **Read an image** with dynamic resolution → `read_image`20- **Read a video** (extract frames) → `read_video`21- **Save specific frame(s)** of a video to file → `save_view` (pass `times=[...]`)22- **Save document page(s)** as images → `save_view` (pass `pages="..."`)2324Producing / annotating (writes an image file):25- **Crop a rectangular region** from an image → `crop`26- **Photo coordinates**: `read_image`, `crop`, and `draw_bbox` apply EXIF orientation. Use 0–1000 coordinates in the displayed image, including boxes returned by `grounding`.27- **Draw bounding boxes** on an image → `draw_bbox`2829## Visualize — Supported Formats3031| Category | Extensions | Notes |32|----------|-----------|-------|33| Documents | `.pdf`, `.svg` | Built-in (pypdfium2 + resvg) |34| Office | `.docx`, `.pptx`, `.vsdx` | Needs `libreoffice` |35| Data | `.csv`, `.xlsx` | Text table + chart image |36| Code | `.js`, `.ts`, `.py`, `.go`, `.rs`, `.md`, ... | Returns text (markdown code block) |37| Plain text | `.txt`, `.text`, `.log` | Returns text (fenced block) |38| Web pages | `.html`, `.htm`, `.mhtml` | Screenshot; needs `playwright` |39| Diagrams | `.drawio` | XML → SVG rendering |40| Subtitles | `.srt`, `.vtt` | Returns text |41| 3D Models | `.obj`, `.stl`, `.glb`, `.gltf`, `.fbx`, `.ply`, `.step`, `.stp` | Built-in; `blender` for best quality |42| Medical volumes | `.nii`, `.nii.gz` | Local/read-only (`nibabel`); 3 center slices; 4D `pages` selects volumes (default 1) |43| GIS/Geo | `.geojson`, `.kml`, `.shp` | Built-in |44| Notebooks | `.ipynb` | Text cells + embedded images |45| LaTeX | `.tex` | Compiles to PDF; falls back to source on failure |46| Images/Videos | `.jpg`, `.mp4`, ... | Delegates to `read_image`/`read_video` |4748The table lists supported formats; files with unknown extensions return an unsupported-type error.4950Use `pages` for page ranges, `budget` for resolution, `max_pages` to cap output.5152NIfTI uses closest-canonical voxel axes without resampling and is intended for inspection, not53clinical diagnosis.5455## Metadata First5657Always run `media_info` on a video/audio file **before** `read_video` and before any clip/trim/concat/mux — it reads only the header, so it is fast even on huge files, and it tells you how to treat the source. `read_video` also samples smarter once you know the fps and duration.5859When juggling heterogeneous assets (action-cam clips, VFX/stock footage, voiceover, SFX), `media_info` is how you catch the traps that silently corrupt an edit — check for:60- **VFR** (variable frame rate — common on phones/action cams): `media_info` flags it as `possibly VFR`. Frame-accurate cuts need a constant-fps conform first (a video-edit step) — never assume a fixed fps on VFR sources.61- **Rotation** (`rotation 90°/180°`): the display orientation differs from stored pixels; bake it in before cropping or overlaying.62- **Mismatched fps / resolution / DAR** across clips → pick one project timebase and conform the outliers.63- **Audio sample rate & channels** (voiceover vs music vs SFX often differ, e.g. 44.1k vs 48k, mono vs stereo) → resample to one project rate before mixing.64- **No audio track** / **odd container start_time** → `media_info` calls these out; both shift or break naive timestamp math.6566## Tips6768**Resolution budgets**: `small` for preview, `normal` (~1024) default, `large` for fine detail.6970**Video strategy**: `fps=0` auto-selects. Skim with `fps=1, budget="normal"` in 5-min chunks, then detail with `fps=2, budget="large"` on interesting segments. Use `start_time`/`end_time` for windowing.7172## Relationship to Other Capabilities7374Core handles file reading, rendering, and basic image operations. Install these capabilities for dedicated model inference or search:75- **Understand media with a model** → `qwen-mm-plugins-api`, grouped by model family: VL (`vision_chat`, `ocr`, `grounding`), Omni A/V (timestamped captioning, multi-speaker ASR, temporal grounding, event counting), plus `transcribe_audio` and `segmentation`. Annotate its `grounding` output with `draw_bbox` here.76- **Confirm a fact / identify an entity** (reverse image + web) → `qwen-mm-plugins-search`. Grab the frame with `save_view` here first.
QwenLM