How do I install the Dspy Adapters Multimodal skill?

Run npx skillmds@latest add omidzamani/dspy-adapters-multimodal in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Dspy Adapters Multimodal safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: executes scripts, makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Dspy Adapters Multimodal?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Dspy Adapters Multimodal free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Dspy Adapters Multimodal?

OmidZamani (@omidzamani) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Dspy Adapters Multimodal by omidzamani

Name: Dspy Adapters Multimodal
Rating: 4.5 (4 reviews)
Author: OmidZamani

File contents

DSPy Adapters and Multimodal I/O

Goal

Choose an adapter deliberately and model image, audio, and file inputs with DSPy's typed primitives.

Adapter Selection

Adapter	Use it for
`dspy.ChatAdapter()`	Default, human-readable field markers, broad model compatibility
`dspy.JSONAdapter()`	Structured JSON output and native function calling where supported
`dspy.XMLAdapter()`	XML-tagged fields when XML is easier for the target LM to follow
`dspy.TwoStepAdapter()`	A separate extraction pass when parsing needs extra help

Configure globally or for a limited scope:

import dspy

dspy.configure(
    lm=dspy.LM("openai/gpt-4o-mini"),
    adapter=dspy.JSONAdapter(),
)

with dspy.context(adapter=dspy.XMLAdapter()):
    result = dspy.Predict("question -> answer")(question="What is DSPy?")

Native Function Calling

JSONAdapter enables native function calling by default. ChatAdapter keeps text parsing by default. Override either behavior explicitly:

chat_native = dspy.ChatAdapter(use_native_function_calling=True)
json_manual = dspy.JSONAdapter(use_native_function_calling=False)

DSPy falls back to manual parsing when the configured LM does not support native function calling.

Image Inputs

class DescribeImage(dspy.Signature):
    image: dspy.Image = dspy.InputField()
    description: str = dspy.OutputField()

describe = dspy.Predict(DescribeImage)
result = describe(image=dspy.Image("./diagram.png"))

Pass a local path, HTTP URL, bytes, PIL image, or existing data URI directly to dspy.Image(...).

Audio and File Inputs

class SummarizeAudio(dspy.Signature):
    audio: dspy.Audio = dspy.InputField()
    summary: str = dspy.OutputField()

audio = dspy.Audio.from_file("./meeting.wav")
summary = dspy.Predict(SummarizeAudio)(audio=audio)

class SummarizeFile(dspy.Signature):
    file: dspy.File = dspy.InputField()
    summary: str = dspy.OutputField()

document = dspy.File.from_path("./research.pdf")
summary = dspy.Predict(SummarizeFile)(file=document)

Provider capabilities vary. Verify that the selected model accepts the media type before deployment.

Best Practices

Start with ChatAdapter; switch only for a measured reason.
Use typed signatures for structured output.
Test adapter behavior against the exact production model.
Avoid deprecated Image.from_file() and Image.from_url() helpers; call dspy.Image(...).
Keep local file handling and uploaded file IDs within provider policy.

Related Skills

Design signatures: dspy-signature-designer
Build tool agents: dspy-react-agent-builder

Official Documentation

Adapters guide: https://dspy.ai/learn/programming/adapters/
Tools guide: https://dspy.ai/learn/programming/tools/
XMLAdapter API: https://dspy.ai/api/adapters/XMLAdapter/
Image API: https://dspy.ai/api/primitives/Image/
Audio API: https://dspy.ai/api/primitives/Audio/

---name: dspy-adapters-multimodaldescription: Use for DSPy adapter selection, JSONAdapter, XMLAdapter, ChatAdapter, native function calling, structured outputs, and multimodal inputs like dspy.Image or dspy.Audio.--- # DSPy Adapters and Multimodal I/O ## Goal Choose an adapter deliberately and model image, audio, and file inputs with DSPy's typed primitives. ## Adapter Selection | Adapter | Use it for | |---------|------------| | `dspy.ChatAdapter()` | Default, human-readable field markers, broad model compatibility | | `dspy.JSONAdapter()` | Structured JSON output and native function calling where supported | | `dspy.XMLAdapter()` | XML-tagged fields when XML is easier for the target LM to follow | | `dspy.TwoStepAdapter()` | A separate extraction pass when parsing needs extra help | Configure globally or for a limited scope: ```python import dspy dspy.configure( lm=dspy.LM("openai/gpt-4o-mini"), adapter=dspy.JSONAdapter(), ) with dspy.context(adapter=dspy.XMLAdapter()): result = dspy.Predict("question -> answer")(question="What is DSPy?") ``` ## Native Function Calling `JSONAdapter` enables native function calling by default. `ChatAdapter` keeps text parsing by default. Override either behavior explicitly: ```python chat_native = dspy.ChatAdapter(use_native_function_calling=True) json_manual = dspy.JSONAdapter(use_native_function_calling=False) ``` DSPy falls back to manual parsing when the configured LM does not support native function calling. ## Image Inputs ```python class DescribeImage(dspy.Signature): image: dspy.Image = dspy.InputField() description: str = dspy.OutputField() describe = dspy.Predict(DescribeImage) result = describe(image=dspy.Image("./diagram.png")) ``` Pass a local path, HTTP URL, bytes, PIL image, or existing data URI directly to `dspy.Image(...)`. ## Audio and File Inputs ```python class SummarizeAudio(dspy.Signature): audio: dspy.Audio = dspy.InputField() summary: str = dspy.OutputField() audio = dspy.Audio.from_file("./meeting.wav") summary = dspy.Predict(SummarizeAudio)(audio=audio) ``` ```python class SummarizeFile(dspy.Signature): file: dspy.File = dspy.InputField() summary: str = dspy.OutputField() document = dspy.File.from_path("./research.pdf") summary = dspy.Predict(SummarizeFile)(file=document) ``` Provider capabilities vary. Verify that the selected model accepts the media type before deployment. ## Best Practices 1. Start with `ChatAdapter`; switch only for a measured reason. 2. Use typed signatures for structured output. 3. Test adapter behavior against the exact production model. 4. Avoid deprecated `Image.from_file()` and `Image.from_url()` helpers; call `dspy.Image(...)`. 5. Keep local file handling and uploaded file IDs within provider policy. ## Related Skills - Design signatures: [dspy-signature-designer](../dspy-signature-designer/SKILL.md) - Build tool agents: [dspy-react-agent-builder](../dspy-react-agent-builder/SKILL.md) ## Official Documentation - **Adapters guide**: https://dspy.ai/learn/programming/adapters/ - **Tools guide**: https://dspy.ai/learn/programming/tools/ - **XMLAdapter API**: https://dspy.ai/api/adapters/XMLAdapter/ - **Image API**: https://dspy.ai/api/primitives/Image/ - **Audio API**: https://dspy.ai/api/primitives/Audio/

Frequently asked questions

Dspy Adapters Multimodal