Image Search

Composed Text + Image Search

Search indexed videos using an image combined with optional text for more precise results. The text narrows image-based results — for example, an image of a car plus "red color" finds only red versions of that car model.

When to Use This Skill

Use this skill when the user:

• Provides a reference image and wants to find similar content in videos
• Wants to combine an image with text to refine results ("find this but in blue")
• Asks to search with a picture or screenshot
• Wants to locate specific objects/scenes matching a visual reference

Prerequisites

• Videos must be indexed with Marengo 3.0 (composed search requires Marengo 3.0)
• A reference image: either a publicly accessible URL or a local file path

Instructions

Step 1: Get the Image and Query

From the user's request, extract:

• Image: A publicly accessible URL or an absolute path to a local image file
• Text query (optional): Additional text to refine the image search

Step 2: Call the Search Tool

Image-only search (find content similar to the image):

With a URL:

Tool: mcp__twelvelabs-mcp__search
Parameters:
  queryMediaUrl: "<image URL>"
  queryMediaType: "image"

With a local file:

Tool: mcp__twelvelabs-mcp__search
Parameters:
  queryMediaFile: "<absolute path to image file>"
  queryMediaType: "image"

Composed search (image + text refinement):

With a URL:

Tool: mcp__twelvelabs-mcp__search
Parameters:
  query: "<text description>"
  queryMediaUrl: "<image URL>"
  queryMediaType: "image"

With a local file:

Tool: mcp__twelvelabs-mcp__search
Parameters:
  query: "<text description>"
  queryMediaFile: "<absolute path to image file>"
  queryMediaType: "image"

To search a specific index, add indexId.

Step 3: Display Results

Format results the same way as regular text search:

For each video with matching segments, display:

• Filename of the video (if available)
• URL of the video stream (if available)
• Bulleted list of start and end times for matching segments

Example output:

Found matches for image search with "red color":

**demo_video.mp4**
Stream: https://stream.url/video.m3u8

Matching segments:
- 00:12 - 00:28 (16 seconds)
- 01:45 - 02:03 (18 seconds)

Step 4: Format Timestamps

Same as regular search:

• Under 1 hour: MM:SS
• 1 hour or more: HH:MM:SS
• Show duration in parentheses

Use Cases

Example Interactions

User: "Find scenes in my videos that look like this image: https://example.com/sunset.jpg" Action: Image-only search with queryMediaUrl

User: "Search for this car but in blue: https://example.com/car.jpg" Action: Composed search with query "blue color" + queryMediaUrl

User: "Find this building at night: https://example.com/building.jpg" Action: Composed search with query "at night" + queryMediaUrl

Important Notes

• Marengo 3.0 Required: Composed search only works with indexes created using Marengo 3.0
• Image Source: Supports both publicly accessible URLs (queryMediaUrl) and local file paths (queryMediaFile)
• Text Refines Image: In composed search, the text narrows results from the image match — it doesn't work independently
• Visual Search: Composed search uses the visual search option