Multimodal Dataprep User

Multimodal DataPrep — User

Run deployment and API commands when authorized, then report their actual output. API base: http://localhost:6007/v1/dataprep.

Multimodal DataPrep creates embeddings and metadata for retrieval. It does not offer a vector-query/search endpoint.

Load project resources as needed

Paths are relative to microservices/visual-data-preparation-for-retrieval/multimodal-dataprep/.

Example scenarios:

1. Select deployment backends

Use MM_DATAPREP_VECTORDB_BACKEND (vdms or milvus) and MM_DATAPREP_STORAGE_BACKEND (minio or local) for custom deployments. Keep the service, retriever, and collection naming consistent.

2. Obtain deployment files

In a repository checkout, run from the microservice root.

Without a checkout, fetch the setup script and the compose file(s) for the selected backend:

RAW='https://raw.githubusercontent.com/open-edge-platform/edge-ai-libraries/main/microservices/visual-data-preparation-for-retrieval/multimodal-dataprep'
mkdir -p multimodal-dataprep/docker
cd multimodal-dataprep
curl -fsSLo setup.sh "$RAW/setup.sh"
curl -fsSLo docker/compose.yaml "$RAW/docker/compose.yaml"

Also fetch docker/compose-milvus.yaml or docker/compose.storage-local.yaml when selected.

3. Start the prebuilt stack

Never commit credentials. For the default VDMS + MinIO deployment:

export MINIO_ROOT_USER='<user>'
export MINIO_ROOT_PASSWORD='<strong-password>'
export EMBEDDING_MODEL_NAME='CLIP/clip-vit-b-32'
export REGISTRY_URL='docker.io/intel'
export TAG='latest'

source ./setup.sh --nosetup
docker compose -f docker/compose.yaml up -d --no-build

For Milvus, replace the compose file with docker/compose-milvus.yaml. For local media storage, layer the storage override after docker/compose.yaml.

setup.sh must be sourced because it exports Compose variables. With no argument it only exports defaults; it does not start containers.

4. Verify readiness and dependencies

Wait for the in-process embedding client:

until curl -fsS http://localhost:6007/v1/dataprep/health \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); raise SystemExit(0 if d.get("status") == "ok" and d.get("embedding_client_status") == "preloaded" else 1)'
do
  sleep 10
done

The current fields are embedding_client_status, model_name, embedding_device, use_openvino, detection_model, and detection_device.

Check dependencies separately:

docker compose -f docker/compose.yaml ps
curl -fsS http://localhost:6010/minio/health/live
timeout 2 bash -c '</dev/tcp/localhost/6020'   # default VDMS

For Milvus, use the Milvus compose file and probe localhost:19530. DataPrep's health response is not a substitute for checking every dependency.

5. Ingest supported media

Upload a video:

curl -fsS -X POST \
  'http://localhost:6007/v1/dataprep/media/upload?frame_interval=15&enable_object_detection=true&tags=camera-1' \
  -F 'file=@/path/to/video.mp4;type=video/mp4'

Upload an image through the same multipart endpoint:

curl -fsS -X POST \
  'http://localhost:6007/v1/dataprep/media/upload?enable_object_detection=true&tags=inspection' \
  -F 'file=@/path/to/image.jpg;type=image/jpeg'

For inline base64 or remote HTTP(S) images, use POST /media/ingest. For media already in the selected storage backend, use POST /media/process.

Asynchronous workflows:

• POST /media/upload/batch
• POST /media/process/batch
• POST /media/ingest/batch
• POST /media/ingest-dir (also store_copy: false to embed files in place without copying them into storage — such media is still listed by GET /media with "stored": false and streamable via GET /media/download — and metadata / meta/<basename>.json sidecars for user-defined filterable fields)
• GET /media/jobs/{job_id}
• DELETE /media/jobs/{job_id} to request cancellation

Clean-up: DELETE /media/{bucket_name}/{video_id} removes one item, DELETE /media/{bucket_name} clears a whole bucket (storage + embeddings).

Read the API reference for exact request schemas and configured batch limits.

6. Add a text summary

Use the video_id returned by media listing/job results and its bucket:

curl -fsS -X POST 'http://localhost:6007/v1/dataprep/summary' \
  -H 'Content-Type: application/json' \
  -d '{
    "bucket_name": "video-summary",
    "video_id": "dp_video_1730000000",
    "video_summary": "forklift narrowly misses pedestrian",
    "video_start_time": 33,
    "video_end_time": 41,
    "tags": ["safety"]
  }'

The selected model must support text embeddings.

7. Manage and observe

curl -fsS 'http://localhost:6007/v1/dataprep/media'
curl -fsS 'http://localhost:6007/v1/dataprep/telemetry?limit=5'
curl -L 'http://localhost:6007/v1/dataprep/media/download?video_id=dp_video_1730000000' \
  -o media.bin

GET /media/download supports HTTP Range requests for seeking.

Deletion removes the entire media directory and its matching vectors:

curl -fsS -X DELETE \
  'http://localhost:6007/v1/dataprep/media/video-summary/dp_video_1730000000'

There is no current single-file video_name deletion option. Deletion is destructive, so obtain explicit confirmation before running it.

Set MM_DATAPREP_METRICS_MANAGER_URL to publish completed-pipeline dataprep_embeddings_per_second values asynchronously. /telemetry remains the direct source for detailed per-ingestion stage timings.

Troubleshooting