Multimodal Dataprep Dev

Multimodal DataPrep — Dev

Work from microservices/visual-data-preparation-for-retrieval/multimodal-dataprep/ inside an edge-ai-libraries checkout. If the request only deploys or consumes the service, use ../multimodal-dataprep-user/SKILL.md.

Start with the relevant reference

Example tasks:

Build-context rule

Use ./build.sh; do not run docker build ... . from this directory. The Dockerfile copies both this service and the sibling multimodal-embedding-serving source, so its context must be microservices/. build.sh supplies that context.

Local development loop

poetry install --with dev
poetry run python -m pytest tests
poetry run coverage run --rcfile ./pyproject.toml -m pytest tests
poetry run coverage report -m
poetry run black --check src tests
poetry run isort --check-only src tests

Run a focused test file while iterating:

poetry run python -m pytest tests/test_vectorstores.py

Do not conceal collection failures or attribute unrelated failures to the current change. See references/testing-and-build.md.

Build and run

setup.sh must be sourced. With no argument it exports defaults and creates the YOLOX model volume; it does not start the stack.

export MINIO_ROOT_USER='<user>'
export MINIO_ROOT_PASSWORD='<strong-password>'
export EMBEDDING_MODEL_NAME='CLIP/clip-vit-b-32'
source ./setup.sh --nosetup

./build.sh
docker compose -f docker/compose.yaml up -d --build

Other supported setup actions are --conf, --down, --build [custom-tag], and --nd (foreground docker compose ... up --build).

For Milvus, use docker/compose-milvus.yaml. For local media storage, layer docker/compose.storage-local.yaml after the default compose file.

Current architecture

src/main.py creates the FastAPI app at /v1/dataprep, starts the optional Metrics Manager publisher, preloads the embedding client and YOLOX detector, and asks the active vector store to update its index during shutdown.

Requests flow through src/endpoints/ into src/core/embedding/embedding_orchestrator.py. Video work is executed by the threaded/shared-memory pipeline in embedding_helper.py; client.py wraps the in-process model from the sibling embedding package and persists vectors through src/core/vectorstores/. Media bytes and metadata go through src/core/storage/.

Change rules

• Preserve backend neutrality. Use get_vector_store() and get_storage() rather than importing a concrete backend in endpoint or orchestration code.
• Keep search/query behavior out of this service; BaseVectorStore covers ingestion-time add, delete, health, and index-update operations.
• Add endpoint schemas in src/common/schema.py and include routers in src/main.py.
• Keep media routes under /media; supported inputs include MP4 video and common image formats, plus text summaries at /summary.
• Mock external storage, model, and vector-store calls in unit tests. The Milvus integration test is opt-in through MILVUS_IT_URI.
• Add the repository SPDX header to every new source, config, test, or documentation file.
• Never commit credentials.

Important configuration facts