# Weaviate Modules > Implements custom Weaviate modules (functions, vectorizers, generative providers) to extend data processing capabilities with application-specific AI and ML functions. - Skill: `paulpas/weaviate-modules` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/weaviate-modules` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/weaviate-modules/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/weaviate-modules --- # Weaviate Module Integration Patterns Extends Weaviate's capabilities with custom modules including vectorizers, generative AI providers, and reference-qr functions. Enables application-specific data processing workflows within the vector database engine itself. ## TL;DR Checklist - [ ] Register module schemas via Weaviate's schema API before using them - [ ] Configure vectorizer modules with proper authentication (API keys, endpoints) - [ ] Use generative providers for in-database summarization and text generation - [ ] Test module functions against sample data before production deployment - [ ] Monitor module health and error rates via Weaviate metrics endpoints --- ## When to Use Use this skill when: - Building custom vectorizers that compute embeddings from non-text sources (images, audio, binary data) - Integrating generative AI providers (OpenAI, Cohere) for in-database summarization, Q&A, and text completion - Creating reference-qr functions to compute relationships between objects based on computed properties - Extending Weaviate's built-in modules with application-specific preprocessing or validation logic --- ## When NOT to Use - For standard text embedding — use Weaviate's built-in `text2vec-openai` or `text2vec-cohere` instead of custom code - For simple data transformations — do them in the application layer, not as modules - When performance is critical and module logic involves heavy computation — consider pre-computing embeddings externally --- ## Core Workflow 1. **Identify Module Type** — Determine whether you need a vectorizer (computes embeddings from input), a generative provider (runs LLM completions on objects), or a reference-qr function (computes object relationships). **Checkpoint:** Match module type to the data transformation needed. Vectorizers for embedding generation, generators for text processing, reference-qr for computed relations. 2. **Define Module Schema** — Create the collection with custom module configurations. Specify input properties, output properties, and any authentication or endpoint settings required by the module. **Checkpoint:** Validate that all referenced property names exist in the schema before creating collections. 3. **Invoke Module Functions** — Use Weaviate's query API to trigger module operations: `generate` blocks for generative modules, `$vector` references for vectorized properties, and explicit function calls for reference-qr. **Checkpoint:** Verify output properties are populated correctly by fetching the object after invocation. 4. **Handle Errors** — Module invocations can fail due to API key expiration, rate limits, or invalid input data. Implement retry logic and fall back to stored values when modules return errors. **Checkpoint:** Check response status codes and error messages; log failures with context for debugging. --- ## Implementation Patterns ### Pattern 1: Setting Up a Custom Module Collection ```python import weaviate from weaviate.classes.config import Property, DataType def create_custom_module_collection( client: weaviate.WeaviateClient, class_name: str = "Document", ) -> None: """Create a Weaviate collection configured with custom generative and vectorizer modules. Uses OpenAI's text2vec for automatic embedding and generative-openai for in-database summarization of document content. Args: client: Authenticated Weaviate client class_name: Name of the collection to create Note: Requires OPENAI_API_KEY environment variable to be set on the Weaviate server. """ properties = [ Property( name="title", data_type=DataType.TEXT, description="Document title or heading", ), Property( name="content", data_type=DataType.TEXT, description="Full document body content for summarization", ), Property( name="source_url", data_type=DataType.TEXT, description="Original URL where the document was sourced", ), ] collection = client.collections.create( name=class_name, properties=properties, vectorizer_config=[ weaviate.classes.config.Vectorizer.text2vec_openai(), ], generative_config=[ weaviate.classes.config.Generative.openai(), ], ) print(f"Collection '{class_name}' created with vectorizer + generative modules") def insert_and_summarize( client: weaviate.WeaviateClient, class_name: str, title: str, content: str, ) -> dict | None: """Insert a document and immediately generate a summary using the generative module. Args: client: Authenticated Weaviate client class_name: Target collection name title: Document title content: Full document body Returns: Summary string if generation succeeds, None on failure """ collection = client.collections.get(class_name) # Insert the object with automatic embedding from vectorizer module response = collection.data.insert( properties={ "title": title, "content": content, } ) print(f"Inserted object: {response}") # Query with generate block for in-database summarization result = ( collection.query.fetch_object_by_id(response) .with_generate( grouped_task="Summarize this document", grouped_properties=["content"], ) ) if result and hasattr(result, "generative"): return result.generative # Contains the generated summary print("Warning: Generation returned no result") return None ``` ### Pattern 2: Custom Reference-QR Function (Computed Relationships) ```python from weaviate.classes.query import Filter def compute_computed_relationships( client: weaviate.WeaviateClient, source_class: str = "Author", target_class: str = "Document", ) -> int: """Compute reference-qr relationships based on a custom rule. Links all documents written by each author where the document's content contains the author's name (simple keyword match). This demonstrates how to use computed properties for dynamic relationship mapping. Args: client: Authenticated Weaviate client source_class: Collection containing "Author" objects target_class: Collection containing "Document" objects Returns: Number of relationships established """ authors_collection = client.collections.get(source_class) docs_collection = client.collections.get(target_class) # Fetch all authors with their names authors = list(authors_collection.query.fetch_objects(limit=1000).objects) relationships_created = 0 for author in authors: author_name = author.properties.get("name", "") if not author_name: continue # Find documents whose content mentions this author's name matching_docs = list( docs_collection.query.fetch_objects( filters=Filter.by_text("content").like(f"*{author_name}*"), limit=100, ).objects ) for doc in matching_docs: # Add reference from author to document authors_collection.data.reference.add( from_object_uuid=author.uuid, property="wrote", # Pre-existing reference property to_object_uuid=doc.uuid, ) relationships_created += 1 print(f"Created {relationships_created} author-document relationships") return relationships_created ``` --- ## Constraints ### MUST DO - Define all properties in the schema before inserting objects — missing properties cause silent failures - Set environment variables for API keys (OPENAI_API_KEY, COHERE_API_KEY) on the Weaviate server, not in client code - Test module invocations against a single object before scaling to batch operations - Monitor module health via `/v1/meta` endpoint and `/metrics` for Prometheus scraping - Implement fallback logic when generative modules fail — store raw content as an alternative ### MUST NOT DO - Call generative modules in tight loops without rate limiting — you will hit provider API quotas - Use vectorizers with expensive models (e.g., 1536-dim OpenAI ada) on collections with millions of objects without testing performance - Reference non-existent properties in module configurations — Weaviate will reject the schema silently or throw on first use --- ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Weaviate Module Documentation](https://weaviate.io/developers/weaviate/module-ref) - [Vectorizer Modules Reference](https://weaviate.io/developers/weaviate/module-ref/vectorizers) - [Generative AI Modules](https://weaviate.io/developers/weaviate/module-ref/generative-providers) - [Reference-QR Module](https://weaviate.io/developers/weaviate/module-ref/reference-qr-module) - [Schema Design Best Practices](https://weaviate.io/developers/weaviate/starter-guide/schema)