RAG Vectorize AI Search

RAG, Vectorize, and AI Search

Use this skill for retrieval augmented generation, semantic search, and document Q&A.

Choose AI Search vs Vectorize

• Use AI Search when you want managed ingestion, retrieval, hybrid search/reranking features, and simpler RAG setup.
• Use Vectorize when you need direct control over embedding generation, chunking, metadata, index operations, and query flow.
• Store source documents and metadata outside the index. Retrieval indexes are derived data.

RAG pipeline

Upload document
  -> R2 stores original bytes
  -> D1 stores document metadata and ingestion status
  -> Queue/Workflow parses and chunks
  -> Workers AI embeds chunks
  -> Vectorize or AI Search indexes chunks with metadata
  -> Query embeds/retrieves top chunks
  -> LLM answers only from retrieved context
  -> Response includes citations/source IDs

Chunking rules

• Chunk by semantic boundaries when possible: headings, paragraphs, sections.
• Include overlap for long passages.
• Store documentId, chunkId, tenantId, sourceUrl, title, and offsets/section metadata.
• Keep chunks short enough to fit prompt budgets after retrieving multiple results.
• Re-embed when chunking, model, or source text changes.

Vectorize index sketch

export interface Env {
  AI: Ai;
  VECTORIZE: VectorizeIndex;
}

export async function indexChunk(env: Env, chunk: {
  id: string;
  tenantId: string;
  documentId: string;
  text: string;
}) {
  const embedding = await env.AI.run("@cf/baai/bge-base-en-v1.5", { text: chunk.text });

  await env.VECTORIZE.upsert([
    {
      id: chunk.id,
      values: embedding.data[0],
      metadata: {
        tenantId: chunk.tenantId,
        documentId: chunk.documentId,
        text: chunk.text
      }
    }
  ]);
}

Query sketch

export async function retrieve(env: Env, tenantId: string, question: string) {
  const embedding = await env.AI.run("@cf/baai/bge-base-en-v1.5", { text: question });

  return env.VECTORIZE.query(embedding.data[0], {
    topK: 6,
    returnMetadata: true,
    filter: { tenantId }
  });
}

Verify filter syntax and model dimensions against current docs and the configured index.

Grounded answer prompt

You answer using only the supplied context.
If the context does not contain the answer, say you do not know.
Cite the chunk IDs used.

Context:
{{retrieved_chunks}}

Question:
{{question}}

Quality checks

• Test retrieval separately from generation.
• Inspect failed questions: no retrieval, wrong retrieval, or bad synthesis.
• Add metadata filters for tenant/user permissions before generation.
• Do not put private chunks from other tenants into the prompt.
• Track retrieval hit rate and answer abstention rate.

Cost controls

• Embed once during ingestion, not on every answer except the query embedding.
• Limit retrieved chunks.
• Use reranking only when it improves quality enough to justify cost.
• Use smaller models for query rewriting/classification and larger models only for final answers when needed.

Anti-patterns

• Model answers without retrieved context for factual document QA.
• Index contains text but no source/citation metadata.
• Tenant filter applied after retrieval instead of during retrieval.
• Entire documents pasted into prompts.
• RAG output claims citations that were not retrieved.