# RAG Pipeline

> Details on the Retrieval Augmented Generation pipeline, Ingestion, and Vector Search.

- Skill: `aiskillstore/rag-pipeline` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add aiskillstore/rag-pipeline`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aiskillstore/rag-pipeline/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: aiskillstore (https://skillmd.com/u/aiskillstore)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/aiskillstore/rag-pipeline

---


# RAG Pipeline Logic

## Ingestion
- **Script**: `backend/ingest.py`
- **Process**:
    1. Scans `docs/`.
    2. Cleans MDX (removes frontmatter/imports).
    3. Chunks text (1000 chars, 100 overlap).
    4. Embeds using `models/text-embedding-004`.
    5. Upserts to Qdrant collection `physical_ai_book`.
- **Run**: `python backend/ingest.py`

## Vector Search (Qdrant)
- **Client**: `qdrant-client`
- **Collection**: `physical_ai_book`
- **Vector Size**: 768 (Gecko-004)
- **Similarity**: Cosine

## Prompt Engineering
- **File**: `backend/utils/helpers.py`.
- **RAG Prompt**: Constructs a prompt containing retrieved context chunks.
- **Personalization**: `backend/personalization.py` creates system instructions based on `software_background` and `hardware_background` of the user.

## Agentic Flow
We use a custom `Agent` class (`backend/agents.py`) that wraps the LLM calls, allowing for future expansion into multi-agent workflows.