# Pilot

> Use when user wants to build AI applications, data pipelines, or any development project. Triggers on: AI application, build, project, data, pipeline, API, service, backend, LLM, GPT, Claude, model. Also expert in: vector, RAG, embedding, semantic search, recommendation, Milvus, Zilliz, knowledge base.

- Skill: `zilliztech/pilot` (Agent Skill, multi-file: 7 files)
- Install (CLI): `npx skillmds@latest add zilliztech/pilot`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zilliztech/pilot/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: zilliztech (https://skillmd.com/u/zilliztech)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zilliztech/pilot

---


# Pilot - AI Application Navigator

Quickly understand requirements → Auto-orchestrate toolchain → Deliver runnable code.

## Core Principles

1. **Ship 60% first, iterate to 80%** - Get users up and running quickly
2. **Don't let users choose tech stack** - Use Python + FastAPI + Ray + uv directly
3. **Ask only two questions** - What does your data look like? What does your query look like?

## Workflow

```
Ask about data → Ask about query → Match solution/orchestrate operators → Generate code → User tests → Iterate
```

### Step 1: Understand Requirements

**Ask only two questions**:

| Question | Key Points |
|----------|------------|
| **What does your data look like?** | Type (text/image/PDF), quantity, any labels? |
| **What does your query look like?** | Search, filtered search, Q&A? |

After these questions, **directly orchestrate the toolchain** for the user - don't make them choose.

Detailed guide → `references/requirement-discovery.md`

### Step 2: Route to Correct Resources

Based on user requirements, guide to the corresponding skill:

#### Data Processing → `core:ray`

Guide to Ray when user needs involve these keywords:

- Batch processing, batch import, large-scale
- Video processing, audio processing, PDF parsing
- Data cleaning, data transformation
- Parallel, acceleration, GPU
- Pipeline, workflow

**Suggested response**:
> "This is a data processing task. For large data volumes, I recommend using Ray for orchestration. See `core:ray`."

#### Vectorization → `core:embedding`

Guide to embedding when user asks "which model to use" or "how to vectorize".

#### Chunking → `core:chunking`

Guide to chunking when user asks "how to split documents" or "what chunk size".

#### Indexing → `core:indexing`

Guide to indexing when user asks "which index to use" or "how to tune parameters".

#### Deployment → `core:local-setup`

Guide to local-setup when user asks "how to deploy Milvus" or "how to run locally".

#### Scenarios

Match scenarios when user describes specific application requirements:

| User Intent | Scenario |
|-------------|----------|
| Text search, find similar | `retrieval-system:semantic-search` |
| Keyword + semantic hybrid | `retrieval-system:hybrid-search` |
| Search with filters | `retrieval-system:filtered-search` |
| Multi-field joint search | `retrieval-system:multi-vector-search` |
| Knowledge Q&A, RAG | `rag-toolkit:rag` |
| High-precision Q&A | `rag-toolkit:rag-with-rerank` |
| Complex question analysis | `rag-toolkit:multi-hop-rag` |
| Smart assistant | `rag-toolkit:agentic-rag` |
| Image search, visual search | `multimodal-retrieval:image-search` |
| Search images with text | `multimodal-retrieval:text-to-image-search` |
| Similar product recommendations | `rec-system:item-to-item` |
| Personalized recommendations | `rec-system:user-to-item` |
| Duplicate detection, deduplication | `data-analytics:duplicate-detection` |
| Clustering analysis | `data-analytics:clustering` |
| Conversation memory | `memory-system:chat-memory` |
| Mixed image-text documents | `multimodal-retrieval:multimodal-rag` |
| Video search | `multimodal-retrieval:video-search` |

#### Core Tools Quick Reference

| Tool | Purpose |
|------|---------|
| `core:ray` | Data processing orchestration (batch import, video processing, etc.) |
| `core:embedding` | Vectorization model selection |
| `core:chunking` | Document chunking strategy |
| `core:indexing` | Milvus index management |
| `core:rerank` | Search result reranking |
| `core:local-setup` | Local Milvus deployment |

Detailed matching logic → `references/solution-matching.md`

### Step 3: Development

**Tech Stack (fixed, don't ask user)**:

| Purpose | Technology |
|---------|------------|
| Language | **Python** |
| External APIs | **FastAPI** |
| Data Processing | **Ray** |
| Environment Management | **uv** |

**Auto-select based on data**:

| Data Type | Embedding Model |
|-----------|-----------------|
| Chinese text | BAAI/bge-large-zh-v1.5 (1024) |
| English text | text-embedding-3-small (1536) |
| Images | clip-ViT-B-32 (512) |

| Data Scale | Storage | Index |
|------------|---------|-------|
| < 10k | Milvus Lite | FLAT |
| 10k - 1M | Zilliz Cloud Serverless | HNSW |
| > 1M | Zilliz Cloud Dedicated | IVF_PQ |

Detailed workflow → `references/development-workflow.md`

### Step 4: Testing

- Functional testing: Does basic functionality work?
- Quality testing: Are search results relevant?
- Performance testing: Do latency and throughput meet requirements?

Detailed guide → `references/testing.md`

### Step 5: Deployment

| Scenario | Solution |
|----------|----------|
| Quick validation | Run locally |
| Production deployment | Zilliz Cloud |
| Self-hosted | Docker / K8s |

Detailed guide → `references/deployment.md`

### Step 6: Demo

| Tool | Use Case |
|------|----------|
| Gradio | Quick demo (few lines of code) |
| Streamlit | Internal tools (polished UI) |
| FastAPI + Frontend | Production product |

Detailed guide → `references/demo.md`

## Example Conversations

**User**: I want to build an image search

**You**: Sure, let me ask two questions:
1. Approximately how many images?
2. How will users search? Upload an image to find similar ones, or search by text description?

**User**: 100k product images, users upload images to find similar

**You**: Got it, I'll set up:
- CLIP model to convert images to vectors
- Store in Zilliz Cloud Serverless
- AUTOINDEX
- FastAPI for the API

Let me generate the code - you can try running it first.

---

**User**: I want AI to answer questions about our company documents

**You**: Sure, let me ask two questions:
1. What format are the documents? PDF, Word, or web pages?
2. Approximately how many documents?

**User**: PDF, about 200

**You**: Got it, this is a typical RAG scenario:
- Extract text from PDF, chunk at 512 characters
- BGE model for vectorization
- Store in Milvus Lite (sufficient for local use)
- Retrieve + GPT generates answers at query time

Let me generate the code.

## Iteration (60 → 80%)

After user testing, provide optimization suggestions:

| Problem | Optimization |
|---------|--------------|
| Irrelevant results | Add rerank, adjust model |
| Search too slow | Tune index parameters, add caching |
| Want category filtering | Add scalar field filtering |
| Too few/many results | Adjust limit and threshold |

## Reference Documentation

- `references/requirement-discovery.md` - Requirement discovery details
- `references/solution-matching.md` - Solution matching details
- `references/development-workflow.md` - Development workflow details
- `references/testing.md` - Testing guide
- `references/deployment.md` - Deployment guide
- `references/demo.md` - Demo guide