LLM App Patterns

LLM Application Patterns

Version: 9.0 | Updated: 01-July-2026 | Architect: Karim Bhalwani | Deps: architect, data-engineering, ops

Dependencies

Load the following via read_file before using this skill:

• ~/.copilot/skills/architect/SKILL.md - module boundary and API contract patterns; LLM app components (retriever, generator, evaluator) are black-box modules
• ~/.copilot/skills/data-engineering/SKILL.md - ingestion and pipeline patterns for corpus preparation, embedding pipelines, and index refresh
• ~/.copilot/skills/ops/SKILL.md - CI/CD and deployment patterns for LLMOps pipelines, model versioning, and evaluation automation

If any dependency SKILL.md cannot be loaded, notify the user of the missing dependency and proceed only with the capabilities that do not require it, clearly stating which guidance may be incomplete.

Expert in production LLM application patterns and architectures.

When to Use This Skill

Use when:

• Building production RAG (Retrieval-Augmented Generation) pipelines
• Implementing AI agents with tool use and multi-step reasoning
• Designing prompt engineering strategies and template systems
• Setting up LLMOps: monitoring, logging, tracing, and evaluation
• Deploying LLM applications with caching, rate limiting, and fallbacks
• Choosing between different agent architectures (ReAct, function calling, plan-execute, multi-agent)
• Optimizing retrieval: chunking strategies, vector databases, hybrid search
• Building production-ready systems: cost optimization, reliability, observability

Core Capabilities

This skill provides production-proven patterns for:

1. RAG Pipelines - Document ingestion, chunking, embedding, retrieval, generation
2. Agent Architectures - ReAct, function calling, plan-execute, multi-agent collaboration
3. Prompt Engineering - Templates, versioning, A/B testing, chaining
4. LLMOps & Monitoring - Metrics, logging, tracing, evaluation frameworks
5. Production Patterns - Caching, rate limiting, retry logic, fallbacks

Pattern References

For detailed implementation guidance, see:

RAG Pipelines

Use when: Building search-augmented LLM applications

Covers:

• Document ingestion and preprocessing
• Chunking strategies (fixed, semantic, sliding window)
• Vector database selection and configuration
• Retrieval patterns (dense, sparse, hybrid, multi-vector)
• Generation with retrieved context

Agent Architectures

Use when: Building agents that use tools or multi-step reasoning

Covers:

• ReAct pattern (Reasoning + Acting)
• Function calling for structured tool use
• Plan-and-execute for complex tasks
• Multi-agent collaboration patterns
• Architecture decision matrix

Prompt Engineering

Use when: Designing prompt templates and chains within an LLM application (not for building a standalone reusable prompt template library - use prompt-library for that)

Covers:

• Prompt templates with variables
• Versioning and A/B testing
• Prompt chaining for multi-step workflows
• Few-shot learning patterns
• Best practices for prompt structure

LLMOps & Observability

Use when: Setting up monitoring and evaluation

Covers:

• Key metrics to track (performance, quality, cost, reliability)
• Logging and distributed tracing
• Evaluation frameworks and benchmarking
• Caching strategies for cost reduction
• Rate limiting and retry patterns
• Fallback strategies for reliability

Quick Decision Guide

Definition of Done

• Selected pattern has clear justification (RAG vs. agent vs. fine-tune decision documented)
• Observability configured: latency, token usage, and error rate metrics defined
• Fallback strategy exists for LLM provider failures
• Prompt templates are versioned and stored outside application code
• Cost estimate provided for expected query volume

Constraints

• Does NOT write production application code or infrastructure configuration. Provides architecture diagrams, decision guidance, and starter prompt/config templates only.
• Does NOT evaluate model quality or fine-tuning strategies
• Does NOT manage infrastructure (consult ops skill)
• Does NOT handle non-LLM ML patterns (classical ML, computer vision, etc.)

Common Pitfalls

• Over-engineering RAG: Start with naive RAG before adding reranking, hybrid search, or agentic retrieval
• Ignoring cost: LLM API costs scale with query volume; always estimate and set budgets
• Missing fallbacks: Every LLM call needs a fallback strategy (retry, cheaper model, cached response)
• Prompt drift: Unversioned prompts silently degrade; version-control all prompt templates
• Evaluation gaps: Deploy without eval framework = flying blind. Measure before and after every change

Integration Points

• architect: Defines overall system architecture into which LLM patterns fit
• data-engineering: Manages data pipelines feeding RAG knowledge bases
• genai-security: Audits LLM applications for security vulnerabilities
• ai-engineer: Implements the patterns defined by this skill
• ops: Deploys and monitors LLM applications in production

References

Load these to apply the correct pattern for the task at hand. If the task spans multiple references (e.g., an agentic RAG system requiring both agent-architectures.md and rag-pipelines.md), load all relevant references and synthesize guidance, noting where the references interact.

• rag-pipelines.md - RAG architecture patterns (naive, advanced, agentic). Load when designing or reviewing any retrieval-augmented generation system.
• agent-architectures.md - LLM agent topology patterns (ReAct, Plan-and-Execute, multi-agent). Load when designing autonomous agent systems or tool-use workflows.
• prompt-engineering.md - Prompt design patterns (few-shot, chain-of-thought, structured output). Load when writing or reviewing system prompts or task prompts.
• llmops-observability.md - LLMOps monitoring and evaluation patterns. Load when setting up eval frameworks, tracing, or production observability for LLM applications.
• pattern_summary.md - One-page pattern decision guide. Load first when unsure which pattern applies - use to select the right reference file before diving deeper.