BejeweledMe LLM System Design
Design an LLM product as a composition of capability, data, control, and release contracts. This skill decides what belongs in the product architecture and why; specialist skills implement the chosen component.
Core Rules
- Start with the user task, error cost, data-access boundary, latency/cost contract, volume, deployment constraints, and fallback before selecting a model or pattern.
- Compare the smallest viable capability path: deterministic flow or prompt, retrieval, agent workflow, model adaptation, then self-hosted runtime work only when its benefits are measurable.
- Treat model/provider, prompt/template, context policy, index, tool policy, routing, limits, and evaluations as an attributable release bundle.
- Keep architecture-level budgets separate from runtime tuning: choose the user
contract here; use
$llm-inference-optimizationto make a self-hosted route meet it. - Do not hide retrieval, agent, safety, or evaluation decisions inside a prompt.
Reference Routing
Read only the files needed for the current request.
- Start with capability selection for a new LLM product, a design review, or a prompt/RAG/agent/adaptation decision.
- For provider/model portfolio, prompts, context windows, structured output, and model routing, read models, prompts, and context.
- For service topology, fallbacks, product budgets, and graceful degradation, read architecture and routing.
- For tenancy, release bundles, rollout, rollback, and operating controls, read release and tenancy.
- For a design or review template, read templates.
Boundaries
This Skill Owns
- whether the product needs an LLM at all, and the choice between prompt, RAG, agent workflow, or behavior adaptation;
- product-level choice between a managed provider and a self-hosted route, model portfolio, and request-routing policy;
- prompt and context architecture, context/token budgets, product latency/cost targets, user-visible degradation, and release topology;
- tenant isolation and versioning of the complete LLM product bundle.
Not This Skill
- Shared registry, replica placement, controller scaling, fleet routing, and
training-job infrastructure belong to
$ai-platform-llmops. Keep product quality, cost, autonomy, and fallback requirements here. - Neural step/memory/collective execution and coherent training-state restart
belong to
$neural-training-systems; model and adaptation choices stay with their modeling owner. - HTTP/OpenAPI artifacts and observable wire/consumer compatibility belong to
$api-contract-engineering. - Retrieval, chunking, index construction, reranking, and recall diagnosis belong
to
$rag-engineering. - Tool loops, state machines, handoffs, retries, and multi-agent execution belong
to
$agent-workflows. - Fine-tuning method, classical NLP/encoder choice, tokenizer changes, training
data, and adaptation evaluation belong to
$nlp-modeling-and-adaptation. - Computer-vision task framing and perception topology belong to
$computer-vision-system-design; visual model training belongs to$computer-vision-modeling-and-training. This skill owns the LLM/VLM product composition around prompts, provider routing, context, tools, budgets, and user-visible fallback. - TTFT, prefill/decode, KV cache, batching, quantization, GPUs, and serving
capacity belong to
$llm-inference-optimization. - Eval harnesses, graders, regression gates, and calibration belong to
$agent-llm-evals; threat modeling and authorized boundary testing belong to$genai-security-testing. - Generic storage, queues, architectural service guarantees, and distributed topology belong
to
$system-design; SLOs, incidents, and on-call operations belong to$sre-reliability-engineering.
Workflow
- Define the task contract: allowed inputs and sources, required evidence, error/abstention behavior, privacy boundary, volume, and product budget.
- Establish a simple non-agent baseline. Select prompt, RAG, agent, adaptation, or a combination based on the failure that remains, not fashion.
- Assign one primary owner for each selected component and request the necessary specialist design. Do not duplicate their implementation guidance here.
- Specify model/provider routing, context and output limits, failure behavior, tenant controls, and the versioned release bundle.
- Require evaluation, security, rollout, rollback, and operating evidence before calling the product ready.
Output
For a new design or review, include the user/task contract; chosen capability pattern and rejected alternatives; model/provider and context architecture; component handoffs; product latency/cost/token budgets; data and tenancy boundary; degradation/fallback behavior; release bundle; evaluation/security/operations plan; risks, assumptions, and next discriminating experiment.
Quality Bar
- Do not use an agent when a bounded deterministic or retrieval flow meets the contract.
- Do not use RAG to disguise an access, data-quality, or model-behavior problem.
- Do not treat a model switch, prompt change, or index update as an isolated release.
- Do not claim a cost or latency improvement without workload-class and fallback evidence.