LLM Generation

LLM Generation

Implement business-facing LLM calls as explicit, independently observable workflows. Keep prompt identity, model policy, structured output, and tracing responsibilities separate.

Locate the Existing Boundary

Before editing a call, inspect:

• packages/prompts for reusable application prompts;
• apps/server/src/services/aiGeneration for the server-side structured generation wrapper;
• packages/const/src/llmGenerationTracing.ts for scenario names;
• packages/llm-generation-tracing for tracing option and registry behavior;
• the owning service for model configuration and business-specific schemas.

Use agent-tracing for execution-snapshot diagnosis and agent-runtime-hooks for lifecycle hook behavior. Neither owns application LLM generation conventions.

Prompt Ownership and Versioning

• Put a reusable generation contract in packages/prompts/src/chains: the message builder, JSON schema, schema name, and prompt version should be exported together. Do not leave substantial system prompts or model-facing input serialization embedded in a service.
• Keep execution concerns in the owning server service: model configuration, AiGenerationService, tracing entity IDs, Zod validation, persistence, and business error handling do not belong in the prompt chain.
• Keep each *_PROMPT_VERSION beside the prompt it versions and export both from the same module.
• Format versions as v<major> or v<major>.<minor>, for example v1 or v1.2.
• Store only the version in promptVersion. Do not include a feature or scenario prefix such as expertise-ingestion-v1; scenario carries workflow identity.
• Bump the version whenever a prompt or output contract changes in a way that should create a separate evaluation or tracing cohort.

export const EXAMPLE_PROMPT_VERSION = 'v1';

export const EXAMPLE_SYSTEM_PROMPT = `...`;

Scenario Semantics

Treat scenario as the stable product workflow and lifecycle-stage partition, not as a label for a prompt, schema, model, or helper.

• Check TRACING_SCENARIOS before adding a call.
• Reuse a scenario only for the same user-visible workflow and lifecycle stage.
• Add a scenario when the business action differs, even if another call shares its prompt or JSON schema. Editable goal-criteria drafting and run-time verification planning are different scenarios.
• Never borrow a nearby scenario as a placeholder. Doing so contaminates latency, cost, success-rate, and quality data.
• Pass schemaName for structured generation and relevant entity IDs when available.

Model Policy

• Resolve the model and provider through the owning service's configuration policy. Do not silently inherit an unrelated chat model.
• When a workflow requires a stable service model, give it an explicit default and expose the corresponding service-model configuration instead of hardcoding the model only at the call site.
• Keep model choice separate from prompt version. Changing a configured model does not rename the prompt or scenario.
• Prefer the shared server generation service when it fits the call so runtime initialization, routing, and tracing remain consistent.

Structured Generation

• Give each JSON schema a stable, workflow-appropriate name.
• Keep prompt instructions and schema requirements aligned; required fields in one must be supplied by the other.
• Validate generated content at the service boundary and preserve the owning service's fallback/error behavior.
• Do not reuse a schema name to justify reusing an unrelated tracing scenario.

Verification

For every new or corrected generation workflow:

1. Assert the emitted scenario, promptVersion, and schemaName where applicable.
2. Test the prompt's important behavioral constraints without snapshotting the entire prose.
3. Test structured-output validation and relevant failure behavior.
4. Search for stale inline prompts, old version strings, and incorrectly reused scenarios.
5. Run bun run check <changed-files...> and bun run check --type for cross-package changes.

Use the testing skill for test mechanics and the typescript skill for TypeScript changes.