LLM Generation
Implement business-facing LLM calls as explicit, independently observable workflows. Keep prompt identity, model policy, structured output, and tracing responsibilities separate.
Locate the Existing Boundary
Before editing a call, inspect:
packages/prompts for reusable application prompts;
apps/server/src/services/aiGeneration for the server-side structured generation wrapper;
packages/const/src/llmGenerationTracing.ts for scenario names;
packages/llm-generation-tracing for tracing option and registry behavior;
- the owning service for model configuration and business-specific schemas.
Use agent-tracing for execution-snapshot diagnosis and agent-runtime-hooks for lifecycle hook behavior. Neither owns application LLM generation conventions.
Prompt Ownership and Versioning
- Put a reusable generation contract in
packages/prompts/src/chains: the message builder, JSON schema, schema name, and prompt version should be exported together. Do not leave substantial system prompts or model-facing input serialization embedded in a service.
- Keep execution concerns in the owning server service: model configuration,
AiGenerationService, tracing entity IDs, Zod validation, persistence, and business error handling do not belong in the prompt chain.
- Keep each
*_PROMPT_VERSION beside the prompt it versions and export both from the same module.
- Format versions as
v<major> or v<major>.<minor>, for example v1 or v1.2.
- Store only the version in
promptVersion. Do not include a feature or scenario prefix such as expertise-ingestion-v1; scenario carries workflow identity.
- Bump the version whenever a prompt or output contract changes in a way that should create a separate evaluation or tracing cohort.
export const EXAMPLE_PROMPT_VERSION = 'v1';
export const EXAMPLE_SYSTEM_PROMPT = `...`;
Scenario Semantics
Treat scenario as the stable product workflow and lifecycle-stage partition, not as a label for a prompt, schema, model, or helper.
- Check
TRACING_SCENARIOS before adding a call.
- Reuse a scenario only for the same user-visible workflow and lifecycle stage.
- Add a scenario when the business action differs, even if another call shares its prompt or JSON schema. Editable goal-criteria drafting and run-time verification planning are different scenarios.
- Never borrow a nearby scenario as a placeholder. Doing so contaminates latency, cost, success-rate, and quality data.
- Pass
schemaName for structured generation and relevant entity IDs when available.
Model Policy
- Resolve the model and provider through the owning service's configuration policy. Do not silently inherit an unrelated chat model.
- When a workflow requires a stable service model, give it an explicit default and expose the corresponding service-model configuration instead of hardcoding the model only at the call site.
- Keep model choice separate from prompt version. Changing a configured model does not rename the prompt or scenario.
- Prefer the shared server generation service when it fits the call so runtime initialization, routing, and tracing remain consistent.
Structured Generation
- Give each JSON schema a stable, workflow-appropriate name.
- Keep prompt instructions and schema requirements aligned; required fields in one must be supplied by the other.
- Validate generated content at the service boundary and preserve the owning service's fallback/error behavior.
- Do not reuse a schema name to justify reusing an unrelated tracing scenario.
Verification
For every new or corrected generation workflow:
- Assert the emitted
scenario, promptVersion, and schemaName where applicable.
- Test the prompt's important behavioral constraints without snapshotting the entire prose.
- Test structured-output validation and relevant failure behavior.
- Search for stale inline prompts, old version strings, and incorrectly reused scenarios.
- Run
bun run check <changed-files...> and bun run check --type for cross-package changes.
Use the testing skill for test mechanics and the typescript skill for TypeScript changes.
1---
2name: llm-generation
3description: 'Use for application prompts, generateObject/generateText, model selection and generation tracing. Excludes provider adapters and agent snapshots.'
4metadata:
5 source: https://skillmd.com/skills/lobehub/llm-generation
6---
7
8# LLM Generation
9
10Implement business-facing LLM calls as explicit, independently observable workflows. Keep prompt identity, model policy, structured output, and tracing responsibilities separate.
11
12## Locate the Existing Boundary
13
14Before editing a call, inspect:
15
16- `packages/prompts` for reusable application prompts;
17- `apps/server/src/services/aiGeneration` for the server-side structured generation wrapper;
18- `packages/const/src/llmGenerationTracing.ts` for scenario names;
19- `packages/llm-generation-tracing` for tracing option and registry behavior;
20- the owning service for model configuration and business-specific schemas.
21
22Use `agent-tracing` for execution-snapshot diagnosis and `agent-runtime-hooks` for lifecycle hook behavior. Neither owns application LLM generation conventions.
23
24## Prompt Ownership and Versioning
25
26- Put a reusable generation contract in `packages/prompts/src/chains`: the message builder, JSON schema, schema name, and prompt version should be exported together. Do not leave substantial system prompts or model-facing input serialization embedded in a service.
27- Keep execution concerns in the owning server service: model configuration, `AiGenerationService`, tracing entity IDs, Zod validation, persistence, and business error handling do not belong in the prompt chain.
28- Keep each `*_PROMPT_VERSION` beside the prompt it versions and export both from the same module.
29- Format versions as `v<major>` or `v<major>.<minor>`, for example `v1` or `v1.2`.
30- Store only the version in `promptVersion`. Do not include a feature or scenario prefix such as `expertise-ingestion-v1`; `scenario` carries workflow identity.
31- Bump the version whenever a prompt or output contract changes in a way that should create a separate evaluation or tracing cohort.
32
33```ts
34export const EXAMPLE_PROMPT_VERSION = 'v1';
35
36export const EXAMPLE_SYSTEM_PROMPT = `...`;
37```
38
39## Scenario Semantics
40
41Treat `scenario` as the stable product workflow and lifecycle-stage partition, not as a label for a prompt, schema, model, or helper.
42
43- Check `TRACING_SCENARIOS` before adding a call.
44- Reuse a scenario only for the same user-visible workflow and lifecycle stage.
45- Add a scenario when the business action differs, even if another call shares its prompt or JSON schema. Editable goal-criteria drafting and run-time verification planning are different scenarios.
46- Never borrow a nearby scenario as a placeholder. Doing so contaminates latency, cost, success-rate, and quality data.
47- Pass `schemaName` for structured generation and relevant entity IDs when available.
48
49## Model Policy
50
51- Resolve the model and provider through the owning service's configuration policy. Do not silently inherit an unrelated chat model.
52- When a workflow requires a stable service model, give it an explicit default and expose the corresponding service-model configuration instead of hardcoding the model only at the call site.
53- Keep model choice separate from prompt version. Changing a configured model does not rename the prompt or scenario.
54- Prefer the shared server generation service when it fits the call so runtime initialization, routing, and tracing remain consistent.
55
56## Structured Generation
57
58- Give each JSON schema a stable, workflow-appropriate name.
59- Keep prompt instructions and schema requirements aligned; required fields in one must be supplied by the other.
60- Validate generated content at the service boundary and preserve the owning service's fallback/error behavior.
61- Do not reuse a schema name to justify reusing an unrelated tracing scenario.
62
63## Verification
64
65For every new or corrected generation workflow:
66
671. Assert the emitted `scenario`, `promptVersion`, and `schemaName` where applicable.
682. Test the prompt's important behavioral constraints without snapshotting the entire prose.
693. Test structured-output validation and relevant failure behavior.
704. Search for stale inline prompts, old version strings, and incorrectly reused scenarios.
715. Run `bun run check <changed-files...>` and `bun run check --type` for cross-package changes.
72
73Use the `testing` skill for test mechanics and the `typescript` skill for TypeScript changes.