LLM Structured Output
Overview
Extract typed, validated data from LLM API responses instead of parsing free-text. This skill covers the three main provider approaches — OpenAI's response_format with JSON Schema, Anthropic's tool_use block for structured extraction, and Google's responseSchema in Gemini — plus constrained decoding for local models. You will learn when each approach works, when it breaks, and how to build retry logic around schema validation failures that every production system encounters.
When to Use
Use this skill when:
- The user needs to extract structured data (JSON objects, arrays, enums) from an LLM response.
- The user is building a pipeline where LLM output feeds directly into code (database writes, API calls, UI rendering).
- The user asks about
response_format, json_mode, json_object, or json_schema in OpenAI.
- The user asks about using Anthropic's
tool_use or tool_result blocks for data extraction (not for actual tool execution).
- The user asks about Zod schemas with
zodResponseFormat() from the openai npm package.
- The user needs to parse LLM output into Pydantic models using
instructor, marvin, or manual validation.
- The user is getting malformed JSON, missing fields, or wrong types from LLM responses and needs a fix.
- The user asks about
controlled generation, constrained decoding, or grammar-based sampling in local models.
Do NOT use this skill when:
- The user wants free-form text generation (summaries, essays, chat).
- The user is asking about Zod for form validation or API input validation (use
zod-validation-expert instead).
- The user needs prompt engineering for better text quality (not structure).
- The user wants to call real external tools/APIs (this skill covers using
tool_use as a structured output hack, not actual tool orchestration).
Prerequisites
- An API key for the target provider (OpenAI, Anthropic, or Google). Use placeholder
YOUR_KEY in examples — never commit live secrets.
- Python 3.10+ with
openai, anthropic, or google-generativeai SDK installed, or Node.js 18+ with the openai npm package.
- For Pydantic workflows:
pip install pydantic openai instructor (or marvin if preferred).
- For Zod workflows:
npm install openai zod.
- For local model constrained decoding:
llama.cpp (GBNF grammars) or vLLM (--json-schema flag).
- Windows host is primary. Use PowerShell for all CLI commands. Path separators in examples use backslash on Windows.
Procedure
Step 1 — Identify the target schema
Ask the user what fields they need extracted. Define every field with its type, whether it is required or optional, and valid enum values if applicable. Do not proceed without a concrete schema.
Step 2 — Choose the provider-appropriate method
| Provider |
Method |
Key Setting |
| OpenAI (gpt-4o, gpt-4o-mini) |
response_format: { type: "json_schema", json_schema: { ... } } |
Set "strict": true for constrained decoding |
| Anthropic (Claude) |
Define a single tool with target schema as input_schema |
Set tool_choice: { type: "tool", name: "extract_data" } |
| Google (Gemini) |
generationConfig.responseSchema + responseMimeType: "application/json" |
Provide a JSON Schema object |
| Local models (llama.cpp, vLLM) |
GBNF grammars or --json-schema flag |
Constrained decoding at token level |
Step 3 — Write the schema in the user's language
- Python: Define a Pydantic
BaseModel.
- TypeScript: Define a Zod schema and convert with
zodResponseFormat().
- Raw API calls: Write JSON Schema directly.
Step 4 — Include field-level descriptions in the schema
Every field must have a description string that tells the model what to put there. Models use these descriptions as implicit prompt instructions. A field described as "The user's sentiment as positive, negative, or neutral" produces better results than a bare sentiment: str with no context.
Step 5 — Set the system prompt to reinforce structure
Tell the model its job is data extraction, not conversation.
Example system prompt:
You are a data extraction system. Analyze the input and return the requested fields. Do not include explanations outside the JSON structure.
Step 6 — Enable strict mode (OpenAI)
If using OpenAI's json_schema mode, set "strict": true in the schema definition. This activates constrained decoding where the model can only output tokens that conform to the schema. Without strict: true, the model may still produce invalid JSON.
Step 7 — Extract data from the correct block (Anthropic)
If using Anthropic's tool_use approach, extract the structured data from response.content by finding the block where type == "tool_use" and reading its input field. Do not parse the text blocks — the structured data lives exclusively in the tool_use block.
Step 8 — Validate the response in application code
Even with constrained decoding, validate with Pydantic's model_validate() or Zod's .parse() before passing data downstream. This catches semantic issues (empty strings, out-of-range numbers) that schema conformance alone cannot prevent.
Step 9 — Build a retry loop for validation failures
When validation fails, send the original input plus the failed output and the validation error back to the model with an instruction like:
Your previous output failed validation: {error}. Fix the output.
Cap retries at 3 attempts.
Step 10 — Log every structured output call
Log: the input, the raw response, the parsed result, and any validation errors. When structured output breaks in production, you need these logs to determine whether the failure was a schema design issue, a prompt issue, or a model regression.
Examples
Example 1: OpenAI Structured Outputs with Pydantic (Python)
from pydantic import BaseModel, Field
from openai import OpenAI
from enum import Enum
class Sentiment(str, Enum):
positive = "positive"
negative = "negative"
neutral = "neutral"
class ReviewAnalysis(BaseModel):
sentiment: Sentiment = Field(description="Overall sentiment of the review")
key_topics: list[str] = Field(description="Main topics mentioned, max 5")
purchase_intent: bool = Field(description="Whether the reviewer would buy again")
confidence_score: float = Field(ge=0.0, le=1.0, description="Model confidence 0-1")
client = OpenAI()
response = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "Extract structured review analysis."},
{"role": "user", "content": "This laptop is amazing. The battery lasts forever and the keyboard feels great. Definitely buying the next version."}
],
response_format=ReviewAnalysis,
)
result = response.choices[0].message.parsed
# result.sentiment == Sentiment.positive
# result.key_topics == ["battery life", "keyboard"]
# result.purchase_intent == True
Example 2: Anthropic tool_use for Structured Extraction (Python)
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
system="You are a data extraction system. Use the provided tool to return structured data.",
tools=[{
"name": "extract_invoice",
"description": "Extract invoice fields from text",
"input_schema": {
"type": "object",
"properties": {
"vendor_name": {"type": "string", "description": "Company that issued the invoice"},
"total_amount": {"type": "number", "description": "Total amount in USD"},
"line_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"description": {"type": "string"},
"quantity": {"type": "integer"},
"unit_price": {"type": "number"}
},
"required": ["description", "quantity", "unit_price"]
}
}
},
"required": ["vendor_name", "total_amount", "line_items"]
}
}],
tool_choice={"type": "tool", "name": "extract_invoice"},
messages=[{"role": "user", "content": "Invoice from Acme Corp: 3x Widget A at $10 each, 1x Widget B at $25. Total: $55."}]
)
# Find the tool_use block — do NOT parse text blocks
tool_block = next(b for b in response.content if b.type == "tool_use")
invoice = tool_block.input
# invoice["vendor_name"] == "Acme Corp"
# invoice["total_amount"] == 55.0
Example 3: TypeScript with Zod + zodResponseFormat
import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";
const EventSchema = z.object({
event_name: z.string().describe("Name of the event"),
date: z.string().describe("ISO 8601 date string"),
location: z.string().describe("City and venue"),
attendee_count: z.number().int().describe("Expected number of attendees"),
is_virtual: z.boolean().describe("Whether the event is online-only"),
});
const client = new OpenAI();
const completion = await client.beta.chat.completions.parse({
model: "gpt-4o-2024-08-06",
messages: [
{ role: "system", content: "Extract event details from the text." },
{ role: "user", content: "Tech Summit 2025 in Austin at the Convention Center on March 15th. Expecting 2000 attendees, in-person only." },
],
response_format: zodResponseFormat(EventSchema, "event_extraction"),
});
const event = completion.choices[0].message.parsed;
// event.event_name === "Tech Summit 2025"
// event.is_virtual === false
Pitfalls
HARD RULES — never violate these
Never use response_format: { type: "json_object" } without a schema. This is OpenAI's legacy JSON mode — it guarantees valid JSON syntax but not schema conformance. The model can return {"result": "hello"} when you expected {"name": str, "age": int}. Always use json_schema with a full schema definition instead.
Never parse Anthropic's text blocks for structured data. When using tool_choice to force structured output, the data is in the tool_use content block, not in any text block. Parsing response.content[0].text will either return empty string or a conversational preamble — never the data you need.
Never define schema fields without descriptions. A field named status with no description can mean HTTP status, order status, or review status. Models use field descriptions as extraction instructions. Omitting them is equivalent to omitting half your prompt.
Never use additionalProperties: true in strict mode schemas. OpenAI's strict mode requires additionalProperties: false on every object in the schema. If you set it to true or omit it, the API rejects the request with a 400 error — not at response time. You will never get a response at all.
Never put extraction instructions only in the user message and not the system prompt. The system prompt has higher attention weight for behavioral instructions. Putting "extract the following fields" only in the user message alongside the source text forces the model to split attention between the instruction and the data. System prompt defines behavior; user message provides input data.
Never assume structured output means correct output. Constrained decoding guarantees the response matches the schema's types and structure. It does not guarantee the values are correct. A model can return {"sentiment": "positive"} for a negative review if the source text is ambiguous. Always validate semantics in application code after schema validation.
Never use recursive or deeply nested schemas without testing. Recursive types ($ref pointing to the same definition) and schemas deeper than 3 levels increase decoding latency significantly and raise the probability of the model hitting max_tokens before completing the JSON structure. Flatten nested schemas where possible.
Edge cases to watch for
Long source text exceeding context window. When the input text is too long, the model may truncate its reading and return incomplete extractions. Split long documents into chunks, extract from each chunk independently, then merge results in application code. Do not rely on the model to handle 50-page documents in a single call.
The model returns a refusal instead of structured data. OpenAI's structured output can return a refusal field when the model considers the request unsafe. Check response.choices[0].message.refusal before accessing .parsed. If refusal is not None, the parsed data will be None and accessing it throws an error.
Array fields returning empty when data exists. Models sometimes return [] for array fields when the source text contains the data but the field description is too vague. Fix by making the description prescriptive: "List of all product names mentioned in the text. Return at least one if any product is referenced.".
Enum values not matching due to casing. If you define an enum as ["Active", "Inactive"] but the model returns "active", validation fails. Either lowercase all enum values in the schema or add a normalization step before validation. OpenAI's strict mode respects exact casing; Anthropic may not.
Streaming with structured output. OpenAI supports streaming structured output where partial JSON arrives chunk by chunk. You cannot parse intermediate chunks as valid JSON. Use the openai SDK's built-in partial parsing or buffer chunks until the stream completes. Anthropic's tool_use blocks arrive complete in a single content_block_stop event — no partial assembly needed.
Verification
After implementing a structured output pipeline, verify correctness with these checks:
Schema conformance check — confirm the API accepts the schema:
# Python: run a single extraction and confirm no 400 error
python -c "from your_module import extract; print(extract('test input'))"
If you see a 400 Bad Request mentioning additionalProperties or strict, your schema is missing "additionalProperties": false on one or more objects.
Parsed result is not None:
result = response.choices[0].message.parsed
assert result is not None, "Parsed result is None — check for refusal or schema error"
Refusal check (OpenAI):
if response.choices[0].message.refusal is not None:
print(f"Model refused: {response.choices[0].message.refusal}")
# Do not proceed — handle safety refusal
Tool_use block exists (Anthropic):
tool_blocks = [b for b in response.content if b.type == "tool_use"]
assert len(tool_blocks) == 1, f"Expected 1 tool_use block, got {len(tool_blocks)}"
Semantic validation passes:
# Pydantic validation with constraints
validated = ReviewAnalysis.model_validate(result)
assert 0.0 <= validated.confidence_score <= 1.0
assert len(validated.key_topics) <= 5
Retry loop fires on bad output:
Inject a deliberately malformed response into your retry path and confirm the loop sends the validation error back to the model and retries up to 3 times.
Logs contain all four fields:
Confirm your log entries include: input text, raw API response, parsed result, and any validation errors. Without all four, production debugging is impossible.
Best Practices
Start with the simplest schema that solves the problem. Flat objects with 3–5 fields produce higher accuracy than nested schemas with 20+ fields. If you need complex data, extract in two passes: first extract top-level entities, then make a second call to extract details for each entity.
Use enums instead of free-form strings for categorical data. A field mood: str can return anything. A field mood: Literal["happy", "sad", "neutral", "angry"] constrains the model to exactly those values. This reduces downstream parsing logic to zero.
Pin the model version in production. gpt-4o is an alias that changes when OpenAI releases new versions. Structured output behavior can change between versions. Use gpt-4o-2024-08-06 explicitly so that your schema+prompt combination remains stable until you deliberately upgrade.
Test schema changes against 20+ real inputs before deploying. Schema changes (adding a field, changing a type, modifying a description) can break extraction on inputs that previously worked. Build a test suite of real inputs with expected outputs and run it on every schema change. This is the structured output equivalent of unit testing.
Use default values in Pydantic models for optional fields. When a field might not have relevant data in the source text, define it as Optional[str] = None in Pydantic or .optional() in Zod. Without defaults, the model is forced to hallucinate a value for fields where the source text has no answer.
Separate extraction schemas from application schemas. Your LLM extraction schema should match what the model can reliably produce. Your application database schema may have additional computed fields, foreign keys, or constraints. Map between them in application code — do not force the LLM to understand your database schema.
Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
Related Skills
zod-validation-expert — for Zod-based form and API input validation (not LLM structured output).
prompt-engineering — for improving free-form text generation quality.
1---
2name: llm-structured-output
3description: "Gets schema-constrained JSON from LLMs via OpenAI json_schema, Anthropic tool_use extraction, Gemini responseSchema, or local GBNF. Use when extracting typed objects, enums, or validated JSON from model APIs (zodResponseFormat, instructor, marvin). Not for free-form prose, Zod form/API validation (zod-validation-expert), or real tool orchestration."
4version: 1.0.1
5risk: safe
6source: community
7date_added: "2026-03-12"
8metadata:
9 source: https://skillmd.com/skills/kayforkind/llm-structured-output
10---
11
12# LLM Structured Output
13
14## Overview
15
16Extract typed, validated data from LLM API responses instead of parsing free-text. This skill covers the three main provider approaches — OpenAI's `response_format` with JSON Schema, Anthropic's `tool_use` block for structured extraction, and Google's `responseSchema` in Gemini — plus constrained decoding for local models. You will learn when each approach works, when it breaks, and how to build retry logic around schema validation failures that every production system encounters.
17
18## When to Use
19
20Use this skill when:
21
22- The user needs to extract structured data (JSON objects, arrays, enums) from an LLM response.
23- The user is building a pipeline where LLM output feeds directly into code (database writes, API calls, UI rendering).
24- The user asks about `response_format`, `json_mode`, `json_object`, or `json_schema` in OpenAI.
25- The user asks about using Anthropic's `tool_use` or `tool_result` blocks for data extraction (not for actual tool execution).
26- The user asks about Zod schemas with `zodResponseFormat()` from the `openai` npm package.
27- The user needs to parse LLM output into Pydantic models using `instructor`, `marvin`, or manual validation.
28- The user is getting malformed JSON, missing fields, or wrong types from LLM responses and needs a fix.
29- The user asks about `controlled generation`, `constrained decoding`, or `grammar-based sampling` in local models.
30
31Do NOT use this skill when:
32
33- The user wants free-form text generation (summaries, essays, chat).
34- The user is asking about Zod for form validation or API input validation (use `zod-validation-expert` instead).
35- The user needs prompt engineering for better text quality (not structure).
36- The user wants to call real external tools/APIs (this skill covers using `tool_use` as a structured output hack, not actual tool orchestration).
37
38## Prerequisites
39
40- An API key for the target provider (OpenAI, Anthropic, or Google). Use placeholder `YOUR_KEY` in examples — never commit live secrets.
41- Python 3.10+ with `openai`, `anthropic`, or `google-generativeai` SDK installed, or Node.js 18+ with the `openai` npm package.
42- For Pydantic workflows: `pip install pydantic openai instructor` (or `marvin` if preferred).
43- For Zod workflows: `npm install openai zod`.
44- For local model constrained decoding: `llama.cpp` (GBNF grammars) or `vLLM` (`--json-schema` flag).
45- Windows host is primary. Use PowerShell for all CLI commands. Path separators in examples use backslash on Windows.
46
47## Procedure
48
49### Step 1 — Identify the target schema
50
51Ask the user what fields they need extracted. Define every field with its type, whether it is required or optional, and valid enum values if applicable. Do not proceed without a concrete schema.
52
53### Step 2 — Choose the provider-appropriate method
54
55| Provider | Method | Key Setting |
56|---|---|---|
57| OpenAI (gpt-4o, gpt-4o-mini) | `response_format: { type: "json_schema", json_schema: { ... } }` | Set `"strict": true` for constrained decoding |
58| Anthropic (Claude) | Define a single tool with target schema as `input_schema` | Set `tool_choice: { type: "tool", name: "extract_data" }` |
59| Google (Gemini) | `generationConfig.responseSchema` + `responseMimeType: "application/json"` | Provide a JSON Schema object |
60| Local models (llama.cpp, vLLM) | GBNF grammars or `--json-schema` flag | Constrained decoding at token level |
61
62### Step 3 — Write the schema in the user's language
63
64- **Python:** Define a Pydantic `BaseModel`.
65- **TypeScript:** Define a Zod schema and convert with `zodResponseFormat()`.
66- **Raw API calls:** Write JSON Schema directly.
67
68### Step 4 — Include field-level descriptions in the schema
69
70Every field must have a `description` string that tells the model what to put there. Models use these descriptions as implicit prompt instructions. A field described as `"The user's sentiment as positive, negative, or neutral"` produces better results than a bare `sentiment: str` with no context.
71
72### Step 5 — Set the system prompt to reinforce structure
73
74Tell the model its job is data extraction, not conversation.
75
76Example system prompt:
77
78```
79You are a data extraction system. Analyze the input and return the requested fields. Do not include explanations outside the JSON structure.
80```
81
82### Step 6 — Enable strict mode (OpenAI)
83
84If using OpenAI's `json_schema` mode, set `"strict": true` in the schema definition. This activates constrained decoding where the model can only output tokens that conform to the schema. Without `strict: true`, the model may still produce invalid JSON.
85
86### Step 7 — Extract data from the correct block (Anthropic)
87
88If using Anthropic's `tool_use` approach, extract the structured data from `response.content` by finding the block where `type == "tool_use"` and reading its `input` field. Do not parse the text blocks — the structured data lives exclusively in the `tool_use` block.
89
90### Step 8 — Validate the response in application code
91
92Even with constrained decoding, validate with Pydantic's `model_validate()` or Zod's `.parse()` before passing data downstream. This catches semantic issues (empty strings, out-of-range numbers) that schema conformance alone cannot prevent.
93
94### Step 9 — Build a retry loop for validation failures
95
96When validation fails, send the original input plus the failed output and the validation error back to the model with an instruction like:
97
98```
99Your previous output failed validation: {error}. Fix the output.
100```
101
102Cap retries at 3 attempts.
103
104### Step 10 — Log every structured output call
105
106Log: the input, the raw response, the parsed result, and any validation errors. When structured output breaks in production, you need these logs to determine whether the failure was a schema design issue, a prompt issue, or a model regression.
107
108## Examples
109
110### Example 1: OpenAI Structured Outputs with Pydantic (Python)
111
112```python
113from pydantic import BaseModel, Field
114from openai import OpenAI
115from enum import Enum
116
117class Sentiment(str, Enum):
118 positive = "positive"
119 negative = "negative"
120 neutral = "neutral"
121
122class ReviewAnalysis(BaseModel):
123 sentiment: Sentiment = Field(description="Overall sentiment of the review")
124 key_topics: list[str] = Field(description="Main topics mentioned, max 5")
125 purchase_intent: bool = Field(description="Whether the reviewer would buy again")
126 confidence_score: float = Field(ge=0.0, le=1.0, description="Model confidence 0-1")
127
128client = OpenAI()
129response = client.beta.chat.completions.parse(
130 model="gpt-4o-2024-08-06",
131 messages=[
132 {"role": "system", "content": "Extract structured review analysis."},
133 {"role": "user", "content": "This laptop is amazing. The battery lasts forever and the keyboard feels great. Definitely buying the next version."}
134 ],
135 response_format=ReviewAnalysis,
136)
137result = response.choices[0].message.parsed
138# result.sentiment == Sentiment.positive
139# result.key_topics == ["battery life", "keyboard"]
140# result.purchase_intent == True
141```
142
143### Example 2: Anthropic tool_use for Structured Extraction (Python)
144
145```python
146import anthropic
147
148client = anthropic.Anthropic()
149response = client.messages.create(
150 model="claude-sonnet-4-20250514",
151 max_tokens=1024,
152 system="You are a data extraction system. Use the provided tool to return structured data.",
153 tools=[{
154 "name": "extract_invoice",
155 "description": "Extract invoice fields from text",
156 "input_schema": {
157 "type": "object",
158 "properties": {
159 "vendor_name": {"type": "string", "description": "Company that issued the invoice"},
160 "total_amount": {"type": "number", "description": "Total amount in USD"},
161 "line_items": {
162 "type": "array",
163 "items": {
164 "type": "object",
165 "properties": {
166 "description": {"type": "string"},
167 "quantity": {"type": "integer"},
168 "unit_price": {"type": "number"}
169 },
170 "required": ["description", "quantity", "unit_price"]
171 }
172 }
173 },
174 "required": ["vendor_name", "total_amount", "line_items"]
175 }
176 }],
177 tool_choice={"type": "tool", "name": "extract_invoice"},
178 messages=[{"role": "user", "content": "Invoice from Acme Corp: 3x Widget A at $10 each, 1x Widget B at $25. Total: $55."}]
179)
180# Find the tool_use block — do NOT parse text blocks
181tool_block = next(b for b in response.content if b.type == "tool_use")
182invoice = tool_block.input
183# invoice["vendor_name"] == "Acme Corp"
184# invoice["total_amount"] == 55.0
185```
186
187### Example 3: TypeScript with Zod + zodResponseFormat
188
189```typescript
190import OpenAI from "openai";
191import { z } from "zod";
192import { zodResponseFormat } from "openai/helpers/zod";
193
194const EventSchema = z.object({
195 event_name: z.string().describe("Name of the event"),
196 date: z.string().describe("ISO 8601 date string"),
197 location: z.string().describe("City and venue"),
198 attendee_count: z.number().int().describe("Expected number of attendees"),
199 is_virtual: z.boolean().describe("Whether the event is online-only"),
200});
201
202const client = new OpenAI();
203const completion = await client.beta.chat.completions.parse({
204 model: "gpt-4o-2024-08-06",
205 messages: [
206 { role: "system", content: "Extract event details from the text." },
207 { role: "user", content: "Tech Summit 2025 in Austin at the Convention Center on March 15th. Expecting 2000 attendees, in-person only." },
208 ],
209 response_format: zodResponseFormat(EventSchema, "event_extraction"),
210});
211const event = completion.choices[0].message.parsed;
212// event.event_name === "Tech Summit 2025"
213// event.is_virtual === false
214```
215
216## Pitfalls
217
218### HARD RULES — never violate these
219
2201. **Never use `response_format: { type: "json_object" }` without a schema.** This is OpenAI's legacy JSON mode — it guarantees valid JSON syntax but not schema conformance. The model can return `{"result": "hello"}` when you expected `{"name": str, "age": int}`. Always use `json_schema` with a full schema definition instead.
221
2222. **Never parse Anthropic's text blocks for structured data.** When using `tool_choice` to force structured output, the data is in the `tool_use` content block, not in any `text` block. Parsing `response.content[0].text` will either return empty string or a conversational preamble — never the data you need.
223
2243. **Never define schema fields without descriptions.** A field named `status` with no description can mean HTTP status, order status, or review status. Models use field descriptions as extraction instructions. Omitting them is equivalent to omitting half your prompt.
225
2264. **Never use `additionalProperties: true` in strict mode schemas.** OpenAI's strict mode requires `additionalProperties: false` on every object in the schema. If you set it to true or omit it, the API rejects the request with a 400 error — not at response time. You will never get a response at all.
227
2285. **Never put extraction instructions only in the user message and not the system prompt.** The system prompt has higher attention weight for behavioral instructions. Putting "extract the following fields" only in the user message alongside the source text forces the model to split attention between the instruction and the data. System prompt defines behavior; user message provides input data.
229
2306. **Never assume structured output means correct output.** Constrained decoding guarantees the response matches the schema's types and structure. It does not guarantee the values are correct. A model can return `{"sentiment": "positive"}` for a negative review if the source text is ambiguous. Always validate semantics in application code after schema validation.
231
2327. **Never use recursive or deeply nested schemas without testing.** Recursive types (`$ref` pointing to the same definition) and schemas deeper than 3 levels increase decoding latency significantly and raise the probability of the model hitting `max_tokens` before completing the JSON structure. Flatten nested schemas where possible.
233
234### Edge cases to watch for
235
2361. **Long source text exceeding context window.** When the input text is too long, the model may truncate its reading and return incomplete extractions. Split long documents into chunks, extract from each chunk independently, then merge results in application code. Do not rely on the model to handle 50-page documents in a single call.
237
2382. **The model returns a `refusal` instead of structured data.** OpenAI's structured output can return a `refusal` field when the model considers the request unsafe. Check `response.choices[0].message.refusal` before accessing `.parsed`. If `refusal` is not None, the parsed data will be None and accessing it throws an error.
239
2403. **Array fields returning empty when data exists.** Models sometimes return `[]` for array fields when the source text contains the data but the field description is too vague. Fix by making the description prescriptive: `"List of all product names mentioned in the text. Return at least one if any product is referenced."`.
241
2424. **Enum values not matching due to casing.** If you define an enum as `["Active", "Inactive"]` but the model returns `"active"`, validation fails. Either lowercase all enum values in the schema or add a normalization step before validation. OpenAI's strict mode respects exact casing; Anthropic may not.
243
2445. **Streaming with structured output.** OpenAI supports streaming structured output where partial JSON arrives chunk by chunk. You cannot parse intermediate chunks as valid JSON. Use the `openai` SDK's built-in partial parsing or buffer chunks until the stream completes. Anthropic's `tool_use` blocks arrive complete in a single `content_block_stop` event — no partial assembly needed.
245
246## Verification
247
248After implementing a structured output pipeline, verify correctness with these checks:
249
2501. **Schema conformance check — confirm the API accepts the schema:**
251
252 ```powershell
253 # Python: run a single extraction and confirm no 400 error
254 python -c "from your_module import extract; print(extract('test input'))"
255 ```
256
257 If you see a `400 Bad Request` mentioning `additionalProperties` or `strict`, your schema is missing `"additionalProperties": false` on one or more objects.
258
2592. **Parsed result is not None:**
260
261 ```python
262 result = response.choices[0].message.parsed
263 assert result is not None, "Parsed result is None — check for refusal or schema error"
264 ```
265
2663. **Refusal check (OpenAI):**
267
268 ```python
269 if response.choices[0].message.refusal is not None:
270 print(f"Model refused: {response.choices[0].message.refusal}")
271 # Do not proceed — handle safety refusal
272 ```
273
2744. **Tool_use block exists (Anthropic):**
275
276 ```python
277 tool_blocks = [b for b in response.content if b.type == "tool_use"]
278 assert len(tool_blocks) == 1, f"Expected 1 tool_use block, got {len(tool_blocks)}"
279 ```
280
2815. **Semantic validation passes:**
282
283 ```python
284 # Pydantic validation with constraints
285 validated = ReviewAnalysis.model_validate(result)
286 assert 0.0 <= validated.confidence_score <= 1.0
287 assert len(validated.key_topics) <= 5
288 ```
289
2906. **Retry loop fires on bad output:**
291
292 Inject a deliberately malformed response into your retry path and confirm the loop sends the validation error back to the model and retries up to 3 times.
293
2947. **Logs contain all four fields:**
295
296 Confirm your log entries include: input text, raw API response, parsed result, and any validation errors. Without all four, production debugging is impossible.
297
298## Best Practices
299
3001. **Start with the simplest schema that solves the problem.** Flat objects with 3–5 fields produce higher accuracy than nested schemas with 20+ fields. If you need complex data, extract in two passes: first extract top-level entities, then make a second call to extract details for each entity.
301
3022. **Use enums instead of free-form strings for categorical data.** A field `mood: str` can return anything. A field `mood: Literal["happy", "sad", "neutral", "angry"]` constrains the model to exactly those values. This reduces downstream parsing logic to zero.
303
3043. **Pin the model version in production.** `gpt-4o` is an alias that changes when OpenAI releases new versions. Structured output behavior can change between versions. Use `gpt-4o-2024-08-06` explicitly so that your schema+prompt combination remains stable until you deliberately upgrade.
305
3064. **Test schema changes against 20+ real inputs before deploying.** Schema changes (adding a field, changing a type, modifying a description) can break extraction on inputs that previously worked. Build a test suite of real inputs with expected outputs and run it on every schema change. This is the structured output equivalent of unit testing.
307
3085. **Use `default` values in Pydantic models for optional fields.** When a field might not have relevant data in the source text, define it as `Optional[str] = None` in Pydantic or `.optional()` in Zod. Without defaults, the model is forced to hallucinate a value for fields where the source text has no answer.
309
3106. **Separate extraction schemas from application schemas.** Your LLM extraction schema should match what the model can reliably produce. Your application database schema may have additional computed fields, foreign keys, or constraints. Map between them in application code — do not force the LLM to understand your database schema.
311
312## Limitations
313
314- Use this skill only when the task clearly matches the scope described above.
315- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
316- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
317
318## Related Skills
319
320- `zod-validation-expert` — for Zod-based form and API input validation (not LLM structured output).
321- `prompt-engineering` — for improving free-form text generation quality.