LLM structured output as JSON you can validate: schemas, Pydantic, retries

Most AI features don’t want an essay. They want data: a ticket category, an order number, a priority, something the rest of the code can act on. If you let the model answer in prose and pick it apart with a regex, one day it fails silently. In 2026 the major providers have a mode that forces output to match a JSON schema. This article covers how to use it properly, and what the schema does not protect you from.

Tested on Python 3.12, anthropic 2.1.0, pydantic 2.14.1 (the validate-and-retry logic was tested against a stubbed client; API parameter names were checked against the Claude and OpenAI docs as of October 2026).

Three levels of “answer in JSON”

Level	How	What it guarantees
1. Ask in the prompt	“Reply as JSON like this…”	nothing; you may get a preamble, a code fence, or missing fields
2. JSON mode	OpenAI {"type": "json_object"}	parseable, but not your schema
3. Structured outputs	send a JSON schema to the API	matches the schema, except for the cases below

Level 3 works by constrained decoding: as the model generates each token, tokens that would break the schema are ruled out, so it cannot produce "priority": "urgent" when the enum only has low and high. The cost: the first request with a new schema waits while a grammar compiles. Claude caches compiled grammars for 24 hours from last use.

Parameter names as of October 2026

These names have changed more than once. Check the docs before you rely on them.

	Claude (Anthropic)	OpenAI
Data as the answer	output_config.format = {"type": "json_schema", "schema": ...}	Responses API: text.format = {"type": "json_schema", "name", "schema", "strict": true} · Chat Completions: response_format
Tool input matching a schema	"strict": true on the tool	strict: true on the function
Python SDK helper	client.messages.parse(output_format=Model) → parsed_output	client.responses.parse(text_format=Model) → output_parsed
Every object	additionalProperties: false	additionalProperties: false, and every field listed in required
Optional fields	leave out of required (max 24 per request)	keep in required, allow null in the type

On Claude, the old top-level output_format parameter has moved to output_config.format; the old form needs the structured-outputs-2025-11-13 beta header or returns a 400. OpenAI’s JavaScript SDK has zodTextFormat to convert a Zod schema.

Schema features the APIs reject

This is where people trip. Full JSON Schema and the subset an API accepts are not the same thing.

• Claude rejects minimum/maximum, minLength/maxLength, minItems above 1, recursive schemas, and additionalProperties other than false; any of these returns a 400. It also caps a request at 20 strict tools and 16 union-typed (anyOf) fields
• OpenAI requires an object at the root with no root-level anyOf, rejects allOf, not, if/then/else, and allows 10 levels of nesting, 5,000 properties and 1,000 enum values in total

Claude’s SDK converts a Pydantic model for you. Run anthropic.transform_schema() on a field summary: str = Field(max_length=200) and you get:

"summary": {"type": "string", "title": "Summary", "description": "{maxLength: 200}"}

The length limit has been moved into the description as text. The model sees it as a hint; nothing enforces it. The same happens to pattern in the example below. That is why you always validate again on your side.

Python example: one model for sending and checking

Triage for customer-support tickets. The Pydantic model is the single source of truth: it generates the schema for the API and validates the reply.

pip install anthropic==2.1.0 pydantic==2.14.1
export ANTHROPIC_API_KEY=sk-ant-...   # your key

from typing import Literal
import anthropic
from pydantic import BaseModel, Field, ValidationError, field_validator

class Ticket(BaseModel):
    category: Literal["billing", "delivery", "refund", "other"]
    priority: Literal["low", "high"]
    order_id: str | None = Field(pattern=r"^ORD-\d{5}$")
    summary: str = Field(max_length=200)

    @field_validator("category", "priority", mode="before")
    @classmethod
    def lower(cls, v):
        return v.lower() if isinstance(v, str) else v

SCHEMA = anthropic.transform_schema(Ticket)   # Pydantic -> JSON schema the API accepts
client = anthropic.Anthropic()                # reads ANTHROPIC_API_KEY

def triage(text: str, tries: int = 3) -> Ticket | None:
    messages = [{"role": "user", "content": f"Triage this support message:\n\n{text}"}]
    for _ in range(tries):
        r = client.messages.create(
            model="claude-haiku-5-5",
            max_tokens=1024,
            messages=messages,
            output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
        )
        if r.stop_reason == "refusal":
            return None                       # not a format problem; retrying won't help
        raw = next((b.text for b in r.content if b.type == "text"), "")
        try:
            return Ticket.model_validate_json(raw)
        except ValidationError as e:
            messages += [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"Validation failed:\n{e}\nReturn corrected JSON."},
            ]
    return None                               # send to a human queue, never guess

if __name__ == "__main__":
    print(triage("Order ORD-48213 arrived broken, I want my money back today."))

What each piece is for:

• field_validator lowercases enums. Claude’s docs warn that the casing of enum values is not guaranteed: you may get "Refund" for "refund", and the response finishes normally with no signal
• Refusals are not retried. You get HTTP 200 with stop_reason: "refusal", the output may not match the schema, and asking again gets the same answer
• Hitting max_tokens cuts the JSON off mid-way, so parsing fails and the loop retries; if it happens often, raise max_tokens instead
• The retry sends the error back. Pydantic’s message says exactly which field failed and why. With the stubbed client, the first “answer” had "order_id": "48213" and got String should match pattern '^ORD-\d{5}$'; the second came back as ORD-48213 and passed
• Out of attempts, hand it to a person. Don’t fill in defaults. A ticket silently filed in the wrong category is worse than one waiting for a human

If you don’t need control over retries, client.messages.parse(output_format=Ticket) converts the schema, sends it and validates in one call. In TypeScript, Zod plays the same role: Ticket.safeParse(JSON.parse(raw)), then check result.success.

Keep enums small

A schema can force a value to be in the enum; it cannot force the right one. The more options and the more alike they are, the easier it is to pick wrong, for models and people alike.

• Start with 3-7 clearly distinct values, always with other, then read what lands in other in the logs before adding a category
• If you truly need dozens, split into two levels (top category, then subcategory in a second call)
• Name values so they read clearly (refund, not cat_07); the model uses the names as information

Worked example: how much do 3 attempts buy you?

Say your eval set shows 2% of answers failing validation (mostly constraints the API doesn’t enforce, like a pattern).

1. If attempts failed independently, the chance all 3 fail = 0.02³ = 0.000008, or 8 in a million messages
2. Average calls per message = 1 + 0.02 + 0.02² = 1.0204, about 2% extra cost (retries are also longer, since they carry the conversation)
3. In reality attempts are not independent: an input that trips the model once tends to trip it again, so the real figure is worse than step 1. That is why you need the human route, and a count of how many messages end there

If more than 1-2% of messages need a retry, don’t add attempts. Fix the schema or prompt and measure it with an eval set.

Common mistakes

• Treating schema-valid as correct. A schema checks shape, not truth: an order number that doesn’t exist can still match the pattern. Look it up before using it
• An over-complex schema. Many optional fields, many unions, deep nesting; Claude may return a 400 “Schema is too complex for compilation”. Split into several calls
• Long reasoning inside a field. Claude’s docs warn that asking for step-by-step reasoning in a property can trigger a reasoning_extraction refusal; use a short explanation field instead
• Changing the schema often. Each change recompiles the grammar, and on Claude changing output_config.format invalidates the prompt cache for that conversation
• Sensitive data in the schema. Schemas can be cached for up to 24 hours separately from messages; keep personal data out of field names and enum values
• Forgetting outside input. Validated output limits the damage, but it does not make the input safe (defending against prompt injection)

The schema enforces shape, Pydantic checks constraints, the database confirms facts. You need all three layers.

Sources

• Structured outputs — Claude Docs
• Structured model outputs — OpenAI API Docs
• Models overview — Claude Docs
• JSON — Pydantic Docs
• Basic usage — Zod