Structured LLM Output with Pydantic: Get Valid JSON Every Time (2026)

Every developer writing an LLM feature hits the same wall. You ask for JSON, you get a friendly paragraph wrapped around it — "Sure! Here's your JSON:" followed by valid-looking data that has a trailing comma you cannot parse. You add "respond with JSON only" to the prompt. It works 95 times out of 100. The other five crash your pipeline at 2 a.m.

The fix is not a better prompt. It is a schema. This tutorial shows how to hand a Pydantic model to the model itself so the output is a typed, validated Python object — not a string you hope parses. Four providers, one schema, and the retry pattern that makes the last few percent reliable.

1. Why "Just Ask for JSON" Breaks

Language models are token predictors, not serializers. Nothing in next-token prediction enforces balanced braces or a required key. Four failure modes show up in production far more often than people expect:

• Prose wrapping. The model explains itself before or after the JSON, so json.loads() raises before you see the data.
• Valid JSON, wrong shape. You asked for {"name": str, "price": float} and got {"Name": "Widget", "price": "12.99 USD"}. It parses. It is still garbage.
• Missing keys under load. The longer and more nested the schema, the more often a field silently disappears from the middle of the object.
• Non-determinism. The same prompt returns a list on Monday and a single object on Tuesday, and your for item in data loop explodes.

Schema-constrained generation removes all four. Instead of hoping, you pass the schema to the provider, which constrains decoding so the model physically cannot emit a token that violates it. Pydantic is the natural way to express that schema in Python because it already gives you types, validation, and serialization for free.

2. Four Ways to Force a Schema

Pick based on your provider and how much provider-independence you need.

Note the distinction that trips people up: JSON mode is not structured output. JSON mode guarantees the output parses; it says nothing about the keys inside. If your downstream code reads result["items"], you need a schema, not JSON mode.

3. Define the Schema First

Always start from the data you want, not from the prompt. If you are shaky on annotations, our Python type hints guide covers the syntax; here is a realistic extraction schema:

from pydantic import BaseModel, Field, field_validator
from enum import Enum
from typing import Literal

class Priority(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"

class Task(BaseModel):
    title: str = Field(description="Short imperative title, max 8 words")
    priority: Priority
    estimate_hours: float = Field(ge=0, le=40, description="Rough effort estimate")
    labels: list[str] = Field(default_factory=list, max_length=5)

    @field_validator("title")
    @classmethod
    def no_trailing_period(cls, v: str) -> str:
        return v.strip().rstrip(".")

class Extraction(BaseModel):
    tasks: list[Task]
    summary: str = Field(max_length=280)
    blocking: Literal["yes", "no"]

Three details make this dramatically more reliable than a bare dataclass:

• description= is not decoration. Providers put it into the schema they constrain against, and it materially improves accuracy. "Short imperative title, max 8 words" beats a bare str.
• Field(ge=0, le=40) and max_length become JSON Schema constraints. The model cannot emit an estimate of 900 hours.
• Enum and Literal collapse free text into a fixed set, which is the single highest-value change for classification-style tasks.

4. OpenAI: Native Structured Outputs

The SDK accepts a Pydantic model directly and returns a parsed instance — no manual json.loads:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract tasks from the meeting notes."},
        {"role": "user", "content": notes},
    ],
    response_format=Extraction,
)

result: Extraction = completion.choices[0].message.parsed
print(result.tasks[0].priority)   # Priority.high  — a real enum member

If your SDK version exposes client.chat.completions.parse without the beta namespace, use that — it is the same method promoted out of beta. Verify with a one-liner:

python -c "import openai; print(openai.__version__)"   # 1.40+ required

5. Anthropic: Tool Use as a Schema Lock

Anthropic does not expose a response_format schema parameter; the reliable path is to declare your Pydantic model as a tool and force the model to call it:

import anthropic

client = anthropic.Anthropic()

schema = Extraction.model_json_schema()   # Pydantic -> JSON Schema

msg = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=2048,
    tools=[{
        "name": "save_extraction",
        "description": "Store the structured extraction result.",
        "input_schema": schema,
    }],
    tool_choice={"type": "tool", "name": "save_extraction"},
    messages=[{"role": "user", "content": notes}],
)

block = next(b for b in msg.content if b.type == "tool_use")
result = Extraction.model_validate(block.input)   # validated Pydantic object

tool_choice is the critical line. Without it the model may answer in prose and you are back to hoping. Pinning the tool name makes the structured call the only legal output. This same pattern underpins the agent work in our build an AI agent in Python guide.

6. instructor: One API for Every Provider

Writing a different adapter per provider gets old fast. instructor patches the client so the call site is identical everywhere:

pip install instructor pydantic

import instructor
from openai import OpenAI

client = instructor.from_openai(OpenAI())

result = client.chat.completions.create(
    model="gpt-4o",
    response_model=Extraction,
    max_retries=3,                 # re-prompts with the validation error
    messages=[{"role": "user", "content": notes}],
)

Change one line and the same code runs against Anthropic or a local model:

from anthropic import Anthropic
client = instructor.from_anthropic(Anthropic())

# or a local Ollama server (see the Ollama guide for setup)
import openai as oa
client = instructor.from_openai(
    oa.OpenAI(base_url="http://localhost:11434/v1", api_key="ollama"),
    mode=instructor.Mode.JSON,
)

The real value is max_retries. On a validation failure, instructor feeds the exact ValidationError message back to the model and asks again, which recovers most of the residual failures. You get a properly typed Extraction object or a raised exception — never a half-valid dict.

7. Local Models with Ollama

Local models are the weakest at instruction following, so schema constraints matter even more. Ollama accepts a format parameter; passing your model's JSON Schema works with recent builds, and passing the literal string "json" gives you JSON mode for older ones:

client = oa.OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

resp = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": notes}],
    response_format={"type": "json_object"},       # JSON mode
    # For schema mode on newer builds:
    # extra_body={"format": Extraction.model_json_schema()},
)
result = Extraction.model_validate_json(resp.choices[0].message.content)

Budget for a lower success rate on small local models. An 8B model with a three-level nested schema will fail more often than a hosted model, so keep local schemas flat and lean on retries. Setup details are in our Ollama local LLM guide.

8. Retries and Real Validation

Even with constrained decoding, treat the response as untrusted input. Wrap the call so failures are explicit and bounded:

from pydantic import ValidationError

def extract(notes: str, attempts: int = 3) -> Extraction:
    last_error = None
    for i in range(attempts):
        try:
            raw = call_model(notes, hint=str(last_error) if last_error else None)
            return Extraction.model_validate_json(raw)
        except ValidationError as e:
            last_error = e
    raise RuntimeError(f"Model failed to produce valid output after {attempts} tries: {last_error}")

Two practical rules. First, pass a trimmed error — e.errors() gives you the field name and reason, which is enough for the model to self-correct without blowing up your token count. Second, never retry forever; three attempts catches almost everything recoverable, and an exception you can alert on beats an infinite loop burning API credit.

Wiring this into an API? Return the Pydantic model directly from a FastAPI route and you get validation on the other side of the boundary too — see how to build a REST API with FastAPI.

9. Common Mistakes

Mistake 1: Using JSON Mode and Calling It Structured Output

response_format={"type": "json_object"} guarantees the braces balance, nothing else. You still need model_validate and still need to handle missing keys. It is a floor, not a ceiling.

Mistake 2: Making Everything Optional

Developers mark fields Optional to stop validation errors, then discover the model now omits them half the time. Optional fields become genuinely optional to the model. Make a field required and fix the underlying prompt instead.

Mistake 3: Ignoring Extra Keys

Pydantic silently drops unknown fields by default, which hides model drift. Set it explicitly:

class Task(BaseModel):
    model_config = {"extra": "forbid"}
    title: str

Now an unexpected key raises instead of quietly vanishing — you find out on the first bad response instead of three weeks later.

Mistake 4: Schemas With No Descriptions

A schema of bare types gives the model almost nothing to work with. Field descriptions are the highest-leverage thirty seconds you can spend on a structured extraction, and they cost nothing at runtime.

Mistake 5: Nesting Five Levels Deep

Accuracy degrades sharply past two or three levels on every provider we have tested. Flatten the model, extract the nested part in a second pass, then join in Python. Two reliable calls beat one unreliable one.

Mistake 6: Assuming Streaming and Structured Output Are Free Together

You can stream partial structured output, but you must parse incrementally and you will not have a validated object until the stream ends. If you already stream tokens, read streaming LLM responses with FastAPI SSE and validate only at completion.

Bottom Line

Structured output is the difference between an LLM feature that demos well and one you can put in front of users. The workflow is short: define a Pydantic model with real types and descriptions, pass it to your provider's structured output or tool-use path, validate the result, and retry with the validation error a maximum of three times.

Start with the schema in section 3 and run it through whichever provider you already pay for. Compare your parse-failure rate before and after — in our testing it goes from a few percent to effectively zero, and that is the kind of change that turns a nightly cron alert into silence.