Get Structured JSON Output from LLMs

Getting an LLM to return well-formed JSON sounds simple until you try it in production. The model adds commentary before the JSON, wraps it in markdown code fences, hallucinates extra fields, or silently changes data types. These small inconsistencies break downstream parsers and make your application fragile.

This guide walks you through reliable techniques for extracting structured output from LLMs, from basic prompt patterns to schema-enforced validation that works at scale.

Why Structured Output Matters

Most real-world LLM applications don’t just need text. They need data your code can work with: extracted entities, classification labels, parsed form fields, or API parameters. Without structured output, you end up writing brittle regex parsers that break whenever the model changes its formatting.

Structured output gives you predictable, machine-readable responses that slot directly into your application logic.

The Basic Prompt Pattern

The simplest approach is telling the model exactly what format you want. Be explicit about the schema, provide an example, and tell the model to return only JSON with no extra text.

import openai

client = openai.OpenAI()

def extract_contact_info(text: str) -> dict:
    prompt = f"""Extract contact information from the following text.
Return ONLY a valid JSON object with these exact fields:
- "name": string (full name)
- "email": string or null
- "phone": string or null
- "company": string or null

Example output:
{{"name": "Jane Smith", "email": "jane@example.com", "phone": null, "company": "Acme Inc"}}

Text to extract from:
{text}

JSON output:"""

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a data extraction assistant. Return only valid JSON, no markdown fences, no commentary."},
            {"role": "user", "content": prompt}
        ],
        temperature=0
    )
    return response.choices[0].message.content

This works most of the time, but “most of the time” is not good enough for production. The model can still add preamble text, use slightly different field names, or return nested structures you didn’t ask for.

Using OpenAI’s Response Format

OpenAI provides a response_format parameter that constrains the model to output valid JSON. This is a significant reliability improvement over prompt-only approaches.

import openai
import json

client = openai.OpenAI()

def extract_with_response_format(text: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": "Extract contact information. Return JSON with fields: name (string), email (string|null), phone (string|null), company (string|null)."
            },
            {"role": "user", "content": text}
        ],
        response_format={"type": "json_object"},
        temperature=0
    )
    return json.loads(response.choices[0].message.content)

The json_object response format guarantees valid JSON syntax, but it does not enforce your schema. The model could still return different field names or unexpected structures. For full schema enforcement, you need structured outputs with a JSON schema.

Schema-Enforced Structured Outputs

OpenAI’s structured outputs feature lets you pass a JSON Schema that the model must conform to. This is the most reliable approach available.

import openai
import json

client = openai.OpenAI()

contact_schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string", "description": "Full name of the person"},
        "email": {"type": ["string", "null"], "description": "Email address if found"},
        "phone": {"type": ["string", "null"], "description": "Phone number if found"},
        "company": {"type": ["string", "null"], "description": "Company or organization"}
    },
    "required": ["name", "email", "phone", "company"],
    "additionalProperties": False
}

def extract_with_schema(text: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Extract contact information from the provided text."},
            {"role": "user", "content": text}
        ],
        response_format={
            "type": "json_schema",
            "json_schema": {
                "name": "contact_extraction",
                "strict": True,
                "schema": contact_schema
            }
        },
        temperature=0
    )
    return json.loads(response.choices[0].message.content)

result = extract_with_schema(
    "Hi, I'm Alex Chen from DataFlow Labs. Reach me at alex@dataflow.io or 555-0142."
)
print(result)
# {"name": "Alex Chen", "email": "alex@dataflow.io", "phone": "555-0142", "company": "DataFlow Labs"}

With strict: True, the model’s output is guaranteed to match your schema. Every required field will be present, types will be correct, and no extra fields will appear.

Validation with Pydantic

Even with schema enforcement at the API level, you should validate responses in your application. Pydantic makes this straightforward and adds type safety to your Python code.

from pydantic import BaseModel, Field, field_validator
from typing import Optional
import openai
import json

class ContactInfo(BaseModel):
    name: str = Field(description="Full name of the person")
    email: Optional[str] = Field(default=None, description="Email address")
    phone: Optional[str] = Field(default=None, description="Phone number")
    company: Optional[str] = Field(default=None, description="Company name")

    @field_validator("email")
    @classmethod
    def validate_email(cls, v):
        if v is not None and "@" not in v:
            raise ValueError(f"Invalid email format: {v}")
        return v

class ExtractionResult(BaseModel):
    contacts: list[ContactInfo]
    confidence: float = Field(ge=0.0, le=1.0)

def extract_and_validate(text: str) -> ExtractionResult:
    client = openai.OpenAI()
    schema = ExtractionResult.model_json_schema()

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": f"Extract all contacts from the text. Return JSON matching this schema:\n{json.dumps(schema, indent=2)}"
            },
            {"role": "user", "content": text}
        ],
        response_format={"type": "json_object"},
        temperature=0
    )

    raw = json.loads(response.choices[0].message.content)
    return ExtractionResult.model_validate(raw)

This approach gives you runtime type checking, custom validation logic, and clear error messages when the model returns something unexpected.

Handling Malformed Responses

In production, you need to handle cases where the model returns invalid output. A retry-with-correction pattern works well for this.

import json
from pydantic import BaseModel, ValidationError
from typing import TypeVar, Type

T = TypeVar("T", bound=BaseModel)

def robust_extract(
    client,
    messages: list[dict],
    response_model: Type[T],
    max_retries: int = 3
) -> T:
    """Extract structured data with automatic retry on validation failure."""
    last_error = None

    for attempt in range(max_retries):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            response_format={"type": "json_object"},
            temperature=0
        )

        raw_text = response.choices[0].message.content

        try:
            raw_json = json.loads(raw_text)
            return response_model.model_validate(raw_json)
        except (json.JSONDecodeError, ValidationError) as e:
            last_error = e
            # Add correction message and retry
            messages.append({"role": "assistant", "content": raw_text})
            messages.append({
                "role": "user",
                "content": f"Your response had an error: {str(e)}. Please fix it and return valid JSON matching the required schema."
            })

    raise ValueError(
        f"Failed to get valid response after {max_retries} attempts. "
        f"Last error: {last_error}"
    )

This function tries to parse and validate the response, and if it fails, it sends the error back to the model as a correction prompt. Most validation errors are fixed on the first retry.

Complex Nested Structures

Real applications often need deeply nested output. Define your schema carefully and provide clear descriptions for each field.

from pydantic import BaseModel, Field
from typing import Optional
from enum import Enum

class Severity(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"

class CodeLocation(BaseModel):
    file: str
    line_start: int
    line_end: Optional[int] = None

class SecurityFinding(BaseModel):
    title: str = Field(description="Short title for the finding")
    severity: Severity
    description: str = Field(description="Detailed explanation of the vulnerability")
    location: CodeLocation
    recommendation: str = Field(description="How to fix this issue")
    cwe_id: Optional[str] = Field(default=None, description="CWE identifier if applicable")

class SecurityReport(BaseModel):
    summary: str = Field(description="Executive summary of the review")
    findings: list[SecurityFinding]
    overall_risk: Severity
    reviewed_files: list[str]

def review_code_security(code: str, filename: str) -> SecurityReport:
    client = openai.OpenAI()
    schema = SecurityReport.model_json_schema()

    return robust_extract(
        client=client,
        messages=[
            {
                "role": "system",
                "content": f"You are a security reviewer. Analyze the code and return findings as JSON.\nSchema:\n{json.dumps(schema, indent=2)}"
            },
            {
                "role": "user",
                "content": f"Review this file ({filename}):\n\n```\n{code}\n```"
            }
        ],
        response_model=SecurityReport
    )

Structured Output with Anthropic Claude

Claude supports structured output through tool use. You define a tool whose input schema matches your desired output, and Claude will fill in the parameters.

import anthropic
import json

client = anthropic.Anthropic()

def extract_with_claude(text: str) -> dict:
    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        tools=[{
            "name": "save_contact",
            "description": "Save extracted contact information",
            "input_schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string", "description": "Full name"},
                    "email": {"type": "string", "description": "Email address"},
                    "phone": {"type": "string", "description": "Phone number"},
                    "company": {"type": "string", "description": "Company name"}
                },
                "required": ["name"]
            }
        }],
        tool_choice={"type": "tool", "name": "save_contact"},
        messages=[
            {"role": "user", "content": f"Extract contact info from: {text}"}
        ]
    )

    for block in response.content:
        if block.type == "tool_use":
            return block.input

    raise ValueError("No tool use in response")

By setting tool_choice to force a specific tool, Claude will always return structured data matching your schema.

Tips for Reliable Structured Output

Here are patterns that consistently improve structured output quality.

Set temperature to 0. Creative variation is the enemy of consistent schemas. Always use temperature=0 for extraction tasks.

Use enums for categorical fields. Instead of letting the model pick arbitrary strings for categories or statuses, define the allowed values explicitly in your schema.

Provide field descriptions. Every field in your schema should have a clear description. The model uses these to understand what data goes where.

Keep schemas flat when possible. Deep nesting increases the chance of errors. If you can flatten your schema without losing meaning, do it.

Test with adversarial inputs. Try empty strings, very long texts, texts in other languages, and texts with no relevant information. Make sure your extraction handles edge cases gracefully.

# Example: handling missing data gracefully
class ProductReview(BaseModel):
    sentiment: str = Field(description="positive, negative, or neutral")
    rating: Optional[float] = Field(
        default=None,
        ge=1.0,
        le=5.0,
        description="Numeric rating if mentioned, 1-5 scale"
    )
    pros: list[str] = Field(default_factory=list, description="Positive points mentioned")
    cons: list[str] = Field(default_factory=list, description="Negative points mentioned")
    summary: str = Field(description="One-sentence summary of the review")

Wrapping Up

Getting reliable structured output from LLMs requires a layered approach. Start with clear prompts that specify your exact schema. Use API-level format constraints like response_format when available. Validate responses with Pydantic or similar libraries. And always have a retry mechanism for the cases where validation fails.

The combination of schema enforcement at the API level and Pydantic validation in your application code gives you the reliability you need for production systems. The model handles the intelligence of extraction and classification, while your code ensures the output is always in the shape your application expects.