Structured Output Extraction with LLMs

The Problem

LLMs generate text. Your application needs data structures. Somewhere between the model’s response and your database, you need to reliably convert free-form text into typed, validated objects. This is structured output extraction.

Consider extracting product information from a customer review: “I bought the Sony WH-1000XM5 headphones for $348 from Amazon last week. Great noise cancellation but the app is buggy.” You need a structured object with product name, price, retailer, sentiment, and specific pros and cons.

Approach 1: JSON Mode

The simplest approach is asking the model to respond in JSON and parsing the result.

OpenAI JSON Mode

from openai import OpenAI
import json

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o",
    response_format={"type": "json_object"},
    messages=[
        {
            "role": "system",
            "content": "Extract product information from the review. Respond in JSON with keys: product_name, price, retailer, sentiment (positive/negative/mixed), pros (list), cons (list).",
        },
        {
            "role": "user",
            "content": "I bought the Sony WH-1000XM5 headphones for $348 from Amazon last week. Great noise cancellation but the app is buggy.",
        },
    ],
)

data = json.loads(response.choices[0].message.content)
print(json.dumps(data, indent=2))

JSON mode guarantees valid JSON but does not guarantee the schema matches your expectations. The model might use product instead of product_name or return a string instead of a list.

Approach 2: Structured Outputs with Schemas

Both OpenAI and Anthropic support schema-constrained outputs that guarantee the response matches your exact structure.

OpenAI Structured Outputs

from openai import OpenAI
from pydantic import BaseModel

class ProductReview(BaseModel):
    product_name: str
    price: float | None
    retailer: str | None
    sentiment: str  # "positive", "negative", or "mixed"
    pros: list[str]
    cons: list[str]

client = OpenAI()

response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[
        {
            "role": "system",
            "content": "Extract product information from the review.",
        },
        {
            "role": "user",
            "content": "I bought the Sony WH-1000XM5 headphones for $348 from Amazon last week. Great noise cancellation but the app is buggy.",
        },
    ],
    response_format=ProductReview,
)

review = response.choices[0].message.parsed
print(f"Product: {review.product_name}")
print(f"Price: ${review.price}")
print(f"Sentiment: {review.sentiment}")

The parse method returns a typed Pydantic object. If the model’s output does not match the schema, parsing fails explicitly rather than silently producing bad data.

Anthropic Tool-Based Extraction

Anthropic does not have a dedicated structured output mode, but you can achieve the same result using tool use. Define a tool whose schema matches your desired output and force the model to call it.

import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "extract_review",
        "description": "Extract structured product review information",
        "input_schema": {
            "type": "object",
            "properties": {
                "product_name": {"type": "string"},
                "price": {"type": "number", "description": "Price in USD, or null"},
                "retailer": {"type": "string", "description": "Store name, or null"},
                "sentiment": {
                    "type": "string",
                    "enum": ["positive", "negative", "mixed"],
                },
                "pros": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "List of positive aspects",
                },
                "cons": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "List of negative aspects",
                },
            },
            "required": ["product_name", "sentiment", "pros", "cons"],
        },
    }
]

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "extract_review"},
    messages=[
        {
            "role": "user",
            "content": "Extract from this review: I bought the Sony WH-1000XM5 headphones for $348 from Amazon last week. Great noise cancellation but the app is buggy.",
        }
    ],
)

for block in response.content:
    if block.type == "tool_use":
        data = block.input
        print(f"Product: {data['product_name']}")
        print(f"Sentiment: {data['sentiment']}")

Validating with Pydantic

Regardless of the provider, always validate LLM output with Pydantic. This catches type mismatches, missing fields, and invalid values.

from pydantic import BaseModel, Field, field_validator
from enum import Enum

class Sentiment(str, Enum):
    positive = "positive"
    negative = "negative"
    mixed = "mixed"

class ProductReview(BaseModel):
    product_name: str = Field(min_length=1, max_length=200)
    price: float | None = Field(None, ge=0, le=100000)
    retailer: str | None = None
    sentiment: Sentiment
    pros: list[str] = Field(default_factory=list, max_length=10)
    cons: list[str] = Field(default_factory=list, max_length=10)

    @field_validator("pros", "cons")
    @classmethod
    def non_empty_strings(cls, v):
        return [item.strip() for item in v if item.strip()]

# Parse and validate
try:
    review = ProductReview.model_validate(raw_data)
    print(f"Valid: {review.product_name}, ${review.price}")
except Exception as e:
    print(f"Validation failed: {e}")

Extracting from Complex Documents

For longer documents, you often need to extract multiple structured objects. Break the task into stages.

Stage 1: Identify Entities

class Entity(BaseModel):
    name: str
    entity_type: str  # "person", "company", "product", "location"
    context: str  # The sentence where the entity appears

class DocumentEntities(BaseModel):
    entities: list[Entity]

Stage 2: Extract Relationships

class Relationship(BaseModel):
    subject: str
    predicate: str  # "works_at", "acquired", "located_in"
    object: str
    confidence: float = Field(ge=0, le=1)

class DocumentRelationships(BaseModel):
    relationships: list[Relationship]

Stage 3: Combine into a Knowledge Graph

def extract_knowledge_graph(document: str) -> dict:
    # Step 1: Extract entities
    entities = extract_with_schema(document, DocumentEntities)

    # Step 2: Extract relationships using entities as context
    entity_names = [e.name for e in entities.entities]
    relationships = extract_with_schema(
        f"Given these entities: {entity_names}\n\nDocument: {document}",
        DocumentRelationships,
    )

    return {
        "entities": [e.model_dump() for e in entities.entities],
        "relationships": [r.model_dump() for r in relationships.relationships],
    }

Handling Extraction Failures

LLM extraction is not perfect. Build retry logic and fallback strategies.

from pydantic import ValidationError
import json

def extract_with_retry(text: str, schema: type[BaseModel], max_retries: int = 3):
    """Extract structured data with retries on failure."""
    last_error = None

    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model="gpt-4o",
                response_format={"type": "json_object"},
                messages=[
                    {
                        "role": "system",
                        "content": f"Extract data matching this schema: {schema.model_json_schema()}. Respond with valid JSON only.",
                    },
                    {"role": "user", "content": text},
                ],
                temperature=0.0 if attempt == 0 else 0.3,
            )

            raw = json.loads(response.choices[0].message.content)
            return schema.model_validate(raw)

        except (json.JSONDecodeError, ValidationError) as e:
            last_error = e
            continue

    raise ValueError(f"Extraction failed after {max_retries} attempts: {last_error}")

Setting temperature to 0 on the first attempt gives the most deterministic output. Raising it slightly on retries adds variation that might avoid the same error pattern.

Batch Extraction

When processing many documents, batch them efficiently.

import asyncio
from openai import AsyncOpenAI

async_client = AsyncOpenAI()

async def extract_one(text: str, schema: type[BaseModel]) -> BaseModel:
    response = await async_client.beta.chat.completions.parse(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Extract structured data from the text."},
            {"role": "user", "content": text},
        ],
        response_format=schema,
    )
    return response.choices[0].message.parsed

async def extract_batch(texts: list[str], schema: type[BaseModel], concurrency: int = 10):
    semaphore = asyncio.Semaphore(concurrency)

    async def limited_extract(text):
        async with semaphore:
            return await extract_one(text, schema)

    tasks = [limited_extract(text) for text in texts]
    return await asyncio.gather(*tasks, return_exceptions=True)

# Usage
reviews = ["review 1...", "review 2...", "review 3..."]
results = asyncio.run(extract_batch(reviews, ProductReview))

for result in results:
    if isinstance(result, Exception):
        print(f"Failed: {result}")
    else:
        print(f"Extracted: {result.product_name}")

Tips for Reliable Extraction

Keep schemas flat when possible. Deeply nested structures are harder for models to populate correctly. If you need nested data, extract the outer structure first and then enrich each item in a second pass.

Use enums for categorical fields. A free-text sentiment field might return “somewhat positive” or “mostly good” instead of your expected values. An enum constrains the output.

Include examples in your system prompt for ambiguous fields. If “price” should be in USD even when the text says euros, state that explicitly.

Set temperature to 0 for extraction tasks. You want consistency, not creativity.

Test with adversarial inputs. What happens when the text contains no relevant information? What if it contains contradictory information? Build your schema with optional fields and handle missing data gracefully.

Summary

Structured output extraction converts free-form LLM responses into typed data structures. Use OpenAI’s parse method or Anthropic’s tool-use pattern for schema-constrained generation. Always validate with Pydantic models that enforce types, ranges, and enums. Build retry logic for production use. Process batches with async concurrency. Keep schemas flat, use enums for categorical fields, and set temperature to 0 for deterministic extraction.