Quick Answer: Pydantic-AI and Instructor both force an LLM to return typed JSON, but they fail differently. In a 2,000-call invoice test, Pydantic-AI retried 6.2% of calls and Instructor retried 11.8%. At 1M calls, that gap costs about ₹4,200 in wasted output tokens and adds roughly 500ms to P95 latency.
Key Takeaways
- Retries are the real cost: A schema failure is not free. Each retry re-sends the prompt and pays for new output tokens, which is where the 1M-call bill grows.
- Pydantic-AI wins on retry rate: Native Pydantic validation and a built-in repair loop produced 6.2% retries versus 11.8% for Instructor in my test.
- Instructor wins on portability: It patches an existing client, so you can adopt it in a legacy codebase without rewriting the call path.
- Validate locally first: A local Pydantic check before the model call catches 30–40% of bad inputs and removes them from the retry budget entirely.
The Test Setup
I ran the same invoice-extraction task through both libraries on the same model and the same 2,000 documents. The documents came from a Surat wholesale client: mixed GST formats, some scanned, some typed, and a long tail of vendor-specific layouts.
| Parameter | Value |
|---|---|
| Model | GPT-4.1 mini class |
| Documents | 2,000 invoices |
| Output schema | 11 fields, 3 nested line items |
| Temperature | 0.0 |
| Max output tokens | 700 |
| Retry cap | 3 attempts |
| Runtime | Python 3.12 |
The model choice matters less than the retry policy. I kept the model fixed so the comparison measures the library, not the vendor.
How Each Library Handles a Bad Output
flowchart TD
A[Prompt + schema] --> B[Model output]
B --> C{Valid JSON?}
C -- No --> D[Pydantic-AI: validation error fed back]
C -- No --> E[Instructor: retry with error message]
D --> F{Valid after repair?}
E --> F
F -- Yes --> G[Return typed object]
F -- No --> H[Raise after 3 attempts]
Pydantic-AI treats the schema as part of the agent contract. It validates the output, converts the validation error into a repair prompt, and calls the model again. Instructor does something similar through a patched client, but the repair prompt and retry behavior depend on the mode you choose.
The Code: Pydantic-AI
# agents/invoice_agent.py
from pydantic import BaseModel, Field
from pydantic_ai import Agent
class LineItem(BaseModel):
description: str
quantity: float = Field(gt=0)
rate_inr: float = Field(ge=0)
gst_rate: float = Field(ge=0, le=28)
class Invoice(BaseModel):
invoice_number: str
vendor_gstin: str
invoice_date: str
total_inr: float = Field(gt=0)
line_items: list[LineItem]
agent = Agent(
"openai:gpt-4.1-mini",
output_type=Invoice,
system_prompt=(
"Extract invoice fields. Use the printed total, not a computed sum. "
"If a field is missing, return an empty string and do not invent a value."
),
)
def extract(text: str) -> Invoice:
result = agent.run_sync(text)
return result.output
The output_type parameter is the whole point. The agent will not return a dict you have to validate by hand. It returns an Invoice or raises after the retry budget.
I keep the schema narrow on purpose. Eleven fields and three nested line items is already near the limit where small models start dropping a key. Every optional field adds a branch the model can get wrong, and every branch adds a retry path you have to test. Start with the fields you will actually store, then add the rest in a second pass.
The Code: Instructor
# agents/invoice_instructor.py
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field
class Invoice(BaseModel):
invoice_number: str
vendor_gstin: str
invoice_date: str
total_inr: float = Field(gt=0)
line_items: list[dict]
client = instructor.from_openai(OpenAI())
def extract(text: str) -> Invoice:
return client.chat.completions.create(
model="gpt-4.1-mini",
response_model=Invoice,
max_retries=3,
temperature=0.0,
messages=[
{"role": "system", "content": "Extract invoice fields. Never invent values."},
{"role": "user", "content": text},
],
)
Instructor is smaller to adopt. If you already have an OpenAI client in a Django or FastAPI service, this is a five-line change. Pydantic-AI asks you to move the call into an agent abstraction, which is more work upfront and less work later when you add tools.
Retry and Cost Results
| Metric | Pydantic-AI | Instructor |
|---|---|---|
| First-try valid | 93.8% | 88.2% |
| Retried once | 5.1% | 9.4% |
| Retried twice | 1.1% | 2.4% |
| Failed after 3 | 0.0% | 0.0% |
| P50 latency | 1.9 s | 2.1 s |
| P95 latency | 2.9 s | 3.4 s |
| Cost per 2,000 calls | ₹1,840 | ₹1,980 |
The cost gap looks small at 2,000 calls. At 1M calls, the 5.6-point retry difference is 56,000 extra model calls. At an average repair output of 110 tokens and ₹720 per 1M output tokens, that is about ₹4,435. I round it to ₹4,200 after prompt caching.
The latency gap matters more than the money for a counter or support agent. A 500ms P95 difference is the difference between a tool that feels instant and a tool that feels broken.
War Story: The GST Field That Broke Both
The Surat client had one vendor that printed GSTIN in a footer image, not as text. Pydantic-AI failed 100% of those invoices because the field was genuinely absent from the text. Instructor did the same. The libraries were not the problem. The input was.
I added a local pre-check that flags documents with no GSTIN pattern and routes them to a human queue. That removed 212 invoices from the model path and cut the retry rate by 38% for both libraries. The lesson is boring: clean the input before you tune the retry loop.
I also added a model_validator that checks whether total_inr equals the sum of line items within a 1% tolerance. That caught 47 invoices where the model picked a subtotal instead of the total. Neither library catches that by default because the schema alone cannot express the arithmetic relationship.
When NOT to Use Either Library
Do not use a structured-output library if your output is free-form prose. You will spend more time fighting the schema than shipping.
Do not use either if you cannot retry. A voice agent with a 2-second budget cannot afford a 3.4s P95 retry. Use a smaller schema and a single attempt, then fall back to a human.
Do not use Pydantic-AI if your team is already deep in a custom client wrapper and has no appetite for an agent abstraction. Instructor is the lower-risk adoption.
Do not use Instructor if you need tool calling and multi-step planning in the same loop. Pydantic-AI's agent model is a better fit once the task needs more than one model call.
Frequently Asked Questions
Is Pydantic-AI better than Instructor for structured output?
Pydantic-AI had a lower retry rate in my 2,000-invoice test, 6.2% versus 11.8%, and a lower P95 latency, 2.9s versus 3.4s. Instructor is easier to add to an existing client. Choose Pydantic-AI for new agent code and Instructor for a legacy service that needs typed JSON quickly.
How much do retries cost at 1M calls?
At a 5.6-point retry gap, 1M calls produce 56,000 extra model calls. With a 110-token repair output at ₹720 per 1M output tokens, the extra cost is about ₹4,200. The number moves with your model price and output length, not with the library alone.
Can I use Pydantic-AI with LangGraph?
Yes. Wrap the Pydantic-AI agent in a LangGraph node and pass the typed output as state. Keep the validation inside the node so a schema failure triggers a retry before the graph advances. This is the pattern I use for invoice and support workflows.
Which library is better for a FastAPI backend?
Instructor is the faster integration because it patches the OpenAI client you already use. Pydantic-AI is better if the endpoint is really an agent with tools, retries, and a typed result. For a single extraction endpoint, Instructor wins on time to ship.
Bottom Line
Pick the library that matches the shape of your code, then fix the input before you tune the retry loop. Pydantic-AI is the better default for agent work. Instructor is the better patch for a legacy client. If you want the deeper agent patterns, read the Pydantic-AI and LangGraph type-safe guide, then the zero-hallucination RAG blueprint and the agentic RAG architecture. If your JSON keeps breaking in production, send me a sample failure and I will tell you whether to fix the schema or the input.