Structured Output Frameworks in Production: Comparing Instructor, BAML, PydanticAI, and Marvin Architecture, Schema Compilation, Validation Retries, and Token Economics

Integrating large language models into production software architectures requires bridging probabilistic text generation with deterministic data structures. While foundational models generate token probability distributions, backend APIs, relational databases, and transactional microservices require strictly validated, type-safe data payloads.

To enforce schema conformance, development teams rely on structured extraction frameworks that manage schema compilation, prompt injection, output deserialization, and validation error recovery. Four major open-source frameworks provide distinct architectural approaches to structured output generation: Instructor, BAML (Boundary ML), PydanticAI, and Marvin.

Each framework operates at a different layer of the application stack, offering distinct trade-offs across runtime validation latency, token overhead during validation retries, type system portability, and agentic orchestration capabilities.

The Structured Output Architecture Matrix

| Feature / Dimension | Instructor | BAML (Boundary ML) | PydanticAI | Marvin | | :--- | :--- | :--- | :--- | :--- | | Architectural Layer | Client SDK Wrapper | Domain-Specific Language & Compiler | Agentic Runtime Framework | Functional Abstraction Layer | | Primary Language / Ecosystem | Python (ports in TS, Go, Ruby) | Multi-language (Rust Core, generates Python, TS, Go, Ruby) | Python (Pydantic v2 Native) | Python (Prefect Native) | | Schema Definition Mechanism | Pydantic BaseModel classes | .baml schema definitions | Pydantic BaseModel & typed return signatures | Python Type Hints & Pydantic Models | | Model Interface Strategy | Tool / Function Calling, JSON Mode, Markdown | Raw Prompting + System Prompt Schema Formatting | Provider Tool Calling & Structured Outputs | Function Calling & JSON Schema Prompting | | Output Parsing Engine | Standard json.loads + Pydantic validation | Schema-Aligned Parsing (SAP) in Rust | Pydantic v2 core parsing | Pydantic parsing & type coercion | | Validation Error Handling | Automatic Re-asking LLM retry loop | Resilient SAP structural recovery + explicit retry policy | Agent reflection & tool retry loops | Internal retry loop with schema prompts | | Streaming Support | Iterable[T], Partial[T] streaming | Partial AST streaming via Rust engine | Streamed structured agent results | Partial object extraction streaming | | Observability Integration | OpenTelemetry, Langfuse, Helicone | BAML Studio, OpenTelemetry traces | Native Pydantic Logfire, OpenTelemetry | Prefect Cloud, OpenTelemetry |

1. Instructor: Client-Side SDK Wrapper and Validation Re-Asking

Created by Jason Liu, Instructor patches standard LLM provider SDKs (including OpenAI, Anthropic, Google Gemini, Groq, and Ollama) to accept a Pydantic response_model argument.

Architecture and Execution Flow

Instructor wraps existing API client instances without introducing external runtime daemons or compilers. When a request is dispatched:

1. Schema Extraction: Instructor inspects the provided Pydantic model and converts its field definitions, type annotations, and docstrings into an OpenAI-compatible JSON Schema or provider-specific tool definition.
2. Request Dispatch: The schema is transmitted via the provider's native function calling interface or JSON mode.
3. Validation & Type Casting: The raw string response from the model is parsed into a JSON object and validated through Pydantic v2.

import instructor
from openai import OpenAI
from pydantic import BaseModel, Field, field_validator
from typing import List

# Patch the standard client
client = instructor.from_openai(OpenAI())

class FinancialEntity(BaseModel):
    name: str = Field(description="Normalized legal name of the entity")
    ticker: str = Field(description="Stock ticker symbol if public, else NONE")
    exposure_usd: float = Field(description="Total capital exposure in USD")
    
    @field_validator("ticker")
    @classmethod
    def validate_ticker(cls, v: str) -> str:
        if v != "NONE" and not v.isupper():
            raise ValueError("Ticker must be fully capitalized or set to NONE")
        return v

class PortfolioRiskReport(BaseModel):
    risk_summary: str = Field(description="Executive risk evaluation")
    entities: List[FinancialEntity]

# Execute extraction with automatic re-asking on validation failure
report = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=PortfolioRiskReport,
    max_retries=2,
    messages=[
        {"role": "user", "content": "Analyze exposure: Acme Corp (ticker acme) has $4.2M debt."}
    ]
)

The Re-Asking Mechanism

When Pydantic encounters a ValidationError (for example, if a @field_validator rejects a lowercase ticker symbol or a regex constraint fails), Instructor intercepts the exception. Rather than crashing, it appends two messages to the conversation history:

• The model's previous invalid output.
• A user/system message detailing the exact ValidationError trace and requesting a corrected payload.

This re-asking cycle repeats up to max_retries. While this guarantees output conformity, each retry incurs additional round-trip latency and compounds input token costs across the full conversation prefix.

2. BAML: Domain-Specific Language, Compiler, and Schema-Aligned Parsing

Developed by Boundary ML, BAML replaces Python-centric schema definitions with a dedicated domain-specific language (.baml) compiled by a high-performance Rust engine.

The .baml Specification and Code Generation

BAML co-locates prompt templates, schema definitions, model parameter configurations, and retry policies within standalone .baml files. The BAML compiler (baml-cli) compiles these specifications into native, strongly typed client code across Python, TypeScript, Go, and Ruby.

// extract_risk.baml
class FinancialEntity {
  name string @description("Normalized legal name of the entity")
  ticker string @description("Stock ticker symbol if public, else NONE")
  exposure_usd float @description("Total capital exposure in USD")
}

class PortfolioRiskReport {
  risk_summary string
  entities FinancialEntity[]
}

function ExtractPortfolioRisk(input_text: string) -> PortfolioRiskReport {
  client CustomGPT4oMini
  prompt #"
    Analyze the following financial report and extract portfolio risk:
    {{ input_text }}

    {{ ctx.output_format }}
  "#
}

In the target application, the generated client is invoked as a standard typed function:

# Generated client usage in Python
from baml_client import b

report = b.ExtractPortfolioRisk(
    input_text="Analyze exposure: Acme Corp (ticker ACME) has $4.2M debt."
)
# Returns a strictly typed PortfolioRiskReport instance
print(report.entities[0].exposure_usd)

Schema-Aligned Parsing (SAP)

BAML's primary technical differentiator is Schema-Aligned Parsing (SAP). Standard frameworks fail when a model wraps JSON in conversational text, includes markdown code blocks, omits trailing brackets, or outputs single values where arrays are expected.

Instead of relying on strict json.loads and triggering expensive LLM re-asking round trips, BAML's Rust parser performs grammar-aware alignment:

• Structural Error Correction: Automatically casts single scalar values into singleton lists if the schema specifies an array.
• Preamble and Postamble Stripping: Extracts valid JSON structures embedded within unstructured conversational preambles.
• Streaming AST Resolution: Parses partially streamed tokens directly into an incomplete Abstract Syntax Tree (AST), allowing frontend interfaces to consume structured partial objects in real time without JSON parse syntax errors.

3. PydanticAI: Agentic Runtime with Typed Return Boundaries

Built by the maintainers of Pydantic, PydanticAI integrates structured data generation directly into an agentic orchestration engine.

Agentic Structured Generation and Reflection

In PydanticAI, structured extraction is not merely a post-processing filter; it defines the terminal state of an agent execution graph via the result_type parameter.

from pydantic import BaseModel, Field
from pydantic_ai import Agent, RunContext
from dataclasses import dataclass

@dataclass
class MarketDeps:
    fx_rate_usd_to_eur: float

class EntityExposure(BaseModel):
    name: str
    exposure_eur: float = Field(description="Exposure converted to EUR")

class RiskAssessment(BaseModel):
    summary: str
    exposures: list[EntityExposure]

# Agent initialized with structured return type and dependency injection
agent = Agent(
    'openai:gpt-4o-mini',
    deps_type=MarketDeps,
    result_type=RiskAssessment,
    system_prompt="You are an enterprise risk modeling agent."
)

@agent.tool
def convert_currency(ctx: RunContext[MarketDeps], amount_usd: float) -> float:
    """Convert USD amount to EUR using injected live market rates."""
    return amount_usd * ctx.deps.fx_rate_usd_to_eur

# Run the agent with dependencies
deps = MarketDeps(fx_rate_usd_to_eur=0.92)
result = agent.run_sync(
    "Analyze debt: Apex Global holds $10M liabilities.",
    deps=deps
)

# Access validated result
validated_data: RiskAssessment = result.data

Reflection Loops and Observability

When an output fails validation in PydanticAI:

1. The framework captures the validation error and formats it as an internal tool-retry or reflection event.
2. The agent re-enters its decision loop, allowing it to invoke auxiliary tools or re-evaluate intermediate reasoning before attempting to generate the final result_type payload again.
3. Every execution step, tool call, validation failure, and token count is natively instrumented via Pydantic Logfire and OpenTelemetry semantic conventions.

4. Marvin: Declarative Functional Abstractions for Data Pipelines

Maintained by the Prefect team, Marvin provides high-level functional primitives designed to integrate LLM transformations into data engineering workflows without explicit prompt engineering.

Functional Primitives

Marvin treats language models as pure functions that perform data extraction, classification, and type casting:

• @ai_fn: Decorator that uses type hints and docstrings to generate function return values.
• @ai_model: Decorator transforming standard Pydantic models into self-extracting data containers.
• marvin.extract() / marvin.classify() / marvin.cast(): Standalone functional primitives for ETL pipelines.

import marvin
from pydantic import BaseModel, Field
from typing import List

class CorporateObligation(BaseModel):
    debtor: str
    creditor: str
    principal_usd: float = Field(description="Principal loan value in USD")
    interest_rate: float = Field(description="Annual interest rate percentage")

# Functional extraction across unstructured corpus
document = """
Agreement dated August 2026: NorthStar Logistics secures $15,000,000 term facility 
from Capital Credit Union at an annual fixed rate of 6.25%.
"""

obligations: List[CorporateObligation] = marvin.extract(
    document,
    target=CorporateObligation
)

print(obligations[0].principal_usd)  # 15000000.0

Architectural Positioning

Marvin targets data pipelines where engineers require rapid schema extraction without maintaining custom agent loops or manual JSON Schema serializations. It abstracts provider selection and prompt formatting, passing structured specifications through OpenAI tools or JSON mode depending on the active configuration.

Systems-Level Trade-Offs and Token Economics

Choosing a structured output engine introduces specific cost and performance trade-offs across four primary axes:

+-----------------------------------------------------------------------------------------+
|                                    TOKEN ECONOMICS & OVERHEAD                           |
|                                                                                         |
|  1. SCHEMA DEFINITION OVERHEAD (Per Request)                                            |
|     - Function Calling / JSON Schema: Adds 50-500 tokens to prompt context              |
|     - Markdown Extraction: Adds 20-80 tokens to system instructions                     |
|                                                                                         |
|  2. VALIDATION FAILURE MULTIPLIER (Re-asking)                                           |
|     Cost per Failure = Input Tokens + First Output + Error Trace + Second Output        |
|     => 1 Validation Error can increase single-call cost by 150% to 220%                 |
|                                                                                         |
|  3. RESILIENT PARSING (Schema-Aligned Parsing)                                          |
|     - Zero Token Retries on structural errors (malformed brackets, markdown wrappers)   |
|     - CPU-bound Rust parsing (<1ms) replaces network-bound LLM re-ask (500-2000ms)     |
+-----------------------------------------------------------------------------------------+

1. Schema Token Footprint

Every structured generation framework must convey the target schema to the LLM. When using provider-native tool calling (Instructor, PydanticAI), the schema is serialized into a JSON Schema definition within the API payload. For complex schemas containing nested models, field descriptions, and enum definitions, this adds between 100 and 800 tokens to every request context. In high-throughput streaming systems, this baseline overhead impacts input token billing.

2. Re-Asking Latency vs. Resilient Deserialization

When an LLM generates a response with minor structural anomalies (such as missing closing brackets, trailing commas, or markdown formatting), frameworks handle the error differently:

• Client-Side Re-Asking (Instructor, PydanticAI): Transmits the error back to the model over the network. A single retry doubles the time-to-first-token (TTFT) and total latency (adding 800ms to 3000ms depending on the provider) and incurs double the output token cost.
• Resilient Parsing (BAML): Resolves structural anomalies locally in the Rust runtime within sub-millisecond CPU time ($<1\text{ms}$), avoiding secondary network requests and token expenditure.

3. Polyglot Architecture Support

In enterprise architectures spanning multiple languages (for example, Go API gateways, Python data pipelines, and TypeScript frontend services), maintaining synchronized Pydantic models across codebases requires custom serialization bridges. BAML provides native compilation from a single .baml schema repository into strongly typed bindings for Python, TypeScript, Go, and Ruby, ensuring schema consistency across microservice boundaries.

Production Framework Selection Matrix

| Workload / System Requirement | Recommended Framework | Rationale | | :--- | :--- | :--- | | Python-Centric Application with Pydantic v2 | Instructor | Native integration with existing Pydantic validation rules, minimal abstraction overhead, and comprehensive provider patching. | | Polyglot Microservices (Python, Go, TypeScript) | BAML | Unified .baml schema definitions compile to native SDKs across languages with sub-millisecond Rust parsing and zero-token retry recovery. | | Multi-Turn Autonomous Agents with Tool Calling | PydanticAI | First-class agent reflection loops, dependency injection, and native Logfire distributed tracing. | | Data Pipelines and ETL Workflows | Marvin | Clean functional abstractions (extract, cast, classify) integrated directly into data processing flows without boilerplate. |

Sources

• Instructor GitHub Repository and Documentation
• BAML GitHub Repository and Documentation
• Boundary ML: Schema-Aligned Parsing Architecture
• PydanticAI GitHub Repository and Framework Documentation
• Marvin GitHub Repository and Prefect Documentation