Structured outputs make model APIs return JSON that is guaranteed to match a JSON Schema you supply, replacing the fragile pattern of pleading for valid JSON in the system prompt and retrying on parse failure. The two most-used cloud AI APIs implement this differently: Anthropic exposes JSON outputs via output_config.format plus an independent strict: true mode for tool inputs, while OpenAI offers a json_schema response format with a strict flag and schema-constrained function calling. Understanding where the two mechanisms agree, and where they silently diverge, determines whether your extraction pipeline needs a retry budget at all.
How constrained decoding works
Both vendors guarantee schema compliance through constrained decoding: at generation time the sampler is restricted to tokens that keep the output parseable against your schema, so invalid enum values or missing required keys cannot be emitted. Anthropic states that structured outputs guarantee schema-compliant responses through constrained decoding, eliminating parsing errors from invalid JSON syntax, missing required fields, inconsistent data types, and schema violations that previously required error handling and retries. OpenAI describes the same guarantee from the consumer side: the model will always generate responses that adhere to your supplied JSON Schema, so you no longer worry about the model omitting a required key or hallucinating an invalid enum value. The engineering consequence is identical on both platforms — the validation-and-retry loop moves from your code into the provider’s serving stack, and the failure mode changes from malformed JSON to schema design errors that surface at request time.
Request shapes and compatibility
The APIs differ in how you attach a schema. OpenAI supports structured outputs in two forms: through function calling, where each tool definition carries its own schema, and through a json_schema response format for structuring the model’s reply to the user, with the documentation recommending strict structured outputs over the older JSON mode whenever possible. Anthropic split the same surface into two independent features — JSON outputs through output_config.format and strict tool use through strict: true — which you can enable separately or together in one request. Anthropic’s feature also landed on multiple clouds from day one: the documentation lists availability on the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud, and Microsoft Foundry, with Bedrock support limited to specific Claude model versions. Teams routing the same schema across clouds should treat the model compatibility matrix as part of the contract, not an afterthought.
Caching and cost interactions
The least obvious trap is how schemas interact with prompt caching. Anthropic’s documentation warns that prompt caches are keyed by the JSON schema structure and by the set of tools in the request, so a schema that changes between requests defeats the cache even when the conversation text is identical. Because schema serialization is part of the cached prefix, embedding volatile values — timestamps, request IDs, per-tenant examples — inside the schema definition turns a cacheable system prompt into an effectively uncached one. On the OpenAI side the same class of problem appears as token cost: the serialized schema is submitted with every request and counted as input tokens, so oversized schemas with deeply nested definitions tax every call. The fix is the same on both platforms: factor schemas into stable, reusable definitions, version them deliberately, and keep anything request-specific out of the schema body.
| Dimension | Anthropic | OpenAI |
|---|---|---|
| Response formatting | output_config.format with json_schema type | json_schema response format with strict: true |
| Tool input guarantees | Separate strict: true flag on tool definitions | Built into function calling with strict mode |
| Older JSON mode | Not part of this surface | JSON mode still available but schema-unaware |
| Cloud availability | Claude API, AWS, Bedrock, Google Cloud, Microsoft Foundry | OpenAI API platform |
| Cache sensitivity | Cache keyed on schema structure and tool set | Schema tokens billed as input on every request |
Migrating and hardening pipelines
Most production incidents after adopting structured outputs come from migration drift, not from invalid JSON. Anthropic moved the beta output_format parameter to output_config.format and dropped the beta header requirement, keeping the old forms working only for a transition period — code still sending the beta header is one deprecation notice away from breaking. Follow this order when you adopt or migrate:
- Define the schema once in a versioned module; mark every field
requiredand setadditionalPropertiesto false so both platforms can enforce strictness. - Pick the enforcement point: response formatting for extraction and classification, strict tool use for agent loops that call your functions.
- Handle explicit refusals separately from parse errors; a refusal is a schema-valid outcome with its own field, not a retryable failure.
- Keep schemas byte-stable to preserve prompt caches, and diff the serialized schema in CI so accidental edits surface before deployment.
- Run one golden-file test per schema against each model version you use, since constrained decoding guarantees shape, not semantic quality.
Schema-constrained decoding removes an entire class of runtime failures, but it shifts reliability work upstream to schema design, cache stability, and compatibility matrices. Budget your engineering time there. For the surrounding infrastructure decisions — routing workloads between managed APIs and self-hosted inference — the guides on what cloud AI means for Portugal and the EU and on choosing deployment models in 2026 cover the broader trade-offs.