Structured Outputs Explained

The problem they solve

You ask the model: "Extract the invoice details as JSON: { vendor, total, due_date, line_items[] }."

Without structured output, you get one of:

• Valid JSON ✅
• JSON wrapped in markdown ✨json ...
• JSON with trailing commas
• JSON missing required fields
• "Sure! Here's the data: ..." then JSON
• Pure prose, no JSON in sight

Now you write parsing code, regex extraction, retry loops, error handling. All to do something the model could've just done correctly.

Structured outputs eliminate this.

How it works

The model emits one token at a time. Structured output mode constrains the next-token distribution to only tokens that keep the output valid JSON matching your schema. The model literally cannot return malformed output.

Three flavors:

• OpenAI: response_format: { type: "json_schema", json_schema: {...} } — strict mode guarantees validity.
• Anthropic: define a tool with the schema, force tool use. Tool input is your structured output.
• Gemini: generation_config.response_schema parameter.

A real example

// Schema
{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "neutral", "negative"]
    },
    "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
    "key_phrases": {
      "type": "array",
      "items": { "type": "string" },
      "maxItems": 5
    }
  },
  "required": ["sentiment", "confidence"]
}

The model is forced to return exactly this shape. No prose, no markdown wrappers, no missing fields.

Schema as prompt

Field descriptions in the schema are part of the prompt. The model reads them. Good descriptions improve quality:

"sentiment": {
  "type": "string",
  "enum": ["positive", "neutral", "negative"],
  "description": "Overall emotional tone of the message. Mixed = whichever side is stronger."
}

Treat schemas like you treat prompts.

When to use it

Always, when you need structured data. Specifically:

• Extracting fields from text (invoices, emails, support tickets)
• Classification with multiple outputs
• Multi-step agent state
• Anything you'd previously parse with regex

When not to use it:

• Free-form writing
• Code generation (use code blocks, not JSON-wrapped strings)
• Pure conversational chat

Common mistakes

• Schemas too deep — flat schemas are more reliable than nested ones.
• Required vs optional confusion — be explicit about what's optional.
• Reusing the same schema name with different shapes — providers cache. Version your schemas.
• Forgetting enum values — open string fields invite hallucination.

The cross-provider trick

Wrap your schema as a tool definition:

{
  "name": "extract_invoice",
  "description": "Extract structured fields from an invoice.",
  "input_schema": { ... }
}

Then force the model to call this tool. The tool input is your structured output. This shape works on Claude, GPT, Gemini, and any open-weights model with tool support — same code, all providers.

Structured outputs are the single highest-ROI prompt engineering upgrade in production. If you're parsing model responses by hand, switch.