Structured outputs that don't break in production

The fastest way to lose trust in an AI feature is for it to return JSON your code can’t parse — at 2am, in front of a user. This guide walks through a small, durable pattern I use to make structured LLM output safe to depend on: define a schema, validate hard, retry once, and always have a deterministic fallback.

We’ll use TypeScript and Zod, but the shape of the idea ports anywhere.

1. Define the contract as a schema

Don’t describe your desired output in prose and hope. Define it once as a schema and let that be the single source of truth — for validation and for the prompt.

import { z } from "zod";

export const Triage = z.object({
  category: z.enum(["billing", "bug", "feature", "other"]),
  urgency: z.number().int().min(1).max(5),
  summary: z.string().max(280),
});

export type Triage = z.infer<typeof Triage>;

2. Ask for exactly that shape

Tell the model the schema and demand JSON-only. Be explicit about the enum — models are far more reliable when the allowed values are spelled out.

const system = `You triage support tickets. Respond with ONLY valid JSON matching:
{ "category": "billing" | "bug" | "feature" | "other",
  "urgency": integer 1-5,
  "summary": string, max 280 chars }
No prose, no code fences.`;

3. Validate hard — never trust the first parse

The model will occasionally wrap JSON in prose or a code fence. Strip defensively, then let the schema be the gatekeeper.

function extractJson(text: string): unknown {
  const fenced = text.match(/```(?:json)?\s*([\s\S]*?)```/i);
  const raw = fenced ? fenced[1] : text;
  const start = raw.indexOf("{");
  const end = raw.lastIndexOf("}");
  return JSON.parse(raw.slice(start, end + 1));
}

4. Retry once, then fall back deterministically

One bad response is normal; two in a row means stop gambling and degrade gracefully. The fallback is what turns “the feature is down” into “the feature is slightly less smart for one request.”

async function triageTicket(text: string): Promise<Triage> {
  for (let attempt = 0; attempt < 2; attempt++) {
    try {
      const reply = await callModel(system, text); // your API call
      return Triage.parse(extractJson(reply));
    } catch (err) {
      if (attempt === 1) break; // out of retries
    }
  }
  // Deterministic fallback: route to humans, low confidence, no crash.
  return { category: "other", urgency: 3, summary: text.slice(0, 280) };
}

5. Log the failures, not just the successes

Every fallback is a signal. Log the raw response that failed validation — that’s your dataset for tightening the prompt or the schema later.

catch (err) {
  console.warn("triage validation failed", { attempt, err });
}

Why this holds up

• The schema is the contract. Validation, types, and the fallback all derive from one definition, so they can’t drift apart.
• The fallback means no hard failure. A flaky model degrades the feature; it never breaks the page.
• You stay in control of blast radius. Retry once, then hand off — you decide the cost ceiling, not the model.

That’s the whole pattern. It’s not clever, and that’s the point: reliability comes from boring, predictable boundaries around an unpredictable component.