Visit Website →
← Back to blog

Structured Error Handling for APIs: A Practical Guide

November 7, 2026
Structured Error Handling for APIs: A Practical Guide

Structured error handling means every failure your API returns is machine-readable, typed and consistent. A client can branch on it without parsing prose. The working recipe: a standard body shape (RFC 9457 problem details), the right HTTP status, a stable error code, a safe request ID, and an explicit retry signal.

Gyrence
Make API Failures Machine-Readable
Gyrence returns typed responses, including failure cases, so agents can reason about web data results instead of guessing.
Explore the API

TL;DR

  • Return one error shape everywhere. RFC 9457's type, title, status, detail and instance are the baseline; add extensions for your own fields.
  • Clients should switch on a stable code, never on the HTTP status alone and never on message text.
  • Be precise about the caller's input and opaque about your system's internals. Log the detail server-side and return a request ID.
  • Mark what is retryable. Retry transient failures with backoff and jitter; never retry permanent ones.
  • For scraping and agent tools, treat "the page said no" as a typed result, not an exception.

Table of Contents

Why a consistent error contract matters

RFC 9457 exists because non-human clients have a hard time interpreting HTML error pages. It defines a JSON or XML problem-details format so a program can understand what went wrong and act on it.

AWS makes the same point from the client side: an explicit error contract lets clients centralize handling, automate retries and generate strongly typed SDKs. Without a contract, every integration reinvents string matching, and every message rewrite breaks someone.

Status codes alone are too coarse. They tell the client which class of problem occurred. The body tells it which problem, where, and what to change. For more on how bodies complement status codes, see this breakdown of API rate limiting decisions, where 429 handling is a good example.

The error body: fields that earn their place

Start from RFC 9457. It defines the application/problem+json media type with five optional members: type, title, status, detail and instance (summary). Add extensions for problem-specific data, such as an errors array with pointers for validation failures.

{
  "type": "https://api.example.com/problems/invalid-field",
  "title": "Invalid field",
  "status": 422,
  "code": "slug_contains_invalid_character",
  "detail": "slug may contain only a-z, 0-9 and hyphens.",
  "errors": [{ "pointer": "/slug", "constraint": "^[a-z0-9-]+$" }],
  "request_id": "req_8f2c1d"
}

The code field matters most. A stable string a client can branch on, combined with a field path and a stated constraint, removes most support conversations. Paymos documents the same rule: switch on code, not on title or detail, which are human text and will be reworded.

Rules we apply:

  1. code is permanent. Add new codes; never rename old ones.
  2. detail is for humans and may change.
  3. Multiple problems go in one response as a list, one entry per problem.
  4. Add a retryable flag when the server knows the answer.

Picking status codes

A compact map, following Zuplo's guidance:

CodeUse it for
400Malformed syntax
401Missing or invalid credentials
403Authenticated but lacking permission
404Resource does not exist
409State conflict
422Valid syntax, failed business validation
429Rate limit, with Retry-After
500Unhandled server error
502Invalid upstream response
503Maintenance or overload, with Retry-After
504Upstream timeout

A 401 must carry a WWW-Authenticate challenge, per the Siemens API guidelines. The 502/504 split matters for any API that wraps other systems: it tells the caller the fault is upstream, not in their request.

Useful messages without leaking internals

Good messages have two parts: the cause in terms of the caller's own request, and the fix. PartnerMatch's three habits are to name the field or value, say what good looks like, and keep one message per problem.

Nielsen Norman Group's error message guidelines say the same for humans: plain language, precise description, constructive advice.

The security rule is simple: be precise about the caller's input, opaque about the system's state. That includes the difference between "you may not access this" and "this does not exist" (source). RFC 9457's security section warns against exposing stack traces, SQL, internal hostnames and secrets, and recommends logging private detail server-side and returning a safe occurrence identifier. CWE-209 lists the usual leaks, including login endpoints that distinguish "invalid password" from "user not found".

Useful messages without leaking internals

Retries: what the client should do

The error body should let the client decide without guessing. The guidance from Zuplo:

  • Retry 429, 500, 502, 503 and 504 only when the operation is safe to repeat or protected by an idempotency key.
  • Respect Retry-After.
  • Use exponential backoff with jitter and cap attempts or total time.
  • Do not retry validation, auth or not-found failures.

Google Cloud adds that repeating non-idempotent operations can create duplicates or side effects, and that permanent authorization or configuration errors need a change before another attempt helps. Our note on idempotent web fetch covers why reads are the easy case.

Request IDs and structured logs

Generate an ID at the edge, pass it through every service, put it in logs and return it in the error. Microsoft's playbook describes a correlation ID as the glue binding a transaction together. Salesforce Engineering describes how a correlation ID in a header lets both developers and support query all related logs.

Request IDs and structured logs

Log route, action, status, timing and retry count as named JSON fields. Never log secrets.

Perspective

Most scraping vendors hide the interesting part. A blocked page, a paywall, a timeout and an empty extraction all collapse into a 200 with empty content, or a generic 500. An agent cannot reason about that. It guesses.

We built Gyrence so every call returns a typed, discriminated-union response, failures included. A blocked fetch is a value with a reason, not an exception. Agents can branch on it, route it, or escalate it.

Web data is a storm. Pretending otherwise is how bills and pipelines surprise you. Naming failure modes is the honest design, and it is the one we ship.

Glen

Applying this to scraping, agents and RAG

Scraping has failures plain REST rarely has: blocked fetches, content that changed shape, extraction that returns nothing for a field. Treat these as first-class variants.

  • Fetch: success, blocked, timeout and unsupported content type are different variants with different retry rules.
  • Extract: a missing field is not a failed document. LandingAI recommends routing missing fields to a separate workflow and terminal errors to a dead-letter queue with the error code and document reference.
  • Crawl: report per-URL outcomes, so one bad page does not hide 500 good ones.

See our patterns for discriminated union responses, agent web tool failure handling and typed scraping failures. If you document the contract in OpenAPI, discriminators model the variants cleanly.

Try typed failures on a real API

Gyrence exposes Search, Traverse, Fetch, Extract and Map, plus a hosted MCP endpoint, and every response is typed, failures included. Billing uses spending caps and bundled LLM extraction with no separate AI charge. Standard is $75 per month, Growth is $299 per month and Scale is $549 per month; Free, Founders and Pay-As-You-Go are also available on the pricing page.

Gyrence

For change monitoring with webhook alerts, see WebDoppler.

FAQ

Should I use RFC 9457 or a custom error format?

Use RFC 9457 as the base. It is a standard, clients and tooling already understand it, and it allows extension members. Add your own stable code and request_id on top.

Is the HTTP status code enough?

No. Status codes classify the failure, but many distinct problems share one code. A stable machine-readable code plus a field pointer lets clients react precisely.

Should error messages say exactly what went wrong?

About the caller's input, yes: name the field and the fix. About your system, no. Keep stack traces, hostnames and SQL out of responses and give a request ID for support instead.

Which errors should clients retry?

Transient ones such as 429, 502, 503 and 504, with backoff, jitter and a cap, and only for safe or idempotent operations. Permanent errors need a changed request first.

Sources