To degrade cleanly

The provider will have incidents. Rate limits will kick in at the worst possible moment. The question isn't how to avoid the outage but to decide in advance what happens when it does.

The fallback chain

Primary model, then a secondary model from a different provider, then a degraded but honest response. A secondary from the same provider protects against nothing: incidents are often systemic.

This assumes you've abstracted the call behind a single interface from day one. Retrofitting that abstraction under pressure, during an incident, is an experience best avoided.

Timeouts and retries

An explicit maximum timeout, always. A retry with exponential backoff and some jitter, so your clients don't all retry at the same second and finish off a service that's recovering.

And above all: never retry an action that has side effects without an idempotency key. A command sent twice is worse than one not sent at all.

The circuit breaker

Beyond a failure threshold, stop calling for a while and serve the fallback directly. Continuing to hammer a struggling service lengthens its outage and yours.

What you tell the user

"An error occurred" is the worst message: it tells them nothing and doesn't say what to do. Prefer: "The analysis service is temporarily unavailable. Your document is saved, and we'll process it as soon as it's back up."

The rule: say what isn't working, what's been saved, and what happens next. Three pieces of information, and the complaint doesn't come.