One API Key for Every Model: How We Use the Vercel AI Gateway

The Vercel AI Gateway is a single endpoint that routes AI SDK calls to any model provider through a plain "provider/model" string. At Devya we removed @ai-sdk/anthropic, @ai-sdk/openai, and three separate provider API keys, and now switch models by editing one string instead of swapping SDK packages.

Key takeaways

The Vercel AI Gateway is a unified API, GA since August 2025, that sits between an application and multiple LLM providers so one credential and one code path reach any model.

With the AI SDK you select a model by a "provider/model" string — 'anthropic/claude-sonnet-5', 'openai/gpt-5' — instead of importing a provider-specific package like @ai-sdk/anthropic.

Model fallbacks let you declare an ordered list of models; if the primary is rate-limited or erroring, the gateway retries the next one on the same request without your code branching.

The gateway adds observability (per-request cost, latency, token counts) and supports zero data retention, so switching providers no longer means re-instrumenting logging each time.

The gateway replaces per-provider SDK wiring and key management — it does not replace the AI SDK itself; you still call generateText, streamText, and the same tool-calling API.

What is the Vercel AI Gateway and what problem does it solve?

The Vercel AI Gateway is a hosted proxy that exposes a single OpenAI-compatible endpoint and forwards requests to whichever provider a model string names. Instead of holding an Anthropic key, an OpenAI key, and a Google key — each with its own SDK and rate-limit behavior — you hold one gateway credential and name the model inline.

The problem it solves is provider lock-in at the code level. When a model call is anthropic('claude-sonnet-5'), trying a different provider means a new import, a new client, and often a new logging shape. When the call is the string 'anthropic/claude-sonnet-5', trying 'openai/gpt-5' is a one-line diff.

How do we switch from a provider SDK to the gateway?

You drop the provider package import and pass a "provider/model" string directly to the AI SDK's model field. The AI SDK resolves any bare string through the gateway by default. In the before version the call is locked to one provider: generateText({ model: anthropic('claude-sonnet-5'), prompt }). In the after version there is no provider import and one credential (AI_GATEWAY_API_KEY) in the environment: generateText({ model: 'anthropic/claude-sonnet-5', prompt }).

Everything downstream — streamText for token streaming, generateObject for schema-constrained output, tool calling — stays identical. The only thing that changed is how the model is named. The model becomes configuration, not code.

When should we use model fallbacks?

Use model fallbacks when a request must succeed even if the preferred provider is rate-limited or having an incident. The gateway lets you pass an ordered list of models; it tries the primary, and on a provider error or rate-limit it retries the next model transparently within the same call. We reach for this on user-facing paths — a chat reply, a generated summary someone is waiting on — where a hard failure is worse than a slightly different model answering.

We do not use fallbacks on background jobs where deterministic model behavior matters. A nightly summarization batch should fail loudly and retry the same model, not silently answer with a different one whose output was never evaluated. Fallbacks trade consistency for availability; spend that trade only where availability matters more.

What does the Vercel AI Gateway replace, and what stays?

The gateway replaces the per-provider plumbing, not the AI SDK. You still write the same SDK calls; you stop writing the provider glue around them.

Model selection: before, anthropic('...') and openai('...') imports; with the gateway, a 'provider/model' string.

Credentials: before, one API key per provider; with the gateway, one gateway key.

Switching provider: before, a new import plus client plus key; with the gateway, edit the model string.

Fallback on outage: before, hand-written try/catch across clients; with the gateway, an ordered fallbacks list.

Cost and latency visibility: before, per-provider dashboards; with the gateway, unified per-request metrics. The SDK call shape (generateText, streamText) is unchanged.

What about cost, data retention, and observability?

The gateway reports cost, latency, and token counts per request in one place, so comparing two models on the same prompt no longer means stitching together two providers' billing pages. It also supports a zero data retention mode, meaning request and response bodies are not stored by the gateway — important when the prompt carries customer data and a clean data-flow answer is needed for a security review.

The practical win is measurement. When model choice is a string and metrics are unified, running a real comparison — same prompt, three models, look at cost and latency — is a config change plus a dashboard read, not an engineering project.

What we would watch before going all-in

One indirection layer means one more dependency in the request path. If the gateway is down, every model call is down, so the availability argument cuts both ways and we still keep the fallback list plus sane timeouts. And a bare string is only provider-agnostic until you rely on a provider-specific feature (a particular tool format, a cache-control header); at that point you are back to provider options, just passed through the gateway instead of a dedicated SDK.

FAQ

Q: Do we still need the AI SDK if we use the Vercel AI Gateway?

A: Yes. The gateway routes model requests; the AI SDK is still what you call in code (generateText, streamText, generateObject). The gateway replaces provider-specific SDK packages like @ai-sdk/anthropic, not the core ai package.

Q: How do we select a model through the gateway?

A: Pass a "provider/model" string to the model field, e.g. 'anthropic/claude-sonnet-5' or 'openai/gpt-5'. The AI SDK resolves bare model strings through the gateway by default.

Q: What happens if the primary model is rate-limited?

A: If an ordered fallbacks list is configured, the gateway retries the next model in that list on the same request. Without fallbacks, the call returns the provider error to handle yourself.

Q: Does the Vercel AI Gateway store prompts?

A: It supports a zero data retention mode in which request and response bodies are not retained by the gateway. Verify the retention setting for your account before sending sensitive data.

Q: Should we use provider-specific SDKs at all in 2026?

A: Default to gateway string routing for portability. Reach for a direct provider SDK only when you need a provider-specific capability the gateway does not surface.