Streaming AI Responses in Next.js: What Actually Breaks in Production

At Devya, every AI feature we have shipped over the past year — chat assistants, summarizers, report generators — streams its output token by token. Users will sit through a long generation when text appears immediately; they abandon a silent spinner much sooner. The streaming code itself has become short. The failure modes around it are what fill our engineering notes, and this post collects them.

Key takeaways

• Streaming in the Next.js App Router is a Route Handler that returns a streamed Response. The AI SDK's streamText plus toUIMessageStreamResponse() emit an SSE-based message stream, and the useChat hook from @ai-sdk/react consumes it on the client.

• Chat interfaces do not use the browser EventSource API. EventSource only supports GET requests with no body, so AI clients send a POST with fetch and read the SSE-formatted response through a ReadableStream reader.

• A stream that works locally but arrives as one block in production is almost always intermediary buffering: compression middleware, nginx proxy buffering, or a corporate proxy. The headers Cache-Control: no-transform and X-Accel-Buffering: no fix most cases.

• Passing the incoming request's AbortSignal into streamText means a closed tab cancels the upstream model call and stops token spend immediately.

• A page refresh kills the default fetch stream. Persisting finished messages in the onFinish callback on the server is the baseline; resumable streams are worth their extra infrastructure only when mid-generation continuity is a real product requirement.

What does it take to stream an AI response in Next.js?

One Route Handler and one hook. On the server, streamText from the AI SDK — Vercel's TypeScript toolkit for calling language models — starts the model call and returns immediately, and toUIMessageStreamResponse() converts the result into a streaming Response that speaks the SDK's SSE-based UI message stream protocol. On the client, useChat from @ai-sdk/react posts the conversation, parses the stream, and re-renders as parts arrive.

Two details matter on serverless. The function must live as long as the generation, so we export a maxDuration from the route — on Vercel, Fluid Compute streams responses and the default execution cap is 300 seconds. And we pass the model as a provider/model string such as anthropic/claude-sonnet-4-6 through the AI Gateway, which keeps provider switching a one-line change instead of a dependency swap.

Should we use SSE, WebSockets, or plain fetch streaming?

For token streaming from server to client, SSE format read over fetch is the default, and it is what the AI SDK implements. Server-Sent Events (SSE) is a plain-HTTP wire format where the server writes data: lines down one long-lived response. WebSockets earn their place only when traffic is genuinely bidirectional — voice conversations, multiplayer cursors, collaborative editing. For a chat box they add connection state and infrastructure without buying anything.

The subtle point: the browser's built-in EventSource API is not what chat apps use. EventSource can only issue GET requests and cannot attach a body or custom headers, while a chat request needs to POST the conversation history. So the AI SDK sends a normal fetch POST and parses the SSE format off the response body with a ReadableStream reader. We get the SSE wire format without the EventSource limitations — at the cost of its automatic reconnection.

• fetch + SSE format: one-way server to client, any HTTP method and body, works on serverless — the default for AI chat.

• EventSource: one-way, GET only with no body, automatic reconnection with Last-Event-ID — fits live tickers and notifications, not chat.

• WebSockets: bidirectional after an upgrade handshake, needs a persistent connection and fits serverless poorly — right for voice and real-time collaboration.

Why does a stream work locally but arrive all at once in production?

Because something between the function and the browser is buffering. Compression middleware is the most common culprit: gzip and brotli want complete chunks to compress, so small token deltas sit in the compressor's buffer until the response ends. Reverse proxies are second: nginx's proxy_buffering collects the entire upstream response by default. Corporate proxies and some antivirus products do the same and are outside our control entirely.

The fix is to mark the response as untransformable and unbufferable: Content-Type: text/event-stream, Cache-Control: no-cache, no-transform, and X-Accel-Buffering: no for nginx, plus excluding the streaming route from any compression middleware. On Vercel, streaming responses pass through unbuffered out of the box — the classic broken case in our experience is self-hosted Next.js behind an nginx config nobody has touched in a year.

How do we stop paying for tokens nobody is reading?

By wiring aborts end to end. The incoming Request carries an AbortSignal that fires when the user closes the tab, navigates away, or hits stop. Passing it as streamText's abortSignal option propagates the cancellation to the model provider, which stops generating — and stops billing.

There is a real trade-off hiding here. The SDK's consumeStream() does the opposite: it detaches the generation from the client connection so the model runs to completion and onFinish can persist the full message even after a disconnect. Abort-on-disconnect saves tokens but loses the partial response; consume-to-completion keeps the response but pays for every token. We wire abort for cheap conversational chat and consume-plus-persist for expensive long-form generations. Choosing per feature, not globally, is the point.

What happens when the user refreshes mid-stream?

With the default setup, the fetch stream dies with the page, and if the abort signal is wired, the server-side generation dies with it. After the reload, the chat shows whatever onFinish had persisted — completed messages survive, the in-flight one vanishes. For most chat products this is acceptable and honest.

When it is not — long report generations a user kicks off and checks back on — resumable streams close the gap: chunks are published to a store such as Redis as they arrive, and a reconnecting client re-subscribes from where it left off. The AI SDK supports this pattern, but it adds real infrastructure and state to operate. Our rule: persist finished messages always; add resumability only when a product requirement names it.

FAQ

Q: Do we need WebSockets to stream ChatGPT-style responses?

A: No. Token streaming is one-way, server to client, and SSE format over a fetch POST handles it on plain HTTP. WebSockets are for bidirectional traffic such as voice or collaborative editing.

Q: Why does an AI stream not work behind nginx?

A: nginx buffers upstream responses by default. Send X-Accel-Buffering: no on the response (or disable proxy_buffering for the route) and make sure no compression layer is re-buffering the stream.

Q: Does streaming work on serverless platforms like Vercel?

A: Yes. Vercel Functions stream responses, and Fluid Compute keeps the function alive for the whole generation — the default execution cap is 300 seconds, configurable per route with maxDuration.

Q: How do we save the AI message if the user closes the tab mid-stream?

A: Call consumeStream() on the server so the generation runs to completion and onFinish persists the full message. Note this conflicts with abort-to-save-tokens — each route picks one behavior.

Q: Can EventSource send a POST body?

A: No. EventSource only issues GET requests with no body or custom headers, which is why AI chat clients read the SSE format from a fetch POST via a ReadableStream reader instead.