For streaming LLM tokens from a hosted model API to a client, server-sent events over plain HTTPS is the right default — it is the transport the providers themselves ship. Anthropic’s Messages API streams responses incrementally using server-sent events when you set stream to true, and Google’s Gemini API mirrors the same split between generateContent and streamGenerateContent. Reserve WebSocket for the narrower case where the client must also push frames over the same connection: collaborative prompt editing, voice pipelines, or agents that interrupt generation mid-stream.
What the providers use
Anthropic documents a fixed event flow for the Messages API: a message_start event opens the stream, content arrives as a sequence of content_block_start, content_block_delta and content_block_stop events, and a final message_stop closes it. Ping events keep intermediaries from timing the connection out, and the API can send error events mid-stream — an overloaded_error, for example, corresponds to what would normally be an HTTP 529 response in a non-streaming call. Token counts in message_delta events are cumulative, which matters if you meter usage from the wire.
Google exposes the same split in the Gemini API: generateContent returns a single complete response, while streamGenerateContent produces a streaming model response from the same request body. Streaming also changes the limits you plan against — a streaming request can stop after 65,536 output tokens with a continuation token, and your client resends the original payload with that token to keep generating instead of appending partial text.
The SDKs push you the same way. Anthropic’s SDKs expose a streaming helper that keeps the HTTP connection alive with server-sent events and can either surface each delta or accumulate them into the final Message object, which is useful for long generations that would otherwise hit client timeouts. In the Google Gen AI SDK, automatic continuation is enabled by default, so token-limit stops can be transparent if you let the SDK manage the loop.
SSE vs WebSocket compared
| Dimension | SSE | WebSocket |
|---|---|---|
| Transport | One long-lived HTTP response with content type text/event-stream | Full-duplex TCP connection after an HTTP upgrade handshake |
| Direction | Server to client only | Bidirectional |
| Browser client | EventSource with automatic reconnection | WebSocket with hand-rolled reconnect logic |
| Authentication | Ordinary HTTP headers and cookies on the request | Headers only during the upgrade; later auth travels in frames |
| Infrastructure | Passes through proxies and CDNs when buffering is off | Needs upgrade support in every proxy, load balancer and ingress |
| Payload | UTF-8 text events with id and retry fields | Any framing you define, including binary |
| Best fit | Token deltas, progress updates, logs | Shared editing, voice, client-driven interruption |
The asymmetry that decides most cases: generation is one-directional. The model produces tokens; the user waits. You pay for WebSocket’s bidirectionality in proxy configuration, reconnection code and auth complexity without ever using the return channel. SSE keeps your endpoint a normal HTTP handler that simply declines to close the response.
The decision procedure
- Start with SSE and the provider SDK’s streaming helper, and keep the provider connection server-side: re-serialize events to your own clients so a provider SDK change never breaks the browser contract.
- Disable response buffering on every hop —
proxy_buffering offin nginx or theX-Accel-Buffering: noheader — or the first token arrives as one giant chunk. - Serve the stream over HTTP/2 so many tabs multiplex one connection instead of fighting the per-domain SSE cap.
- Budget the payload before the call: counting tokens before inference tells you whether the stream finishes in seconds or runs for minutes.
- Move to WebSocket only when a concrete requirement needs the return channel — shared document state, duplex audio, or cancel-and-redirect gestures that must reach the server without a new request.
Failure modes and recovery
The client side is easier than it looks. In the browser, EventSource reconnects for free: if the connection between client and server closes, the browser restarts it automatically, and only an explicit close call terminates the stream. A server can also steer the retry interval with the retry field. WebSocket gives you none of this; the reconnect state machine is yours to write and test.
The trap is tab-heavy users. Without HTTP/2, browsers allow only 6 open SSE connections per domain across all tabs, and both Chrome and Firefox have marked the cap as won’t fix. Six background tabs each holding a stream is a real support ticket. Terminate streams server-side after message_stop, serve everything over HTTP/2, and send periodic comment lines as keep-alives so idle proxies do not reap the socket.
Mid-stream provider failure is the third mode. Anthropic’s documented recovery strategy is capture-and-resume: keep every delta you received, then issue a continuation request that contains the partial response, and continue streaming from the interruption. Google’s equivalent is the continuationToken: when the output limit stops the decode, resend the original request with the token rather than appending partial text to the contents. Both patterns assume you kept the partial state, which is another argument for terminating the provider stream in your backend, where you can log it, meter it and resume it — and for planning around capacity interruptions that end streams from the infrastructure side.