As asked
You are adding an AI chat feature to your product. The LLM API supports streaming. Walk me through the full stack implementation from the API call to the user seeing tokens appear in real time, including how you handle errors mid-stream.
Sample answer outline
On the server you call the LLM with stream:true, pipe the response as a ReadableStream through a Next.js Route Handler with the correct content-type of text/event-stream, and flush chunks as SSE events. On the client you use the Fetch API with a ReadableStream reader or the Vercel AI SDK's useChat hook which handles reconnection and parsing. Error handling mid-stream is the tricky part: if the stream closes with an error event you need to surface that in the UI rather than silently stopping, and you should add a timeout so a stalled stream does not block the user indefinitely. A strong answer also mentions storing partial completions for resumability if the user refreshes.
Expect these follow-ups
- How would you rate-limit streaming endpoints to prevent abuse without killing legitimate long sessions?