Skip to the solution
MediumFrontend13 min read

Vercel AI SDK 5 streamText: how do you stream LLM chat to React?

59 views

The problem

Question presented to candidate: "We are building a chat page in Next.js with the Vercel AI SDK 5. Walk me through the server route and the React side: how the tokens get from streamText into the component, what the messages look like, and how you handle tools, stopping, errors and saving the conversation."

What a strong answer should cover:

  • Server: a route handler that receives UIMessage[], converts them with convertToModelMessages, calls streamText({ model, system, messages, tools, stopWhen, abortSignal }) and returns result.toUIMessageStreamResponse()
  • The wire format: the UI message stream protocol, SSE data: frames of typed chunks (start, text-start, text-delta, tool-input-available, tool-output-available, finish, ...) ending in data: [DONE], with a x-vercel-ai-ui-message-stream: v1 header
  • Client: useChat from @ai-sdk/react (default transport POSTs to /api/chat) returning messages, sendMessage, status (submitted, streaming, ready, error), stop, regenerate, error; rendering message.parts by type, including typed tool parts with states
  • Production concerns: persisting with originalMessages + onFinish, stable chat and message ids, propagating abort, error masking (the default sends only "An error occurred."), throttling re-renders, and not trusting client-sent history blindly
  • Version awareness: v5 introduced this UIMessage / transport design; v6 made convertToModelMessages async; v7 deprecates the result helpers in favour of createUIMessageStreamResponse + toUIMessageStream

Clarifying questions expected:

  • Do conversations need to be saved and reloaded, and do answers need to survive a page refresh mid-stream?
  • Do tools run on the server, or do some need the browser or a human confirmation?
  • Which AI SDK major version and Next.js router is the app on?

Code / implementation expected: Yes โ€” a route handler and a client component, or the protocol and message-parts reducer they rely on.

  • ai-sdk
  • streaming

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

13 min read ยท 12 sections

Read the code

UI message stream: server chunks to message parts
// The AI SDK 5 chat pipeline in miniature: model parts -> UI message chunks
// (the shapes toUIMessageStreamResponse sends, as SSE) -> a useChat-style
// reducer that builds message.parts. Stub model, fake clock, no "ai" package.

const words = "It is 18 degrees in Paris right now, mild enough for a light jacket.".split(" ");

// 1) Server: what a streamText result would turn into, one UI chunk per event.
const chunks = [
  { type: "start", messageId: "msg_1" },
  { type: "start-step" },
  { type: "tool-input-start", toolCallId: "call_1", toolName: "getWeather" },
  { type: "tool-input-delta", toolCallId: "call_1", inputTextDelta: '{"city":' },
  { type: "tool-input-delta", toolCallId: "call_1", inputTextDelta: '"Paris"}' },
  { type: "tool-input-available", toolCallId: "call_1", toolName: "getWeather", input: { city: "Paris" } },
  { type: "tool-output-available", toolCallId: "call_1", output: { tempC: 18 } },
  { type: "finish-step" },
  { type: "start-step" },
  { type: "text-start", id: "txt_1" },
  ...words.map((w, i) => ({ type: "text-delta", id: "txt_1", delta: (i ? " " : "") + w })),
  { type: "text-end", id: "txt_1" },
  { type: "finish-step" },
  { type: "finish" },
];
const sse = chunks.map((c) => "data: " + JSON.stringify(c) + "\n\n").join("") + "data: [DONE]\n\n";
console.log("server sent " + chunks.length + " chunks as " + sse.length + " characters of SSE");
console.log("first frame:  " + sse.split("\n\n")[0]);
console.log("a text frame: " + sse.split("\n\n")[11]);

// 2) Client: parse frames and fold them into one assistant UIMessage.
const msg = { id: null, role: "assistant", parts: [] };
const statusLog = ["submitted"];
let status = "submitted";
const setStatus = (s) => { if (s !== status) { status = s; statusLog.push(s); } };
const toolPart = (id) => msg.parts.find((p) => p.toolCallId === id);
const textPart = () => msg.parts[msg.parts.length - 1];

function apply(c) {
  switch (c.type) {
    case "start": msg.id = c.messageId; break;
    case "start-step": msg.parts.push({ type: "step-start" }); break;
    case "tool-input-start": msg.parts.push({ type: "tool-" + c.toolName, toolCallId: c.toolCallId, state: "input-streaming" }); break;
    case "tool-input-delta": break; // a real client may parse partial JSON for a live preview
    case "tool-input-available": Object.assign(toolPart(c.toolCallId), { state: "input-available", input: c.input }); break;
    case "tool-output-available": Object.assign(toolPart(c.toolCallId), { state: "output-available", output: c.output }); break;
    case "text-start": msg.parts.push({ type: "text", text: "", state: "streaming" }); break;
    case "text-delta": textPart().text += c.delta; break;
    case "text-end": textPart().state = "done"; break;
    case "finish": setStatus("ready"); break;
  }
}

// Chunks arrive 10 ms apart starting at t = 400 ms. Count React re-renders
// with no throttle, and with a 50 ms throttle (leading + trailing).
const frames = sse.split("\n\n").filter(Boolean).map((f) => f.slice(6));
let renders = 0, throttled = 0, lastRender = -Infinity, pending = null;
frames.forEach((data, i) => {
  const now = 400 + i * 10;
  if (data === "[DONE]") return;
  if (pending !== null && pending <= now) { throttled++; lastRender = pending; pending = null; }
  apply(JSON.parse(data));
  if (status === "submitted") setStatus("streaming");
  renders++;
  if (now - lastRender >= 50) { throttled++; lastRender = now; } else if (pending === null) pending = lastRender + 50;
});
if (pending !== null) throttled++;

console.log("");
console.log("status: " + statusLog.join(" -> "));
console.log("message.id: " + msg.id);
msg.parts.forEach((p, i) => console.log("parts[" + i + "] " + JSON.stringify(p)));
console.log("");
console.log("re-renders, one per chunk:      " + renders);
console.log("re-renders, 50 ms throttle:     " + throttled);

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 60 of 83 in AI engineering

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextMedium

What should you log and trace in an LLM application?

Continue