Chapter DStreaming responsesPage 2 of 8

Streaming responses

Set up interfaces and contracts

Create the project boundary: build an HTTP endpoint that relays model text as Server-Sent Events while preserving cancellation and final usage as a bounded, testable system whose behavior does not depend on trusting plausible model output.

~13 minSetup

Before you start

Why this matters

Before coding, write the single observable result this page must add to the previous page. Then name one failure that the implementation must reject. For Streaming responses, use the running project—not a toy explanation—as your test: an HTTP endpoint that relays model text as Server-Sent Events while preserving cancellation and final usage. Predict what the CLI, test, or HTTP client will observe when the page is complete.

1Learn the idea

Read

Create the project boundary

Use Node.js 20 or newer, TypeScript in strict mode, native fetch, Zod for runtime validation, and Vitest for tests. Install current package releases with npm install zod and npm install -D typescript tsx vitest @types/node; do not copy fictional version numbers. Put secrets in .env.local, commit only .env.example, and inject configuration at startup.

Design interfaces around unknown data. JSON from a model, provider, tool, or remote process is unknown until runtime parsing succeeds. Keep provider wire types at the adapter edge and expose a small domain type to the rest of the program.

The central domain contract is ordered start, delta, error, and done events with an explicit terminal state. Start from type StreamEvent = {type:'start';requestId:string}|{type:'delta';text:string}|{type:'error';code:string}|{type:'done';finishReason:string}. That declaration documents compile-time intent; pair it with a Zod schema or an equivalent handwritten parser at every external boundary. Keep configuration such as AI_BASE_URL, AI_API_KEY, CHAT_MODEL, STREAM_IDLE_TIMEOUT_MS outside source control. Variables are placeholders: consult your selected provider or local server documentation for real endpoint names and supported model IDs.

Define a Dependencies interface with parseInput, execute, parseOutput, now, and log. The production adapter may read the provider response body with a Web ReadableStream and emit framed SSE events; tests inject a deterministic fake. This split prevents network availability from deciding whether domain tests pass and makes cancellation, latency, and malformed responses reproducible.

Read

Implementation

import { z } from "zod";

const Env = z.object({
  AI_BASE_URL: z.string().url().optional(),
  AI_API_KEY: z.string().min(1).optional(),
  REQUEST_TIMEOUT_MS: z.coerce.number().int().positive().default(15000),
});
export const env = Env.parse(process.env);

type StreamEvent = {type:'start';requestId:string}|{type:'delta';text:string}|{type:'error';code:string}|{type:'done';finishReason:string};

The code is intentionally provider-neutral at the adapter boundary. AI_BASE_URL and model names are environment placeholders, not invented services. If the topic uses MCP, install the current official SDK and verify its exported paths against its release documentation. If a provider offers an official SDK, it can replace fetch inside execute without changing the domain contract. Always inspect the real provider’s documented request and response fields before connecting credentials.

Read

Debug the boundary

Debug from deterministic code outward. First print the parsed configuration with secret values replaced by [set]. Next run the parser against a local fixture. Then run the orchestration with a fake dependency. Only after those pass should you inspect the real boundary. Capture status, content type, request ID, elapsed time, and a redacted response shape. Do not “fix” malformed data with a permissive cast; save it as a failing fixture and decide whether the contract or adapter is wrong.

For this chapter, the fastest diagnostic is to assert that every accepted stream ends exactly once as done, error, or client cancellation. When that assertion fails, stop before downstream work. Expected behavior is concrete: The client renders “Hello 👋” in order, receives done once, and closes cleanly. A different but plausible sentence is not enough if the structural and policy checks fail.

Read

Verification notes

  • Strict TypeScript and runtime schemas are enabled.
  • .env.example names variables but contains no secrets.
  • Provider or protocol code is behind an injected adapter.
  • Run npm run typecheck and npm test without provider credentials.
  • Record expected terminal behavior beside the fixture so future changes remain reviewable.

A passing test is necessary but does not establish production quality. Review the fixture set for realistic distributions, stale assumptions, tenant boundaries, and expensive edge cases. When outputs are probabilistic, assert schemas, citations, chosen capabilities, stop reasons, and task rubric scores. Reserve exact string assertions for deterministic adapters and protocol framing.

Checking tutor…

Continue learning · glossary & guides
  • Can you point to the executable check proving that every accepted stream ends exactly once as done, error, or client cancellation?
  • Does malformed or unauthorized data stop before external or privileged work?
  • Is the observable success criterion exactly this clear: The client renders “Hello 👋” in order, receives done once, and closes cleanly.
  • Glossary: tool · Glossary: structured output · Cheatsheet: production ops signals
  • Previous · Next