Page 2 of 8~104 min topic

Streaming responses

Define the SSE token relay input contract

Page 2 hardens the boundary around the SSE relay for model tokens so bad inputs fail before the interesting algorithm runs.

~13 min this pageData contract

1Learn the idea

Read

Define what may enter

The accepted input remains: upstream chunk iterator, response headers, abort signal. Keep parsing and normalization in functions that do not score, train, or call a model. That split lets a test fail the boundary without blaming the core logic. The user-facing decision stays: stream tokens to a client while honoring cancel and partial failure.

Read

Reject at the boundary

const Env = { requireAbort: true, maxChunkMs: 2000 };
function assertStreamContract(headers) {
  if (!String(headers['content-type']||'').includes('text/event-stream')) throw new Error('SSE content-type required');
}
console.log({Env, ok: true});

Expected evidence: SSE contract flags. If the contract is silent on a bad value, later debugging will look like an algorithm bug when it is really a data bug.

Read

Keep transforms testable

Write one assertion for a neighboring valid input to the SSE token relay so tightening the boundary does not over-reject. Document field names and types the way a teammate would need them on day two of streaming-responses—not as comments you plan to delete.

Read

Lab notebook: name the fields

List every field in fake chunk iterator yielding three tokens then done and mark each as required, optional, or forbidden. Required fields must fail loudly when missing; optional fields need defaults you can quote in a test; forbidden fields (secrets, raw PII, path escapes) must never be accepted silently. This list is the contract for the SSE token relay.

Add one sentence about encoding, units, or timezones if relevant to upstream chunk iterator, response headers, abort signal. Contracts that ignore units create “correct” programs that still ship wrong decisions when someone tries to stream tokens to a client while honoring cancel and partial failure.

Read

Worked judgment

Write the error string you want for the most likely bad input. Prefer ValueError('threshold out of range')-style messages over generic invalid input. The contract’s job is to make buffering the whole answer before first byte, or ignoring client disconnect harder to confuse with a model or algorithm bug later.

Read

Why this stage matters for the SSE token relay

At the data contract stage for streaming-responses, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about fake chunk iterator yielding three tokens then done that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: non-streaming JSON endpoint latency for the same prompt.

For this page specifically, success looks like malformed inputs rejected with field-named errors while still centering the user decision to stream tokens to a client while honoring cancel and partial failure. If you cannot point to a file, command, or assertion that proves that for the SSE token relay, stay on this page instead of advancing.

Glossary: tool · Glossary: structured output · Cheatsheet: production ops signals

Previous · Next

Go deeper

Before you start

Why this matters

Invent one malformed input that the SSE relay for model tokens might accidentally accept. Predict the exception or rejection message. After you run the contract code, compare your prediction with the real failure text.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Check your understanding

Page assessment

Answer from memory. Completion is saved from this evidence, not from opening the next page.

1. Which malformed values die before core logic?
2. Can transform and prediction/search be tested separately?
3. Does the error name the violated field or shape?
4. Is the accepted input still exactly: upstream chunk iterator, response headers, abort signal?

All responses are required.