Instrument the SSE token relay

Instrument the SSE relay for model tokens so a run records enough structure to debug offline: counts, latency if relevant, pass/fail of time-to-first-byte, event count, cancel stops within N events, and a stable stage name. Redact secrets and raw credentials from every event.

const event = { stage: 'sse-relay', ttfb_ms: 42, chunks: 3, cancelled: false };
console.log(JSON.stringify(event));
if (!('ttfb_ms' in event)) throw new Error('missing ttfb');

Expected evidence: SSE telemetry event. Prefer JSON or structured text you can grep in CI over prose logs for streaming-responses.

Turn one historical failure—especially buffering the whole answer before first byte—into a test that fails if the signal disappears for the SSE token relay. Observability without a failing test is optional decoration; observability with a test is part of the streaming-responses artifact.

Draft a three-field event for the SSE token relay: stage, ok, and one domain field derived from time-to-first-byte, event count, cancel stops within N events. Add fixture_id or docs_version when content can change. Explicitly list fields that must never appear (tokens, passwords, raw prompts) because streaming secrets from upstream error bodies to the browser is in scope for this lab.

Wire one assertion that fails if the SSE token relay event is missing after a run. Observability that cannot fail a test will not survive contact with a busy streaming-responses repository.

Imagine a teammate opens only your event stream after a bad deploy. Could they tell whether fake chunk iterator yielding three tokens then done was wrong, whether buffering the whole answer before first byte, or ignoring client disconnect returned, or whether streaming secrets from upstream error bodies to the browser slipped through? If not, rename fields until those three stories are distinguishable.

At the testing and observability stage for streaming-responses, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about fake chunk iterator yielding three tokens then done that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: non-streaming JSON endpoint latency for the same prompt.

For this page specifically, success looks like a structured event schema locked by a test while still centering the user decision to stream tokens to a client while honoring cancel and partial failure. If you cannot point to a file, command, or assertion that proves that for the SSE token relay, stay on this page instead of advancing.

Glossary: tool · Glossary: structured output · Cheatsheet: production ops signals

Previous · Next