Reference · Glossary
Streaming (LLM responses)
Last updated
Sending model output **token-by-token** (or chunk-by-chunk) over HTTP instead of waiting for the full completion.
#When to use
Chat UIs where users should see text appear immediately — improves perceived speed.
#When not to
Batch jobs that need the full JSON object before parsing — wait for the complete response.
#Example
`stream=True` on chat completions → read SSE events → append each delta to the UI as it arrives.