Reference · Glossary
Small language model (SLM)
Last updated
A language model built to run with far fewer parameters (roughly under ~10B) than flagship models — small enough to run on a laptop, phone, or cheap server — trading some raw capability for speed, cost, and the ability to run **on-device** or fully offline.
#When to use
Latency-sensitive or offline features (on-device assistants, privacy-sensitive tasks), narrow tasks that don't need frontier reasoning, or cost-sensitive high-volume routing where a big model is overkill.
#When not to
Open-ended reasoning, long documents, or tasks needing broad world knowledge — flagship models still win on raw capability and context length.
#Example
Running a 3B-parameter model on-device to classify support tickets by urgency instantly, while routing only the genuinely hard tickets to a larger hosted model.