Reference · Glossary
Local LLM
Last updated
A **local LLM** runs on your machine or private hardware instead of a vendor chat API. Useful for privacy, offline experiments, and learning serving tradeoffs.
#When to use
Sensitive prototypes, air-gapped exploration, cost control for high-volume low-stakes tasks, and learning model limits hands-on.
#When not to
Expecting frontier quality on a laptop GPU for hard reasoning, or skipping evals because “it’s private so it’s fine.”
#Quality checklist
- Hardware fits model size (RAM/VRAM)
- Pin model revision
- Compare against a cloud baseline on golden tasks
- Know quantization’s quality cost
#Learn next
- Compare: Local LLM vs API
- How-to: Run a local LLM with Ollama