Guide · Builder
Your first local LLM
Get a model running on your machine, prove it with golden tasks, and decide whether local, API, or hybrid fits the job — before you build infrastructure.
Last reviewed: 2026-08-02.
Checklist
- Pick Ollama (CLI) or LM Studio (UI) — not both on day one
- Pull a small chat model that fits your RAM/VRAM
- Run three golden tasks you already know the answers to
- Confirm the app is not sending prompts off-machine unexpectedly
- Write a one-line quality note: keep, escalate to API, or hybrid
Which tool should I open first?
- Ollama — CLI-friendly runner; follow run a local LLM with Ollama.
- LM Studio — desktop UI for downloading and chatting locally.
Decision tradeoffs: Local LLM vs API. Concept lesson: Local LLMs mental model.
What next?
If local quality is enough for drafts, keep going on the Builder path. If you need frontier reasoning, escalate to ChatGPT or Claude after redaction.
FAQ
What is the easiest way to run a local LLM?
Start with Ollama (CLI) or LM Studio (desktop UI). Pull a small model, chat once, then run the same three golden tasks you use for hosted APIs.
Is a local LLM private?
It can be if inference stays on-device and apps do not phone home. Read network behavior and avoid pasting secrets until you trust the setup.
Will local quality match ChatGPT?
Usually not on hard reasoning. Local wins on privacy, offline use, and predictable cost. Compare honestly with Local LLM vs API.