Compare · Local models · Research notes
Best free local LLM tools — scored
Local stacks look interchangeable until you hit VRAM, tool calls, or a browser-only constraint. This page is a 15-minute protocol, not a permanent ranking.
Last reviewed: 2026-08-15. Re-run the protocol after major model or pricing changes.
How do you test local LLM tools in 15 minutes?
- Write the constraint first: desktop chat, OpenAI-compatible API, or on-device/browser.
- Estimate VRAM for one 7B Q4 model before you download anything larger.
- Run the same three prompts: a short rewrite, a date/number fact check, and one tool-shaped instruction.
- Log install time, tokens/sec feel, and whether the model stayed on-device.
What is the scoring rubric?
| Criterion | Weight | How to score |
|---|---|---|
| Fits the constraint | Gate | Fail if you needed a cloud fallback to finish the job. |
| Install / first reply | 1–5 | Time to a useful answer on a 7B-class model. |
| Quality on a fixed prompt | 1–5 | Same prompt, same temperature. Score edit time, not vibes. |
| VRAM honesty | Pass/Fail | Fail if the UI hid memory needs until download finished. |
Which local LLM tools should I start with?
| Tool | Best fit | First-reply feel | Best constraint | Note |
|---|---|---|---|---|
| Ollama | Desktop serve / API | Usually fastest path | Chat + OpenAI-compatible API | Default when you want a local server without compiling runtimes. |
| LM Studio | Desktop UI | Friendly first hour | Point-and-click local chat | Strong when you want a GUI more than a CLI. |
| MLC LLM | On-device / browser | Slower setup | Edge and web runtimes | Pick when portability matters more than a polished chat app. |
| LocalAI | Drop-in API | Ops-shaped | OpenAI-compatible local API | Use when an existing client already speaks the OpenAI API. |
Learn the craft, not just the tool
Practice the transferable habit in Local LLMs explained, then Local VRAM estimator, Local LLM vs API, First local LLM guide.
FAQ
What is the best free local LLM tool?
Ollama is the usual desktop default. LM Studio wins for a GUI. MLC LLM wins when the job is on-device or in-browser. Hold one 7B prompt constant before you standardize.
How much VRAM do I need?
A 7B Q4 model often fits in 6–8 GB. Use the VRAM estimator, then confirm on your card. Long context needs more than the weight file suggests.