How much VRAM do I need for a local LLM?

A 7B Q4 model often fits in 6–8 GB. 13B Q4 wants ~10–12 GB. 70B class models usually need 40 GB+ or CPU offload. Always leave headroom for context.

Does quantization always work?

Q4 is the usual starting point for chat. Quality drops on hard reasoning and code. If a model sounds broken, try Q5 or a smaller dense model before buying a GPU.

Lab · Local models · ~2 min

Local VRAM estimator

Pick a model size and quantization to estimate VRAM before you download weights. Then compare local stacks like Ollama and MLC LLM.

How to read this

Weights plus a KV/overhead allowance — not a measured llama.cpp dump.
Long context and image models need more than this table shows.
Compare Ollama, LM Studio, and MLC LLM after you know the memory budget.

Continue in Local LLMs explained →All labs