Reference · Glossary
GPU
Last updated
Graphics Processing Unit — hardware that **parallelizes matrix math**, making neural network inference and training much faster than CPU alone.
When to use
Running local LLMs, training models, or high-throughput serving.
When not to
Tiny batch jobs where CPU + API is cheaper and simpler.
Example
A 7B parameter local model feels usable on a GPU; on CPU alone it may be slow for chat.