Reference · Glossary
Loss functions (overview)
Last updated
A **loss function** scores how wrong a model's predictions are — training minimizes this score so the model improves on the task you actually care about.
#When to use
Choosing the right objective for regression (MSE), classification (cross-entropy), ranking, or generation — each task needs a loss that matches the output type.
#When not to
When you're only doing inference with a frozen model — loss is a training-time concept unless you're fine-tuning or evaluating.
#Example
| Task | Common loss | Intuition |
|------|-------------|-----------|
| Predict price | MSE | Penalize big numeric errors |
| Spam filter | Cross-entropy | Penalize confident wrong class |
| Next token | Cross-entropy | Penalize unlikely correct word |
Pick loss → compute gradient → update weights. Wrong loss = model optimizes the wrong goal.