Reference · Glossary
Loss function
Last updated
A **loss function** scores how wrong a model’s prediction is during training. Optimizers push parameters to reduce that score.
#When to use
Every supervised training run — classification, regression, sequence modeling. The loss must match the job (e.g. cross-entropy for classes).
#When not to
Treating training loss alone as product success. Low loss can still mean bad UX, bias, or poor calibration on real traffic.
#Practical tips
- Match loss to metric stakeholders care about
- Watch train vs validation loss for overfitting
- For LLMs, SFT loss ≠ chat quality — use eval tasks too
#Learn next
- Lesson: `loss-functions`
- How-to: Pick a loss function
- Related: gradient descent