Tutorials · Chapter D (4/4) · ~10 min

Loss functions

Try it → see it → read → next

Turn prediction mistakes into one training signal, then see how the choice changes learning.

Try yourself

Playground

Loss landscape (lite)

Slide weight w. Loss = (w − 2)² is the training signal — gradient steps walk downhill.

Weight w: -1.50

Loss = (w − 2)² = 12.250

Recap

What you just did

LossLandscapeLite moved a weight downhill so loss fell. Loss is the training signal that tells parameters which way to step.

Teach

How it works

Mean squared error is common for numeric predictions:

truth = [2.0, 4.0, 6.0]
predictions = [2.5, 3.0, 7.0]

squared_errors = [
    (prediction - target) ** 2
    for prediction, target in zip(predictions, truth)
]
loss = sum(squared_errors) / len(squared_errors)
print(loss)

Classification often uses cross-entropy, which strongly penalizes confident probability assigned to the wrong class.

Compare each prediction with its target
Penalize the difference using a chosen rule
Aggregate example penalties into one score
Optimize parameters to push that score down

Mental model: loss is the scoreboard the training algorithm is trying to shrink.

Use it

When you'd use this

Training regression and classification models
Comparing training runs on the same objective
Choosing whether large errors deserve extra punishment

Watch out

Low training loss can coexist with poor real-world performance or unfair outcomes. Loss scale differs across functions, so raw values are not always comparable. Watch validation metrics that match the actual product goal.

Try next

Try this next

Replace one prediction with a large outlier. Recalculate squared error and absolute error. Which score reacts more?