Tutorials · Chapter D (4/4) · ~10 min
Random forests
Try it → see it → read → next
Combine varied decision trees so one brittle path does not control the prediction.
Try yourself
Playground
Forest vote
Five shallow trees vote. Compare majority vote to one deep tree that overfits.
Recap
What you just did
ForestVoteSim let shallow trees vote against one overfit deep tree. Ensembles win by averaging diverse mistakes.
Teach
How it works
The prediction step looks like this:
tree_predictions = ["spam", "not spam", "spam", "spam", "not spam"]
spam_votes = tree_predictions.count("spam")
prediction = (
"spam"
if spam_votes > len(tree_predictions) / 2
else "not spam"
)
print(prediction)
During training, randomness creates useful disagreement:
- Bootstrap rows give each tree a different practice set
- Sample features so trees consider different split options
- Grow trees independently
- Vote or average their predictions
Mental model: ask a diverse panel, then combine the answers instead of trusting one loud judge.
Use it
When you'd use this
- Strong baselines for spreadsheet-like data
- Classification or regression with nonlinear relationships
- Estimating which features the ensemble relies on
Watch out
Watch out
More trees increase compute and do not repair biased data. Forests are harder to explain than one small tree, and feature-importance scores can mislead when features overlap. Always validate on fresh rows.
Try next
Try this next
Flip one tree’s vote in the example. Then flip three. Identify when the forest’s final answer changes.