Chapter DRandom forestsPage 1 of 8

Random forests

Define the lab goal and success criteria

A random forest trains many decision trees on varied row and feature samples, then combines their votes or numeric averages to reduce dependence on one brittle tree.

~14 minLab goal

1Try it yourself

Playground

Forest vote

Five shallow trees vote. Compare majority vote to one deep tree that overfits.

Tree 1 (depth 2): ?
Tree 2 (depth 2): ?
Tree 3 (depth 2): ?
Tree 4 (depth 2): ?
Tree 5 (depth 2): ?
Deep tree (depth 12): ?

Before you start

Why this matters

Before running anything, predict one observable result from the case: a small tabular classifier must remain stable when one noisy training row changes. Write the prediction beside the command or code line that should cause it. This makes the session an experiment rather than a transcription exercise.

2Learn the idea

Read

Explain the chapter step

Begin by writing the success condition in observable terms. For this case, success is not familiarity with the vocabulary; it is producing a reproducible forest classifier with held-out predictions, vote probabilities, and an out-of-bag estimate. Record the starting state so you can distinguish an improvement from a result that was already present.

On this page, the practical job is to state a measurable outcome before changing anything. The running case is a small tabular classifier must remain stable when one noisy training row changes.

Read

Run the working example

from sklearn.ensemble import RandomForestClassifier

X = [[1, 0], [2, 1], [3, 0], [4, 2], [5, 1], [6, 3], [7, 2], [8, 4]]
y = [0, 0, 0, 1, 1, 1, 1, 1]

forest = RandomForestClassifier(
    n_estimators=200, max_depth=3, oob_score=True,
    random_state=7, n_jobs=-1,
)
forest.fit(X, y)
prediction = forest.predict([[5, 2]])
probabilities = forest.predict_proba([[5, 2]])
print("predicted class:", prediction[0])
print("probability shape:", probabilities.shape)
print("probabilities sum to one:", probabilities[0].sum().round(6) == 1)
print("oob in range:", 0 <= forest.oob_score_ <= 1)

Expected evidence:

predicted class: 1
probability shape: (1, 2)
probabilities sum to one: True
oob in range: True

The output may include version-specific details such as hashes, paths, fitted thresholds, or final decimal places. Compare the structural facts described here rather than copying placeholders. If the structure differs, stop and inspect the earliest unexpected line.

Read

Read it line by line

  1. n_estimators sets the number of independently grown trees whose results are combined.
  2. max_depth limits each member's complexity.
  3. a fixed random_state makes bootstrap and feature sampling reproducible.
  4. predict_proba reports vote fractions by class; out-of-bag scoring uses rows omitted from each tree's bootstrap sample.

These lines form one chain: rows of [hours, practice_tests] and binary readiness labels becomes majority-vote classes, class probabilities, and evaluation evidence from rows not used by individual trees. Change only one input first. When several values change together, you cannot tell which change caused the new behavior.

Read

Common errors and fixes

  • First failure: very small datasets can produce unreliable out-of-bag scores and warnings because too few independent votes exist. Re-run the smallest command that proves the repair.
  • Second failure: feature order drift still changes predictions even though forests accept the numeric shape. Preserve the failing input as a test when it represents a realistic mistake.
  • Misleading success: more trees reduce Monte Carlo variation but cannot correct wrong labels, leakage, or a biased sample. A clean-looking final line cannot cancel contradictory intermediate evidence.

When debugging, copy the exact error text and inspect names, paths, shapes, types, and versions. Explain the cause in one sentence before changing code. That discipline prevents a guessed repair from creating a second defect.

Read

Evidence for this stage

Use a real held-out set in addition to out-of-bag evidence, compare with one constrained tree and a simple baseline, inspect confusion counts, and test stability across several seeds.

For the current explain step, save the smallest useful evidence: the relevant command, its output, and the input that produced it. Do not use a screenshot as the only record when text can be copied and searched. Keep generated artifacts separate from source inputs so rerunning the example does not destroy the evidence it is meant to evaluate.

Record parameters, scikit-learn version, feature schema, class ordering, baseline metrics, and resource needs. If serializing, load artifacts only from trusted sources and verify their version compatibility.

Read

Reflect on the result

Return to your opening prediction. Mark it correct or rewrite it with the condition you missed. Then explain the difference between a successful execution and a trustworthy result for this specific example.

Checking tutor…

Continue learning · glossary & guides
  1. Which line or command establishes the current step's most important fact?
  2. What output would reveal that very small datasets can produce unreliable out-of-bag scores and warnings because too few independent votes exist?
  3. Can a new user reproduce a reproducible forest classifier with held-out predictions, vote probabilities, and an out-of-bag estimate from the stated setup?