Page 4 of 8~113 min topic

NumPy and pandas basics

Filtering rows with boolean masks

`df[condition]` reads as "keep only the rows where this is True" — the single most-used pattern in pandas.

~14 min this pageCore skill

1Learn the idea

Read

Combining conditions

Combining multiple conditions uses & (and) and | (or) — not Python's and/or keywords, which don't work element-wise on pandas/NumPy data. Each condition also needs its own parentheses:

df[(df["score"] >= 80) & (df["hours_studied"] < 5)]

Forgetting the parentheses around each condition is the single most common error here — Python's operator precedence will otherwise try to evaluate & before the comparisons finish, producing a confusing error rather than the answer you expected.

Read

Selecting rows and columns together with `.loc`

Filtering rows and picking specific columns can be combined in one step using .loc:

df.loc[df["score"] >= 80, ["name", "score"]]

Read this as: "rows where score is at least 80, but show me only the name and score columns." .loc is worth learning early because it's the pattern you'll see in almost every real-world pandas example online, even though the two-step version (filter, then select columns) works just as well while you're building intuition.

Read

A subtlety worth knowing: views vs. copies

Filtering a DataFrame sometimes returns a view into the original data and sometimes a copy — and pandas isn't always obvious about which. This matters when you try to modify the filtered result afterward:

passing = df[df["score"] >= 80]
passing["bonus"] = 5   # may raise a SettingWithCopyWarning

The safe habit, once you intend to modify a filtered result, is to be explicit about wanting a copy:

passing = df[df["score"] >= 80].copy()
passing["bonus"] = 5   # safe — passing is unambiguously its own DataFrame

You don't need to fully understand pandas' internal memory model to work safely here — just remember: if you're about to modify a filtered DataFrame, add .copy() when you create it.

Read

Filtering is the gateway to real data cleaning

Almost every data-cleaning task decomposes into: find the rows with a problem (a mask), then either fix them, drop them, or handle them separately. "Remove rows with a missing score," "keep only requests from the last 30 days," "flag orders over $1,000 for review" are all the same boolean-masking pattern with a different condition.

Go deeper

Before you start

Why this matters

The previous page introduced df["score"] >= 80, which returns a column of True/False values — one per row. On its own, that's just a Series of booleans. The move that makes it powerful is putting that boolean Series back inside the DataFrame's own brackets:

mask = df["score"] >= 80
print(mask)          # True, True, False — one per row

passing = df[mask]   # only the rows where mask is True
print(passing)

You'll see this written both across two lines (as above, useful while learning) and inline in one line once it's familiar: df[df["score"] >= 80]. Both do exactly the same thing.

In the wild

See how this idea shows up as a product and a company — then come back to the lesson. Skills transfer across vendors.

Continue learning · glossary & guides