Filtering with boolean masks

The idea

Most questions start with "only": only completed orders, only Jakarta, only orders over Rp 300,000 in March. In a spreadsheet you'd click a filter. In pandas you build a boolean mask: a Series of True/False, one per row, and use it to keep the True rows.

import pandas as pd

sales = pd.read_csv("sales.csv")

is_completed = sales["status"] == "completed"
print(is_completed.head())
print(is_completed.sum())

0    True
1    True
2    True
3    True
4    True
Name: status, dtype: bool
45

The comparison runs on every row at once (vectorised, like the maths in module 1) and gives back a boolean Series. Because True counts as 1, .sum() counts the matching rows: 45 completed lines.

Applying the mask

Put the mask inside square brackets, and pandas keeps the rows where it's True:

completed = sales[is_completed]
print(completed.shape)
print(completed["revenue"].sum())

(45, 12)
11836000

Compare with the unfiltered total of 17,502,000 from the last module: including cancelled and pending orders would overstate revenue by almost 50%. Filtering correctly is the difference between a right and a wrong report.

You'll usually write the condition inline:

big = sales[sales["revenue"] >= 500000]
print(big[["customer", "product", "revenue"]])

         customer             product  revenue
2       Eka Putri    Pour-over Kettle   960000
6   Kartika Rahma           USB-C Hub   578000
9    Lukman Hakim           USB-C Hub   578000
22  Hendra Wijaya    Pour-over Kettle   640000
26  Hendra Wijaya       Desk Lamp LED   735000
31  Kartika Rahma           USB-C Hub   578000
36  Hendra Wijaya           USB-C Hub   867000
40  Kartika Rahma           USB-C Hub   867000
54    Ayu Lestari  French Press 600ml   555000
57  Kartika Rahma       Desk Lamp LED   735000
67     Citra Dewi    Pour-over Kettle   640000

Read sales[sales["revenue"] >= 500000] as "sales, where revenue is at least 500,000".

Combining conditions: & | ~

Combine masks with & (and), | (or) and ~ (not). Each condition needs its own brackets:

jakarta_completed = sales[(sales["city"] == "Jakarta") & (sales["status"] == "completed")]
coffee_or_kitchen = sales[(sales["category"] == "Coffee") | (sales["category"] == "Kitchen")]
not_cancelled = sales[~(sales["status"] == "cancelled")]
print(len(jakarta_completed), len(coffee_or_kitchen), len(not_cancelled))

13 29 52

For longer filters, name the masks. It reads like the business rule:

is_completed = sales["status"] == "completed"
is_business = sales["segment"] == "Business"
is_big = sales["revenue"] >= 300000
target = sales[is_completed & is_business & is_big]
print(target[["customer", "product", "revenue"]])

         customer           product  revenue
22  Hendra Wijaya  Pour-over Kettle   640000
26  Hendra Wijaya     Desk Lamp LED   735000
31  Kartika Rahma         USB-C Hub   578000
36  Hendra Wijaya         USB-C Hub   867000
40  Kartika Rahma         USB-C Hub   867000
41  Kartika Rahma  Pour-over Kettle   320000
57  Kartika Rahma     Desk Lamp LED   735000

Membership: isin

Say you want only the orders from cities on Java. You could write (city == "Jakarta") | (city == "Bandung") | ... for all five cities, but that's long and easy to get wrong. .isin() checks whether each value is in a list you give it:

java = ["Jakarta", "Bandung", "Semarang", "Surabaya", "Yogyakarta"]
in_java = sales[sales["city"].isin(java)]
print(len(in_java))
print(sales.loc[~sales["city"].isin(java), "city"].unique())

49
['Makassar' nan 'Medan']

Notice NaN (missing city) is not in the list, so ~isin keeps those rows. Missing values never equal anything, which is worth remembering whenever you filter. .isna() and .notna() give masks for missing and present values.

Ranges: between

For "prices from 50 to 120 thousand", you'd normally need two conditions joined with &. .between() does it in one, which reads much more like the question you're asking:

mid_range = sales[sales["unit_price"].between(50000, 120000)]
print(mid_range["unit_price"].min(), mid_range["unit_price"].max())

55000 120000

between(low, high) includes both ends by default.

Text conditions: .str

For text columns, .str methods produce masks too (module 11 covers .str in depth):

coffee_items = sales[sales["product"].str.contains("250g")]
print(coffee_items["product"].unique())

['Arabica Gayo 250g' 'Toraja Kalosi 250g' 'Robusta Lampung 250g']

query: filters as a sentence

.query() lets you write the condition as a string, which is often easier to read for long filters:

min_revenue = 300000
result = sales.query("status == 'completed' and segment == 'Business' and revenue >= @min_revenue")
print(len(result))

7

Inside query, you can use and/or/not, and @name refers to a Python variable. It's a matter of taste; both styles are common in professional code. Use whichever makes the rule clearest.

Counting and summarising with masks

Masks aren't only for filtering. Because they're 0/1 values, they answer "how many" and "what share" directly:

print((sales["status"] == "cancelled").sum())
print(round((sales["status"] == "cancelled").mean(), 3))

19
0.268

The .mean() of a boolean Series is the proportion that's True: here 26.8% of lines were cancelled.

Why this matters for clean code

A filter is a business rule ("a sale counts if it's completed and not a test order"). Give important masks names, define them once, and reuse them, so the rule lives in one place. When finance changes the definition of a valid sale, you edit one line instead of hunting through every analysis.

Recap

• df["col"] == value gives a boolean mask; df[mask] keeps the True rows.
• mask.sum() counts matches; mask.mean() gives the share.
• Combine with &, |, ~, and put brackets around every condition.
• isin([...]) for membership, between(a, b) for ranges, .str.contains() for text, .isna()/.notna() for missing values.
• Missing values never match == or isin.
• df.query("...") writes filters as readable strings; @var uses Python variables.