Pandas
Pandas Missing Values
What is a Missing Value?
A missing value means data is not available for a particular column/record.
In Pandas, missing values are represented as NaN (Not a Number).
Detecting Missing Values
Python
df.isnull()
df.isna()
df.notnull()
df.notna()
isnull() / isna() → returns True where value is missingnotnull() / notna() → returns True where value is present
Counting Missing Values
Python
df.isnull().sum() # missing values per column
df.isnull().sum().sum() # total missing values
df.isna().sum()
True is treated as 1 and False as 0, so
.sum() gives the count of missing values.
Dropping Missing Values
Python
df.dropna() # drop rows with any missing value
df.dropna(axis=0) # same as above (rows)
df.dropna(axis=1) # drop columns with any missing value
df.dropna(how="any") # drop if any value is missing
df.dropna(how="all") # drop only if all values are missing
df.dropna(subset=["Age", "Salary"])
dropna() removes rows or columns that contain missing values.Original DataFrame is not changed unless you use
inplace=True.
Filling Missing Values
Python
df.fillna(0) # fill all NaN with 0
df["Age"].fillna(df["Age"].mean())
df["Salary"].fillna(df["Salary"].median())
df.fillna({"Age": 30, "Salary": 50000})
We can fill missing values with a constant, mean, median, or different values for different columns.
Forward Fill (ffill)
Forward fill takes the previous valid value and fills the missing value with it.
Python
df["credit_score"].ffill()
df.fillna(method="ffill") # older syntax
Output
0 780.0 1 720.0 2 690.0 3 690.0 4 810.0 5 700.0 6 650.0 7 650.0 Name: credit_score, dtype: float64
The missing value is replaced by the last known valid value above it.
Backward Fill (bfill)
Backward fill takes the next valid value and fills the missing value with it.
Python
df["credit_score"].bfill()
Output
0 780.0 1 720.0 2 690.0 3 810.0 4 810.0 5 700.0 6 650.0 7 NaN Name: credit_score, dtype: float64
The missing value is replaced by the next known valid value below it.
If there is no next value, it remains NaN.
If there is no next value, it remains NaN.
Summary
isnull()/isna()→ detect missing valuesisnull().sum()→ count missing valuesdropna()→ remove rows/columns with missing valuesfillna()→ fill missing values with a valueffill()→ forward fillbfill()→ backward fill
Missing Values in Pandas
Detect, drop and fill missing data using isnull, dropna, fillna, ffill and bfill.
Select any topic from the left sidebar to begin.
Start Learning →
