typestar

Missing data in Python

Finding gaps, filling them, and deciding when to drop the row instead.

import pandas as pd

frame = pd.DataFrame({
    "lang": ["python", "rust", None],
    "tpm": [104.0, None, 88.0],
})

print(frame.isna().sum())
print(frame.fillna({"lang": "unknown", "tpm": frame["tpm"].mean()}))
print(frame.dropna(subset=["lang"]))
print(frame["tpm"].ffill().tolist())

How it works

  1. isna gives a boolean frame; summing it counts per column.
  2. fillna takes a value, a dict per column, or a method.
  3. dropna with subset only cares about the columns you name.

Keywords and builtins used here

The run, in numbers

Lines
11
Characters to type
278
Tokens
117
Three-star pace
110 tpm

At the three-star pace of 110 tokens a minute, this run takes about 64 seconds.

Type this snippet

Step 5 of 5 in Grouping, step 18 of 26 in pandas.

← Previous Next →

Missing data in other languages