Gradient boosting in Python

Trees fitted one after another, each correcting what the last got wrong.

from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=0, stratify=y)

model = HistGradientBoostingClassifier(
    learning_rate=0.1, max_iter=200, early_stopping=True, random_state=0)
model.fit(X_train, y_train)

print(f"train {model.score(X_train, y_train):.3f}")
print(f"test  {model.score(X_test, y_test):.3f}")
print("stages", model.n_iter_)

How it works

1. learning_rate trades accuracy against the number of stages.
2. HistGradientBoostingClassifier is the fast histogram version.
3. staged_predict lets you watch the score improve per stage.

Keywords and builtins used here

• print

The run, in numbers

Lines

15

Characters to type

580

Tokens

130

Three-star pace

110 tpm

At the three-star pace of 110 tokens a minute, this run takes about 71 seconds.

Type this snippet

Step 2 of 4 in Models, step 13 of 25 in Machine learning with scikit-learn.

← Previous Next →