Machine learning with scikit-learn
25 steps in 8 sets of Python.
scikit-learn's real contribution is the estimator API: fit, predict, transform, and the fact that every model in the library obeys it. Learn that shape once and the rest of the library is a catalog.
So the tour starts there, then preprocessing and the pipelines that stop you leaking test data into training. Validation, a run through the models, scoring that goes past accuracy, unsupervised methods, and a final set on the things you reach for when the basics stop being enough.
The estimator API
- The estimator APIEvery model in scikit-learn is the same shape: construct, fit, predict.
- Splitting the dataNever score a model on rows it trained on: split first, always.
- Linear regressionFitting a line, then reading the coefficients and the error.
Preprocessing
- Scaling featuresModels that measure distance need features on comparable scales.
- Filling in missing valuesSimpleImputer replaces gaps with a statistic learned from the training data.
- Different columns, different treatmentColumnTransformer routes numeric and categorical columns down separate paths.
- PipelinesA pipeline chains preprocessing and a model into one estimator, so nothing leaks.
Validation
- Cross validationOne split is a coin toss; cross validation scores every fold and averages.
- Choosing the foldsKFold and StratifiedKFold decide how the data is carved up.
- Grid searchGridSearchCV tries every combination with cross validation and keeps the best.
- Does more data help?A learning curve answers whether to gather data or change the model.
Models
- Random forestsAn ensemble of trees, and the feature importances that come with it.
- Gradient boostingTrees fitted one after another, each correcting what the last got wrong.
- Nearest neighboursThe model that does no work until you ask it a question.
- Decision treesA model you can read: depth, splits, and the rules it learned.
Scoring
- Classification metricsAccuracy alone hides the interesting failures: look at precision and recall too.
- The confusion matrixFour counts that explain every classification score you might quote.
- Probabilities, thresholds and AUCpredict_proba gives scores; the threshold is a decision you make, not the model.
- Regression metricsError in the units you care about, and the share of variance explained.
Unsupervised
- KMeans clusteringUnsupervised: no labels, just centroids and the points nearest each one.
- PCARotating the data so the first axes carry the most variance.
Beyond the basics
- A transformer of your ownImplement fit and transform and your code slots into any pipeline.
- Imbalanced classesWhen one class is rare, accuracy lies and the weights need saying.
- Saving a fitted modeljoblib writes the whole fitted pipeline, preprocessing included.
Encore
- sk_churn_model.pyAn end-to-end model: split, preprocess by column type, grid search, report.