Book Image

scikit-learn Cookbook - Second Edition

By : Trent Hauck

Book Image

scikit-learn Cookbook - Second Edition

By: Trent Hauck

Overview of this book

Python is quickly becoming the go-to language for analysts and data scientists due to its simplicity and flexibility, and within the Python data space, scikit-learn is the unequivocal choice for machine learning. This book includes walk throughs and solutions to the common as well as the not-so-common problems in machine learning, and how scikit-learn can be leveraged to perform various machine learning tasks effectively. The second edition begins with taking you through recipes on evaluating the statistical properties of data and generates synthetic data for machine learning modelling. As you progress through the chapters, you will comes across recipes that will teach you to implement techniques like data pre-processing, linear regression, logistic regression, K-NN, Naïve Bayes, classification, decision trees, Ensembles and much more. Furthermore, you’ll learn to optimize your models with multi-class classification, cross validation, model evaluation and dive deeper in to implementing deep learning with scikit-learn. Along with covering the enhanced features on model section, API and new features like classifiers, regressors and estimators the book also contains recipes on evaluating and fine-tuning the performance of your model. By the end of this book, you will have explored plethora of features offered by scikit-learn for Python to solve any machine learning problem you come across.

Preface

What this book covers

Who this book is for

What you need for this book

Reader feedback

Customer support

Free Chapter

High-Performance Machine Learning – NumPy

High-Performance Machine Learning – NumPy

Loading the iris dataset

Viewing the iris dataset

Viewing the iris dataset with Pandas

Plotting with NumPy and matplotlib

A minimal machine learning recipe – SVM classification

Introducing cross-validation

Putting it all together

Machine learning overview – classification versus regression

Pre-Model Workflow and Pre-Processing

Pre-Model Workflow and Pre-Processing

Creating sample data for toy analysis

Scaling data to the standard normal distribution

Creating binary features through thresholding

Working with categorical variables

Imputing missing values through various strategies

A linear model in the presence of outliers

Putting it all together with pipelines

Using Gaussian processes for regression

Using SGD for regression

Dimensionality Reduction

Dimensionality Reduction

Reducing dimensionality with PCA

Using factor analysis for decomposition

Using kernel PCA for nonlinear dimensionality reduction

Using truncated SVD to reduce dimensionality

Using decomposition to classify with DictionaryLearning

Doing dimensionality reduction with manifolds – t-SNE

Testing methods to reduce dimensionality with pipelines

Linear Models with scikit-learn

Linear Models with scikit-learn

Fitting a line through data

Fitting a line through data with machine learning

Evaluating the linear regression model

Using ridge regression to overcome linear regression's shortfalls

Optimizing the ridge regression parameter

Using sparsity to regularize models

Taking a more fundamental approach to regularization with LARS

Linear Models – Logistic Regression

Linear Models – Logistic Regression

Loading data from the UCI repository

Viewing the Pima Indians diabetes dataset with pandas

Looking at the UCI Pima Indians dataset web page

Machine learning with logistic regression

Examining logistic regression errors with a confusion matrix

Varying the classification threshold in logistic regression

Receiver operating characteristic – ROC analysis

Plotting an ROC curve without context

Putting it all together – UCI breast cancer dataset

Building Models with Distance Metrics

Building Models with Distance Metrics

Using k-means to cluster data

Optimizing the number of centroids

Assessing cluster correctness

Using MiniBatch k-means to handle more data

Quantizing an image with k-means clustering

Finding the closest object in the feature space

Probabilistic clustering with Gaussian mixture models

Using k-means for outlier detection

Using KNN for regression

Cross-Validation and Post-Model Workflow

Cross-Validation and Post-Model Workflow

Selecting a model with cross-validation

K-fold cross validation

Balanced cross-validation

Cross-validation with ShuffleSplit

Time series cross-validation

Grid search with scikit-learn

Randomized search with scikit-learn

Classification metrics

Regression metrics

Clustering metrics

Using dummy estimators to compare results

Feature selection

Feature selection on L1 norms

Persisting models with joblib or pickle

Support Vector Machines

Support Vector Machines

Classifying data with a linear SVM

Optimizing an SVM

Multiclass classification with SVM

Support vector regression

Tree Algorithms and Ensembles

Tree Algorithms and Ensembles

Doing basic classifications with decision trees

Visualizing a decision tree with pydot

Tuning a decision tree

Using decision trees for regression

Reducing overfitting with cross-validation

Implementing random forest regression

Bagging regression with nearest neighbors

Tuning gradient boosting trees

Tuning an AdaBoost regressor

Writing a stacking aggregator with scikit-learn

Text and Multiclass Classification with scikit-learn

Text and Multiclass Classification with scikit-learn

Using LDA for classification

Working with QDA – a nonlinear LDA

Using SGD for classification

Classifying documents with Naive Bayes

Label propagation with semi-supervised learning

Neural Networks

Neural Networks

Perceptron classifier

Neural network – multilayer perceptron

Stacking with a neural network

Create a Simple Estimator

Create a Simple Estimator

Create a simple estimator

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Writing a stacking aggregator with scikit-learn

In this section, we will write a stacking aggregator with scikit-learn. A stacking aggregator mixes models of potentially very different types. Many of the ensemble algorithms we have seen mix models of the same type, usually decision trees.

The fundamental process in the stacking aggregator is that we use the predictions of several machine learning algorithms as input for the training of another machine learning algorithm.

In more detail, we train two or more machine learning algorithms using a pair of X and y sets (X_1, y_1). Then we make predictions on a second X set (X_stack), y_pred_1, y_pred_2, and so on.

These predictions, y_pred_1 and y_pred_2, become inputs to a machine learning algorithm with the training output y_stack. Finally, the error can be measured on a third input set, X_3, and a ground truth set, y_3.

It will be...