Book Image

Data Cleaning and Exploration with Machine Learning

By : Michael Walker
Book Image

Data Cleaning and Exploration with Machine Learning

By: Michael Walker

Overview of this book

Many individuals who know how to run machine learning algorithms do not have a good sense of the statistical assumptions they make and how to match the properties of the data to the algorithm for the best results. As you start with this book, models are carefully chosen to help you grasp the underlying data, including in-feature importance and correlation, and the distribution of features and targets. The first two parts of the book introduce you to techniques for preparing data for ML algorithms, without being bashful about using some ML techniques for data cleaning, including anomaly detection and feature selection. The book then helps you apply that knowledge to a wide variety of ML tasks. You’ll gain an understanding of popular supervised and unsupervised algorithms, how to prepare data for them, and how to evaluate them. Next, you’ll build models and understand the relationships in your data, as well as perform cleaning and exploration tasks with that data. You’ll make quick progress in studying the distribution of variables, identifying anomalies, and examining bivariate relationships, as you focus more on the accuracy of predictions in this book. By the end of this book, you’ll be able to deal with complex data problems using unsupervised ML algorithms like principal component analysis and k-means clustering.

Related Content you might be interested in

Current Title:

Small book image
Data Cleaning and Exploration with Machine Learning
Michael Walker
Aug, 2022 | 542 pages
Small book image
Python Data Cleaning Cookbook
Michael Walker
Dec, 2020 | 436 pages
Small book image
scikit-learn Cookbook
Julian Avila | Trent Hauck
Nov, 2017 | 374 pages
Small book image
Data Science Projects with Python
Stephen Klosterman
Apr, 2019 | 374 pages
Table of Contents (23 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Section 1 – Data Cleaning and Machine Learning Algorithms
Section 1 – Data Cleaning and Machine Learning Algorithms
Free Chapter
2
Chapter 1: Examining the Distribution of Features and Targets
Chapter 1: Examining the Distribution of Features and Targets
Technical requirements
Subsetting data
Generating frequencies for categorical features
Generating summary statistics for continuous and discrete features
Identifying extreme values and outliers in univariate analysis
Using histograms, boxplots, and violin plots to examine the distribution of features
Summary
3
Chapter 2: Examining Bivariate and Multivariate Relationships between Features and Targets
Chapter 2: Examining Bivariate and Multivariate Relationships between Features and Targets
Technical requirements
Identifying outliers and extreme values in bivariate relationships
Using scatter plots to view bivariate relationships between continuous features
Using grouped boxplots to view bivariate relationships between continuous and categorical features
Using linear regression to identify data points with significant influence
Using K-nearest neighbors to find outliers
Using Isolation Forest to find outliers
Summary
4
Chapter 3: Identifying and Fixing Missing Values
Chapter 3: Identifying and Fixing Missing Values
Technical requirements
Identifying missing values
Cleaning missing values
Imputing values with regression
Using KNN imputation
Using random forest for imputation
Summary
5
Section 2 – Preprocessing, Feature Selection, and Sampling
Section 2 – Preprocessing, Feature Selection, and Sampling
6
Chapter 4: Encoding, Transforming, and Scaling Features
Chapter 4: Encoding, Transforming, and Scaling Features
Technical requirements
Creating training datasets and avoiding data leakage
Removing redundant or unhelpful features
Encoding categorical features
Encoding categorical features with medium or high cardinality
Using mathematical transformations
Feature binning
Feature scaling
Summary
7
Chapter 5: Feature Selection
Chapter 5: Feature Selection
Technical requirements
Selecting features for classification models
Selecting features for regression models
Using forward and backward feature selection
Using exhaustive feature selection
Eliminating features recursively in a regression model
Eliminating features recursively in a classification model
Using Boruta for feature selection
Using regularization and other embedded methods
Using principal component analysis
Summary
8
Chapter 6: Preparing for Model Evaluation
Chapter 6: Preparing for Model Evaluation
Technical requirements
Measuring accuracy, sensitivity, specificity, and precision for binary classification
Examining CAP, ROC, and precision-sensitivity curves for binary classification
Evaluating multiclass models
Evaluating regression models
Using K-fold cross-validation
Preprocessing data with pipelines
Summary
9
Section 3 – Modeling Continuous Targets with Supervised Learning
Section 3 – Modeling Continuous Targets with Supervised Learning
10
Chapter 7: Linear Regression Models
Chapter 7: Linear Regression Models
Technical requirements
Key concepts
Linear regression and gradient descent
Using classical linear regression
Using lasso regression
Using non-linear regression
Regression with gradient descent
Summary
11
Chapter 8: Support Vector Regression
Chapter 8: Support Vector Regression
Technical requirements
Key concepts of SVR
SVR with a linear model
Using kernels for nonlinear SVR
Summary
12
Chapter 9: K-Nearest Neighbors, Decision Tree, Random Forest, and Gradient Boosted Regression
Chapter 9: K-Nearest Neighbors, Decision Tree, Random Forest, and Gradient Boosted Regression
Technical requirements
Key concepts for K-nearest neighbors regression
K-nearest neighbors regression
Key concepts for decision tree and random forest regression
Decision tree and random forest regression
Using gradient boosted regression
Summary
13
Section 4 – Modeling Dichotomous and Multiclass Targets with Supervised Learning
Section 4 – Modeling Dichotomous and Multiclass Targets with Supervised Learning
14
Chapter 10: Logistic Regression
Chapter 10: Logistic Regression
Technical requirements
Key concepts of logistic regression
Binary classification with logistic regression
Regularization with logistic regression
Multinomial logistic regression
Summary
15
Chapter 11: Decision Trees and Random Forest Classification
Chapter 11: Decision Trees and Random Forest Classification
Technical requirements
Key concepts
Decision tree models
Implementing random forest
Implementing gradient boosting
Summary
16
Chapter 12: K-Nearest Neighbors for Classification
Chapter 12: K-Nearest Neighbors for Classification
Technical requirements
Key concepts of KNN
KNN for binary classification
KNN for multiclass classification
Summary
17
Chapter 13: Support Vector Machine Classification
Chapter 13: Support Vector Machine Classification
Technical requirements
Key concepts for SVC
Linear SVC models
Nonlinear SVM classification models
SVMs for multiclass classification
Summary
18
Chapter 14: Naïve Bayes Classification
Chapter 14: Naïve Bayes Classification
Technical requirements
Key concepts
Naïve Bayes classification models
Naïve Bayes for text classification
Summary
19
Section 5 – Clustering and Dimensionality Reduction with Unsupervised Learning
Section 5 – Clustering and Dimensionality Reduction with Unsupervised Learning
20
Chapter 15: Principal Component Analysis
Chapter 15: Principal Component Analysis
Technical requirements
Key concepts of PCA
Feature extraction with PCA
Using kernels with PCA
Summary
21
Chapter 16: K-Means and DBSCAN Clustering
Chapter 16: K-Means and DBSCAN Clustering
Technical requirements
The key concepts of k-means and DBSCAN clustering
Implementing k-means clustering
Implementing DBSCAN clustering
Summary
Why subscribe?
22
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Using Boruta for feature selection

The Boruta package takes a unique approach to feature selection, though it has some similarities with wrapper methods. For each feature, Boruta creates a shadow feature, one with the same range of values as the original feature but with shuffled values. It then evaluates whether the original feature offers more information than the shadow feature, gradually removing features providing the least information. Boruta outputs confirmed, tentative, and rejected features with each iteration.

Let's use the Boruta package to select features for a classification model of bachelor's degree completion (you can install the Boruta package with pip if you have not yet installed it):

  1. We start by loading the necessary libraries:
    import pandas as pd
from feature_engine.encoding import OneHotEncoder
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier...
Previous Section
End of Section 9
Next Section