Book Image

Python Feature Engineering Cookbook

By : Soledad Galli

Book Image

Python Feature Engineering Cookbook

By: Soledad Galli

Overview of this book

Feature engineering is invaluable for developing and enriching your machine learning models. In this cookbook, you will work with the best tools to streamline your feature engineering pipelines and techniques and simplify and improve the quality of your code. Using Python libraries such as pandas, scikit-learn, Featuretools, and Feature-engine, you’ll learn how to work with both continuous and discrete datasets and be able to transform features from unstructured datasets. You will develop the skills necessary to select the best features as well as the most suitable extraction techniques. This book will cover Python recipes that will help you automate feature engineering to simplify complex processes. You’ll also get to grips with different feature engineering strategies, such as the box-cox transform, power transform, and log transform across machine learning, reinforcement learning, and natural language processing (NLP) domains. By the end of this book, you’ll have discovered tips and practical solutions to all of your feature engineering problems.

Preface

Who this book is for

What this book covers

To get the most out of this book

Foreseeing Variable Problems When Building ML Models

Foreseeing Variable Problems When Building ML Models

Technical requirements

Identifying numerical and categorical variables

Quantifying missing data

Determining cardinality in categorical variables

Pinpointing rare categories in categorical variables

Identifying a linear relationship

Identifying a normal distribution

Distinguishing variable distribution

Highlighting outliers

Comparing feature magnitude

Free Chapter

Imputing Missing Data

Imputing Missing Data

Technical requirements

Removing observations with missing data

Performing mean or median imputation

Implementing mode or frequent category imputation

Replacing missing values with an arbitrary number

Capturing missing values in a bespoke category

Replacing missing values with a value at the end of the distribution

Implementing random sample imputation

Adding a missing value indicator variable

Performing multivariate imputation by chained equations

Assembling an imputation pipeline with scikit-learn

Assembling an imputation pipeline with Feature-engine

Encoding Categorical Variables

Encoding Categorical Variables

Technical requirements

Creating binary variables through one-hot encoding

Performing one-hot encoding of frequent categories

Replacing categories with ordinal numbers

Replacing categories with counts or frequency of observations

Encoding with integers in an ordered manner

Encoding with the mean of the target

Encoding with the Weight of Evidence

Grouping rare or infrequent categories

Performing binary encoding

Performing feature hashing

Transforming Numerical Variables

Transforming Numerical Variables

Technical requirements

Transforming variables with the logarithm

Transforming variables with the reciprocal function

Using square and cube root to transform variables

Using power transformations on numerical variables

Performing Box-Cox transformation on numerical variables

Performing Yeo-Johnson transformation on numerical variables

Performing Variable Discretization

Performing Variable Discretization

Technical requirements

Dividing the variable into intervals of equal width

Sorting the variable values in intervals of equal frequency

Performing discretization followed by categorical encoding

Allocating the variable values in arbitrary intervals

Performing discretization with k-means clustering

Using decision trees for discretization

Working with Outliers

Working with Outliers

Technical requirements

Trimming outliers from the dataset

Performing winsorization

Capping the variable at arbitrary maximum and minimum values

Performing zero-coding – capping the variable at zero

Deriving Features from Dates and Time Variables

Deriving Features from Dates and Time Variables

Technical requirements

Extracting date and time parts from a datetime variable

Deriving representations of the year and month

Creating representations of day and week

Extracting time parts from a time variable

Capturing the elapsed time between datetime variables

Working with time in different time zones

Performing Feature Scaling

Performing Feature Scaling

Technical requirements

Standardizing the features

Performing mean normalization

Scaling to the maximum and minimum values

Implementing maximum absolute scaling

Scaling with the median and quantiles

Scaling to vector unit length

Applying Mathematical Computations to Features

Applying Mathematical Computations to Features

Technical requirements

Combining multiple features with statistical operations

Combining pairs of features with mathematical functions

Performing polynomial expansion

Deriving new features with decision trees

Carrying out PCA

Creating Features with Transactional and Time Series Data

Creating Features with Transactional and Time Series Data

Technical requirements

Aggregating transactions with mathematical operations

Aggregating transactions in a time window

Determining the number of local maxima and minima

Deriving time elapsed between time-stamped events

Creating features from transactions with Featuretools

Extracting Features from Text Variables

Extracting Features from Text Variables

Technical requirements

Counting characters, words, and vocabulary

Estimating text complexity by counting sentences

Creating features with bag-of-words and n-grams

Implementing term frequency-inverse document frequency

Cleaning and stemming text variables

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Capturing missing values in a bespoke category

Missing data in categorical variables can be treated as a different category, so it is common to replace missing values with the Missing string. In this recipe, we will learn how to do so using pandas, scikit-learn, and Feature-engine.

How to do it...

To proceed with the recipe, let's import the required tools and prepare the dataset:

Import pandas and the required functions and classes from scikit-learn and Feature-engine:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from feature_engine.missing_data_imputers import CategoricalVariableImputer

Let's load the dataset:

data = pd.read_csv('creditApprovalUCI...