Book Image

Feature Engineering Made Easy

By : Sinan Ozdemir, Divya Susarla
Book Image

Feature Engineering Made Easy

By: Sinan Ozdemir, Divya Susarla

Overview of this book

Feature engineering is the most important step in creating powerful machine learning systems. This book will take you through the entire feature-engineering journey to make your machine learning much more systematic and effective. You will start with understanding your data—often the success of your ML models depends on how you leverage different feature types, such as continuous, categorical, and more, You will learn when to include a feature, when to omit it, and why, all by understanding error analysis and the acceptability of your models. You will learn to convert a problem statement into useful new features. You will learn to deliver features driven by business needs as well as mathematical insights. You'll also learn how to use machine learning on your machines, automatically learning amazing features for your data. By the end of the book, you will become proficient in Feature Selection, Feature Learning, and Feature Optimization.

Related Content you might be interested in

Current Title:

Small book image
Feature Engineering Made Easy
Sinan Ozdemir | Divya Susarla
Jan, 2018 | 316 pages
Small book image
Principles of Data Science
Mr. Sinan Ozdemir
Jan, 2024 | 326 pages
Small book image
Python Data Mining Quick Start Guide
Nathan Greeneltch
Apr, 2019 | 188 pages
Small book image
scikit-learn Cookbook , Second Edition
Trent Hauck | Mr. Julian Avila
Nov, 2017 | 374 pages
Table of Contents (14 chapters)
Title Page
Title Page
Copyright and Credits
Copyright and Credits
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Feature Engineering
Introduction to Feature Engineering
Motivating example – AI-powered communications
Why feature engineering matters
What is feature engineering?
Evaluation of machine learning algorithms and feature engineering procedures
Feature understanding – what’s in my dataset?
Feature improvement – cleaning datasets
Feature selection – say no to bad attributes
Feature construction – can we build it?
Feature transformation – enter math-man
Feature learning – using AI to better our AI
Summary
2
Feature Understanding – What's in My Dataset?
Feature Understanding – What's in My Dataset?
The structure, or lack thereof, of data
An example of unstructured data – server logs
Quantitative versus qualitative data
The four levels of data
Recap of the levels of data
Summary
3
Feature Improvement - Cleaning Datasets
Feature Improvement - Cleaning Datasets
Identifying missing values in data
Dealing with missing values in a dataset
Standardization and normalization
Summary
4
Feature Construction
Feature Construction
Examining our dataset
Imputing categorical features
Encoding categorical variables
Extending numerical features
Text-specific feature construction
Summary
5
Feature Selection
Feature Selection
Achieving better performance in feature engineering
Creating a baseline machine learning pipeline
The types of feature selection
Choosing the right feature selection method
Summary
6
Feature Transformations
Feature Transformations
Dimension reduction – feature transformations versus feature selection versus feature construction
Principal Component Analysis
Scikit-learn's PCA
How centering and scaling data affects PCA
A deeper look into the principal components
Linear Discriminant Analysis
LDA versus PCA – iris dataset
Summary
7
Feature Learning
Feature Learning
Parametric assumptions of data
Restricted Boltzmann Machines
The BernoulliRBM
Extracting RBM components from MNIST
Using RBMs in a machine learning pipeline
Learning text features – word vectorizations
Summary
8
Case Studies
Case Studies
Case study 1 - facial recognition
Case study 2 - predicting topics of hotel reviews data
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Imputing categorical features

Now that we have an understanding of the data we are working with, let's take a look at our missing values:

  • To do this, we can use the isnull method available to us in pandas for DataFrames. This method returns a boolean same-sized object indicating if the values are null.
  • We will then sum these to see which columns have missing data:
X.isnull().sum()
>>>>
boolean                1
city                   1
ordinal_column         0
quantitative_column    1
dtype: int64

Here, we can see that three of our columns are missing values. Our course of action will be to impute these missing values.

If you recall, we implemented scikit-learn's Imputer class in a previous chapter to fill in numerical data. Imputer does have a categorical option, most_frequent, however it only works on categorical data that has been encoded as integers.

We may not always want to transform our categorical data this way, as it can change how we interpret the categorical information,...

Previous Section
End of Section 3
Next Section