Book Image

Feature Engineering Made Easy

By : Sinan Ozdemir, Divya Susarla
Book Image

Feature Engineering Made Easy

By: Sinan Ozdemir, Divya Susarla

Overview of this book

Feature engineering is the most important step in creating powerful machine learning systems. This book will take you through the entire feature-engineering journey to make your machine learning much more systematic and effective. You will start with understanding your data—often the success of your ML models depends on how you leverage different feature types, such as continuous, categorical, and more, You will learn when to include a feature, when to omit it, and why, all by understanding error analysis and the acceptability of your models. You will learn to convert a problem statement into useful new features. You will learn to deliver features driven by business needs as well as mathematical insights. You'll also learn how to use machine learning on your machines, automatically learning amazing features for your data. By the end of the book, you will become proficient in Feature Selection, Feature Learning, and Feature Optimization.

Related Content you might be interested in

Current Title:

Small book image
Feature Engineering Made Easy
Sinan Ozdemir | Divya Susarla
Jan, 2018 | 316 pages
Small book image
Principles of Data Science
Sinan Ozdemir
Jan, 2024 | 326 pages
Small book image
Python Data Mining Quick Start Guide
Nathan Greeneltch
Apr, 2019 | 188 pages
Small book image
scikit-learn Cookbook
Julian Avila | Trent Hauck
Nov, 2017 | 374 pages
Table of Contents (14 chapters)
Title Page
Title Page
Copyright and Credits
Copyright and Credits
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Feature Engineering
Introduction to Feature Engineering
Motivating example – AI-powered communications
Why feature engineering matters
What is feature engineering?
Evaluation of machine learning algorithms and feature engineering procedures
Feature understanding – what’s in my dataset?
Feature improvement – cleaning datasets
Feature selection – say no to bad attributes
Feature construction – can we build it?
Feature transformation – enter math-man
Feature learning – using AI to better our AI
Summary
2
Feature Understanding – What's in My Dataset?
Feature Understanding – What's in My Dataset?
The structure, or lack thereof, of data
An example of unstructured data – server logs
Quantitative versus qualitative data
The four levels of data
Recap of the levels of data
Summary
3
Feature Improvement - Cleaning Datasets
Feature Improvement - Cleaning Datasets
Identifying missing values in data
Dealing with missing values in a dataset
Standardization and normalization
Summary
4
Feature Construction
Feature Construction
Examining our dataset
Imputing categorical features
Encoding categorical variables
Extending numerical features
Text-specific feature construction
Summary
5
Feature Selection
Feature Selection
Achieving better performance in feature engineering
Creating a baseline machine learning pipeline
The types of feature selection
Choosing the right feature selection method
Summary
6
Feature Transformations
Feature Transformations
Dimension reduction – feature transformations versus feature selection versus feature construction
Principal Component Analysis
Scikit-learn's PCA
How centering and scaling data affects PCA
A deeper look into the principal components
Linear Discriminant Analysis
LDA versus PCA – iris dataset
Summary
7
Feature Learning
Feature Learning
Parametric assumptions of data
Restricted Boltzmann Machines
The BernoulliRBM
Extracting RBM components from MNIST
Using RBMs in a machine learning pipeline
Learning text features – word vectorizations
Summary
8
Case Studies
Case Studies
Case study 1 - facial recognition
Case study 2 - predicting topics of hotel reviews data
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Summary

In this chapter, we learned a great deal about methodologies for selecting subsets of features in order to increase the performance of our machine learning pipelines in both a predictive capacity as well in-time-complexity.

The dataset that we chose had a relatively low number of features. If selecting, however, from a very large set of features (over a hundred), then the methods in this chapter will likely start to become entirely too cumbersome. We saw that in this chapter, when attempting to optimize a CountVectorizer pipeline, the time it would take to run a univariate test on every feature is not only astronomical; we would run a greater risk of experiencing multicollinearity in our features by sheer coincidence. 

In the next chapter, we will introduce purely mathematical transformations that we may apply to our data matrices in order to alleviate the trouble of working with vast quantities of features, or even a few highly uninterpretable features. We will begin to work with...

Previous Section
End of Chapter 5
Next Chapter