Book Image

Feature Engineering Made Easy

By : Sinan Ozdemir, Divya Susarla
Book Image

Feature Engineering Made Easy

By: Sinan Ozdemir, Divya Susarla

Overview of this book

Feature engineering is the most important step in creating powerful machine learning systems. This book will take you through the entire feature-engineering journey to make your machine learning much more systematic and effective. You will start with understanding your data—often the success of your ML models depends on how you leverage different feature types, such as continuous, categorical, and more, You will learn when to include a feature, when to omit it, and why, all by understanding error analysis and the acceptability of your models. You will learn to convert a problem statement into useful new features. You will learn to deliver features driven by business needs as well as mathematical insights. You'll also learn how to use machine learning on your machines, automatically learning amazing features for your data. By the end of the book, you will become proficient in Feature Selection, Feature Learning, and Feature Optimization.

Related Content you might be interested in

Current Title:

Small book image
Feature Engineering Made Easy
Sinan Ozdemir | Divya Susarla
Jan, 2018 | 316 pages
Small book image
Principles of Data Science
Sinan Ozdemir
Jan, 2024 | 326 pages
Small book image
Python Data Mining Quick Start Guide
Nathan Greeneltch
Apr, 2019 | 188 pages
Small book image
scikit-learn Cookbook
Julian Avila | Trent Hauck
Nov, 2017 | 374 pages
Table of Contents (14 chapters)
Title Page
Title Page
Copyright and Credits
Copyright and Credits
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Feature Engineering
Introduction to Feature Engineering
Motivating example – AI-powered communications
Why feature engineering matters
What is feature engineering?
Evaluation of machine learning algorithms and feature engineering procedures
Feature understanding – what’s in my dataset?
Feature improvement – cleaning datasets
Feature selection – say no to bad attributes
Feature construction – can we build it?
Feature transformation – enter math-man
Feature learning – using AI to better our AI
Summary
2
Feature Understanding – What's in My Dataset?
Feature Understanding – What's in My Dataset?
The structure, or lack thereof, of data
An example of unstructured data – server logs
Quantitative versus qualitative data
The four levels of data
Recap of the levels of data
Summary
3
Feature Improvement - Cleaning Datasets
Feature Improvement - Cleaning Datasets
Identifying missing values in data
Dealing with missing values in a dataset
Standardization and normalization
Summary
4
Feature Construction
Feature Construction
Examining our dataset
Imputing categorical features
Encoding categorical variables
Extending numerical features
Text-specific feature construction
Summary
5
Feature Selection
Feature Selection
Achieving better performance in feature engineering
Creating a baseline machine learning pipeline
The types of feature selection
Choosing the right feature selection method
Summary
6
Feature Transformations
Feature Transformations
Dimension reduction – feature transformations versus feature selection versus feature construction
Principal Component Analysis
Scikit-learn's PCA
How centering and scaling data affects PCA
A deeper look into the principal components
Linear Discriminant Analysis
LDA versus PCA – iris dataset
Summary
7
Feature Learning
Feature Learning
Parametric assumptions of data
Restricted Boltzmann Machines
The BernoulliRBM
Extracting RBM components from MNIST
Using RBMs in a machine learning pipeline
Learning text features – word vectorizations
Summary
8
Case Studies
Case Studies
Case study 1 - facial recognition
Case study 2 - predicting topics of hotel reviews data
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Using RBMs in a machine learning pipeline

Of course, we want to see how the RBM performs in our machine learning pipelines to not just visualize the workings of the model, but to see concrete results of the feature learning. To do this, we will create and run three pipelines:

  • A logistic regression model by itself running on the raw pixel strengths
  • A logistic regression running on extracted PCA components
  • A logistic regression running on extracted RBM components

Each of these pipelines will be grid-searched across a number of components (for PCA and RBM) and the C parameter for logistic regression. Let's start with our simplest pipeline. We will run the raw pixel values through a logistic regression to see if the linear model is enough to separate out the digits.

Using a linear model on raw pixel values

To begin, we will run the raw pixel values through a logistic regression model in order to obtain something of a baseline model. We want to see if utilizing PCA or RBM components will allow the...

Previous Section
End of Section 6
Next Section