Book Image

Python Feature Engineering Cookbook - Second Edition

By : Soledad Galli
Book Image

Python Feature Engineering Cookbook - Second Edition

By: Soledad Galli

Overview of this book

Feature engineering, the process of transforming variables and creating features, albeit time-consuming, ensures that your machine learning models perform seamlessly. This second edition of Python Feature Engineering Cookbook will take the struggle out of feature engineering by showing you how to use open source Python libraries to accelerate the process via a plethora of practical, hands-on recipes. This updated edition begins by addressing fundamental data challenges such as missing data and categorical values, before moving on to strategies for dealing with skewed distributions and outliers. The concluding chapters show you how to develop new features from various types of data, including text, time series, and relational databases. With the help of numerous open source Python libraries, you'll learn how to implement each feature engineering method in a performant, reproducible, and elegant manner. By the end of this Python book, you will have the tools and expertise needed to confidently build end-to-end and reproducible feature engineering pipelines that can be deployed into production.
Table of Contents (14 chapters)

Comparing features to reference variables

In the previous recipe, Combining features with mathematical functions, we created new features by applying mathematical or statistical functions, such as the sum or the mean, to a group of variables. Some mathematical operations, however, such as subtraction or division, make more sense when performed between two features, or when considering multiple features against one reference variable. These operations are very useful to derive ratios, such as the debt-to-income ratio:

debt-to-income ratio = total debt / total income

Alternatively, we can use them for differences, for example, to calculate disposable income:

disposable income = income - total debt

In this recipe, we will learn how to create new features via subtraction or division while utilizing pandas, and then automate the procedure for multiple variables by using Feature-engine.

How to do it…

Let’s begin by loading the necessary Python libraries and...