Book Image

Regression Analysis with Python

By : Luca Massaron, Alberto Boschetti
4 (1)
Book Image

Regression Analysis with Python

4 (1)
By: Luca Massaron, Alberto Boschetti

Overview of this book

Regression is the process of learning relationships between inputs and continuous outputs from example data, which enables predictions for novel inputs. There are many kinds of regression algorithms, and the aim of this book is to explain which is the right one to use for each set of problems and how to prepare real-world data for it. With this book you will learn to define a simple regression problem and evaluate its performance. The book will help you understand how to properly parse a dataset, clean it, and create an output matrix optimally built for regression. You will begin with a simple regression algorithm to solve some data science problems and then progress to more complex algorithms. The book will enable you to use regression models to predict outcomes and take critical business decisions. Through the book, you will gain knowledge to use Python for building fast better linear models and to apply the results in Python or in any computer language you prefer.

Related Content you might be interested in

Current Title:

Small book image
Regression Analysis with Python
Luca Massaron | Alberto Boschetti
Feb, 2016 | 312 pages
No titles found
Table of Contents (16 chapters)
Regression Analysis with Python
Regression Analysis with Python
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Regression – The Workhorse of Data Science
Regression – The Workhorse of Data Science
Regression analysis and data science
Python for data science
Python packages and functions for linear models
Summary
2
Approaching Simple Linear Regression
Approaching Simple Linear Regression
Defining a regression problem
Starting from the basics
Extending to linear regression
Minimizing the cost function
Summary
3
Multiple Regression in Action
Multiple Regression in Action
Using multiple features
Revisiting gradient descent
Estimating feature importance
Interaction models
Polynomial regression
Summary
4
Logistic Regression
Logistic Regression
Defining a classification problem
Defining a probability-based approach
Revisiting gradient descent
Multiclass Logistic Regression
An example
Summary
5
Data Preparation
Data Preparation
Numeric feature scaling
Qualitative feature encoding
Numeric feature transformation
Missing data
Outliers
Summary
6
Achieving Generalization
Achieving Generalization
Checking on out-of-sample data
Greedy selection of features
Regularization optimized by grid-search
Stability selection
Summary
7
Online and Batch Learning
Online and Batch Learning
Batch learning
Online mini-batch learning
Summary
8
Advanced Regression Methods
Advanced Regression Methods
Least Angle Regression
Bayesian regression
SGD classification with hinge loss
Regression trees (CART)
Bagging and boosting
Gradient Boosting Regressor with LAD
Summary
9
Real-world Applications for Regression Models
Real-world Applications for Regression Models
Downloading the datasets
A regression problem
An imbalanced and multiclass classification problem
A ranking problem
A time series problem
Summary
Index
Index

Least Angle Regression

Although very similar to Lasso (seen in Chapter 6, Achieving Generalization), Least Angle Regression, or simply LARS, is a regression algorithm that, in a fast and smart way, selects the best features to use in the model, even though they're very closely correlated to each other. LARS is an evolution of the Forward Selection (also called Forward Stepwise Regression) algorithm and of the Forward Stagewise Regression algorithm.

Here is how the Forward Selection algorithm works, based on the hypothesis that all the variables, including the target one, have been previously normalized:

  1. Of all the possible predictors for a problem, the one with the largest absolute correlation with the target variable y is selected (that is, the one with the most explanatory capability). Let's call it p1.

  2. All the other predictors are now projected onto p1 Least Angle Regression, and the projection is removed, creating a vector of residuals orthogonal to p1.

  3. Step 1 is repeated on the residual...

Previous Section
End of Section 2
Next Section