Book Image

Machine Learning Algorithms

Book Image

Machine Learning Algorithms

Overview of this book

In this book, you will learn all the important machine learning algorithms that are commonly used in the field of data science. These algorithms can be used for supervised as well as unsupervised learning, reinforcement learning, and semi-supervised learning. The algorithms that are covered in this book are linear regression, logistic regression, SVM, naïve Bayes, k-means, random forest, TensorFlow and feature engineering. In this book, you will how to use these algorithms to resolve your problems, and how they work. This book will also introduce you to natural language processing and recommendation systems, which help you to run multiple algorithms simultaneously. On completion of the book, you will know how to pick the right machine learning algorithm for clustering, classification, or regression for your problem
Table of Contents (22 chapters)
Title Page
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface

Ridge, Lasso, and ElasticNet


Ridge regression imposes an additional shrinkage penalty to the ordinary least squares loss function to limit its squared L2 norm:

In this case, X is a matrix containing all samples as columns and the term w represents the weight vector. The additional term (through the coefficient alpha—if large it implies a stronger regularization and smaller values) forces the loss function to disallow an infinite growth of w, which can be caused by multicollinearity or ill-conditioning. In the following figure, there's a representation of what happens when a Ridge penalty is applied:

The gray surface represents the loss function (here, for simplicity, we're working with only two weights), while the circle center O is the boundary imposed by the Ridge condition. The minimum will have smaller w values and potential explosions are avoided.

In the following snippet, we're going to compare LinearRegression and Ridge with a cross-validation:

from sklearn.datasets import load_diabetes...