Book Image

The Deep Learning with Keras Workshop

By : Matthew Moocarme, Mahla Abdolahnejad, Ritesh Bhagwat
1 (1)
Book Image

The Deep Learning with Keras Workshop

1 (1)
By: Matthew Moocarme, Mahla Abdolahnejad, Ritesh Bhagwat

Overview of this book

New experiences can be intimidating, but not this one! This beginner’s guide to deep learning is here to help you explore deep learning from scratch with Keras, and be on your way to training your first ever neural networks. What sets Keras apart from other deep learning frameworks is its simplicity. With over two hundred thousand users, Keras has a stronger adoption in industry and the research community than any other deep learning framework. The Deep Learning with Keras Workshop starts by introducing you to the fundamental concepts of machine learning using the scikit-learn package. After learning how to perform the linear transformations that are necessary for building neural networks, you'll build your first neural network with the Keras library. As you advance, you'll learn how to build multi-layer neural networks and recognize when your model is underfitting or overfitting to the training data. With the help of practical exercises, you’ll learn to use cross-validation techniques to evaluate your models and then choose the optimal hyperparameters to fine-tune their performance. Finally, you’ll explore recurrent neural networks and learn how to train them to predict values in sequential data. By the end of this book, you'll have developed the skills you need to confidently train your own neural network models.
Table of Contents (11 chapters)
Preface

L1 and L2 Regularization

The most common type of regularization for deep learning models is the one that keeps the weights of the network small. This type of regularization is called weight regularization and has two different variations: L2 regularization and L1 regularization. In this section, you will learn about these regularization methods in detail, along with how to implement them in Keras. Additionally, you will practice applying them to real-life problems and observe how they can improve the performance of a model.

L1 and L2 Regularization Formulation

In weight regularization, a penalizing term is added to the loss function. This term is either the L2 norm (the sum of the squared values) of the weights or the L1 norm (the sum of the absolute values) of the weights. If the L1 norm is used, then it will be called L1 regularization. If the L2 norm is used, then it will be called L2 regularization. In each case, the sum is multiplied by a hyperparameter called a regularization...