Book Image

Deep Learning Quick Reference

By : Mike Bernico
Book Image

Deep Learning Quick Reference

By: Mike Bernico

Overview of this book

Deep learning has become an essential necessity to enter the world of artificial intelligence. With this book deep learning techniques will become more accessible, practical, and relevant to practicing data scientists. It moves deep learning from academia to the real world through practical examples. You will learn how Tensor Board is used to monitor the training of deep neural networks and solve binary classification problems using deep learning. Readers will then learn to optimize hyperparameters in their deep learning models. The book then takes the readers through the practical implementation of training CNN's, RNN's, and LSTM's with word embeddings and seq2seq models from scratch. Later the book explores advanced topics such as Deep Q Network to solve an autonomous agent problem and how to use two adversarial networks to generate artificial images that appear real. For implementation purposes, we look at popular Python-based deep learning frameworks such as Keras and Tensorflow, Each chapter provides best practices and safe choices to help readers make the right decision while training deep neural networks. By the end of this book, you will be able to solve real-world problems quickly with deep neural networks.

Related Content you might be interested in

Current Title:

Small book image
Deep Learning Quick Reference
Mike Bernico
Mar, 2018 | 272 pages
Small book image
Keras Deep Learning Cookbook
Manpreet Singh Ghotra | Rajdeep Dua
Oct, 2018 | 252 pages
Small book image
Machine Learning for Finance
Jannes Klaas
May, 2019 | 456 pages
Small book image
Deep Learning with Keras
Sujit Pal | Antonio Gulli
Apr, 2017 | 318 pages
Table of Contents (15 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
The Building Blocks of Deep Learning
The Building Blocks of Deep Learning
The deep neural network architectures
Optimization algorithms for deep learning
Deep learning frameworks
Building datasets for deep learning
Summary
2
Using Deep Learning to Solve Regression Problems
Using Deep Learning to Solve Regression Problems
Regression analysis and deep neural networks
Using deep neural networks for regression
Building an MLP in Keras
Building a deep neural network in Keras
Saving and loading a trained Keras model
Summary
3
Monitoring Network Training Using TensorBoard
Monitoring Network Training Using TensorBoard
A brief overview of TensorBoard
Setting up TensorBoard
Connecting Keras to TensorBoard
Using TensorBoard
Summary
4
Using Deep Learning to Solve Binary Classification Problems
Using Deep Learning to Solve Binary Classification Problems
Binary classification and deep neural networks
Case study – epileptic seizure recognition
Building a binary classifier in Keras
Using the checkpoint callback in Keras
Measuring ROC AUC in a custom callback
Measuring precision, recall, and f1-score
Summary
5
Using Keras to Solve Multiclass Classification Problems
Using Keras to Solve Multiclass Classification Problems
Multiclass classification and deep neural networks
Case study - handwritten digit classification
Building a multiclass classifier in Keras
Controlling variance with dropout
Controlling variance with regularization
Summary
6
Hyperparameter Optimization
Hyperparameter Optimization
Should network architecture be considered a hyperparameter?
Which hyperparameters should we optimize?
Hyperparameter optimization strategies
Summary
7
Training a CNN from Scratch
Training a CNN from Scratch
Introducing convolutions
Training a convolutional neural network in Keras
Using data augmentation
Summary
8
Transfer Learning with Pretrained CNNs
Transfer Learning with Pretrained CNNs
Overview of transfer learning
When transfer learning should be used
The impact of source/target volume and similarity
Transfer learning in Keras
Summary
9
Training an RNN from scratch
Training an RNN from scratch
Introducing recurrent neural networks
A refresher on time series problems
Using an LSTM for time series prediction
Summary
10
Training LSTMs with Word Embeddings from Scratch
Training LSTMs with Word Embeddings from Scratch
An introduction to natural language processing
Vectorizing text
Word embedding
Keras embedding layer
1D CNNs for natural language processing
Case studies for document classifications
Summary
11
Training Seq2Seq Models
Training Seq2Seq Models
Sequence-to-sequence models
Machine translation
Summary
12
Using Deep Reinforcement Learning
Using Deep Reinforcement Learning
Reinforcement learning overview
The Keras reinforcement learning framework
Building a reinforcement learning agent in Keras
Summary
13
Generative Adversarial Networks
Generative Adversarial Networks
An overview of the GAN
Deep Convolutional GAN architecture
How GANs can fail
Safe choices for GAN
Generating MNIST images using a Keras GAN
Generating CIFAR-10 images using a Keras GAN
Summary
14
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Controlling variance with regularization

Regularization is another way to control overfitting, that penalizes individual weights in the model as they grow larger. If you're familiar with linear models such as linear and logistic regression, it's exactly the same technique applied at the neuron level. Two flavors of regularization, called L1 and L2, can be used to regularize neural networks. However, because it is more computationally efficient L2 regularization is almost always used in neural networks.

Quickly, we need to first regularize our cost function. If we imagine C0, categorical cross-entropy, as the original cost function, then the regularized cost function would be as follows:

Here, ; is a regularization parameter that can be increased or decreased to change the amount of regularization applied. This regularization parameter penalizes big values for weights...

Previous Section
End of Section 6
Next Section