Deep Learning Quick Reference

By : Mike Bernico

Deep Learning Quick Reference

By: Mike Bernico

Overview of this book

Deep learning has become an essential necessity to enter the world of artificial intelligence. With this book deep learning techniques will become more accessible, practical, and relevant to practicing data scientists. It moves deep learning from academia to the real world through practical examples. You will learn how Tensor Board is used to monitor the training of deep neural networks and solve binary classification problems using deep learning. Readers will then learn to optimize hyperparameters in their deep learning models. The book then takes the readers through the practical implementation of training CNN's, RNN's, and LSTM's with word embeddings and seq2seq models from scratch. Later the book explores advanced topics such as Deep Q Network to solve an autonomous agent problem and how to use two adversarial networks to generate artificial images that appear real. For implementation purposes, we look at popular Python-based deep learning frameworks such as Keras and Tensorflow, Each chapter provides best practices and safe choices to help readers make the right decision while training deep neural networks. By the end of this book, you will be able to solve real-world problems quickly with deep neural networks.

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Free Chapter

The Building Blocks of Deep Learning

The deep neural network architectures

Optimization algorithms for deep learning

Deep learning frameworks

Building datasets for deep learning

Summary

Using Deep Learning to Solve Regression Problems

Regression analysis and deep neural networks

Using deep neural networks for regression

Building an MLP in Keras

Building a deep neural network in Keras

Saving and loading a trained Keras model

Summary

Monitoring Network Training Using TensorBoard

A brief overview of TensorBoard

Setting up TensorBoard

Connecting Keras to TensorBoard

Using TensorBoard

Summary

Using Deep Learning to Solve Binary Classification Problems

Binary classification and deep neural networks

Case study – epileptic seizure recognition

Building a binary classifier in Keras

Using the checkpoint callback in Keras

Measuring ROC AUC in a custom callback

Measuring precision, recall, and f1-score

Summary

Using Keras to Solve Multiclass Classification Problems

Multiclass classification and deep neural networks

Case study - handwritten digit classification

Building a multiclass classifier in Keras

Controlling variance with dropout

Controlling variance with regularization

Summary

Hyperparameter Optimization

Should network architecture be considered a hyperparameter?

Which hyperparameters should we optimize?

Hyperparameter optimization strategies

Summary

Training a CNN from Scratch

Introducing convolutions

Training a convolutional neural network in Keras

Using data augmentation

Summary

Transfer Learning with Pretrained CNNs

Overview of transfer learning

When transfer learning should be used

The impact of source/target volume and similarity

Transfer learning in Keras

Summary

Training an RNN from scratch

Introducing recurrent neural networks

A refresher on time series problems

Using an LSTM for time series prediction

Summary

Training LSTMs with Word Embeddings from Scratch

An introduction to natural language processing

Vectorizing text

Word embedding

Keras embedding layer

1D CNNs for natural language processing

Case studies for document classifications

Summary

Training Seq2Seq Models

Sequence-to-sequence models

Machine translation

Summary

Using Deep Reinforcement Learning

Reinforcement learning overview

The Keras reinforcement learning framework

Building a reinforcement learning agent in Keras

Summary

Generative Adversarial Networks

An overview of the GAN

Deep Convolutional GAN architecture

How GANs can fail

Safe choices for GAN

Generating MNIST images using a Keras GAN

Generating CIFAR-10 images using a Keras GAN

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Optimization algorithms for deep learning

The gradient descent algorithm is not the only optimization algorithm available to optimize our network weights, however it's the basis for most other algorithms. While understanding every optimization algorithm out there is likely a PhD worth of material, we will devote a few sentences to some of the most practical.

Using momentum with gradient descent

Using gradient descent with momentum speeds up gradient descent by increasing the speed of learning in directions the gradient has been constant in direction while slowing learning in directions the gradient fluctuates in direction. It allows the velocity of gradient descent to increase.

Momentum works by introducing a velocity term, and using a weighted moving average of that term in the update rule, as follows:

Most typically is set to 0.9 in the case of momentum, and usually this is not a hyper-parameter that needs to be changed.

The RMSProp algorithm

RMSProp is another algorithm that can speed up gradient descent by speeding up learning in some directions, and dampening oscillations in other directions, across the multidimensional space that the network weights represent:

This has the effect of reducing oscillations more in directions where is large.

The Adam optimizer

Adam is one of the best performing known optimizer and it's my first choice. It works well across a wide variety of problems. It combines the best parts of both momentum and RMSProp into a single update rule: