Book Image

Hands-On Deep Learning Algorithms with Python

By : Sudharsan Ravichandiran
Book Image

Hands-On Deep Learning Algorithms with Python

By: Sudharsan Ravichandiran

Overview of this book

Deep learning is one of the most popular domains in the AI space that allows you to develop multi-layered models of varying complexities. This book introduces you to popular deep learning algorithms—from basic to advanced—and shows you how to implement them from scratch using TensorFlow. Throughout the book, you will gain insights into each algorithm, the mathematical principles involved, and how to implement it in the best possible manner. The book starts by explaining how you can build your own neural networks, followed by introducing you to TensorFlow, the powerful Python-based library for machine learning and deep learning. Moving on, you will get up to speed with gradient descent variants, such as NAG, AMSGrad, AdaDelta, Adam, and Nadam. The book will then provide you with insights into recurrent neural networks (RNNs) and LSTM and how to generate song lyrics with RNN. Next, you will master the math necessary to work with convolutional and capsule networks, widely used for image recognition tasks. You will also learn how machines understand the semantics of words and documents using CBOW, skip-gram, and PV-DM. Finally, you will explore GANs, including InfoGAN and LSGAN, and autoencoders, such as contractive autoencoders and VAE. By the end of this book, you will be equipped with all the skills you need to implement deep learning in your own projects.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Deep Learning Algorithms with Python
Sudharsan Ravichandiran
Jul, 2019 | 512 pages
Small book image
Deep Reinforcement Learning with Python
Sudharsan Ravichandiran
Sep, 2020 | 760 pages
Small book image
Hands-On Meta Learning with Python
Sudharsan Ravichandiran
Dec, 2018 | 226 pages
Small book image
Hands-On Reinforcement Learning with Python
Sudharsan Ravichandiran
Jun, 2018 | 318 pages
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Getting Started with Deep Learning
Section 1: Getting Started with Deep Learning
2
Introduction to Deep Learning
Introduction to Deep Learning
What is deep learning?
Biological and artificial neurons
ANN and its layers
Exploring activation functions
Forward propagation in ANN
How does ANN learn?
Debugging gradient descent with gradient checking
Putting it all together
Summary
Questions
Further reading
3
Getting to Know TensorFlow
Getting to Know TensorFlow
What is TensorFlow?
Understanding computational graphs and sessions
Variables, constants, and placeholders
Introducing TensorBoard
Handwritten digit classification using TensorFlow
Introducing eager execution
Math operations in TensorFlow
TensorFlow 2.0 and Keras
Should we use Keras or TensorFlow?
Summary
Questions
Further reading
4
Section 2: Fundamental Deep Learning Algorithms
Section 2: Fundamental Deep Learning Algorithms
5
Gradient Descent and Its Variants
Gradient Descent and Its Variants
Demystifying gradient descent
Gradient descent versus stochastic gradient descent
Momentum-based gradient descent
Adaptive methods of gradient descent
Summary
Questions
Further reading
6
Generating Song Lyrics Using RNN
Generating Song Lyrics Using RNN
Introducing RNNs
Generating song lyrics using RNNs
Different types of RNN architectures
Summary
Questions
Further reading
7
Improvements to the RNN
Improvements to the RNN
LSTM to the rescue
Gated recurrent units
Bidirectional RNN
Going deep with deep RNN
Language translation using the seq2seq model
Summary
Questions
Further reading
8
Demystifying Convolutional Networks
Demystifying Convolutional Networks
What are CNNs?
The architecture of CNNs
The math behind CNNs
Implementing a CNN in TensorFlow
CNN architectures
Capsule networks
Building Capsule networks in TensorFlow
Summary
Questions
Further reading
9
Learning Text Representations
Learning Text Representations
Understanding the word2vec model
Building the word2vec model using gensim
Visualizing word embeddings in TensorBoard
Doc2vec
Understanding skip-thoughts algorithm
Quick-thoughts for sentence embeddings
Summary
Questions
Further reading
10
Section 3: Advanced Deep Learning Algorithms
Section 3: Advanced Deep Learning Algorithms
11
Generating Images Using GANs
Generating Images Using GANs
Differences between discriminative and generative models
Say hello to GANs!
DCGAN – Adding convolution to a GAN
Least squares GAN
GANs with Wasserstein distance
Summary
Questions
Further reading
12
Learning More about GANs
Learning More about GANs
Conditional GANs
Understanding InfoGAN
Translating images using a CycleGAN
StackGAN
Summary
Questions
Further reading
13
Reconstructing Inputs Using Autoencoders
Reconstructing Inputs Using Autoencoders
What is an autoencoder?
Autoencoders with convolutions
Exploring denoising autoencoders
Understanding sparse autoencoders
Learning to use contractive autoencoders
Dissecting variational autoencoders
Summary
Questions
Further reading
14
Exploring Few-Shot Learning Algorithms
Exploring Few-Shot Learning Algorithms
What is few-shot learning?
Siamese networks
Architecture of siamese networks
Prototypical networks
Relation networks
Matching networks
Summary
Questions
Further reading
15
Assessments
Assessments
Chapter 1 - Introduction to Deep Learning
Chapter 2 - Getting to Know TensorFlow
Chapter 3 - Gradient Descent and Its Variants
Chapter 4 - Generating Song Lyrics Using an RNN
Chapter 5 - Improvements to the RNN
Chapter 6 - Demystifying Convolutional Networks
Chapter 7 - Learning Text Representations
Chapter 8 - Generating Images Using GANs
Chapter 9 - Learning More about GANs
Chapter 10 - Reconstructing Inputs Using Autoencoders
Chapter 11 - Exploring Few-Shot Learning Algorithms
16
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Adaptive methods of gradient descent

In this section, we will learn about several adaptive versions of gradient descent.

Setting a learning rate adaptively using Adagrad

When we build a deep neural network, we have many parameters. Parameters are basically the weights of the network, so when we build a network with many layers, we will have many weights, say, . Our goal is to find the optimal values for all these weights. In all of the previous methods we learned about, the learning rate was a common value for all the parameters of the network. However Adagrad (short for adaptive gradient) adaptively sets the learning rate according to a parameter.

Parameters that have frequent updates or high gradients will have a slower...

Previous Section
End of Section 5
Next Section