Book Image

Advanced Deep Learning with Python

By : Ivan Vasilev
Book Image

Advanced Deep Learning with Python

By: Ivan Vasilev

Overview of this book

In order to build robust deep learning systems, you’ll need to understand everything from how neural networks work to training CNN models. In this book, you’ll discover newly developed deep learning models, methodologies used in the domain, and their implementation based on areas of application. You’ll start by understanding the building blocks and the math behind neural networks, and then move on to CNNs and their advanced applications in computer vision. You'll also learn to apply the most popular CNN architectures in object detection and image segmentation. Further on, you’ll focus on variational autoencoders and GANs. You’ll then use neural networks to extract sophisticated vector representations of words, before going on to cover various types of recurrent networks, such as LSTM and GRU. You’ll even explore the attention mechanism to process sequential data without the help of recurrent neural networks (RNNs). Later, you’ll use graph neural networks for processing structured data, along with covering meta-learning, which allows you to train neural networks with fewer training samples. Finally, you’ll understand how to apply deep learning to autonomous vehicles. By the end of this book, you’ll have mastered key deep learning concepts and the different applications of deep learning models in the real world.

Related Content you might be interested in

Current Title:

Small book image
Advanced Deep Learning with Python
Ivan Vasilev
Dec, 2019 | 468 pages
Small book image
Hands-On One-shot Learning with Python
Shruti Jadon | Ankush Garg
Apr, 2020 | 156 pages
Small book image
Hands-On Mathematics for Deep Learning
Jay Dawani
Jun, 2020 | 364 pages
Small book image
Deep Learning with TensorFlow and Keras
Sujit Pal | Amita Kapoor | Antonio Gulli
Oct, 2022 | 698 pages
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Core Concepts
Section 1: Core Concepts
2
The Nuts and Bolts of Neural Networks
The Nuts and Bolts of Neural Networks
The mathematical apparatus of NNs
A short introduction to NNs
Training NNs
Summary
3
Section 2: Computer Vision
Section 2: Computer Vision
4
Understanding Convolutional Networks
Understanding Convolutional Networks
Understanding CNNs
Introducing transfer learning
Summary
5
Advanced Convolutional Networks
Advanced Convolutional Networks
Introducing AlexNet
An introduction to Visual Geometry Group
Understanding residual networks
Understanding Inception networks
Introducing Xception
Introducing MobileNet
An introduction to DenseNets
The workings of neural architecture search
Introducing capsule networks
Summary
6
Object Detection and Image Segmentation
Object Detection and Image Segmentation
Introduction to object detection
Introducing image segmentation
Summary
7
Generative Models
Generative Models
Intuition and justification of generative models
Introduction to VAEs
Introduction to GANs
Types of GAN
Introducing artistic style transfer
Summary
8
Section 3: Natural Language and Sequence Processing
Section 3: Natural Language and Sequence Processing
9
Language Modeling
Language Modeling
Understanding n-grams
Introducing neural language models
Implementing language models
Summary
10
Understanding Recurrent Networks
Understanding Recurrent Networks
Introduction to RNNs
Introducing long short-term memory
Introducing gated recurrent units
Implementing text classification
Summary
11
Sequence-to-Sequence Models and Attention
Sequence-to-Sequence Models and Attention
Introducing seq2seq models
Seq2seq with attention
Understanding transformers
Transformer language models
Summary
12
Section 4: A Look to the Future
Section 4: A Look to the Future
13
Emerging Neural Network Designs
Emerging Neural Network Designs
Introducing Graph NNs
Introducing memory-augmented NNs
Summary
14
Meta Learning
Meta Learning
Introduction to meta learning
Metric-based meta learning
Optimization-based learning
Summary
15
Deep Learning for Autonomous Vehicles
Deep Learning for Autonomous Vehicles
Introduction to AVs
Components of an AV system
Introduction to 3D data processing
Imitation driving policy
Driving policy with ChauffeurNet
Summary
16
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Introducing long short-term memory

Hochreiter and Schmidhuber studied the problems of vanishing and exploding gradients extensively and came up with a solution called Long Short-Term Memory (LSTM, https://www.bioinf.jku.at/publications/older/2604.pdf). LSTMs can handle long-term dependencies due to a specially crafted memory cell. In fact, they work so well that most of the current accomplishments in training RNNs on a variety of problems are due to the use of LSTMs. In this section, we'll explore how this memory cell works and how it solves the vanishing gradients issue.

The key idea of LSTM is the cell state, ct (in addition to the hidden RNN state, ht), where the information can only be explicitly written in or removed so that the state stays constant if there is no outside interference. The cell state can only be modified by specific gates, which are a way to let information...

Previous Section
End of Section 3
Next Section