Book Image

Hands-On Deep Learning Architectures with Python

By : Yuxi (Hayden) Liu, Saransh Mehta
Book Image

Hands-On Deep Learning Architectures with Python

By: Yuxi (Hayden) Liu, Saransh Mehta

Overview of this book

Deep learning architectures are composed of multilevel nonlinear operations that represent high-level abstractions; this allows you to learn useful feature representations from the data. This book will help you learn and implement deep learning architectures to resolve various deep learning research problems. Hands-On Deep Learning Architectures with Python explains the essential learning algorithms used for deep and shallow architectures. Packed with practical implementations and ideas to help you build efficient artificial intelligence systems (AI), this book will help you learn how neural networks play a major role in building deep architectures. You will understand various deep learning architectures (such as AlexNet, VGG Net, GoogleNet) with easy-to-follow code and diagrams. In addition to this, the book will also guide you in building and training various deep architectures such as the Boltzmann mechanism, autoencoders, convolutional neural networks (CNNs), recurrent neural networks (RNNs), natural language processing (NLP), GAN, and more—all with practical implementations. By the end of this book, you will be able to construct deep models using popular frameworks and datasets with the required design patterns for each architecture. You will be ready to explore the potential of deep architectures in today's world.
Table of Contents (15 chapters)
Free Chapter
1
Section 1: The Elements of Deep Learning
5
Section 2: Convolutional Neural Networks
8
Section 3: Sequence Modeling
10
Section 4: Generative Adversarial Networks (GANs)
12
Section 5: The Future of Deep Learning and Advanced Artificial Intelligence

The evolution path of RNNs

RNNs actually have a long history and were first developed in the 1980s. The Hopfield network, as the first neural network with recurrent links, was invented by John Hopfield in Neurons with graded response have collective computational properties like those of two-state neurons (PNAS. 1984 May; 81(10): 3088-3092).

Inspired by the Hopfield network, the fully connected neural network—the Elman network—was introduced in Finding structure in time (Cognitive Science, 1990 March; 14(2): 179-211). The Elman network has one hidden layer and a set of context units connected to the hidden layer. At each time step, the context units keep track of the previous values of the hidden units.

In 1992, Schmidhuber discovered the vanishing gradient problem due to memorizing long-term dependencies. Five years later, the long short-term memory (LSTM...