Book Image

Neural Networks with Keras Cookbook

By : V Kishore Ayyadevara
Book Image

Neural Networks with Keras Cookbook

By: V Kishore Ayyadevara

Overview of this book

This book will take you from the basics of neural networks to advanced implementations of architectures using a recipe-based approach. We will learn about how neural networks work and the impact of various hyper parameters on a network's accuracy along with leveraging neural networks for structured and unstructured data. Later, we will learn how to classify and detect objects in images. We will also learn to use transfer learning for multiple applications, including a self-driving car using Convolutional Neural Networks. We will generate images while leveraging GANs and also by performing image encoding. Additionally, we will perform text analysis using word vector based techniques. Later, we will use Recurrent Neural Networks and LSTM to implement chatbot and Machine Translation systems. Finally, you will learn about transcribing images, audio, and generating captions and also use Deep Q-learning to build an agent that plays Space Invaders game. By the end of this book, you will have developed the skills to choose and customize multiple neural network architectures for various deep learning problems you might encounter.

Related Content you might be interested in

Current Title:

Small book image
Neural Networks with Keras Cookbook
V Kishore Ayyadevara
Feb, 2019 | 568 pages
Small book image
Modern Computer Vision with PyTorch
Yeshwanth Reddy | V Kishore Ayyadevara
Nov, 2020 | 824 pages
Small book image
Deep Learning with Keras
Sujit Pal | Antonio Gulli
Apr, 2017 | 318 pages
Small book image
Python Deep Learning Cookbook
Indra den Bakker
Oct, 2017 | 330 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Sections
Get in touch
Free Chapter
1
Building a Feedforward Neural Network
Building a Feedforward Neural Network
Introduction
Architecture of a simple neural network
Applications of a neural network
Feed-forward propagation from scratch in Python
Building back-propagation from scratch in Python
Building a neural network in Keras
2
Building a Deep Feedforward Neural Network
Building a Deep Feedforward Neural Network
Training a vanilla neural network
Scaling the input dataset
Impact on training when the majority of inputs are greater than zero
Impact of batch size on model accuracy
Building a deep neural network to improve network accuracy
Varying the learning rate to improve network accuracy
Varying the loss optimizer to improve network accuracy
Understanding the scenario of overfitting
Speeding up the training process using batch normalization
3
Applications of Deep Feedforward Neural Networks
Applications of Deep Feedforward Neural Networks
Introduction
Predicting credit default
Assigning weights for classes
Predicting house prices
Categorizing news articles into topics
Classifying common audio
Stock price prediction
Leveraging a functional API
Defining weights for rows
4
Building a Deep Convolutional Neural Network
Building a Deep Convolutional Neural Network
Introduction
Inaccuracy of traditional neural networks when images are translated
Building a CNN from scratch using Python
CNNs to improve accuracy in the case of image translation
Gender classification using CNNs
Data augmentation to improve network accuracy
5
Transfer Learning
Transfer Learning
Gender classification of the person in an image using CNNs
Gender classification of the person in image using the VGG16 architecture-based model
Visualizing the output of the intermediate layers of a neural network
Gender classification of the person in image using the VGG19 architecture-based model
Gender classification using the Inception v3 architecture-based model
Gender classification of the person in image using the ResNet 50 architecture-based model
Detecting the key points within image of a face
6
Detecting and Localizing Objects in Images
Detecting and Localizing Objects in Images
Introduction
Creating the dataset for a bounding box
Generating region proposals within an image, using selective search
Calculating an intersection over a union between two images
Detecting objects, using region proposal-based CNN
Performing non-max suppression
Detecting a person using an anchor box-based algorithm
7
Image Analysis Applications in Self-Driving Cars
Image Analysis Applications in Self-Driving Cars
Traffic sign identification
Predicting the angle within which a car needs to be turned
Instance segmentation using the U-net architecture
Semantic segmentation of objects in an image
8
Image Generation
Image Generation
Introduction
Generating images that can fool a neural network using adversarial attack
DeepDream algorithm to generate images
Neural style transfer between images
Generating images of digits using Generative Adversarial Networks
Generating images using a Deep Convolutional GAN
Face generation using a Deep Convolutional GAN
Face transition from one to another
Performing vector arithmetic on generated images
9
Encoding Inputs
Encoding Inputs
Introduction
Need for encoding
Encoding an image
Encoding for recommender systems
10
Text Analysis Using Word Vectors
Text Analysis Using Word Vectors
Introduction
Building a word vector from scratch in Python
Building a word vector using the skip-gram and CBOW models
Performing vector arithmetic using pre-trained word vectors
Creating a document vector
Building word vectors using fastText
Building word vectors using GloVe
Building sentiment classification using word vectors
11
Building a Recurrent Neural Network
Building a Recurrent Neural Network
Introduction
Building an RNN from scratch in Python
Implementing RNN for sentiment classification
Building a LSTM Network from scratch in Python
Implementing LSTM for sentiment classification
Implementing stacked LSTM for sentiment classification
12
Applications of a Many-to-One Architecture RNN
Applications of a Many-to-One Architecture RNN
Generating text
Movie recommendations
Topic-modeling, using embeddings
Forecasting the value of a stock's price
13
Sequence-to-Sequence Learning
Sequence-to-Sequence Learning
Introduction
Returning sequences of outputs from a network
Building a chatbot
Machine translation
Encoder decoder architecture for machine translation
Encoder decoder architecture with attention for machine translation
14
End-to-End Learning
End-to-End Learning
Introduction
Connectionist temporal classification (CTC)
Handwritten-text recognition
Image caption generation
Generating captions, using beam search
15
Audio Analysis
Audio Analysis
Classifying a song by genre
Generating music using deep learning
Transcribing audio into text
16
Reinforcement Learning
Reinforcement Learning
The optimal action to take in a simulated game with a non-negative reward
The optimal action to take in a state in a simulated game
Q-learning to maximize rewards when playing Frozen Lake
Deep Q-learning to balance a cart pole
Deep Q-learning to play Space Invaders game
17
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Speeding up the training process using batch normalization

In the previous section on the scaling dataset, we learned that optimization is slow when the input data is not scaled (that is, it is not between zero and one).

The hidden layer value could be high in the following scenarios:

  • Input data values are high
  • Weight values are high
  • The multiplication of weight and input are high

Any of these scenarios can result in a large output value on the hidden layer.

Note that the hidden layer is the input layer to output layer. Hence, the phenomenon of high input values resulting in a slow optimization holds true when hidden layer values are large as well.

Batch normalization comes to the rescue in this scenario. We have already learned that, when input values are high, we perform scaling to reduce the input values. Additionally, we have learned that scaling can also be performed using...

Previous Section
End of Chapter 2
Next Chapter