Hands-On Neural Networks with Keras

By : Niloy Purkait

Hands-On Neural Networks with Keras

By: Niloy Purkait

Overview of this book

Neural networks are used to solve a wide range of problems in different areas of AI and deep learning. Hands-On Neural Networks with Keras will start with teaching you about the core concepts of neural networks. You will delve into combining different neural network models and work with real-world use cases, including computer vision, natural language understanding, synthetic data generation, and many more. Moving on, you will become well versed with convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) networks, autoencoders, and generative adversarial networks (GANs) using real-world training datasets. We will examine how to use CNNs for image recognition, how to use reinforcement learning agents, and many more. We will dive into the specific architectures of various networks and then implement each of them in a hands-on manner using industry-grade frameworks. By the end of this book, you will be highly familiar with all prominent deep learning models and frameworks, and the options you have when applying deep learning to real-world scenarios and embedding artificial intelligence as the core fabric of your organization.

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Free Chapter

Section 1: Fundamentals of Neural Networks

Overview of Neural Networks

Defining our goal

Knowing our tools

The fundamentals of neural learning

The fundamentals of data science

Summary

Further reading

A Deeper Dive into Neural Networks

From the biological to the artificial neuron – the perceptron

Building a perceptron

Learning through errors

Training a perceptron

Backpropagation

Scaling the perceptron

A single layered network

Summary

Signal Processing - Data Analysis with Neural Networks

Processing signals

Images as numbers

Feeding a neural network

Examples of tensors

Building a model

Compiling the model

Evaluating model performance

Implementing weight regularization in Keras

Weight regularization experiments

Implementing dropout regularization in Keras

Language processing

The internet movie reviews dataset

Plotting a single training instance

Accessing model predictions

Probing the predictions

Feature-wise normalization

Cross validation with scikit-learn API

Summary

Exercises

Section 2: Advanced Neural Network Architectures

Convolutional Neural Networks

Why CNNs?

The birth of vision

Understanding biological vision

Conceptualizing spatial invariance

Defining receptive fields of neurons

Implementing a hierarchy of neurons

The birth of the modern CNN

Designing a CNN

The convolution operation

Visualizing feature extraction with filters

Looking at complex filters

Summarizing the convolution operation

Understanding pooling layers

Implementing CNNs in Keras

Convolutional layer

Leveraging a fully connected layer for classification

Summarizing our model

Checking model accuracy

The problem with detecting smiles

Introducing Keras's functional API

Verifying the number of channels per layer

Understanding saliency

Visualizing saliency maps with ResNet50

Loading pictures from a local directory

Using Keras's visualization module

Searching through layers

Exercise

Gradient weighted class activation mapping

Visualizing class activations with Keras-vis

Using the pretrained model for prediction

Visualizing maximal activations per output class

Converging a model

Using multiple filter indices to hallucinate

Problems with CNNs

Neural network pareidolia

Summary

Recurrent Neural Networks

Modeling sequences

Using RNNs for sequential modeling

Summarizing different types of sequence processing tasks

Predicting an output per time step

Backpropagation through time

Exploding and vanishing gradients

GRUs

Building character-level language models in Keras

Statistics of character modeling

The purpose of controlling stochasticity

Testing different RNN models

Building a SimpleRNN

Building GRUs

On processing reality sequentially

Bi-directional layer in Keras

Visualizing output values

Summary

Further reading

Exercise

Long Short-Term Memory Networks

On processing complex sequences

The LSTM network

Dissecting the LSTM

LSTM memory block

Visualizing the flow of information

Computing contender memory

Computing activations per timestep

Variations of LSTM and performance

Understanding peephole connections

Importance of timing and counting

Putting our knowledge to use

On modeling stock market data

Denoising the data

Implementing exponential smoothing

The problem with one-step-ahead predictions

Creating sequences of observations

Building LSTMs

Closing comments

Summary

Exercises

Reinforcement Learning with Deep Q-Networks

On reward and gratification

Conditioning machines with reinforcement learning

The explore-exploit dilemma

Path to artificial general intelligence

Simulating environments

A self-driving taxi cab

Trade-off between immediate and future rewards

Discounting future rewards

Markov decision process

Understanding policy functions

Assessing the value of a state

Assessing the quality of an action

Using the Bellman equation

Updating the Bellman equation iteratively

Why use neural networks?

Performing a forward pass in Q-learning

Performing a backward pass in Q-Learning

Deep Q-learning in Keras

Balancing exploration with exploitation

Initializing the deep Q-learning agent

Double Q-learning

Dueling network architecture

Exercise

Summary

Section 3: Hybrid Model Architecture

Autoencoders

Why autoencoders?

Automatically encoding information

Understanding the limitations of autoencoders

Breaking down the autoencoder

Training an autoencoder

Overviewing autoencoder archetypes

Network size and representational power

Understanding regularization in autoencoders

Regularization with sparse autoencoders

Probing the data

Building the verification model

Designing a deep autoencoder

Using functional API to design autoencoders

Deep convolutional autoencoder

Compiling and training the model

Testing and visualizing the results

Denoising autoencoders

Training the denoising network

Summary

Exercise

Generative Networks

Replicating versus generating content

Understanding the notion of latent space

Diving deeper into generative networks

Using randomness to augment outputs

Sampling from the latent space

Understanding types of generative networks

Understanding VAEs

Designing a VAE in Keras

Building the encoding module in a VAE

Building the decoder module

Visualizing the latent space

Latent space sampling and output generation

Exploring GANs

Diving deeper into GANs

Designing a GAN in Keras

Designing the generator module

Designing the discriminator module

Putting the GAN together

The training function

Defining the discriminator labels

Training the generator per batch

Executing the training session

Conclusion

Summary

Section 4: Road Ahead

Contemplating Present and Future Developments

Sharing representations with transfer learning

Concluding our experiments

Learning representations

Limits of current neural networks

Encouraging sparse representation learning

Tuning hyperparameters

Automatic optimization and evolutionary algorithms

Multi-network predictions and ensemble models

The future of AI and neural networks

The road ahead

Problems with classical computing

The advent of quantum computing

Quantum neural networks

Technology and society

Contemplating our future

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

4 star

3 star

2 star

1 star

To get the most out of this book

Having some prior knowledge of Python will be beneficial.

Download the example code files

You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

Log in or register at www.packt.com.
Select the SUPPORT tab.
Click on Code Downloads & Errata.
Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Hands-On-Neural-Networks-with-Keras. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781789536089_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "This refers to the type of data that's stored in the tensor, and can be checked by calling the type() method on a tensor of interest."

A block of code is set as follows:

import numpy as np
import keras
from keras.datasets import mnist
from keras.utils import np_utils

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

keras.utils.print_summary(model, line_length=None, positions=None,    
                          print_fn=None)

Any command-line input or output is written as follows:

! pip install keras-vis

Bold: Indicates a new term, an important word, or words that you see on screen.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Hands-On Neural Networks with Keras

By : Niloy Purkait

Hands-On Neural Networks with Keras

By: Niloy Purkait

Overview of this book

Related Content you might be interested in

Current Title:

Hands-On Neural Networks with Keras

Hands-On Neural Networks

Deep Learning with Keras

Deep Learning for Beginners