TensorFlow 1.x Deep Learning Cookbook

Overview of this book

Deep neural networks (DNNs) have achieved a lot of success in the field of computer vision, speech recognition, and natural language processing. This exciting recipe-based guide will take you from the realm of DNN theory to implementing them practically to solve real-life problems in the artificial intelligence domain. In this book, you will learn how to efficiently use TensorFlow, Google’s open source framework for deep learning. You will implement different deep learning networks, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Q-learning Networks (DQNs), and Generative Adversarial Networks (GANs), with easy-to-follow standalone recipes. You will learn how to use TensorFlow with Keras as the backend. You will learn how different DNNs perform on some popularly used datasets, such as MNIST, CIFAR-10, and Youtube8m. You will not only learn about the different mobile and embedded platforms supported by TensorFlow, but also how to set up cloud platforms for deep learning applications. You will also get a sneak peek at TPU architecture and how it will affect the future of DNNs. By using crisp, no-nonsense recipes, you will become an expert in implementing deep learning techniques in growing real-world applications and research areas such as reinforcement learning, GANs, and autoencoders.

Preface

What this book covers

What you need for this book

Free Chapter

TensorFlow - An Introduction

Introduction

Installing TensorFlow

Hello world in TensorFlow

Understanding the TensorFlow program structure

Working with constants, variables, and placeholders

Performing matrix manipulations using TensorFlow

Using a data flow graph

Migrating from 0.x to 1.x

Using XLA to enhance computational performance

Invoking CPU/GPU devices

TensorFlow for Deep Learning

Different Python packages required for DNN-based problems

Regression

Introduction

Choosing loss functions

Optimizers in TensorFlow

Reading from CSV files and preprocessing data

House price estimation-simple linear regression

House price estimation-multiple linear regression

Logistic regression on the MNIST dataset

Neural Networks - Perceptron

Introduction

Activation functions

Single layer perceptron

Calculating gradients of backpropagation algorithm

MNIST classifier using MLP

Function approximation using MLP-predicting Boston house prices

Tuning hyperparameters

Higher-level APIs-Keras

See also

Convolutional Neural Networks

Introduction

Creating a ConvNet to classify handwritten MNIST numbers

Creating a ConvNet to classify CIFAR-10

Transferring style with VGG19 for image repainting

Using a pretrained VGG16 net for transfer learning

Creating a DeepDream network

Advanced Convolutional Neural Networks

Introduction

Creating a ConvNet for Sentiment Analysis

Inspecting what filters a VGG pre-built network has learned

Classifying images with VGGNet, ResNet, Inception, and Xception

Recycling pre-built Deep Learning models for extracting features

Very deep InceptionV3 Net used for Transfer Learning

Generating music with dilated ConvNets, WaveNet, and NSynth

Answering questions about images (Visual Q&A)

Classifying videos with pre-trained nets in six different ways

Recurrent Neural Networks

Introduction

Neural machine translation - training a seq2seq RNN

Neural machine translation - inference on a seq2seq RNN

All you need is attention - another example of a seq2seq RNN

Learning to write as Shakespeare with RNNs

Learning to predict future Bitcoin value with RNNs

Many-to-one and many-to-many RNN examples

Unsupervised Learning

Introduction

Principal component analysis

k-means clustering

Self-organizing maps

Restricted Boltzmann Machine

Recommender system using RBM

DBN for Emotion Detection

Autoencoders

Introduction

Vanilla autoencoders

Sparse autoencoder

Denoising autoencoder

Convolutional autoencoders

Stacked autoencoder

Reinforcement Learning

Introduction

Learning OpenAI Gym

Implementing neural network agent to play Pac-Man

Q learning to balance Cart-Pole

Game of Atari using Deep Q Networks

Policy gradients to play the game of Pong

Mobile Computation

Introduction

Installing TensorFlow mobile for macOS and Android

Playing with TensorFlow and Android examples

Installing TensorFlow mobile for macOS and iPhone

Optimizing a TensorFlow graph for mobile devices

Profiling a TensorFlow graph for mobile devices

Transforming a TensorFlow graph for mobile devices

Generative Models and CapsNet

Introduction

Learning to forge MNIST images with simple GANs

Learning to forge MNIST images with DCGANs

Learning to forge Celebrity Faces and other datasets with DCGAN

Implementing Variational Autoencoders

Learning to beat the previous MNIST state-of-the-art results with Capsule Networks

Distributed TensorFlow and Cloud Deep Learning

Introduction

Working with TensorFlow and GPUs

Playing with Distributed TensorFlow: multiple GPUs and one CPU

Playing with Distributed TensorFlow: multiple servers

Training a Distributed TensorFlow MNIST classifier

Working with TensorFlow Serving and Docker

Running Distributed TensorFlow on Google Cloud (GCP) with Compute Engine

Running Distributed TensorFlow on Google CloudML

Running Distributed TensorFlow on Microsoft Azure

Running Distributed TensorFlow on Amazon AWS

Learning to Learn with AutoML (Meta-Learning)

Meta-learning with recurrent networks and with reinforcement learning

Meta-learning blocks

Meta-learning novel tasks

Siamese Network

TensorFlow Processing Units

Components of TPUs

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Using XLA to enhance computational performance

Accelerated linear algebra (XLA) is a domain-specific compiler for linear algebra. According to https://www.tensorflow.org/performance/xla/, it is still in the experimental stage and is used to optimize TensorFlow computations. It can provide improvements in execution speed, memory usage, and portability on the server and mobile platforms. It provides two-way JIT (Just In Time) compilation or AoT (Ahead of Time) compilation. Using XLA, you can produce platform-dependent binary files (for a large number of platforms like x64, ARM, and so on), which can be optimized for both memory and speed.

Getting ready

At present, XLA is not included in the binary distributions of TensorFlow. One needs to build it from source. To build TensorFlow from source, knowledge of LLVM and Bazel along with TensorFlow is required. TensorFlow.org supports building from source in only MacOS and Ubuntu. The steps needed to build TensorFlow from the source are as follows (https://www.tensorflow.org/install/install_sources):

Determine which TensorFlow you want to install--TensorFlow with CPU support only or TensorFlow with GPU support.
Clone the TensorFlow repository:

git clone https://github.com/tensorflow/tensorflow 
cd tensorflow 
git checkout Branch #where Branch is the desired branch

Install the following dependencies:

- Bazel
- TensorFlow Python dependencies
- For the GPU version, NVIDIA packages to support TensorFlow

Configure the installation. In this step, you need to choose different options such as XLA, Cuda support, Verbs, and so on:

./configure

Next, use bazel-build:
For CPU only version you use:

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

If you have a compatible GPU device and you want the GPU Support, then use:

bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

On a successful run, you will get a script, build_pip_package.
Run this script as follows to build the whl file:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

Install the pip package:

sudo pip install /tmp/tensorflow_pkg/tensorflow-1.1.0-py2-none-any.whl

Now you are ready to go.

How to do it...

TensorFlow generates TensorFlow graphs. With the help of XLA, it is possible to run the TensorFlow graphs on any new kind of device.

JIT Compilation: This is to turn on JIT compilation at session level:

# Config to turn on JIT compilation
 config = tf.ConfigProto()
 config.graph_options.optimizer_options.global_jit_level = tf.OptimizerOptions.ON_1

 sess = tf.Session(config=config)

This is to turn on JIT compilation manually:

jit_scope = tf.contrib.compiler.jit.experimental_jit_scope

x = tf.placeholder(np.float32)
with jit_scope():
   y = tf.add(x, x)  # The "add" will be compiled with XLA.

We can also run computations via XLA by placing the operator on a specific XLA device XLA_CPU or XLA_GPU:

with tf.device \ ("/job:localhost/replica:0/task:0/device:XLA_GPU:0"):
   output = tf.add(input1, input2)

AoT Compilation: Here, we use tfcompile as standalone to convert TensorFlow graphs into executable code for different devices (mobile).

TensorFlow.org tells about tfcompile:

tfcompile takes a subgraph, identified by the TensorFlow concepts of feeds and fetches, and generates a function that implements that subgraph. The feeds are the input arguments for the function, and the fetches are the output arguments for the function. All inputs must be fully specified by the feeds; the resulting pruned subgraph cannot contain placeholder or variable nodes. It is common to specify all placeholders and variables as feeds, which ensures the resulting subgraph no longer contains these nodes. The generated function is packaged as a cc_library, with a header file exporting the function signature, and an object file containing the implementation. The user writes code to invoke the generated function as appropriate.

For advanced steps to do the same, you can refer to https://www.tensorflow.org/performance/xla/tfcompile.

TensorFlow 1.x Deep Learning Cookbook

TensorFlow 1.x Deep Learning Cookbook

Overview of this book

Related Content you might be interested in

Current Title:

TensorFlow 1.x Deep Learning Cookbook

Deep Learning with TensorFlow 2 and Keras

Deep Learning with Keras

Deep Learning with TensorFlow and Keras