Book Image

TensorFlow 1.x Deep Learning Cookbook

Book Image

TensorFlow 1.x Deep Learning Cookbook

Overview of this book

Deep neural networks (DNNs) have achieved a lot of success in the field of computer vision, speech recognition, and natural language processing. This exciting recipe-based guide will take you from the realm of DNN theory to implementing them practically to solve real-life problems in the artificial intelligence domain. In this book, you will learn how to efficiently use TensorFlow, Google’s open source framework for deep learning. You will implement different deep learning networks, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Q-learning Networks (DQNs), and Generative Adversarial Networks (GANs), with easy-to-follow standalone recipes. You will learn how to use TensorFlow with Keras as the backend. You will learn how different DNNs perform on some popularly used datasets, such as MNIST, CIFAR-10, and Youtube8m. You will not only learn about the different mobile and embedded platforms supported by TensorFlow, but also how to set up cloud platforms for deep learning applications. You will also get a sneak peek at TPU architecture and how it will affect the future of DNNs. By using crisp, no-nonsense recipes, you will become an expert in implementing deep learning techniques in growing real-world applications and research areas such as reinforcement learning, GANs, and autoencoders.

Related Content you might be interested in

Current Title:

Small book image
TensorFlow 1.x Deep Learning Cookbook
Dec, 2017 | 536 pages
Small book image
Deep Learning with TensorFlow 2 and Keras
Sujit Pal | Amita Kapoor | Antonio Gulli
Dec, 2019 | 646 pages
Small book image
Deep Learning with Keras
Sujit Pal | Antonio Gulli
Apr, 2017 | 318 pages
Small book image
Deep Learning with TensorFlow and Keras
Sujit Pal | Amita Kapoor | Antonio Gulli
Oct, 2022 | 698 pages
Table of Contents (15 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Conventions
Reader feedback
Customer support
Free Chapter
1
TensorFlow - An Introduction
TensorFlow - An Introduction
Introduction
Installing TensorFlow
Hello world in TensorFlow
Understanding the TensorFlow program structure
Working with constants, variables, and placeholders
Performing matrix manipulations using TensorFlow
Using a data flow graph
Migrating from 0.x to 1.x
Using XLA to enhance computational performance
Invoking CPU/GPU devices
TensorFlow for Deep Learning
Different Python packages required for DNN-based problems
2
Regression
Regression
Introduction
Choosing loss functions
Optimizers in TensorFlow
Reading from CSV files and preprocessing data
House price estimation-simple linear regression
House price estimation-multiple linear regression
Logistic regression on the MNIST dataset
3
Neural Networks - Perceptron
Neural Networks - Perceptron
Introduction
Activation functions
Single layer perceptron
Calculating gradients of backpropagation algorithm
MNIST classifier using MLP
Function approximation using MLP-predicting Boston house prices
Tuning hyperparameters
Higher-level APIs-Keras
See also
4
Convolutional Neural Networks
Convolutional Neural Networks
Introduction
Creating a ConvNet to classify handwritten MNIST numbers
Creating a ConvNet to classify CIFAR-10
Transferring style with VGG19 for image repainting
Using a pretrained VGG16 net for transfer learning
Creating a DeepDream network
5
Advanced Convolutional Neural Networks
Advanced Convolutional Neural Networks
Introduction
Creating a ConvNet for Sentiment Analysis
Inspecting what filters a VGG pre-built network has learned
Classifying images with VGGNet, ResNet, Inception, and Xception
Recycling pre-built Deep Learning models for extracting features
Very deep InceptionV3 Net used for Transfer Learning
Generating music with dilated ConvNets, WaveNet, and NSynth
Answering questions about images (Visual Q&A)
Classifying videos with pre-trained nets in six different ways
6
Recurrent Neural Networks
Recurrent Neural Networks
Introduction
Neural machine translation - training a seq2seq RNN
Neural machine translation - inference on a seq2seq RNN
All you need is attention - another example of a seq2seq RNN
Learning to write as Shakespeare with RNNs
Learning to predict future Bitcoin value with RNNs
Many-to-one and many-to-many RNN examples
7
Unsupervised Learning
Unsupervised Learning
Introduction
Principal component analysis
k-means clustering
Self-organizing maps
Restricted Boltzmann Machine
Recommender system using RBM
DBN for Emotion Detection
8
Autoencoders
Autoencoders
Introduction
Vanilla autoencoders
Sparse autoencoder
Denoising autoencoder
Convolutional autoencoders
Stacked autoencoder
9
Reinforcement Learning
Reinforcement Learning
Introduction
Learning OpenAI Gym
Implementing neural network agent to play Pac-Man
Q learning to balance Cart-Pole
Game of Atari using Deep Q Networks
Policy gradients to play the game of Pong
10
Mobile Computation
Mobile Computation
Introduction
Installing TensorFlow mobile for macOS and Android
Playing with TensorFlow and Android examples
Installing TensorFlow mobile for macOS and iPhone
Optimizing a TensorFlow graph for mobile devices
Profiling a TensorFlow graph for mobile devices
Transforming a TensorFlow graph for mobile devices
11
Generative Models and CapsNet
Generative Models and CapsNet
Introduction
Learning to forge MNIST images with simple GANs
Learning to forge MNIST images with DCGANs
Learning to forge Celebrity Faces and other datasets with DCGAN
Implementing Variational Autoencoders
Learning to beat the previous MNIST state-of-the-art results with Capsule Networks
12
Distributed TensorFlow and Cloud Deep Learning
Distributed TensorFlow and Cloud Deep Learning
Introduction
Working with TensorFlow and GPUs
Playing with Distributed TensorFlow: multiple GPUs and one CPU
Playing with Distributed TensorFlow: multiple servers
Training a Distributed TensorFlow MNIST classifier
Working with TensorFlow Serving and Docker
Running Distributed TensorFlow on Google Cloud (GCP) with Compute Engine
Running Distributed TensorFlow on Google CloudML
Running Distributed TensorFlow on Microsoft Azure
Running Distributed TensorFlow on Amazon AWS
13
Learning to Learn with AutoML (Meta-Learning)
Learning to Learn with AutoML (Meta-Learning)
Meta-learning with recurrent networks and with reinforcement learning
Meta-learning blocks
Meta-learning novel tasks
Siamese Network
14
TensorFlow Processing Units
TensorFlow Processing Units
Components of TPUs

Neural machine translation - inference on a seq2seq RNN

In this recipe, we use the results of the previous recipe to translate from a source language into a target language. The idea is very simple: a source sentence is given the two combined RNNs (encoder + decoder) as input . As soon as the sentence concludes, the decoder will emit logit values and we greedily emit the word associated with the maximum value. As an example, the word moi is emitted as the first token from the decoder because this word has the maximum logit value. After that, the word suis is emitted, and so on:

An example of sequence models for NMT with probabilities as seen in https://github.com/lmthang/thesis/blob/master/thesis.pdf

There are multiple strategies for using the output of a decoder:

  • Greedy: The word corresponding to the maximum logit is emitted
  • Sampling: A word is emitted by sampling the logit...
Previous Section
End of Section 4
Next Section