Book Image

Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

By : Amita Kapoor, Antonio Gulli, Sujit Pal

5 (2)

Book Image

Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

5 (2)

By: Amita Kapoor, Antonio Gulli, Sujit Pal

Overview of this book

Deep Learning with TensorFlow and Keras teaches you neural networks and deep learning techniques using TensorFlow (TF) and Keras. You'll learn how to write deep learning applications in the most powerful, popular, and scalable machine learning stack available. TensorFlow 2.x focuses on simplicity and ease of use, with updates like eager execution, intuitive higher-level APIs based on Keras, and flexible model building on any platform. This book uses the latest TF 2.0 features and libraries to present an overview of supervised and unsupervised machine learning models and provides a comprehensive analysis of deep learning and reinforcement learning models using practical examples for the cloud, mobile, and large production environments. This book also shows you how to create neural networks with TensorFlow, runs through popular algorithms (regression, convolutional neural networks (CNNs), transformers, generative adversarial networks (GANs), recurrent neural networks (RNNs), natural language processing (NLP), and graph neural networks (GNNs)), covers working example apps, and then dives into TF in production, TF mobile, and TensorFlow with AutoML.

Preface

Who this book is for

What this book covers

Neural Network Foundations with TF

Neural Network Foundations with TF

What is TensorFlow (TF)?

Introduction to neural networks

Multi-layer perceptron: our first example of a network

A real example: recognizing handwritten digits

Playing with Google Colab: CPUs, GPUs, and TPUs

Sentiment analysis

Predicting output

A practical overview of backpropagation

What have we learned so far?

Toward a deep learning approach

Free Chapter

Regression and Classification

Regression and Classification

What is regression?

Prediction using linear regression

Neural networks for linear regression

Classification tasks and decision boundaries

Convolutional Neural Networks

Convolutional Neural Networks

Deep convolutional neural networks

An example of DCNN: LeNet

Recognizing CIFAR-10 images with deep learning

Very deep convolutional networks for large-scale image recognition

Deep Inception V3 for transfer learning

Other CNN architectures

Word Embeddings

Word Embeddings

Word embedding ‒ origins and fundamentals

Distributed representations

Static embeddings

Creating your own embeddings using Gensim

Exploring the embedding space with Gensim

Using word embeddings for spam detection

Neural embeddings – not just for words

Character and subword embeddings

Dynamic embeddings

Sentence and paragraph embeddings

Language model-based embeddings

Recurrent Neural Networks

Recurrent Neural Networks

The basic RNN cell

RNN cell variants

Encoder-decoder architecture – seq2seq

Attention mechanism

Transformers

Transformers’ architectures

An overview of popular and well-known models

Common pitfalls: dos and don’ts

The future of transformers

Unsupervised Learning

Unsupervised Learning

Principal component analysis

K-means clustering

Self-organizing maps

Restricted Boltzmann machines

Autoencoders

Introduction to autoencoders

Vanilla autoencoders

Sparse autoencoder

Denoising autoencoders

Stacked autoencoder

Variational autoencoders

Generative Models

Generative Models

Deep convolutional GAN (DCGAN)

Some interesting GAN architectures

Cool applications of GANs

CycleGAN in TensorFlow

Flow-based models for data generation

Diffusion models for data generation

Self-Supervised Learning

Self-Supervised Learning

Self-supervised learning

Self-prediction

Contrastive learning

Reinforcement Learning

Reinforcement Learning

An introduction to RL

Simulation environments for RL

An introduction to OpenAI Gym

Deep Q-networks

Deep deterministic policy gradient

Probabilistic TensorFlow

Probabilistic TensorFlow

TensorFlow Probability

TensorFlow Probability distributions

An Introduction to AutoML

An Introduction to AutoML

What is AutoML?

Achieving AutoML

Automatic data preparation

Automatic feature engineering

Automatic model generation

Google Cloud AutoML and Vertex AI

The Math Behind Deep Learning

The Math Behind Deep Learning

Some mathematical tools

Activation functions

Backpropagation

A note on TensorFlow and automatic differentiation

Tensor Processing Unit

Tensor Processing Unit

C/G/T processing units

Four generations of TPUs, plus Edge TPU

TPU performance

How to use TPUs with Colab

Using pretrained TPU models

Other Useful Deep Learning Libraries

Other Useful Deep Learning Libraries

Graph Neural Networks

Graph Neural Networks

Graph machine learning

Graph convolutions – the intuition behind GNNs

Common graph layers

Common graph applications

Graph customizations

Future directions

Machine Learning Best Practices

Machine Learning Best Practices

The need for best practices

Data best practices

Model best practices

TensorFlow 2 Ecosystem

TensorFlow 2 Ecosystem

TensorFlow Datasets

TensorFlow Lite

Pretrained models in TensorFlow Lite

An overview of federated learning at the edge

Advanced Convolutional Neural Networks

Advanced Convolutional Neural Networks

Composing CNNs for complex tasks

Application zoos with tf.Keras and TensorFlow Hub

Answering questions about images (visual Q&A)

Creating a DeepDream network

Inspecting what a network has learned

Audio and music

A summary of convolution operations

Capsule networks

Other Books You May Enjoy

Other Books You May Enjoy

Index

Customer Reviews

5 (2)

5 star

100%

4 star

0

3 star

0

2 star

0

1 star

0

TPU performance

Discussing performance is always difficult because it is important to first define the metrics that we are going to measure, and the set of workloads that we are going to use as benchmarks. For instance, Google reported an impressive linear scaling for TPU v2 used with ResNet-50 [4] (see Figure 15.9 and Figure 15.10):

Chart, line chart Description automatically generated

Figure 15.9: Linear scalability in the number of TPUs v2 when increasing the number of images

In addition, you can find online a comparison of ResNet-50 [4] where a full Cloud TPU v2 Pod is >200x faster than a V100 NVIDIA Tesla GPU for ResNet-50 training:

A picture containing timeline Description automatically generated

Figure 15.10: A full Cloud TPU v2 Pod is >200x faster than a V100 NVIDIA Tesla GPU for training a ResNet-50 model

According to Google, TPU v4 givse top-line results for MLPerf1.0 [5] when compared with Nvidia A100 GPUs (see Figure 15.11). Indeed, these accelerators are designed by keeping in mind the latest large models encompassing billions and sometimes trillions of...