Book Image

Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

By : Amita Kapoor, Antonio Gulli, Sujit Pal

5 (2)

Book Image

Deep Learning with TensorFlow and Keras – 3rd edition - Third Edition

5 (2)

By: Amita Kapoor, Antonio Gulli, Sujit Pal

Overview of this book

Deep Learning with TensorFlow and Keras teaches you neural networks and deep learning techniques using TensorFlow (TF) and Keras. You'll learn how to write deep learning applications in the most powerful, popular, and scalable machine learning stack available. TensorFlow 2.x focuses on simplicity and ease of use, with updates like eager execution, intuitive higher-level APIs based on Keras, and flexible model building on any platform. This book uses the latest TF 2.0 features and libraries to present an overview of supervised and unsupervised machine learning models and provides a comprehensive analysis of deep learning and reinforcement learning models using practical examples for the cloud, mobile, and large production environments. This book also shows you how to create neural networks with TensorFlow, runs through popular algorithms (regression, convolutional neural networks (CNNs), transformers, generative adversarial networks (GANs), recurrent neural networks (RNNs), natural language processing (NLP), and graph neural networks (GNNs)), covers working example apps, and then dives into TF in production, TF mobile, and TensorFlow with AutoML.

Preface

Who this book is for

What this book covers

Neural Network Foundations with TF

Neural Network Foundations with TF

What is TensorFlow (TF)?

Introduction to neural networks

Multi-layer perceptron: our first example of a network

A real example: recognizing handwritten digits

Playing with Google Colab: CPUs, GPUs, and TPUs

Sentiment analysis

Predicting output

A practical overview of backpropagation

What have we learned so far?

Toward a deep learning approach

Free Chapter

Regression and Classification

Regression and Classification

What is regression?

Prediction using linear regression

Neural networks for linear regression

Classification tasks and decision boundaries

Convolutional Neural Networks

Convolutional Neural Networks

Deep convolutional neural networks

An example of DCNN: LeNet

Recognizing CIFAR-10 images with deep learning

Very deep convolutional networks for large-scale image recognition

Deep Inception V3 for transfer learning

Other CNN architectures

Word Embeddings

Word Embeddings

Word embedding ‒ origins and fundamentals

Distributed representations

Static embeddings

Creating your own embeddings using Gensim

Exploring the embedding space with Gensim

Using word embeddings for spam detection

Neural embeddings – not just for words

Character and subword embeddings

Dynamic embeddings

Sentence and paragraph embeddings

Language model-based embeddings

Recurrent Neural Networks

Recurrent Neural Networks

The basic RNN cell

RNN cell variants

Encoder-decoder architecture – seq2seq

Attention mechanism

Transformers

Transformers’ architectures

An overview of popular and well-known models

Common pitfalls: dos and don’ts

The future of transformers

Unsupervised Learning

Unsupervised Learning

Principal component analysis

K-means clustering

Self-organizing maps

Restricted Boltzmann machines

Autoencoders

Introduction to autoencoders

Vanilla autoencoders

Sparse autoencoder

Denoising autoencoders

Stacked autoencoder

Variational autoencoders

Generative Models

Generative Models

Deep convolutional GAN (DCGAN)

Some interesting GAN architectures

Cool applications of GANs

CycleGAN in TensorFlow

Flow-based models for data generation

Diffusion models for data generation

Self-Supervised Learning

Self-Supervised Learning

Self-supervised learning

Self-prediction

Contrastive learning

Reinforcement Learning

Reinforcement Learning

An introduction to RL

Simulation environments for RL

An introduction to OpenAI Gym

Deep Q-networks

Deep deterministic policy gradient

Probabilistic TensorFlow

Probabilistic TensorFlow

TensorFlow Probability

TensorFlow Probability distributions

An Introduction to AutoML

An Introduction to AutoML

What is AutoML?

Achieving AutoML

Automatic data preparation

Automatic feature engineering

Automatic model generation

Google Cloud AutoML and Vertex AI

The Math Behind Deep Learning

The Math Behind Deep Learning

Some mathematical tools

Activation functions

Backpropagation

A note on TensorFlow and automatic differentiation

Tensor Processing Unit

Tensor Processing Unit

C/G/T processing units

Four generations of TPUs, plus Edge TPU

TPU performance

How to use TPUs with Colab

Using pretrained TPU models

Other Useful Deep Learning Libraries

Other Useful Deep Learning Libraries

Graph Neural Networks

Graph Neural Networks

Graph machine learning

Graph convolutions – the intuition behind GNNs

Common graph layers

Common graph applications

Graph customizations

Future directions

Machine Learning Best Practices

Machine Learning Best Practices

The need for best practices

Data best practices

Model best practices

TensorFlow 2 Ecosystem

TensorFlow 2 Ecosystem

TensorFlow Datasets

TensorFlow Lite

Pretrained models in TensorFlow Lite

An overview of federated learning at the edge

Advanced Convolutional Neural Networks

Advanced Convolutional Neural Networks

Composing CNNs for complex tasks

Application zoos with tf.Keras and TensorFlow Hub

Answering questions about images (visual Q&A)

Creating a DeepDream network

Inspecting what a network has learned

Audio and music

A summary of convolution operations

Capsule networks

Other Books You May Enjoy

Other Books You May Enjoy

Index

Customer Reviews

5 (2)

5 star

100%

4 star

0

3 star

0

2 star

0

1 star

0

Pretraining

As you have learned earlier, the original transformer had an encoder-decoder architecture. However, the research community understood that there are situations where it is beneficial to have only the encoder, or only the decoder, or both.

Encoder pretraining

As discussed, these models are also called auto-encoding and they use only the encoder during the pretraining. Pretraining is carried out by masking words in the input sequence and training the model to reconstruct the sequence. Typically, the encoder can access all the input words. Encoder-only models are generally used for classification.

Decoder pretraining

Decoder models are referred to as autoregressive. During pretraining, the decoder is optimized to predict the next word. In particular, the decoder can only access all the words positioned before a given word in the sequence. Decoder-only models are generally used for text generation.

Encoder-decoder pretraining

In this case, the model...