Book Image

PyTorch Deep Learning Hands-On

By : Sherin Thomas, Sudhanshu Passi
Book Image

PyTorch Deep Learning Hands-On

By: Sherin Thomas, Sudhanshu Passi

Overview of this book

PyTorch Deep Learning Hands-On is a book for engineers who want a fast-paced guide to doing deep learning work with PyTorch. It is not an academic textbook and does not try to teach deep learning principles. The book will help you most if you want to get your hands dirty and put PyTorch to work quickly. PyTorch Deep Learning Hands-On shows how to implement the major deep learning architectures in PyTorch. It covers neural networks, computer vision, CNNs, natural language processing (RNN), GANs, and reinforcement learning. You will also build deep learning workflows with the PyTorch framework, migrate models built in Python to highly efficient TorchScript, and deploy to production using the most sophisticated available tools. Each chapter focuses on a different area of deep learning. Chapters start with a refresher on how the model works, before sharing the code you need to implement it in PyTorch. This book is ideal if you want to rapidly add PyTorch to your deep learning toolset.
Table of Contents (11 chapters)
10
Index

Autoregressive models

Autoregressive models use information from the previous steps and create the next output. RNNs generating text for a language modeling task is a typical example of the autoregressive model.

Autoregressive models

Figure 6.3: Autoregressive model for RNN language modeling

Autoregressive models generate the first input independently, or we give this to the network. For example, in the case of RNNs, we give the first word to the network and the network uses the first word we provided to assume what the second word would be. Then it uses the first and second word to predict the third word and so on.

Although most generation tasks are done on images, our autoregressive generation is on audio. We will build WaveNet, a research result from Google DeepMind, which is the current state-of-the-art implementation of audio generation and especially for text-to-speech processing. Through this, we'll be exploring what the PyTorch APIs for audio processing are. But before looking at WaveNet,...