Book Image

Python Deep Learning Cookbook

By : Indra den Bakker
Book Image

Python Deep Learning Cookbook

By: Indra den Bakker

Overview of this book

Deep Learning is revolutionizing a wide range of industries. For many applications, deep learning has proven to outperform humans by making faster and more accurate predictions. This book provides a top-down and bottom-up approach to demonstrate deep learning solutions to real-world problems in different areas. These applications include Computer Vision, Natural Language Processing, Time Series, and Robotics. The Python Deep Learning Cookbook presents technical solutions to the issues presented, along with a detailed explanation of the solutions. Furthermore, a discussion on corresponding pros and cons of implementing the proposed solution using one of the popular frameworks like TensorFlow, PyTorch, Keras and CNTK is provided. The book includes recipes that are related to the basic concepts of neural networks. All techniques s, as well as classical networks topologies. The main purpose of this book is to provide Python programmers a detailed list of recipes to apply deep learning to common and not-so-common scenarios.
Table of Contents (21 chapters)
Title Page
About the Author
About the Reviewer
Customer Feedback

Using PyTorch’s dynamic computation graphs for RNNs

PyTorch is the Python deep learning framework and it's getting a lot of traction lately. PyTorch is the implementation of Torch, which uses Lua. It is by Facebook and is fast thanks to GPU-accelerated tensor computations. A huge benefit of using over other frameworks is that graphs are created on the fly and are not static. This means networks are dynamic and you can adjust your network without having to start over again. As a result, the graph that is created on the fly can be different for each example. PyTorch supports multiple GPUs and you can manually set which computation needs to be performed on which device (CPU or GPU).

How to do it...

  1. First, we install in our Anaconda environment, as follows:
conda install pytorch torchvision cuda80 -c soumith

If you want to install on another platform, you can have a look at the PyTorch website for clear guidance:

  1. Let's import PyTorch into our Python environment:
import torch
  1. While Keras provides higher-level abstraction for building neural networks, PyTorch has this feature built in. This means one can build with higher-level building blocks or can even build the forward and backward pass manually. In this introduction, we will use the higher-level abstraction. First, we need to set the size of our random training data:
 batch_size = 32
 input_shape = 5
 output_shape = 10
  1. To make use of GPUs, we will cast the tensors as follows:

This ensures that all computations will use the attached GPU. 

  1. We can use this to generate random training data:
from torch.autograd import Variable
X = Variable(torch.randn(batch_size, input_shape))
y = Variable(torch.randn(batch_size, output_shape), requires_grad=False)
  1. We will use a simple neural network having one hidden layer with 32 units and an output layer:
model = torch.nn.Sequential(
          torch.nn.Linear(input_shape, 32),
          torch.nn.Linear(32, output_shape),

We use the .cuda() extension to make sure the model runs on the GPU. 

  1. Next, we the MSE loss function:
loss_function = torch.nn.MSELoss()
  1. We are now ready to start training our model for 10 epochs with the following code:
learning_rate = 0.001
for i in range(10):
    y_pred = model(x)
    loss = loss_function(y_pred, y)
    # Zero gradients

    # Update weights
    for param in model.parameters(): -= learning_rate *


The PyTorch framework gives a lot of freedom to implement simple neural networks and more complex deep learning models. What we didn't demonstrate in this introduction, is the use of dynamic graphs in PyTorch. This is a really powerful feature that we will demonstrate in other chapters of this book.