Book Image

Hands-On Deep Learning with Go

By : Gareth Seneque, Darrell Chua
Book Image

Hands-On Deep Learning with Go

By: Gareth Seneque, Darrell Chua

Overview of this book

Go is an open source programming language designed by Google for handling large-scale projects efficiently. The Go ecosystem comprises some really powerful deep learning tools such as DQN and CUDA. With this book, you'll be able to use these tools to train and deploy scalable deep learning models from scratch. This deep learning book begins by introducing you to a variety of tools and libraries available in Go. It then takes you through building neural networks, including activation functions and the learning algorithms that make neural networks tick. In addition to this, you'll learn how to build advanced architectures such as autoencoders, restricted Boltzmann machines (RBMs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and more. You'll also understand how you can scale model deployments on the AWS cloud infrastructure for training and inference. By the end of this book, you'll have mastered the art of building, training, and deploying deep learning models in Go to solve real-world problems.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Deep Learning with Go
Gareth Seneque | Darrell Chua
Aug, 2019 | 242 pages
Small book image
Go Machine Learning Projects
Xuanyi Chew
Nov, 2018 | 348 pages
Small book image
Machine Learning With Go
Daniel Whitenack
Sep, 2017 | 304 pages
Small book image
Hands-On Mathematics for Deep Learning
Jay Dawani
Jun, 2020 | 364 pages
Table of Contents (15 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Deep Learning in Go, Neural Networks, and How to Train Them
Section 1: Deep Learning in Go, Neural Networks, and How to Train Them
2
Introduction to Deep Learning in Go
Introduction to Deep Learning in Go
Introducing DL
Overview of ML in Go
Using Gorgonia
Summary
3
What Is a Neural Network and How Do I Train One?
What Is a Neural Network and How Do I Train One?
A basic neural network
Activation functions
Gradient descent and backpropagation
Advanced gradient descent algorithms
Summary
4
Beyond Basic Neural Networks - Autoencoders and RBMs
Beyond Basic Neural Networks - Autoencoders and RBMs
Loading data – MNIST
Building a neural network for handwriting recognition
Building an autoencoder – generating MNIST digits
Building an RBM for Netflix-style collaborative filtering
Summary
Further reading
5
CUDA - GPU-Accelerated Training
CUDA - GPU-Accelerated Training
CPUs versus GPUs
Understanding Gorgonia and CUDA
Building a model in Gorgonia with CUDA support
Performance benchmarking of CPU versus GPU models for training and inference
Summary
6
Section 2: Implementing Deep Neural Network Architectures
Section 2: Implementing Deep Neural Network Architectures
7
Next Word Prediction with Recurrent Neural Networks
Next Word Prediction with Recurrent Neural Networks
Vanilla RNNs
Training RNNs
RNNs and vanishing gradients
Augmenting your RNN with GRU/LSTM units
Building an LSTM in Gorgonia
Summary
8
Object Recognition with Convolutional Neural Networks
Object Recognition with Convolutional Neural Networks
Introduction to CNNs
Building an example CNN
Assessing the results
Summary
Further reading
9
Maze Solving with Deep Q-Networks
Maze Solving with Deep Q-Networks
What is a DQN?
Solving a maze using a DQN in Gorgonia
Summary
Further reading
10
Generative Models with Variational Autoencoders
Generative Models with Variational Autoencoders
Introduction to VAEs
Building a VAE on MNIST
Assessing the results
Summary
Further reading
11
Section 3: Pipeline, Deployment, and Beyond!
Section 3: Pipeline, Deployment, and Beyond!
12
Building a Deep Learning Pipeline
Building a Deep Learning Pipeline
Exploring Pachyderm
Integrating our CNN
Summary
13
Scaling Deployment
Scaling Deployment
Lost (and found) in the cloud
Building deployment templates
Running a model on a K8s cluster
Summary
14
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

RNNs and vanishing gradients

RNNs themselves are an important architectural innovation, but run into problems in terms of their gradients vanishing. When gradient values become so small that the updates are equally tiny, this slows or even halts learning. Your digital neurons die, and your network doesn't do what you want it to do. But is a neural network with a bad memory better than one with no memory at all?

Let's zoom in a bit and discuss what's actually going on when you run into this problem. Recall the formula for calculating the value for a given weight during backpropagation:

W = W - LR*G

Here, the weight value equals the weight minus (learning rate multiplied by the gradient).

Your network is propagating error derivatives across layers and across timesteps. The larger your dataset, the greater the number of timesteps and parameters, and so the greater...

Previous Section
End of Section 4
Next Section