Book Image

Deep Learning Quick Reference

By : Mike Bernico
Book Image

Deep Learning Quick Reference

By: Mike Bernico

Overview of this book

Deep learning has become an essential necessity to enter the world of artificial intelligence. With this book deep learning techniques will become more accessible, practical, and relevant to practicing data scientists. It moves deep learning from academia to the real world through practical examples. You will learn how Tensor Board is used to monitor the training of deep neural networks and solve binary classification problems using deep learning. Readers will then learn to optimize hyperparameters in their deep learning models. The book then takes the readers through the practical implementation of training CNN's, RNN's, and LSTM's with word embeddings and seq2seq models from scratch. Later the book explores advanced topics such as Deep Q Network to solve an autonomous agent problem and how to use two adversarial networks to generate artificial images that appear real. For implementation purposes, we look at popular Python-based deep learning frameworks such as Keras and Tensorflow, Each chapter provides best practices and safe choices to help readers make the right decision while training deep neural networks. By the end of this book, you will be able to solve real-world problems quickly with deep neural networks.
Table of Contents (15 chapters)

Safe choices for GAN

I've previously mentioned Soumith Chintala's GAN hacks Git (https://github.com/soumith/ganhacks), which is an excellent place to start when you're trying to make your GAN stable. Now that we've talked about how difficult it can be to train a stable GAN, let's talk about some of the safe choices that will likely help you succeed that you can find there. While there are quite a few hacks out there, here are my top recommendations that haven't been covered already in the chapter:

  • Batch norm: When using batch normalization, construct different minibatches for both real and fake data and make the updates separately.
  • Leaky ReLU: Leaky ReLU is a variation of the ReLU activation function. Recall the the ReLU function is .

Leaky ReLU, however, is formulated as:

Leaky ReLU allows very small, non-zero gradients when the unit isn&apos...