Book Image

Python Deep Learning Cookbook

By : Indra den Bakker
Book Image

Python Deep Learning Cookbook

By: Indra den Bakker

Overview of this book

Deep Learning is revolutionizing a wide range of industries. For many applications, deep learning has proven to outperform humans by making faster and more accurate predictions. This book provides a top-down and bottom-up approach to demonstrate deep learning solutions to real-world problems in different areas. These applications include Computer Vision, Natural Language Processing, Time Series, and Robotics. The Python Deep Learning Cookbook presents technical solutions to the issues presented, along with a detailed explanation of the solutions. Furthermore, a discussion on corresponding pros and cons of implementing the proposed solution using one of the popular frameworks like TensorFlow, PyTorch, Keras and CNTK is provided. The book includes recipes that are related to the basic concepts of neural networks. All techniques s, as well as classical networks topologies. The main purpose of this book is to provide Python programmers a detailed list of recipes to apply deep learning to common and not-so-common scenarios.
Table of Contents (21 chapters)
Title Page
About the Author
About the Reviewer
Customer Feedback

Comparing optimizers

In Chapter 2, Feed-Forward Neural Networks, we briefly demonstrated the use of optimizers. Of course, we only used a test set of size 1. As for other machine algorithms, the most used and well-known optimizer for deep learning is Stochastic Gradient Descent (SGD). Other optimizers are variants of SGD that try to speed up convergence by adding heuristics. Also, some optimizers have fewer hyperparameters to tune. The table shown in the Chapter 2Feed-Forward Neural Networks, is an overview of the most commonly used optimizers in deep learning.

One could argue that the choice largely depends on the user's ability to tune the optimizer. There is definitely no ideal solution that works best for all problems. However, some optimizers have fewer parameters to tune and have proven to outperform other optimizers with default settings. In addition to our test in Chapter 2, Feed-Forward Neural Networks, we will perform another test to compare optimizers in the following recipe...