Book Image

Deep Learning with TensorFlow - Second Edition

By : Giancarlo Zaccone, Md. Rezaul Karim
Book Image

Deep Learning with TensorFlow - Second Edition

By: Giancarlo Zaccone, Md. Rezaul Karim

Overview of this book

Deep learning is a branch of machine learning algorithms based on learning multiple levels of abstraction. Neural networks, which are at the core of deep learning, are being used in predictive analytics, computer vision, natural language processing, time series forecasting, and to perform a myriad of other complex tasks. This book is conceived for developers, data analysts, machine learning practitioners and deep learning enthusiasts who want to build powerful, robust, and accurate predictive models with the power of TensorFlow, combined with other open source Python libraries. Throughout the book, you’ll learn how to develop deep learning applications for machine learning systems using Feedforward Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks, Autoencoders, and Factorization Machines. Discover how to attain deep learning programming on GPU in a distributed way. You'll come away with an in-depth knowledge of machine learning techniques and the skills to apply them to real-world projects.
Table of Contents (15 chapters)
Deep Learning with TensorFlow - Second Edition
Contributors
Preface
Other Books You May Enjoy
Index

Tuning hyperparameters and advanced FFNNs


The flexibility of neural networks is also one of their main drawbacks: there are many hyperparameters to tweak. Even in a simple MLP, you can change the number of layers, the number of neurons per layer, and the type of activation function to use in each layer. You can also change the weight initialization logic, the drop out keep probability, and so on.

Additionally, some common problems in FFNNs, such as the gradient vanishing problem, and selecting the most suitable activation function, learning rate, and optimizer, are of prime importance.

Tuning FFNN hyperparameters

Hyperparameters are parameters that are not directly learned within estimators. It is possible and recommended that you search the hyperparameter space for the best cross-validation (http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation) score. Any parameter provided when constructing an estimator may be optimized in this manner. Now, the question is: how you...