Book Image

Neural Networks with R

By : Balaji Venkateswaran, Giuseppe Ciaburro
Book Image

Neural Networks with R

By: Balaji Venkateswaran, Giuseppe Ciaburro

Overview of this book

Neural networks are one of the most fascinating machine learning models for solving complex computational problems efficiently. Neural networks are used to solve wide range of problems in different areas of AI and machine learning. This book explains the niche aspects of neural networking and provides you with foundation to get started with advanced topics. The book begins with neural network design using the neural net package, then you’ll build a solid foundation knowledge of how a neural network learns from data, and the principles behind it. This book covers various types of neural network including recurrent neural networks and convoluted neural networks. You will not only learn how to train neural networks, but will also explore generalization of these networks. Later we will delve into combining different neural network models and work with the real-world use cases. By the end of this book, you will learn to implement neural network models in your applications with the help of practical examples in the book.
Table of Contents (14 chapters)
Title Page
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Which activation functions to use?


Given that neural networks are to support nonlinearity and more complexity, the activation function to be used has to be robust enough to have the following:

  • It should be differential; we will see why we need differentiation in backpropagation. It should not cause gradients to vanish.
  • It should be simple and fast in processing.
  • It should not be zero centered.

The sigmoid is the most used activation function, but it suffers from the following setbacks:

  • Since it uses logistic model, the computations are time consuming and complex
  • It cause gradients to vanish and no signals pass through the neurons at some point of time
  • It is slow in convergence
  • It is not zero centered

These drawbacks are solved by ReLU. ReLU is simple and is faster to process. It does not have the vanishing gradient problem and has shown vast improvements compared to the sigmoid and tanh functions. ReLU is the most preferred activation function for neural networks and DL problems.

ReLU is used for hidden layers, while the output layer can use a softmax function for logistic problems and a linear function of regression problems.