-
Book Overview & Buying
-
Table Of Contents
Deep Learning with C++
By :
In this chapter we have used a couple of concepts like Activation functions and Optimization techniques without explicitly defining them. In this section, we will digress a little and formally define these two concepts. Learning these concepts and developing an intuitive understanding of the same will be crucial in your deep learning journey.
Throughout this chapter, we have used the ReLU function in our MLP implementations. However, the choice of activation function is far from arbitrary—it can significantly impact your network’s ability to learn and generalize. Activation functions are the mathematical gateways that introduce non-linearity into neural networks, transforming the weighted sum of inputs into meaningful outputs that enable networks to learn complex patterns and relationships in data.
The absence of activation functions would reduce any multi-layer network to a simple linear model...