This chapter is probably the most theory intensive of this whole book; however, it is required that you have at least an intuitive idea of the building blocks of neural networks and of the various algorithms that are used in machine learning so that you can start developing a meaningful understanding of what's going on.
We have looked at what a neural network is, what it means to train it, and how to perform a parameter update with some of the most common update strategies. You should now have a basic understanding of how the chain rule can be applied in order to compute the gradient of a function efficiently.
We haven't explicitly talked about deep learning, but in practice, that is what we did; keep in mind that stacking layers of neural networks is like stacking different classifiers that combine their expressive power. We indicated this with the term deep...