Going deep with deep learning
In this book, we have worked with relatively “classical” ML models so far, which rely on a variety of different mathematical and statistical approaches to learn from data. These algorithms in general are not modeled on any biological theory of learning and are at their heart motivated by finding procedures to explicitly optimize the loss function in different ways. A slightly different approach that the reader will likely be aware of, and that we met briefly in the section on Learning about learning in Chapter 3, From Model to Model Factory, is that taken by Artificial Neural Networks (ANNs), which originated in the 1950s and were based on idealized models of neuronal activity in the brain. The core concept of an ANN is that through connecting relatively simple computational units called neurons or nodes (modeled on biological neurons), we can build systems that can effectively model any mathematical function (see the information box below...