Let's now return to supervised learning and discuss a family of algorithms known as artificial neural networks. Early studies of neural networks go back to the 1940s when Warren McCulloch and Walter Pitts first described how biological nerve cells (or neurons) in the brain might work. More recently, artificial neural networks have seen a revival under the buzzword deep learning, which powers state-of-the-art technologies such as Google's DeepMind and Facebook's DeepFace algorithms.
In this chapter, we want to wrap our heads around some simple versions of artificial neural networks, such as the McCulloch-Pitts neuron, the perceptron, and the multilayer perceptron. Once we have familiarized ourselves with the basics, we will be ready to implement a more sophisticated deep neural network to classify handwritten digits...