Deep learning theory

Here is our plan of attack to go pro and tackle deep learning:

1. The neuron
2. The activation function
3. How do neural networks work?
4. How do neural networks learn?
5. Forward-propagation and back-propagation
6. Gradient descent, including Batch, Stochastic, and Mini-Batch methods

I hope you're excited about this section—deep learning is an awesome and powerful field to study.

The neuron

The neuron is the basic building block of Artificial Neural Networks, and they are based on the neuron cells found the brain.

Biological neurons

In the following images are real-life neurons that have been smeared onto a slide, colored a little bit, and observed through a microscope:

Figure 8: The neuron

As you can see, they have the structure of a central body with lots of different branches coming out of it. The question is: How can we recreate that in a machine? We really want to recreate it in a machine, since the whole...