## Artificial neural networks

**Neural networks** are a class of algorithm that was originally designed based on the way that human brains work. However, modern advances are generally based on mathematics rather than biological insights. A neural network is a collection of **neurons** that are connected together. Each neuron is a simple function of its inputs, which generates an output:

The functions that define a neuron's processing can be any standard function, such as a linear combination of the inputs, and are called the **activation function**. For the commonly used learning algorithms to work, we need the activation function to be derivable and smooth. A frequently used activation function is the logistic function, which is defined by the following equation (*k* is often simply 1, *x* is the inputs into the neuron, and *L* is normally 1, that is, the maximum value of the function):

The value of this graph, from -6 to +6, is shown as follows:

The red lines indicate that the value is 0.5 when *x* is zero.

Each...