Neural networks learn by adjusting the connections between the neurons, namely the weights. As mentioned in the neural structure section, weights represent the neural network knowledge. Different weights cause the network to produce different results for the same inputs. So, a neural network can improve its results by adapting its weights according to a learning rule. The general schema of learning is depicted in the following figure:
The process depicted in the previous figure is called supervised learning because there is a desired output, but neural networks can also learn by the input data, without any desired output (supervision). In Chapter 2, Getting Neural Networks to Learn, we are going to dive deeper into the neural network learning process.