We have seen how to implement a feed-forward neural network (ffnn) architecture for an image classification problem.
An ffnn is characterized by a set of input units, a set of output units, and one or more hidden units that connect the input level from that output. The connections between the levels are total and in a single direction: each unit receives a signal from all the units of the previous layer and transmits its output value, suitably weighed to all units of the next layer. For each layer a transfer function (sigmoid, softmax, ReLU) must be defined: the choice of the transfer function depends on the architecture and then the addressed problem.
Then we implemented four different ffnn models, the first model with a single hidden layer with softmax activation function, and then three other more complex models, with five hidden layers in total, but with different activation functions:
- Four sigmoid...