When you first encounter a variety of architectures of CNNs, you feel overwhelmed by the abundance of new terms, different layers, and their hyperparameters. In fact, at the moment, only a few architectures have found broad application, and the number of designs suitable for mobile development is even smaller.
There are five basic types of layers plus an input layer, which usually does nothing except passing data forward:
- Input layer: The first layer in the neural network. It does nothing, only takes the input and passes it downstream.
- Convolution layers: Where convolutions happen
- Fully: Connected or dense layers
- Nonlinearity layers: These are layers which apply activation functions to the output of the previous layer: sigmoid, ReLU, tanh, softmax and so on.
- Pooling layers: Downsample their input.
- Regularization layers: layers to fight an overfitting.