An autoencoder is a type of neural network that is trained to attempt to copy its input to its output. It has a hidden layer (let's call it h) that describes a code used to represent the input. The network may be viewed as consisting of two parts:
- Encoder function: h = f (x)
- Decoder that produces a reconstruction: r = g(h)
The following figure shows a basic autoencoder with input n and a hidden layer with neurons m:
Basic representation of an autoencoder
Autoencoders are designed to be unable to learn to copy perfectly. They are restricted in ways that allow them to copy only approximately, and to copy only input that resembles the training data. As the model is forced to prioritize which aspects of the input should be copied, it often learns useful properties of the data.
The following topics will be covered in this chapter:
- Autoencoder algorithms
- Under-complete autoencoders
- Basic autoencoders
- Additive Gaussian Noise autoencoders
- Sparse autoencoders