In this chapter, we continue our journey into deep learning with R with autoencoders.
A classical autoencoder consists of three parts:
- An encoding function, which compresses your data
- A decoding function, which reconstructs data from a compressed version
- A metric or distance, which calculates the difference between the information lost by compression on your data
We typically assume that all these involved functions are smooth enough to be able to use backpropagation or other gradient-based methods, although they need not be and we could use derivative-free methods to train them.
Note
Autoencoding is the process of summarizing information from a potentially large feature set into a smaller feature set.
Although the compression bit might remind you of algorithms, such as the MP3 compression algorithm, an important difference is that autoencoders are data specific. An autoencoder trained in pictures of cats and dogs will likely perform poorly in pictures...