The first generative model we will look at is the autoencoder model. An autoencoder is a simple neural network that is composed of two parts: an encoder and a decoder. The idea is that the encoder part will compress your input into a smaller dimension. From this smaller dimension, it then tries to reconstruct the input using the decoder part of the model. This smaller dimension is often called by many names such as latent space, hidden space, an embedding, or a coding.
If the autoencoder is able to reproduce its input, then, in theory, this latent space should encode all the important information needed to represent the original data, but with the advantage of being a smaller dimension than the input. The encoder can be thought of as a way of compressing the input data while the decoder is the way to uncompress it. We can see what a simple autoencoder looks like in the following illustration. Our latent space or coding is the part in the middle labeled z.
Traditionally, in an...