In this section, we'll explore a popular deep learning model: the recurrent neural network (RNN), and how it can be used in the generation of sequence data. The universal way to create sequence data in deep learning is to train a model (usually a RNN or a ConvNet) to predict the next token or next few tokens in a series, based on the previous tokens as input. For instance, let's imagine that we're given the sentence with these words as input: I love to work in deep learning. We will train the network to predict the next character as our target.
When working with textual data, tokens are typically words or characters, and any network that can model the probability of the next token given the previous ones is called a language model that can capture the latent space of language.
Upon training the language model, we can then proceed to feed...