RNN is a deep learning model architecture specifically designed for sequential data. The purpose of this type of model is to extract relevant features of words and characters of text by using a small window that traverses the corpus.
RNN applies a non-linear function to each item in the sequence. This is called the RNN cell or step and, in our case, the items are words or characters in the sequence. The layer's output in an RNN is derived from the output of the RNN cell, which is applied to each element in the sequence. With regard to NLP and chatbots that use text data as input, the outputs of the model are successive characters or words.
Each RNN cell holds an internal memory that summarizes the history of the sequence it has seen so far.
This diagram helps us to visualize the RNN model architecture:
Vanilla version of RNN model architecture.
At the heart...