In this chapter, we will introduce a new type of neural network. Before, we've only considered neural networks where the information flows through the network in one direction. Next, we will introduce recurrent neural networks. In these networks, data is processed the same way for every element in a sequence, and the output depends on the previous computations. This structure has proven to be very for many applications, such as for natural language processing (NLP) and time series predictions. We will introduce the important building blocks that revolutionized how we process temporal or other forms of sequence data in neural networks.
A simple RNN unit is shown in Figure 4.1:
Figure 4.1: Example of the flow in an RNN unit
As we can see in the figure, the output of a RNN does not only depend on the current input Xt, but also on past inputs (Xt-1). Basically, this gives the network a type of memory. ;
There are multiple types of RNNs where the input and output dimension can differ...