The anatomy of RNNs
In the previous section, we talked about RNNs’ ability to handle sequential data; let’s drill down into how an RNN does this. The key differentiator between RNNs and feed-forward networks is their internal memory, as shown in Figure 11.1, which enables RNNs to process input sequences while retaining information from previous steps. This attribute empowers RNNs to suitably exploit the temporal dependencies in sequences such as text data.
Figure 11.1 – The anatomy of an RNN
Figure 11.2 shows a clearer picture of an RNN and its inner workings. Here, we can see a series of interconnected units through which data flows in a sequential fashion, one element at a time. As each unit processes the input data, it sends the output to the next unit in a similar fashion to how feed-forward networks work. The key difference lies in the feedback loop, which equips RNNs with the memory of previous inputs, empowering them with the...