In the previous chapter, we learned about CNNs and their effectiveness on image- and time series-related tasks that have data with a grid-like structure. We also saw how CNNs are inspired by how the human visual cortex processes visual input. Similarly, the RNNs that we will learn about in this chapter are also biologically inspired.
The need for this form of neural network arises from the fact that fuzzy neural networks (FNNs) are unable to capture time-based dependencies in data.
The first model of an RNN was created by John Hopfield in 1982 in an attempt to understand how associative memory in our brains works. This is known as a Hopfield network. It is a fully connected single-layer recurrent network and it stores and accesses information similarly to how we think our brains do.