In this chapter, we provided an overview of RNNs. These are a class of neural networks where the connections between the units form direct cycles, thus giving the possibility to manage temporal and sequential data. We have described the LSTM architecture. The basic idea of this architecture is to improve the RNN providing it with an explicit memory.
LSTM networks are equipped with special hidden units, said memory cells, whose behavior is to remember the previous input for a long time. These cells take in input, at each instant of time, the previous state, and the current input of the network. Combining them with the current contents of memory, and deciding by a gating mechanism by other units what to keep and which to delete things from memory, LSTM have proved very useful and effective learning of long-term dependency.
We have therefore implemented two models of neural networks--the LSTM for a classification...