Understanding Long Short-Term Memory Networks
In this section, we will first explain how an LSTM cell operates. We will see that in addition to the hidden states, a gating mechanism is in place to control information flow inside the cell.
Then we will work through a detailed example and see how gates and states help at various stages of the example to achieve desired behaviors, finally leading to the desired output. Finally, we will compare an LSTM against a standard RNN to learn how an LSTM differs from a standard RNN.
What is an LSTM?
LSTMs can be seen as a more complex and capable family of RNNs. Though LSTMs are a complicated beast, the underlying principles of LSTMs are as same as of RNNs; they process a sequence of items by working on one input at a time in a sequential order. An LSTM is mainly composed of five different components: