LSTM Network is computationally expensive, hence, researchers found an almost equally effective configuration of RNNs, known as Gated Recurrent Unit (GRU) architecture.
In GRU, instead of a working and a long-term memory, only one kind of memory is used, indicated with h (hidden state). The GRU cell adds information to this state memory or removes information from this state memory through reset and update gates.
Following diagram depicts the GRU cell (explanation follows the diagram):
The GRU Cell
The internal flow through the gates in the GRU cell is as follows:
- Update gate u( ): The input
and 
flows to the u( ) gate as per the following equation:
- Reset Gate r( ): The input
and 
flows to the r( ) gate as per the following equation:
- Candidate State Memory: The candidate long-term memory is computed from the output of the r( ) gate,
, and 
, as per the following...