Feedforward networks, such as densely connected networks, have no memory and treat each input as a whole. For example, an image input represented as a vector of pixels gets processed by a feedforward network in one single step. But time series data, such as stock prices for the last 10 or 20 days, are better processed with a network with memory; assume the prices of the past 10 days are X1, X2, ..., X10, with X1 being the oldest and X10 the latest, then all 10-day prices can be treated as one sequence input, and when RNN processes such an input, the following steps occur:
- A specific RNN cell, connected to the first element, X1, in the sequence, processes X1 and gets its output, y1
- Another RNN cell, connected to the next element, X2, in the sequence input, uses X2, as well as the previous output, y1, to get the next output,...