We have learned enough about RNNs; now, we will look at how to generate song lyrics using RNNs. To do this, we simply build a character-level RNN, meaning that on every time step, we predict a new character.
Let's consider a small sentence, What a beautiful d.
At the first time step, the RNN predicts a new character as a. The sentence will be updated to, What a beautiful da.
At the next time step, it predicts a new character as y, and the sentence becomes, What a beautiful day.
In this manner, we predict a new character at each time step and generate a song. Instead of predicting a new character every time, we can also predict a new word every time, which is called word level RNN. For simplicity, let's start with a character level RNN.
But how does RNN predicts a new character on each time step? Let's suppose at a time step t=0...