In Chapter 2, Generating Drum Sequences with the Drums RNN, we saw how we can use an RNN (LSTM) and a beam search to iteratively generate a sequence, by taking an input and then predicting, note by note, which next note is the most probable. That enabled us to use a primer as a basis for the generation, using it to set a starting melody or a certain key.
Using that technique is useful, but it has its limitations. What if we wanted to start with a primer and explore variations around it, and not just in a random way, but in a desired specific direction? For example, we could have a two-bars melody for a bass line, and we would like to hear how it sounds when played more as an arpeggio. Another example would be transitioning smoothly between two melodies. This is where the RNN models we previously saw fall short and where VAEs comes into play.
Before...