You can refer to the following links for more insight:
The Unreasonable Effectiveness of Recurrent Neural Networks, Andrej Karpathy May 21, 2015 (http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
Understanding LSTM Networks on Christopher Colah's blog's, 2015 (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)
Use of LSTM for audio classification: Connectionist Temporal Classification and Deep Speech: Scaling up end-to-end speech recognition (https://arxiv.org/abs/1412.5567)
Handwriting demo at http://www.cs.toronto.edu/~graves/handwriting.html
General Sequence Learning using Recurrent Neural Networks tutorial at https://www.youtube.com/watch?v=VINCQghQRuM
On the difficulty of training Recurrent Neural Networks Razvan Pascanu, Tomas Mikolov, Yoshua Bengio 2012
Recurrent Neural Networks Tutorial:
Introduction to RNNS
Implementing RNN with Python, NumPy, and Theano
Backpropagation through time and vanishing gradients
Implementing a GRU/LSTM RNN with Python and Theano...