In this section, we'll build a language model that operates over characters. For this implementation, we will use an Anna Karenina novel and see how the network will learn to implement the structure and style of the text:
Figure 10: General architecture for the character-level RNN (source: http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
This network is based off of Andrej Karpathy's post on RNNs (link: http://karpathy.github.io/2015/05/21/rnn-effectiveness/) and implementation in Torch (link: https://github.com/karpathy/char-rnn). Also, there's some information here at r2rt (link: http://r2rt.com/recurrent-neural-networks-in-tensorflow-ii.html) and from Sherjil Ozairp (link: https://github.com/sherjilozair/char-rnn-tensorflow) on GitHub. The following is the general architecture of the character-wise RNN.
We&apos...