The crux of our effort will be the training, which is shown in the second file we encountered earlier—translate.py. The prepare_wmt_dataset function we reviewed earlier is, of course, the starting point as it creates our two datasets and tokenizes them into nice clean numbers.
The training starts as follows:
After preparing the data, we will create a TensorFlow session, as usual, and construct our model. We'll get to the model later; for now, let's look at our preparation and training loop.
We will define a dev set and a training set later, but for now, we will define a scale that is a floating point score ranging from 0 to 1. Nothing complex here; the real work comes in the following training loop. This is very different from what we've done in previous chapters, so close attention is required.
Our main training loop is seeking to minimize...