Training the NMT
Now that we have defined the NMT architecture and preprocessed the training data, it is quite straightforward to train the model. Here, we will define and illustrate (see Figure 9.15) the exact process used for training:
Figure 9.15: The training procedure for NMT
For the model training, we’re going to define a custom training loop, as there is a special metric we’d like to track. Unfortunately, this metric is not a readily available TensorFlow metric. But before that, there are several utility functions we need to define:
def prepare_data(de_lookup_layer, train_xy, valid_xy, test_xy):
""" Create a data dictionary from the dataframes containing data
"""
data_dict = {}
for label, data_xy in zip(['train', 'valid', 'test'], [train_xy,
valid_xy, test_xy]):
data_x, data_y = data_xy
en_inputs = data_x
de_inputs = data_y[...