Unique challenges of generative models

Given the powerful applications that generative models have, what are the major challenges in implementing them? As described, most of these models utilize complex data, requiring us to fit large models to capture all the nuances of their features and distribution. This has implications both for the number of examples that we must collect to adequately represent the kind of data we are trying to generate, and the computational resources needed to build the model. We will discuss techniques in Chapter 2, Setting up a TensorFlow Lab, to parallelize the training of these models using cloud computing frameworks and graphics processing units (GPUs).

A more subtle problem that comes from having complex data, and the fact that we are trying to generate data rather than a numerical label or value, is that our notion of model accuracy is much more complicated: we cannot simply calculate the distance to a single label or scores.

We will discuss...