Introduction to Generative AI: Drawing Data from Models

At the Colorado State Fair in 2022, the winning entry was a fantastical sci-fi landscape created by video game designer Jason Allen titled Théâtre D’opéra Spatial (Figure 1.1). The first-prize art was remarkable both from the dramatic subject matter and due to the unusual origin of this image. Unlike the majority of other artworks entered into the competition, Théâtre D’opéra Spatial was not painted using oil or watercolors, nor was its “creator” even human; rather, it is an entirely digital image produced by a sophisticated machine learning algorithm called Midjourney. Jason used Midjourney, which has been trained on diverse images, along with natural language instructions to create the image, rather than a brush and canvas.

Figure 1.1: Théâtre D’opéra Spatial1

Visual art is far from the only area in which machine learning has demonstrated astonishing results. Indeed, if you have paid attention to the news in the last few years, you have likely seen many stories about the groundbreaking results of modern AI systems applied to diverse problems, from the hard sciences to online avatars and interactive chat. Deep neural network models, such as the one powering Midjourney, have shown amazing abilities to generate realistic human language2, author computer code3, and solve school exams with human-level ability2. Such models can also classify X-ray images of human anatomy on the level of trained physicians4, beat human masters at both classic board games such as Go (an Asian form of chess) as well as multiplayer computer games5, 6, and translate French into English with amazing sensitivity to grammatical nuances7.