Building Blocks of Deep Neural Networks

The wide range of generative AI models that we will implement in this book are all built on the foundation of advances over the last decade in deep learning and neural networks. While in practice we could implement these projects without reference to historical developments, it will give you a richer understanding of how and why these models work to retrace their underlying components. In this chapter, we will dive into this background, showing you how generative AI models are built from the ground up, how smaller units are assembled into complex architectures, how the loss functions in these models are optimized, and some current theories as to why these models are so effective. Armed with this background knowledge, you should be able to understand in greater depth the reasoning behind the more advanced models and topics that start in Chapter 4, Teaching Networks to Generate Digits, of this book. Generally speaking, we can group the building...