This chapter will cover a slightly different kind of model to what we have seen so far. All the models presented until now belong to a type of model called a discriminative model. Discriminative models aim to find the boundaries between different classes. They are interested in finding P(Y|X)—the probability of output Y given some input X. This is the natural probability distribution to work with for classification, as you usually want to find a label Y, given some input X.
However, there is another type of model called a generative model. Generative models are built to model the distributions of different classes. They are interested in finding P(Y, X)—the probability distribution of output Y and input X occurring together. In theory, if you can capture the probability distribution of classes in your data, you will know more about it, and you will be able to calculate P(Y|X) using Bayes rule.
Generative...