In the last chapter, we talked, very briefly, about Bayesian ways of thinking. In short, when speaking about Bayes, you are speaking about the following three things and how they all interact with each other:
- A prior distribution
- A posterior distribution
- A likelihood
Basically, we are concerned with finding the posterior. That's the thing we want to know. Another way to phrase the Bayesian way of thinking is that data shapes and updates our belief. We have a prior probability, or what we naively think about a hypothesis, and then we have a posterior probability, which is what we think about a hypothesis, given some data.
Bayes' theorem is the big result of Bayesian inference. Let's see how it even comes about. Recall that we previously defined the following:
- P(A) = The probability that event A occurs
- P(A|B) = The probability that A occurs, given that B occurred
- P(A, B) = The probability that A and B occur
- P(A, B) = P(A) * P(B|A)
That last...