We can prove this theorem by using elementary set theory on the probability spaces of the events A and B. In other words, here, a probability event will be defined as a set of the possible outcomes in the probability space:
Figure 2.1: Probability space for the two events
As you can see in the preceding diagram, we can state the following relationships:
Rearranging these relationships, we get the following:
This is, in fact, Bayes' theorem:
This concludes the proof.
We can extend Bayes' theorem by taking more probability events into consideration. Suppose that the events B1,…,Bn are conditionally independent given A. Let ~A denote the complement of A. Then, we have the following: