In statistics today, there are two schools of thought as to how we interpret data and make statistical inferences. The classic and more dominant approach to date has been what is termed the Frequentist approach (refer to Chapter 7, A Tour of Statistics – The Classical Approach), while we are looking at the Bayesian approach in this chapter.
At the heart of the debate between the Bayesian and Frequentist worldview is the question—how do we define probability?
In the Frequentist worldview, probability is a notion that is derived from the frequencies of repeated events. For example, when we define the probability of getting heads when a fair coin is tossed as being equal to half. This is because when we repeatedly toss a fair coin, the number of heads divided by the total number of coin tosses approaches 0.5 when the number of coin tosses is sufficiently large.
The Bayesian worldview is different, and the notion of probability...