Naïve Bayes as a machine learning algorithm
The key idea behind the Naïve Bayes algorithm is that you can estimate the likelihood of some outcome given a set of observations by using conditional probabilities and linking the individual observations to the outcome. Defining what conditional probability is turns out to be surprisingly slippery because the notion of probability itself is very slippery. Probabilities are often defined as something similar to proportions, but this view becomes difficult to maintain when you are looking at unique or unbounded sets, which is usually the case when you want to make use of them.
Suppose, for instance, that I am trying to work out how likely it is that France will win the FIFA 2022 World Cup (this is being written 2 days before the final, between France and Argentina, is to be played). In some sense, it is reasonable to ask about this probability – if the bookmakers are offering 3 to 1 against France and the probability that...