The simplest way to find out an approximation of the probability density function is based on a frequency count. If we have a dataset X containing m samples xi ∈ ℜ (for simplicity, we are considering only univariate distributions, but the process is exactly equivalent for multidimensional samples), we can define m and M as follows:
The interval (m, M) can be split into a fixed number b of bins (which can have either the same or different widths denoted as w(bj) so that np(bj) corresponds to the number of samples included into the bin bj. At this point, given a test sample xt, it's easy to understand that the approximation of the probability can be easily obtained by detecting the bin containing xt and using the following formula:
Before analyzing the pros and cons of this approach, let's consider a simple example based on the distribution of...