The study of kernel-based methods is a very productive and active area of research, with entire books dedicated to the subject. Their popularity relies on some interesting mathematical properties of kernels. For the sake of our current introduction to kernels, we are just going to say that we can use kernels as the basis of flexible non-linear models that also are relatively easy to compute. Two popular kernel-based methods are the support vector machine (SVM) and the Gaussian processes. The later is a probabilistic method and is the topic of this chapter while the former is a non-probabilistic method that we are not going to discuss it here, you can read more about it in the following books Python Data Science Handbook, Jake Vanderplas and Python Machine Learning, Sebastian Raschka. Before discussing Gaussian Processes, let's explore what kernels are and how we can use them.

You may find more than one definition of kernel in the statistical literature, and according to...