Support Vector Machines (SVMs) is arguably the most used ML technique for classification. The main idea behind SVM is that we find an optimal hyperplane with maximum margin separating the two classes. If the data is linearly separable, the process of finding the hyperplane is straightforward, but if it isn't linearly separable, then kernel trick is used to make the data linearly separable in some transformed high-dimensional feature space.
SVM is considered a non-parametric supervised learning algorithm. The main idea of SVM is to find a maximal margin separator: a separating hyperplane that is farthest from the training samples presented.
Consider the following diagram; the red dots represent class 1 for which the output should be 1, and the blue dots represent the class 2 for which the output should be -1. There can be many lines which can separate the red dots from the blue ones; the diagram demonstrates three such lines: A, B, and C respectively...