k-nearest neighbours (kNN) is considered one of the simplest algorithms in the category of supervised learning. kNN can be used for both classification and regression problems. In the training phase, kNN stores both the feature vectors and class labels of all of the training samples. In the classification phase, an unlabeled vector (a query or test vector in the same multidimensional feature space as the training examples) is classified as the class label that is most frequent among the k training samples nearest to the unlabeled vector to be classified, where k is a user-defined constant.
This can be seen graphically in the next diagram:
In the previous diagram, if k = 3, the green circle (the unlabeled test sample) will be classified as a triangle because there are two triangles and only one square inside the inner circle. If k = 5, the green circle will...