Book Image

Hands-On Ensemble Learning with R

By : Prabhanjan Narayanachar Tattar
Book Image

Hands-On Ensemble Learning with R

By: Prabhanjan Narayanachar Tattar

Overview of this book

Ensemble techniques are used for combining two or more similar or dissimilar machine learning algorithms to create a stronger model. Such a model delivers superior prediction power and can give your datasets a boost in accuracy. Hands-On Ensemble Learning with R begins with the important statistical resampling methods. You will then walk through the central trilogy of ensemble techniques – bagging, random forest, and boosting – then you'll learn how they can be used to provide greater accuracy on large datasets using popular R packages. You will learn how to combine model predictions using different machine learning algorithms to build ensemble models. In addition to this, you will explore how to improve the performance of your ensemble models. By the end of this book, you will have learned how machine learning algorithms can be combined to reduce common problems and build simple efficient ensemble models with the help of real-world examples.
Table of Contents (17 chapters)
Hands-On Ensemble Learning with R
Contributors
Preface
12
What's Next?
Index

k-NN classifier


In Chapter 1, Introduction to Ensemble Techniques, we became familiar with a variety of classification models. Some readers might already be familiar with the k-NN model. The k-NN classifier is one of the most simple, intuitive, and non-assumptive models. The name of the model itself suggests how it might be working - nearest neighborhoods! And that's preceded by k! Thus, if we have N points in a study, we find the k-nearest points in neighborhood, and then make a note of the class of the k-neighbors. The majority class of the k-neighbors is then assigned to the unit. In case of regression, the average of the neighbors is assigned to the unit. The following is a visual depiction of k-NN:

Figure 4: Visual depiction of k-NN

The top left part of the visual depiction of k-NN shows the scatterplot of 27 observations, 16 of which are circles and the remaining 11 are squares. The circles are marked in orange while the squares are marked in blue . Suppose we choose to set up a classifier...