Book Image

Hands-On Ensemble Learning with R

By : Prabhanjan Narayanachar Tattar
Book Image

Hands-On Ensemble Learning with R

By: Prabhanjan Narayanachar Tattar

Overview of this book

Ensemble techniques are used for combining two or more similar or dissimilar machine learning algorithms to create a stronger model. Such a model delivers superior prediction power and can give your datasets a boost in accuracy. Hands-On Ensemble Learning with R begins with the important statistical resampling methods. You will then walk through the central trilogy of ensemble techniques – bagging, random forest, and boosting – then you'll learn how they can be used to provide greater accuracy on large datasets using popular R packages. You will learn how to combine model predictions using different machine learning algorithms to build ensemble models. In addition to this, you will explore how to improve the performance of your ensemble models. By the end of this book, you will have learned how machine learning algorithms can be combined to reduce common problems and build simple efficient ensemble models with the help of real-world examples.
Table of Contents (17 chapters)
Hands-On Ensemble Learning with R
Contributors
Preface
12
What's Next?
Index

Variable importance


Boosting methods essentially use trees as base learners, and hence the idea of variable importance gets carried over here the same as with trees, bagging, and random forests. We simply add the importance of the variables across the trees as we do with bagging or random forests.

For a boosting fitted object from the adabag package, the variable importance is extracted as follows:

> AB1$importance
 x1  x2 
100   0 

This means that the boosting method has not used the x2 variable at all. For the gradient boosting objects, the importance is given by the summary function:

> summary(sin_gbm)
  var rel.inf
x   x     100

It is now apparent that we only have one variable and so it is important to explain the regressand and we certainly did not require some software to tell us. Of course, it is useful in complex cases. Comparisons are for different ensembling methods based on trees. Let us move on to the next section.