Book Image

Hands-On Ensemble Learning with R

By : Prabhanjan Narayanachar Tattar
Book Image

Hands-On Ensemble Learning with R

By: Prabhanjan Narayanachar Tattar

Overview of this book

Ensemble techniques are used for combining two or more similar or dissimilar machine learning algorithms to create a stronger model. Such a model delivers superior prediction power and can give your datasets a boost in accuracy. Hands-On Ensemble Learning with R begins with the important statistical resampling methods. You will then walk through the central trilogy of ensemble techniques – bagging, random forest, and boosting – then you'll learn how they can be used to provide greater accuracy on large datasets using popular R packages. You will learn how to combine model predictions using different machine learning algorithms to build ensemble models. In addition to this, you will explore how to improve the performance of your ensemble models. By the end of this book, you will have learned how machine learning algorithms can be combined to reduce common problems and build simple efficient ensemble models with the help of real-world examples.
Table of Contents (17 chapters)
Hands-On Ensemble Learning with R
Contributors
Preface
12
What's Next?
Index

Adaptive boosting


Schapire and Freund invented the adaptive boosting method. Adaboost is a popular abbreviation of this technique.

The generic adaptive boosting algorithm is as follows:

  • Initialize the observation weights uniformly:

  • For m, classifier hm, from 1 to m number of passes over with the data, perform the following tasks:

    • Fit a classifier hm to the training data using the weights
    • Compute the error for each classifier as follows:

    • Compute the voting power of the classifier hm:

    • Set
  • Output:

Simply put, the algorithm unfolds as follows:

  1. Initially, we start with uniform weights for all observations.
  2. In the next step, we calculate the weighted error for each of the classifiers under consideration.
  3. A classifier (usually stumps, or decision trees with a single split) needs to be selected and the practice is to select the classifier with the maximum accuracy.

  4. In Improve distribution and Combine outputs case of ties, any accuracy tied classifier is selected.

  5. Next, the misclassified observations...