Book Image

Hands-On Ensemble Learning with R

By : Prabhanjan Narayanachar Tattar
Book Image

Hands-On Ensemble Learning with R

By: Prabhanjan Narayanachar Tattar

Overview of this book

Ensemble techniques are used for combining two or more similar or dissimilar machine learning algorithms to create a stronger model. Such a model delivers superior prediction power and can give your datasets a boost in accuracy. Hands-On Ensemble Learning with R begins with the important statistical resampling methods. You will then walk through the central trilogy of ensemble techniques – bagging, random forest, and boosting – then you'll learn how they can be used to provide greater accuracy on large datasets using popular R packages. You will learn how to combine model predictions using different machine learning algorithms to build ensemble models. In addition to this, you will explore how to improve the performance of your ensemble models. By the end of this book, you will have learned how machine learning algorithms can be combined to reduce common problems and build simple efficient ensemble models with the help of real-world examples.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Ensemble Learning with R
Prabhanjan Narayanachar Tattar
Jul, 2018 | 376 pages
Small book image
Statistical Application Development with R and Python
Prabhanjan Narayanachar Tattar
Aug, 2017 | 432 pages
Small book image
Machine Learning with R Cookbook
YuWei Chiu David Chiu | AshishSingh Bhatia
Oct, 2017 | 572 pages
Small book image
Practical Data Science Cookbook, Second Edition
Anthony Ojeda | ABHIJIT DASGUPTA | Sean Patrick Murphy | Bhushan Purushottam Joshi | Prabhanjan Narayanachar Tattar
Jun, 2017 | 434 pages
Table of Contents (17 chapters)
Hands-On Ensemble Learning with R
Hands-On Ensemble Learning with R
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to Ensemble Techniques
Introduction to Ensemble Techniques
Datasets
Statistical/machine learning models
The right model dilemma!
An ensemble purview
Complementary statistical tests
Summary
2
Bootstrapping
Bootstrapping
Technical requirements
The jackknife technique
Bootstrap – a statistical method
The boot package
Bootstrap and testing hypotheses
Bootstrapping regression models
Bootstrapping survival models*
Bootstrapping time series models*
Summary
3
Bagging
Bagging
Technical requirements
Classification trees and pruning
Bagging
k-NN classifier
k-NN bagging
Summary
4
Random Forests
Random Forests
Technical requirements
Random Forests
Variable importance
Proximity plots
Random Forest nuances
Comparisons with bagging
Missing data imputation
Clustering with Random Forest
Summary
5
The Bare Bones Boosting Algorithms
The Bare Bones Boosting Algorithms
Technical requirements
The general boosting algorithm
Adaptive boosting
Gradient boosting
Using the adabag and gbm packages
Variable importance
Comparing bagging, random forests, and boosting
Summary
6
Boosting Refinements
Boosting Refinements
Technical requirements
Why does boosting work?
The gbm package
The xgboost package
The h2o package
Summary
7
The General Ensemble Technique
The General Ensemble Technique
Technical requirements
Why does ensembling work?
Ensembling by voting
Ensembling by averaging
Stack ensembling
Summary
8
Ensemble Diagnostics
Ensemble Diagnostics
Technical requirements
What is ensemble diagnostics?
Ensemble diversity
Pairwise measure
Interrating agreement
Summary
9
Ensembling Regression Models
Ensembling Regression Models
Technical requirements
Pre-processing the housing data
Visualization and variable reduction
Regression models
Bagging and Random Forests
Boosting regression models
Stacking methods for regression models
Summary
10
Ensembling Survival Models
Ensembling Survival Models
Core concepts of survival analysis
Nonparametric inference
Regression models – parametric and Cox proportional hazards models
Survival tree
Ensemble survival models
Summary
11
Ensembling Time Series Models
Ensembling Time Series Models
Technical requirements
Time series datasets
Time series visualization
Core concepts and metrics
Essential time series models
Bagging and time series
Ensemble time series models
Summary
12
What's Next?
What's Next?
Bibliography
Bibliography
References
R package references
Index
Index

References

Abraham, B. and Ledolter, J. (https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316610), 1983. Statistical methods for forecasting. J. Wiley

Andersen, P.K., Klein, J.P. and Rosthøj, S., (https://doi.org/10.1093/biomet/90.1.15) 2003. Generalised linear models for correlated pseudo-observations, with applications to multi-state models. Biometrika, 90(1), pp.15-27.

Berk, R.A., (https://www.springer.com/in/book/9783319440477) 2016. Statistical learning from a regression perspective, Second Edition. New York: Springer.

Bou-Hamad, I., Larocque, D. and Ben-Ameur, H., (https://projecteuclid.org/euclid.ssu/1315833185) 2011. A review of survival trees. Statistics Surveys, 5, pp.44-71.

Box, G.E., Jenkins, G.M., Reinsel, G.C. and Ljung, G.M., (https://onlinelibrary.wiley.com/doi/book/10.1002/9781118619193)2015. Time series analysis: forecasting and control. John Wiley & Sons.

Breiman, L., (https://link.springer.com/article/10.1007/BF00058655) 1996. Bagging predictors. Machine learning, 24(2), pp.123-140.

Breiman, L., Friedman, J.H., Olshen, R.A. and Stone, C.J., (https://www.taylorfrancis.com/books/9781351460491) 1984. Classification and regression trees. Routledge.

Broemeling, L.D., (https://www.crcpress.com/Bayesian-Methods-for-Measures-of-Agreement/Broemeling/p/book/9781420083415)2009. Bayesian methods for measures of agreement. Chapman and Hall/CRC.

Bühlmann, P. and Van De Geer, S., (https://www.springer.com/in/book/9783642201912) 2011. Statistics for high-dimensional data: methods, theory and applications. Springer Science & Business Media.

Chatterjee, S. and Hadi, A.S., (https://www.wiley.com/en-us/Regression+Analysis+by+Example%2C+5th+Edition-p-9780470905845) 2012. Regression Analysis by Example, Fifth edition. John Wiley & Sons.

Ciaburro, G., (https://www.packtpub.com/big-data-and-business-intelligence/regression-analysis-r) 2018. Regression Analysis with R, Packt Publishing Ltd.

Cleveland, R.B., Cleveland, W.S., McRae, J.E. and Terpenning, I., (http://www.nniiem.ru/file/news/2016/stl-statistical-model.pdf) 1990. STL: A Seasonal-Trend Decomposition. Journal of Official Statistics, 6(1), pp.3-73.

Cox, D.R., (https://eclass.uoa.gr/modules/document/file.php/MATH394/Papers/%5BCox(1972)%5D Regression Models and Life Tables.pdf) 1972. Regression models and life-tables. Journal of the Royal Statistical Society. Series B (Methodological), 34, pp. 187-220

Cox, D.R., (https://academic.oup.com/biomet/article-abstract/62/2/269/337051) 1975. Partial likelihood. Biometrika, 62(2), pp.269-276.

Dixit, A., (https://www.packtpub.com/big-data-and-business-intelligence/ensemble-machine-learning)2017. Ensemble Machine Learning: A beginner's guide that combines powerful machine learning algorithms to build optimized models. Packt Publishing Ltd.

Draper, N.R. and Smith, H., (https://onlinelibrary.wiley.com/doi/book/10.1002/9781118625590)1999/2014. Applied regression analysis, (Vol. 326). John Wiley & Sons.

Efron, B. (https://projecteuclid.org/download/pdf_1/euclid.aos/1176344552) 1979. Bootstrap methods (https://link.springer.com/chapter/10.1007/978-1-4612-4380-9_41): another look at the jackknife, The Annals of Statistics, 7, 1-26.

Efron, B. and Hastie, T., 2016. (https://web.stanford.edu/~hastie/CASI_files/PDF/casi.pdf) Computer age statistical inference (Vol. 5). Cambridge University Press.

Efron, B. and Tibshirani, R.J., (https://www.crcpress.com/An-Introduction-to-the-Bootstrap/Efron-Tibshirani/p/book/9780412042317) 1994. An introduction to the bootstrap. CRC press.

Friedman, J.H., Hastie, T. and Tibshirani, R. (https://projecteuclid.org/download/pdf_1/euclid.aos/1016218223)2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5):1189–1232.

Gordon, L. and Olshen, R.A., (https://europepmc.org/abstract/med/4042086)1985. Tree-structured survival analysis. Cancer treatment reports, 69(10), pp.1065-1069.

Hastie, T., Tibshirani, R., and Friedman, J. (https://www.springer.com/in/book/9780387848570) ,2009., The Elements of Statistical Learning, Second Edition, Springer.

Haykin, S.S, 2009. (https://www.pearson.com/us/higher-education/program/Haykin-Neural-Networks-and-Learning-Machines-3rd-Edition/PGM320370.html) Neural networks and learning machines (Vol. 3). Upper Saddle River, NJ, USA:: Pearson.

Kalbfleisch, J.D. and Prentice, R.L. (https://onlinelibrary.wiley.com/doi/abs/10.2307/3315078), 2002. The statistical analysis of failure time data. John Wiley & Sons.

Kuncheva, L.I., (https://www.wiley.com/en-us/Combining+Pattern+Classifiers%3A+Methods+and+Algorithms%2C+2nd+Edition-p-9781118315231) 2014. Combining pattern classifiers: methods and algorithms. Second Edition. John Wiley & Sons.

LeBlanc, M. and Crowley, J., (https://www.jstor.org/stable/2532300)1992. Relative risk trees for censored survival data. Biometrics, pp.411-425.

Lee, S.S. and Elder, J.F., (http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=6B151AAB29C69A4D4C35C8C4BBFC67F5?doi=10.1.1.34.1753&rep=rep1&type=pdf) 1997. Bundling heterogeneous classifiers with advisor perceptrons. White Paper.

Mardia, K. , Kent, J., and Bibby, J.M.., (https://www.elsevier.com/books/multivariate-analysis/mardia/978-0-08-057047-1) 1979. Multivariate analysis. Academic Press.

Montgomery, D.C., Peck, E.A. and Vining, G.G., (https://www.wiley.com/en-us/Introduction+to+Linear+Regression+Analysis%2C+5th+Edition-p-9781118627365) 2012. Introduction to linear regression analysis (Vol. 821). John Wiley & Sons.

Perrone, M.P., and Cooper, L.N., (https://www.worldscientific.com/doi/abs/10.1142/9789812795885_0025)1993. When Networks Disagree: Ensemble Methods for Hybrid Neural Networks. In Mammone, R.J. (Ed.), Neural Networks for Speech and Image Processing. Chapman Hall.

Ripley, B.D., (http://admin.cambridge.org/fk/academic/subjects/statistics-probability/computational-statistics-machine-learning-and-information-sc/pattern-recognition-and-neural-networks)2007. Pattern recognition and neural networks. Cambridge university press.

Quenouille, M.H., (https://www.cambridge.org/core/journals/mathematical-proceedings-of-the-cambridge-philosophical-society/article/approximate-tests-of-correlation-in-timeseries-3/F6D24B2A8574F1716E44BE788696F9C7) 1949, July. Approximate tests of correlation in time-series 3. In Mathematical Proceedings of the Cambridge Philosophical Society (Vol. 45, No. 3, pp. 483-484). Cambridge University Press.

Quinlan, J. R. (1993), (https://www.elsevier.com/books/c45/quinlan/978-0-08-050058-4) C4.5: Programs for Machine Learning, Morgan Kaufmann.

Ridgeway, G., Madigan, D. and Richardson, T., (http://dimacs.rutgers.edu/archive/Research/MMS/PAPERS/BNBR.pdf) 1999, January. Boosting methodology for regression problems. In AISTATS.

Schapire, R.E. and Freund, Y., (http://dimacs.rutgers.edu/archive/Research/MMS/PAPERS/BNBR.pdf) 2012. Boosting: Foundations and algorithms. MIT press.

Seni, G. and Elder, J.F., (https://www.morganclaypool.com/doi/abs/10.2200/S00240ED1V01Y200912DMK002)2010. Ensemble methods in data mining: improving accuracy through combining predictions. Synthesis Lectures on Data Mining and Knowledge Discovery, 2(1), pp.1-126.

Tattar, P.N., Ramaiah, S. and Manjunath, B.G., (https://onlinelibrary.wiley.com/doi/book/10.1002/9781119152743) 2016. A Course in Statistics with R. John Wiley & Sons.

Tattar, P.N., 2017. (https://www.packtpub.com/big-data-and-business-intelligence/statistical-application-development-r-and-python-second-edition) Statistical Application Development with R and Python. Packt Publishing Ltd.

Tattar, P., Ojeda, T., Murphy, S.P., Bengfort, B. and Dasgupta, A., (https://www.packtpub.com/big-data-and-business-intelligence/practical-data-science-cookbook-second-edition) 2017. Practical Data Science Cookbook. Packt Publishing Ltd.

Zhang, H. and Singer, B.H., (https://www.springer.com/in/book/9781441968234)2010. Recursive partitioning and applications. Springer Science & Business Media.

Zemel, R.S. and Pitassi, T., (http://papers.nips.cc/paper/1797-a-gradient-based-boosting-algorithm-for-regression-problems.pdf )2001. A gradient-based boosting algorithm for regression problems. In Advances in neural information processing systems (pp. 696-702).

Zhou, Z.H., (https://www.crcpress.com/Ensemble-Methods-Foundations-and-Algorithms/Zhou/p/book/9781439830031)2012. Ensemble methods: foundations and algorithms. Chapman and Hall/CRC.

Previous Section
End of Section
Next Section