Book Image

Big Data Analytics with R

By : Simon Walkowiak
Book Image

Big Data Analytics with R

By: Simon Walkowiak

Overview of this book

Big Data analytics is the process of examining large and complex data sets that often exceed the computational capabilities. R is a leading programming language of data science, consisting of powerful functions to tackle all problems related to Big Data processing. The book will begin with a brief introduction to the Big Data world and its current industry standards. With introduction to the R language and presenting its development, structure, applications in real world, and its shortcomings. Book will progress towards revision of major R functions for data management and transformations. Readers will be introduce to Cloud based Big Data solutions (e.g. Amazon EC2 instances and Amazon RDS, Microsoft Azure and its HDInsight clusters) and also provide guidance on R connectivity with relational and non-relational databases such as MongoDB and HBase etc. It will further expand to include Big Data tools such as Apache Hadoop ecosystem, HDFS and MapReduce frameworks. Also other R compatible tools such as Apache Spark, its machine learning library Spark MLlib, as well as H2O.
Table of Contents (16 chapters)
Big Data Analytics with R
Credits
About the Author
Acknowledgement
About the Reviewers
www.PacktPub.com
Preface

GLM example with Spark and R on the HDInsight cluster


In the first practical example of this chapter we will use the HDInsight cluster with Spark and Hadoop, and run a Generalized Linear Model (GLM) on the flight data available for you to download from the Packt Publishing website created for this book.

Preparing the Spark cluster and reading the data from HDFS

Before carrying out any analytics on the data, let's firstly double-check whether you have all of the required resources in place. In this tutorial, we will be using the same multi-node HDInsight cluster that you previously deployed following the instructions in Chapter 7, Faster than Hadoop: Spark with R and specifically the section on Launching HDInsight with Spark and R/RStudio. If you don't remember how to launch the HDInsight cluster on Microsoft Azure, detailed step-by-step guidelines have been provided earlier in the HDInsight - A multi-node Hadoop cluster on Azure section in Chapter 4, Hadoop and MapReduce Framework for R. As...