R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Data analytics with R has emerged as a very important focus for organizations of all kinds. R enables even those with only an intuitive grasp of the underlying concepts, without a deep mathematical background, to unleash powerful and detailed examinations of their data. This book will show you how you can put your data analysis skills in R to practical use, with recipes catering to the basic as well as advanced data analysis tasks. Right from acquiring your data and preparing it for analysis to the more complex data analysis techniques, the book will show you how you can implement each technique in the best possible manner. You will also visualize your data using the popular R packages like ggplot2 and gain hidden insights from it. Starting with implementing the basic data analysis concepts like handling your data to creating basic plots, you will master the more advanced data analysis techniques like performing cluster analysis, and generating effective analysis reports and visualizations. Throughout the book, you will get to know the common problems and obstacles you might encounter while implementing each of the data analysis techniques in R, with ways to overcoming them in the easiest possible way. By the end of this book, you will have all the knowledge you need to become an expert in data analysis with R, and put your skills to test in real-world scenarios.

Preface

What this book covers

What you need for this book

Free Chapter

Acquire and Prepare the Ingredients - Your Data

Introduction

Working with data

Reading data from CSV files

Reading XML data

Reading JSON data

Reading data from fixed-width formatted files

Reading data from R files and R libraries

Removing cases with missing values

Replacing missing values with the mean

Removing duplicate cases

Rescaling a variable to specified min-max range

Normalizing or standardizing data in a data frame

Binning numerical data

Creating dummies for categorical variables

Handling missing data

Correcting data

Imputing data

Detecting outliers

What's in There - Exploratory Data Analysis

Introduction

Creating standard data summaries

Extracting a subset of a dataset

Splitting a dataset

Creating random data partitions

Generating standard plots, such as histograms, boxplots, and scatterplots

Generating multiple plots on a grid

Creating plots with the lattice package

Creating charts that facilitate comparisons

Creating charts that help to visualize possible causality

Where Does It Belong? Classification

Introduction

Generating error/classification confusion matrices

Principal Component Analysis

Generating receiver operating characteristic charts

Building, plotting, and evaluating with classification trees

Using random forest models for classification

Classifying using the support vector machine approach

Classifying using the Naive Bayes approach

Classifying using the KNN approach

Using neural networks for classification

Classifying using linear discriminant function analysis

Classifying using logistic regression

Text classification for sentiment analysis

Give Me a Number - Regression

Introduction

Computing the root-mean-square error

Building KNN models for regression

Performing linear regression

Performing variable selection in linear regression

Building regression trees

Building random forest models for regression

Using neural networks for regression

Performing k-fold cross-validation

Performing leave-one-out cross-validation to limit overfitting

Can you Simplify That? Data Reduction Techniques

Introduction

Performing cluster analysis using hierarchical clustering

Performing cluster analysis using partitioning clustering

Image segmentation using mini-batch K-means

Partitioning around medoids

Clustering large application

Performing cluster validation

Performing Advance clustering

Model-based clustering with the EM algorithm

Reducing dimensionality with principal component analysis

Lessons from History - Time Series Analysis

Introduction

Exploring finance datasets

Creating and examining date objects

Operating on date objects

Performing preliminary analyses on time series data

Using time series objects

Decomposing time series

Filtering time series data

Smoothing and forecasting using the Holt-Winters method

Building an automated ARIMA model

How does it look? - Advanced data visualization

Introduction

Creating scatter plots

Creating line graphs

Creating bar graphs

Making distributions plots

Creating mosaic graphs

Making treemaps

Plotting a correlations matrix

Creating heatmaps

Plotting network graphs

Labeling and legends

Coloring and themes

Creating multivariate plots

Creating 3D graphs and animation

Selecting a graphics device

This may also interest you - Building Recommendations

Introduction

Building collaborative filtering systems

Performing content-based systems

Building hybrid systems

Performing similarity measures

Application of ML algorithms - image recognition system

Evaluating models and optimization

A practical example - fraud detection system

It's All About Your Connections - Social Network Analysis

Introduction

Downloading social network data using public APIs

Creating adjacency matrices and edge lists

Plotting social network data

Computing important network metrics

Cluster analysis

Force layout

YiFan Hu layout

Put Your Best Foot Forward - Document and Present Your Analysis

Introduction

Generating reports of your data analysis with R Markdown and knitr

Creating interactive web applications with shiny

Creating PDF presentations of your analysis with R presentation

Generating dynamic reports

Work Smarter, Not Harder - Efficient and Elegant R Code

Introduction

Exploiting vectorized operations

Processing entire rows or columns using the apply function

Applying a function to all elements of a collection with lapply and sapply

Applying functions to subsets of a vector

Using the split-apply-combine strategy with plyr

Slicing, dicing, and combining data with data tables

Where in the World? Geospatial Analysis

Introduction

Downloading and plotting a Google map of an area

Overlaying data on the downloaded Google map

Importing ESRI shape files to R

Using the sp package to plot geographic data

Getting maps from the maps package

Creating spatial data frames from regular data frames containing spatial and other data

Creating spatial data frames by combining regular data frames with spatial objects

Adding variables to an existing spatial data frame

Spatial data analysis with R and QGIS

Playing Nice - Connecting to Other Systems

Introduction

Using Java objects in R

Using JRI to call R functions from Java

Using Rserve to call R functions from Java

Executing R scripts from Java

Using the xlsx package to connect to Excel

Reading data from relational databases - MySQL

Reading data from NoSQL databases - MongoDB

Working with in-memory data processing with Apache Spark

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Reading XML data

You may sometimes need to extract data from websites. Many providers also supply data in XML and JSON formats. In this recipe, we learn about reading XML data.

Getting ready

Make sure you have downloaded the files for this chapters and the files cd_catalog.xml and WorldPopulation-wiki.htm are in working directory of R. If the XML package is not already installed in your R environment, install the package now, as follows:

> install.packages("XML")

How to do it...

XML data can be read by following these steps:

Load the library and initialize:

> library(XML) 
> url <- "cd_catalog.xml"

Parse the XML file and get the root node:

> xmldoc <- xmlParse(url) 
> rootNode <- xmlRoot(xmldoc) 
> rootNode[1]

Extract the XML data:

> data <- xmlSApply(rootNode,function(x) xmlSApply(x, xmlValue))

Convert the extracted data into a data frame:

> cd.catalog <- data.frame(t(data),row.names=NULL)

Verify the results:

> cd.catalog[1:2,]

How it works...

The xmlParse function returns an object of the XMLInternalDocument class, which is a C-level internal data structure.

The xmlRoot() function gets access to the root node and its elements. Let us check the first element of the root node:

> rootNode[1] 
 
$CD 
<CD> 
  <TITLE>Empire Burlesque</TITLE> 
  <ARTIST>Bob Dylan</ARTIST> 
  <COUNTRY>USA</COUNTRY> 
  <COMPANY>Columbia</COMPANY> 
  <PRICE>10.90</PRICE> 
  <YEAR>1985</YEAR> 
</CD> 
attr(,"class") 
[1] "XMLInternalNodeList" "XMLNodeList"

To extract data from the root node, we use the xmlSApply() function iteratively over all the children of the root node. The xmlSApply function returns a matrix.

To convert the preceding matrix into a data frame, we transpose the matrix using the t() function and then extract the first two rows from the cd.catalog data frame:

> cd.catalog[1:2,] 
             TITLE       ARTIST COUNTRY     COMPANY PRICE YEAR 
1 Empire Burlesque    Bob Dylan     USA    Columbia 10.90 1985 
2  Hide your heart Bonnie Tyler      UK CBS Records  9.90 1988

There's more...

XML data can be deeply nested and hence can become complex to extract. Knowledge of XPath is helpful to access specific XML tags. R provides several functions, such as xpathSApply and getNodeSet, to locate specific elements.

Extracting HTML table data from a web page

Though it is possible to treat HTML data as a specialized form of XML, R provides specific functions to extract data from HTML tables, as follows:

> url <- "WorldPopulation-wiki.htm" 
> tables <- readHTMLTable(url) 
> world.pop <- tables[[6]]

The readHTMLTable() function parses the web page and returns a list of all the tables that are found on the page. For tables that have an id attribute, the function uses the id attribute as the name of that list element.

We are interested in extracting the "10 most populous countries", which is the fifth table, so we use tables[[6]].

Extracting a single HTML table from a web page

A single table can be extracted using the following command:

> table <- readHTMLTable(url,which=5)

Specify which to get data from a specific table. R returns a data frame.

R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Related Content you might be interested in

Current Title:

R Data Analysis Cookbook - Second Edition

Machine Learning with R Cookbook

Mastering Machine Learning with R

R Programming Fundamentals