R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Data analytics with R has emerged as a very important focus for organizations of all kinds. R enables even those with only an intuitive grasp of the underlying concepts, without a deep mathematical background, to unleash powerful and detailed examinations of their data. This book will show you how you can put your data analysis skills in R to practical use, with recipes catering to the basic as well as advanced data analysis tasks. Right from acquiring your data and preparing it for analysis to the more complex data analysis techniques, the book will show you how you can implement each technique in the best possible manner. You will also visualize your data using the popular R packages like ggplot2 and gain hidden insights from it. Starting with implementing the basic data analysis concepts like handling your data to creating basic plots, you will master the more advanced data analysis techniques like performing cluster analysis, and generating effective analysis reports and visualizations. Throughout the book, you will get to know the common problems and obstacles you might encounter while implementing each of the data analysis techniques in R, with ways to overcoming them in the easiest possible way. By the end of this book, you will have all the knowledge you need to become an expert in data analysis with R, and put your skills to test in real-world scenarios.

Preface

What this book covers

What you need for this book

Free Chapter

Acquire and Prepare the Ingredients - Your Data

Introduction

Working with data

Reading data from CSV files

Reading XML data

Reading JSON data

Reading data from fixed-width formatted files

Reading data from R files and R libraries

Removing cases with missing values

Replacing missing values with the mean

Removing duplicate cases

Rescaling a variable to specified min-max range

Normalizing or standardizing data in a data frame

Binning numerical data

Creating dummies for categorical variables

Handling missing data

Correcting data

Imputing data

Detecting outliers

What's in There - Exploratory Data Analysis

Introduction

Creating standard data summaries

Extracting a subset of a dataset

Splitting a dataset

Creating random data partitions

Generating standard plots, such as histograms, boxplots, and scatterplots

Generating multiple plots on a grid

Creating plots with the lattice package

Creating charts that facilitate comparisons

Creating charts that help to visualize possible causality

Where Does It Belong? Classification

Introduction

Generating error/classification confusion matrices

Principal Component Analysis

Generating receiver operating characteristic charts

Building, plotting, and evaluating with classification trees

Using random forest models for classification

Classifying using the support vector machine approach

Classifying using the Naive Bayes approach

Classifying using the KNN approach

Using neural networks for classification

Classifying using linear discriminant function analysis

Classifying using logistic regression

Text classification for sentiment analysis

Give Me a Number - Regression

Introduction

Computing the root-mean-square error

Building KNN models for regression

Performing linear regression

Performing variable selection in linear regression

Building regression trees

Building random forest models for regression

Using neural networks for regression

Performing k-fold cross-validation

Performing leave-one-out cross-validation to limit overfitting

Can you Simplify That? Data Reduction Techniques

Introduction

Performing cluster analysis using hierarchical clustering

Performing cluster analysis using partitioning clustering

Image segmentation using mini-batch K-means

Partitioning around medoids

Clustering large application

Performing cluster validation

Performing Advance clustering

Model-based clustering with the EM algorithm

Reducing dimensionality with principal component analysis

Lessons from History - Time Series Analysis

Introduction

Exploring finance datasets

Creating and examining date objects

Operating on date objects

Performing preliminary analyses on time series data

Using time series objects

Decomposing time series

Filtering time series data

Smoothing and forecasting using the Holt-Winters method

Building an automated ARIMA model

How does it look? - Advanced data visualization

Introduction

Creating scatter plots

Creating line graphs

Creating bar graphs

Making distributions plots

Creating mosaic graphs

Making treemaps

Plotting a correlations matrix

Creating heatmaps

Plotting network graphs

Labeling and legends

Coloring and themes

Creating multivariate plots

Creating 3D graphs and animation

Selecting a graphics device

This may also interest you - Building Recommendations

Introduction

Building collaborative filtering systems

Performing content-based systems

Building hybrid systems

Performing similarity measures

Application of ML algorithms - image recognition system

Evaluating models and optimization

A practical example - fraud detection system

It's All About Your Connections - Social Network Analysis

Introduction

Downloading social network data using public APIs

Creating adjacency matrices and edge lists

Plotting social network data

Computing important network metrics

Cluster analysis

Force layout

YiFan Hu layout

Put Your Best Foot Forward - Document and Present Your Analysis

Introduction

Generating reports of your data analysis with R Markdown and knitr

Creating interactive web applications with shiny

Creating PDF presentations of your analysis with R presentation

Generating dynamic reports

Work Smarter, Not Harder - Efficient and Elegant R Code

Introduction

Exploiting vectorized operations

Processing entire rows or columns using the apply function

Applying a function to all elements of a collection with lapply and sapply

Applying functions to subsets of a vector

Using the split-apply-combine strategy with plyr

Slicing, dicing, and combining data with data tables

Where in the World? Geospatial Analysis

Introduction

Downloading and plotting a Google map of an area

Overlaying data on the downloaded Google map

Importing ESRI shape files to R

Using the sp package to plot geographic data

Getting maps from the maps package

Creating spatial data frames from regular data frames containing spatial and other data

Creating spatial data frames by combining regular data frames with spatial objects

Adding variables to an existing spatial data frame

Spatial data analysis with R and QGIS

Playing Nice - Connecting to Other Systems

Introduction

Using Java objects in R

Using JRI to call R functions from Java

Using Rserve to call R functions from Java

Executing R scripts from Java

Using the xlsx package to connect to Excel

Reading data from relational databases - MySQL

Reading data from NoSQL databases - MongoDB

Working with in-memory data processing with Apache Spark

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Normalizing or standardizing data in a data frame

Distance computations play a big role in many data analytics techniques. We know that variables with higher values tend to dominate distance computations and you may want to use the standardized (or z) values.

Getting ready

Download the BostonHousing.csv data file and store it in your R environment's working directory. Then read the data:

> housing <- read.csv("BostonHousing.csv")

How to do it...

To standardize all the variables in a data frame containing only numeric variables, use:

> housing.z <- scale(housing)

You can only use the scale() function on data frames that contain all numeric variables. Otherwise, you will get an error.

How it works...

When invoked in the preceding example, the scale() function computes the standard z score for each value (ignoring NAs) of each variable. That is, from each value it subtracts the mean and divides the result by the standard deviation of the associated variable.

The scale() function takes two optional arguments, center and scale, whose default values are TRUE. The following table shows the effect of these arguments:

Argument	Effect
`center = TRUE`, `scale = TRUE`	Default behavior described earlier
`center = TRUE`, `scale = FALSE`	From each value, subtract the mean of the concerned variable
`center = FALSE`, `scale = TRUE`	Divide each value by the root mean square of the associated variable, where root mean square is sqrt(sum(x^2)/(n-1))
`center = FALSE`, `scale = FALSE`	Return the original values unchanged

There's more...

When using distance-based techniques, you may need to rescale several variables. You may find it tedious to standardize one variable at a time.

Standardizing several variables simultaneously

If you have a data frame with some numeric and some non-numeric variables, or want to standardize only some of the variables in a fully numeric data frame, then you can either handle each variable separately, which would be cumbersome, or use a function such as the following to handle a subset of variables:

scale.many <- function(dat, column.nos) { 
  nms <- names(dat) 
  for(col in column.nos) { 
    name <- paste(nms[col],".z", sep = "") 
    dat[name] <- scale(dat[,col]) 
  } 
  cat(paste("Scaled ", length(column.nos), " variable(s)n")) 
  dat 
}

With this function, you can now do things like:

> housing <- read.csv("BostonHousing.csv") 
> housing <- scale.many(housing, c(1,3,5:7))

This will add the z values for variables 1, 3, 5, 6, and 7, with .z appended to the original column names:

> names(housing) 
 
[1] "CRIM"    "ZN"      "INDUS"   "CHAS"    "NOX"     "RM" 
[7] "AGE"     "DIS"     "RAD"     "TAX"     "PTRATIO" "B" 
[13] "LSTAT"   "MEDV"    "CRIM.z"  "INDUS.z" "NOX.z"   "RM.z" 
[19] "AGE.z"

R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Related Content you might be interested in

Current Title:

R Data Analysis Cookbook - Second Edition

Machine Learning with R Cookbook

Mastering Machine Learning with R

R Programming Fundamentals