R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Data analytics with R has emerged as a very important focus for organizations of all kinds. R enables even those with only an intuitive grasp of the underlying concepts, without a deep mathematical background, to unleash powerful and detailed examinations of their data. This book will show you how you can put your data analysis skills in R to practical use, with recipes catering to the basic as well as advanced data analysis tasks. Right from acquiring your data and preparing it for analysis to the more complex data analysis techniques, the book will show you how you can implement each technique in the best possible manner. You will also visualize your data using the popular R packages like ggplot2 and gain hidden insights from it. Starting with implementing the basic data analysis concepts like handling your data to creating basic plots, you will master the more advanced data analysis techniques like performing cluster analysis, and generating effective analysis reports and visualizations. Throughout the book, you will get to know the common problems and obstacles you might encounter while implementing each of the data analysis techniques in R, with ways to overcoming them in the easiest possible way. By the end of this book, you will have all the knowledge you need to become an expert in data analysis with R, and put your skills to test in real-world scenarios.

Preface

What this book covers

What you need for this book

Free Chapter

Acquire and Prepare the Ingredients - Your Data

Introduction

Working with data

Reading data from CSV files

Reading XML data

Reading JSON data

Reading data from fixed-width formatted files

Reading data from R files and R libraries

Removing cases with missing values

Replacing missing values with the mean

Removing duplicate cases

Rescaling a variable to specified min-max range

Normalizing or standardizing data in a data frame

Binning numerical data

Creating dummies for categorical variables

Handling missing data

Correcting data

Imputing data

Detecting outliers

What's in There - Exploratory Data Analysis

Introduction

Creating standard data summaries

Extracting a subset of a dataset

Splitting a dataset

Creating random data partitions

Generating standard plots, such as histograms, boxplots, and scatterplots

Generating multiple plots on a grid

Creating plots with the lattice package

Creating charts that facilitate comparisons

Creating charts that help to visualize possible causality

Where Does It Belong? Classification

Introduction

Generating error/classification confusion matrices

Principal Component Analysis

Generating receiver operating characteristic charts

Building, plotting, and evaluating with classification trees

Using random forest models for classification

Classifying using the support vector machine approach

Classifying using the Naive Bayes approach

Classifying using the KNN approach

Using neural networks for classification

Classifying using linear discriminant function analysis

Classifying using logistic regression

Text classification for sentiment analysis

Give Me a Number - Regression

Introduction

Computing the root-mean-square error

Building KNN models for regression

Performing linear regression

Performing variable selection in linear regression

Building regression trees

Building random forest models for regression

Using neural networks for regression

Performing k-fold cross-validation

Performing leave-one-out cross-validation to limit overfitting

Can you Simplify That? Data Reduction Techniques

Introduction

Performing cluster analysis using hierarchical clustering

Performing cluster analysis using partitioning clustering

Image segmentation using mini-batch K-means

Partitioning around medoids

Clustering large application

Performing cluster validation

Performing Advance clustering

Model-based clustering with the EM algorithm

Reducing dimensionality with principal component analysis

Lessons from History - Time Series Analysis

Introduction

Exploring finance datasets

Creating and examining date objects

Operating on date objects

Performing preliminary analyses on time series data

Using time series objects

Decomposing time series

Filtering time series data

Smoothing and forecasting using the Holt-Winters method

Building an automated ARIMA model

How does it look? - Advanced data visualization

Introduction

Creating scatter plots

Creating line graphs

Creating bar graphs

Making distributions plots

Creating mosaic graphs

Making treemaps

Plotting a correlations matrix

Creating heatmaps

Plotting network graphs

Labeling and legends

Coloring and themes

Creating multivariate plots

Creating 3D graphs and animation

Selecting a graphics device

This may also interest you - Building Recommendations

Introduction

Building collaborative filtering systems

Performing content-based systems

Building hybrid systems

Performing similarity measures

Application of ML algorithms - image recognition system

Evaluating models and optimization

A practical example - fraud detection system

It's All About Your Connections - Social Network Analysis

Introduction

Downloading social network data using public APIs

Creating adjacency matrices and edge lists

Plotting social network data

Computing important network metrics

Cluster analysis

Force layout

YiFan Hu layout

Put Your Best Foot Forward - Document and Present Your Analysis

Introduction

Generating reports of your data analysis with R Markdown and knitr

Creating interactive web applications with shiny

Creating PDF presentations of your analysis with R presentation

Generating dynamic reports

Work Smarter, Not Harder - Efficient and Elegant R Code

Introduction

Exploiting vectorized operations

Processing entire rows or columns using the apply function

Applying a function to all elements of a collection with lapply and sapply

Applying functions to subsets of a vector

Using the split-apply-combine strategy with plyr

Slicing, dicing, and combining data with data tables

Where in the World? Geospatial Analysis

Introduction

Downloading and plotting a Google map of an area

Overlaying data on the downloaded Google map

Importing ESRI shape files to R

Using the sp package to plot geographic data

Getting maps from the maps package

Creating spatial data frames from regular data frames containing spatial and other data

Creating spatial data frames by combining regular data frames with spatial objects

Adding variables to an existing spatial data frame

Spatial data analysis with R and QGIS

Playing Nice - Connecting to Other Systems

Introduction

Using Java objects in R

Using JRI to call R functions from Java

Using Rserve to call R functions from Java

Executing R scripts from Java

Using the xlsx package to connect to Excel

Reading data from relational databases - MySQL

Reading data from NoSQL databases - MongoDB

Working with in-memory data processing with Apache Spark

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Creating dummies for categorical variables

In situations where we have categorical variables (factors) but need to use them in analytical methods that require numbers (for example, K nearest neighbors (KNN), Linear Regression), we need to create dummy variables.

Getting ready

Read the data-conversion.csv file and store it in the working directory of your R environment. Install the dummies package. Then read the data:

> install.packages("dummies") 
> library(dummies) 
> students <- read.csv("data-conversion.csv")

How to do it...

Create dummies for all factors in the data frame:

> students.new <- dummy.data.frame(students, sep = ".") 
> names(students.new) 
 
[1] "Age"      "State.NJ" "State.NY" "State.TX" "State.VA" 
[6] "Gender.F" "Gender.M" "Height"   "Income"

The students.new data frame now contains all the original variables and the newly added dummy variables. The dummy.data.frame() function has created dummy variables for all four levels of State and two levels of Gender factors. However, we will generally omit one of the dummy variables for State and one for Gender when we use machine learning techniques.

We can use the optional argument all = FALSE to specify that the resulting data frame should contain only the generated dummy variables and none of the original variables.

How it works...

The dummy.data.frame() function creates dummies for all the factors in the data frame supplied. Internally, it uses another dummy() function which creates dummy variables for a single factor. The dummy() function creates one new variable for every level of the factor for which we are creating dummies. It appends the variable name with the factor level name to generate names for the dummy variables. We can use the sep argument to specify the character that separates them; an empty string is the default:

> dummy(students$State, sep = ".") 
 
      State.NJ State.NY State.TX State.VA 
 [1,]        1        0        0        0 
 [2,]        0        1        0        0 
 [3,]        1        0        0        0 
 [4,]        0        0        0        1 
 [5,]        0        1        0        0 
 [6,]        0        0        1        0 
 [7,]        1        0        0        0 
 [8,]        0        0        0        1 
 [9,]        0        0        1        0 
[10,]        0        0        0        1

There's more...

In situations where a data frame has several factors, and you plan on using only a subset of them, you create dummies only for the chosen subset.

Choosing which variables to create dummies for

To create a dummy only for one variable or a subset of variables, we can use the names argument to specify the column names of the variables we want dummies for:

> students.new1 <- dummy.data.frame(students,     names = c("State","Gender") , sep = ".")

R Data Analysis Cookbook - Second Edition

By : Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

R Data Analysis Cookbook - Second Edition

By: Kuntal Ganguly, Shanthi Viswanathan, Viswa Viswanathan

Overview of this book

Related Content you might be interested in

Current Title:

R Data Analysis Cookbook - Second Edition

Machine Learning with R Cookbook

Mastering Machine Learning with R

R Programming Fundamentals