Data Analysis with R, Second Edition - Second Edition

Book Image

Data Analysis with R, Second Edition - Second Edition

Book Image

Data Analysis with R, Second Edition - Second Edition

Overview of this book

Frequently the tool of choice for academics, R has spread deep into the private sector and can be found in the production pipelines at some of the most advanced and successful enterprises. The power and domain-specificity of R allows the user to express complex analytics easily, quickly, and succinctly. Starting with the basics of R and statistical reasoning, this book dives into advanced predictive analytics, showing how to apply those techniques to real-world data though with real-world examples. Packed with engaging problems and exercises, this book begins with a review of R and its syntax with packages like Rcpp, ggplot2, and dplyr. From there, get to grips with the fundamentals of applied statistics and build on this knowledge to perform sophisticated and powerful analytics. Solve the difficulties relating to performing data analysis in practice and find solutions to working with messy data, large data, communicating results, and facilitating reproducibility. This book is engineered to be an invaluable resource through many stages of anyone’s career as a data analyst.

Title Page

Copyright and Credits

Copyright and Credits

Packt Upsell

Contributors

Preface

Free Chapter

RefresheR

Navigating the basics

Getting help in R

Loading data into R

Working with packages

The Shape of Data

The Shape of Data

Univariate data

Frequency distributions

Central tendency

Populations, samples, and estimation

Probability distributions

Visualization methods

Describing Relationships

Describing Relationships

Multivariate data

Relationships between a categorical and continuous variable

Relationships between two categorical variables

The relationship between two continuous variables

Visualization methods

Probability

Basic probability

A tale of two interpretations

Sampling from distributions

The normal distribution

Using Data To Reason About The World

Using Data To Reason About The World

Estimating means

The sampling distribution

Interval estimation

Smaller samples

Testing Hypotheses

Testing Hypotheses

The null hypothesis significance testing framework

Testing the mean of one sample

Testing two means

Testing more than two means

Testing independence of proportions

What if my assumptions are unfounded?

Bayesian Methods

Bayesian Methods

The big idea behind Bayesian analysis

Choosing a prior

Who cares about coin flips

Enter MCMC – stage left

Using JAGS and runjags

Fitting distributions the Bayesian way

The Bayesian independent samples t-test

The Bootstrap

What's... uhhh... the deal with the bootstrap?

Performing the bootstrap in R (more elegantly)

Confidence intervals

A one-sample test of means

Bootstrapping statistics other than the mean

Busting bootstrap myths

Predicting Continuous Variables

Predicting Continuous Variables

Simple linear regression

Simple linear regression with a binary predictor

Multiple regression

Regression with a non-binary predictor

Kitchen sink regression

The bias-variance trade-off

Linear regression diagnostics

Advanced topics

Predicting Categorical Variables

Predicting Categorical Variables

k-Nearest neighbors

Logistic regression

Choosing a classifier

Predicting Changes with Time

Predicting Changes with Time

What is a time series?

What is forecasting?

Creating and plotting time series

Components of time series

Time series decomposition

Autocorrelation

ETS and the state space model

Interventions for improvement

What we didn't cover

Citations for the climate change data

Sources of Data

Sources of Data

Relational databases

Other data formats

Online repositories

Dealing with Missing Data

Dealing with Missing Data

Analysis with missing data

Visualizing missing data

Types of missing data

Unsophisticated methods for dealing with missing data

So how does mice come up with the imputed values?

Dealing with Messy Data

Dealing with Messy Data

Checking unsanitized data

Regular expressions

Other tools for messy data

Dealing with Large Data

Dealing with Large Data

Wait to optimize

Using a bigger and faster machine

Be smart about your code

Using optimized packages

Using another R implementation

Using parallelization

Being smarter about your code

Working with Popular R Packages

Working with Popular R Packages

The data.table package

Using dplyr and tidyr to manipulate data

Functional programming as a main tidyverse principle

Reshaping data with tidyr

Reproducibility and Best Practices

Reproducibility and Best Practices

Version control

Communicating results

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Index

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Being smarter about your code

In a blog post that I penned showcasing the performance of this task under various optimization methods, I took it for granted that calculating the distances on the full dataset with the unparallelized/un-Rcpp-ed code would be a multi-hour affair, but I was seriously mistaken.

Shortly after publishing the post, a clever R programmer commented on it stating that they were able to slightly rework the code so that the serial/pure-R code took less than 20 seconds to complete, with all the 13,429 observations. How? Vectorization. The following code illustrates the technique:

single.core.improved <- function(airport.locs){ 
  numrows <- nrow(airport.locs) 
  running.sum <- 0

  for (i in 1:(numrows-1)) { 
    this.dist <- sum(haversine(airport.locs[i,2], 
                               airport.locs[i, 3], 
                               airport.locs[(i+1):numrows, 2], 
                               airport.locs[(i+1):numrows, 3])) 
    running.sum <-...