Book Image

Mastering Data analysis with R

By : Gergely Daróczi
Book Image

Mastering Data analysis with R

By: Gergely Daróczi

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Mastering Data analysis with R
Gergely Daróczi
Sep, 2015 | 396 pages
No titles found
Table of Contents (19 chapters)
Mastering Data Analysis with R
Mastering Data Analysis with R
Credits
Credits
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Hello, Data!
Hello, Data!
Loading text files of a reasonable size
Benchmarking text file parsers
Loading a subset of text files
Loading data from databases
Importing data from other statistical systems
Loading Excel spreadsheets
Summary
2
Getting Data from the Web
Getting Data from the Web
Loading datasets from the Internet
Other popular online data formats
Reading data from HTML tables
Scraping data from other online sources
R packages to interact with data source APIs
Summary
3
Filtering and Summarizing Data
Filtering and Summarizing Data
Drop needless data
Aggregation
Running benchmarks
Summary functions
Summary
4
Restructuring Data
Restructuring Data
Transposing matrices
Filtering data by string matching
Rearranging data
dplyr versus data.table
Computing new variables
Merging datasets
Reshaping data in a flexible way
The evolution of the reshape packages
Summary
5
Building Models (authored by Renata Nemeth and Gergely Toth)
Building Models (authored by Renata Nemeth and Gergely Toth)
The motivation behind multivariate models
Linear regression with continuous predictors
Model assumptions
How well does the line fit in the data?
Discrete predictors
Summary
6
Beyond the Linear Trend Line (authored by Renata Nemeth and Gergely Toth)
Beyond the Linear Trend Line (authored by Renata Nemeth and Gergely Toth)
The modeling workflow
Logistic regression
Models for count data
Summary
7
Unstructured Data
Unstructured Data
Importing the corpus
Cleaning the corpus
Visualizing the most frequent words in the corpus
Further cleanup
Analyzing the associations among terms
Some other metrics
The segmentation of documents
Summary
8
Polishing Data
Polishing Data
The types and origins of missing data
Identifying missing data
By-passing missing values
Getting rid of missing data
Filtering missing data before or during the actual analysis
Data imputation
Extreme values and outliers
Using robust methods
Summary
9
From Big to Small Data
From Big to Small Data
Adequacy tests
Principal Component Analysis
Factor analysis
Principal Component Analysis versus Factor Analysis
Multidimensional Scaling
Summary
10
Classification and Clustering
Classification and Clustering
Cluster analysis
Latent class models
Discriminant analysis
Logistic regression
Machine learning algorithms
Summary
11
Social Network Analysis of the R Ecosystem
Social Network Analysis of the R Ecosystem
Loading network data
Centrality measures of networks
Visualizing network data
Further network analysis resources
Summary
12
Analyzing Time-series
Analyzing Time-series
Creating time-series objects
Visualizing time-series
Seasonal decomposition
Holt-Winters filtering
Autoregressive Integrated Moving Average models
Outlier detection
More complex time-series objects
Advanced time-series analysis
Summary
13
Data Around Us
Data Around Us
Geocoding
Visualizing point data in space
Finding polygon overlays of point data
Plotting thematic maps
Rendering polygons around points
Satellite maps
Interactive maps
Alternative map designs
Spatial statistics
Summary
14
Analyzing the R Community
Analyzing the R Community
R Foundation members
R package maintainers
The R-help mailing list
Analyzing overlaps between our lists of R users
The number of R users in social media
R-related posts in social media
Summary
References
References
General good readings on R
Chapter 1 – Hello, Data!
Chapter 2 – Getting Data from the Web
Chapter 3 – Filtering and Summarizing Data
Chapter 4 – Restructuring Data
Chapter 5 – Building Models (authored by Renata Nemeth and Gergely Toth)
Chapter 6 – Beyond the Linear Trend Line (authored by Renata Nemeth and Gergely Toth)
Chapter 7 – Unstructured Data
Chapter 8 – Polishing Data
Chapter 9 – From Big to Smaller Data
Chapter 10 – Classification and Clustering
Chapter 11 – Social Network Analysis of the R Ecosystem
Chapter 12 – Analyzing Time-series
Chapter 13 – Data Around Us
Chapter 14 – Analysing the R Community

Model assumptions

Linear regression models with standard estimation techniques make a number of assumptions about the outcome variable, the predictor variables, and also about their relationship:

  1. Y is a continuous variable (not binary, nominal, or ordinal)

  2. The errors (the residuals) are statistically independent

  3. There is a stochastic linear relationship between Y and each X

  4. Y has a normal distribution, holding each X fixed

  5. Y has the same variance, regardless of the fixed value of the Xs

A violation of assumption 2 occurs in trend analysis, if we use time as the predictor. Since the consecutive years are not independent, the errors will not be independent from each other. For example, if we have a year with high mortality from a specific illness, then we can expect the mortality for the next year to also be high.

A violation of assumption (3) says that the relationship is not exactly linear, but there is a deviation from the linear trend line. Assumption 4 and 5 require the conditional distribution...

Previous Section
End of Section 4
Next Section