Book Image

Practical Predictive Analytics

By : Ralph Winters
Book Image

Practical Predictive Analytics

By: Ralph Winters

Overview of this book

This is the go-to book for anyone interested in the steps needed to develop predictive analytics solutions with examples from the world of marketing, healthcare, and retail. We'll get started with a brief history of predictive analytics and learn about different roles and functions people play within a predictive analytics project. Then, we will learn about various ways of installing R along with their pros and cons, combined with a step-by-step installation of RStudio, and a description of the best practices for organizing your projects. On completing the installation, we will begin to acquire the skills necessary to input, clean, and prepare your data for modeling. We will learn the six specific steps needed to implement and successfully deploy a predictive model starting from asking the right questions through model development and ending with deploying your predictive model into production. We will learn why collaboration is important and how agile iterative modeling cycles can increase your chances of developing and deploying the best successful model. We will continue your journey in the cloud by extending your skill set by learning about Databricks and SparkR, which allow you to develop predictive models on vast gigabytes of data.

Related Content you might be interested in

Current Title:

Small book image
Practical Predictive Analytics
Ralph Winters
Jun, 2017 | 576 pages
Small book image
Machine Learning with R Cookbook
YuWei Chiu David Chiu | AshishSingh Bhatia
Oct, 2017 | 572 pages
Small book image
Big Data Analytics with Hadoop 3
Sridhar Alla
May, 2018 | 482 pages
Small book image
Hands-On Ensemble Learning with R
Prabhanjan Narayanachar Tattar
Jul, 2018 | 376 pages
Table of Contents (19 chapters)
Title Page
Title Page
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Preface
Preface
Free Chapter
1
Getting Started with Predictive Analytics
Getting Started with Predictive Analytics
Predictive analytics are in so many industries
Skills and roles that are important in Predictive Analytics
Predictive analytics software
Other helpful tools
R
How is a predictive analytics project organized?
GUIs
Getting started with RStudio
The R console
The source window
Our first predictive model
Your second script
R packages
References
Summary
2
The Modeling Process
The Modeling Process
Advantages of a structured approach
Analytic process methodologies
An analytics methodology outline specific steps
Step 2 data understanding
Step 3 data preparation
Step 4 modeling
Step 5 evaluation
Step 6 deployment
References
Summary
3
Inputting and Exploring Data
Inputting and Exploring Data
Data input
Joining data
Exploring the hospital dataset
Transposing a dataframe
Missing values
Imputing categorical variables
Outliers
Data transformations
Variable reduction/variable importance
References
Summary
4
Introduction to Regression Algorithms
Introduction to Regression Algorithms
Supervised versus unsupervised learning models
Regression techniques
Generalized linear models
Logistic regression
Summary
5
Introduction to Decision Trees, Clustering, and SVM
Introduction to Decision Trees, Clustering, and SVM
Decision tree algorithms
Cluster analysis
Support vector machines
References
Summary
6
Using Survival Analysis to Predict and Analyze Customer Churn
Using Survival Analysis to Predict and Analyze Customer Churn
What is survival analysis?
Our customer satisfaction dataset
Partitioning into training and test data
Setting the stage by creating survival objects
Examining survival curves
Cox regression modeling
Time-based variables
Comparing the models
Variable selection
Summary
7
Using Market Basket Analysis as a Recommender Engine
Using Market Basket Analysis as a Recommender Engine
What is market basket analysis?
Examining the groceries transaction file
The sample market basket
Association rule algorithms
Antecedents and descendants
Evaluating the accuracy of a rule
Preparing the raw data file for analysis
Analyzing the input file
Scrubbing and cleaning the data
Removing colors automatically
Filtering out single item transactions
Merging the results back into the original data
Compressing descriptions using camelcase
Creating the test and training datasets
Creating the market basket transaction file
Method two Creating a physical transactions file
Converting to a document term matrix
K-means clustering of terms
Predicting cluster assignments
Running the apriori algorithm on the clusters
Summarizing the metrics
References
Summary
8
Exploring Health Care Enrollment Data as a Time Series
Exploring Health Care Enrollment Data as a Time Series
Time series data
Health insurance coverage dataset
Housekeeping
Read the data in
Subsetting the columns
Description of the data
Target time series variable
Saving the data
Determining all of the subset groups
Merging the aggregate data back into the original data
Checking the time intervals
Picking out the top groups in terms of average population size
Plotting the data using lattice
Plotting the data using ggplot
Sending output to an external file
Examining the output
Detecting linear trends
Automating the regressions
Ranking the coefficients
Merging scores back into the original dataframe
Plotting the data with the trend lines
Plotting all the categories on one graph
Performing some automated forecasting using the ets function
Smoothing the data using moving averages
Simple moving average
Verifying the SMA calculation
Exponential moving average
Using the ets function
Forecasting using ALL AGES
Plotting the predicted and actual values
The forecast (fit) method
Plotting future values with confidence bands
Modifying the model to include a trend component
Running the ets function iteratively over all of the categories
Accuracy measures produced by onestep
Comparing the Test and Training for the "UNDER 18 YEARS" group
Accuracy measures
References
Summary
9
Introduction to Spark Using R
Introduction to Spark Using R
About Spark
Spark environments
SparkR
Building our first Spark dataframe
Importing the sample notebook
Creating a new notebook
Becoming large by starting small
Running the code
Running the initialization code
Extracting the Pima Indians diabetes dataset
Simulating the data
Simulating the negative cases
Running summary statistics
Saving your work
Summary
10
Exploring Large Datasets Using Spark
Exploring Large Datasets Using Spark
Performing some exploratory analysis on positives
Cleaning up and caching the table in memory
Some useful Spark functions to explore your data
Creating new columns
Constructing a cross-tab
Contrasting histograms
Plotting using ggplot
Spark SQL
Exporting data from Spark back into R
Running local R packages
Some tips for using Spark
Summary
11
Spark Machine Learning - Regression and Cluster Models
Spark Machine Learning - Regression and Cluster Models
About this chapter/what you will learn
Splitting the data into train and test datasets
Spark machine learning using logistic regression
Running predictions for the test data
Combining the training and test dataset
Exposing the three tables to SQL
Validating the regression results
Calculating goodness of fit measures
Confusion matrix for test group
Plotting outside of Spark
Creating some global views
Normalizing the data
Characterizing the clusters by their mean values
Summary
12
Spark Models – Rule-Based Learning
Spark Models – Rule-Based Learning
Loading the stop and frisk dataset
Reading the table
Discovering the important features
Running the OneR model
Another OneR example
Constructing a decision tree using Rpart
Running an alternative model in Python
Indexing the classification features
Summary

Performing some exploratory analysis on positives

Before we move on to exploring the entire Spark dataframe, we can look at some of the data already generated for positive cases. As you may recall from the prior chapter, this is stored in the Spark dataframe out_sd1.

We have generated some random sample bins specifically so that we can do some exploratory analysis.

We can use the filter command to extract random sample 1, and take the first 1,000 records:

  • The filter is a SparkR command that allows you to subset a Spark dataframe
  • The display command is a databricks command that is equivalent to the View command we have previously used and you can also use the head function as well to limit the number of rows that are displayed:

This code chunk extracts 1000 records from the positives and displays them:

        small_pos <- head(SparkR::filter(out_sd1,out_sd1$sample_bin==1),1000) 
        nrow(small_pos)

        display(small_pos)

The data appears in tabular form, and you can scroll up/down...

Previous Section
End of Section 2
Next Section