What is meant by prediction? The answer, of course, depends on the field and the algorithms used, but this explanation is true most of the time—given the attested reliable relationships between indicators (predictors) and an outcome, the presence (or level) of the indicators for similar cases is a reliable clue to the presence (or level) of the outcome in the future. Here are some examples of relationships, starting with the most obvious:

In the following section, we provide minimal definitions of the distinctions between supervised and unsupervised learning and classification and regression problems.

Two broad families of algorithms will be discussed in this book:

There are basically two types of problems that predictive modeling deals with:

Of course, analyzing data without knowledge of the field is not a serious way to proceed. This is okay to show how some algorithms work, how to make use of them, and to exercise. However, for real-life applications, be sure that you know the topic well, or else consult experts for help. The Cross Industry Standard Process for Data Mining (CRISP-DM, Shearer, 2000) underlines the importance of field knowledge. The steps of the process are depicted as follows:

The Cross Industry Standard Process for Data Mining

As stressed upon in the preceding diagram, field knowledge (here called Business Understanding) informs and is informed by data understanding. The understanding of the data then informs how the data has to be prepared. The next step is data modeling, which can also lead to further data preparation. Data models have to be evaluated, and this evaluation can be informed by field knowledge (this is also stressed upon in the diagram), which is also updated through the data mining process. Finally, if the evaluation is satisfactory, the models are deployed for prediction. This book will focus on the data modeling and evaluation stages.

Of course, predictions are not always accurate, and some have written about the caveats of data science. What do you think about the relationship between the attributes titled Predictor and Outcome on the following plot? It seems like there is a relationship between the two. For the statistically inclined, I tested its significance: r = 0.4195, p = .0024. The value p is the probability of obtaining a relationship of this strength or stronger if there is actually no relationship between the attributes. We could conclude that the relationship between these variables in the population they come from is quite reliable, right?

The relationship between the attributes titled Predictor and Outcome

Believe it or not, the population these observations come from is that of randomly generated numbers. We generated a data frame of 50 columns of 50 randomly generated numbers. We then examined all the correlations (manually) and generated a scatterplot of the two attributes with the largest correlation we found. The code is provided here, in case you want to check it yourself—line 1 sets the seed so that you find the same results as we did, line 2 generates to the data frame, line 3 fills it with random numbers, column by column, line 4 generates the scatterplot, line 5 fits the regression line, and line 6 tests the significance of the correlation:

1  set.seed(1)
2  DF = data.frame(matrix(nrow=50,ncol=50))
3  for (i in 1:50) DF[,i] = runif(50)
4  plot(DF[[2]],DF[[16]], xlab = "Predictor", ylab = "Outcome")
5  abline(lm(DF[[2]]~DF[[16]]))
6  cor.test(DF[[2]], DF[[16]])

How could this relationship happen given that the odds were 2.4 in 1000 ? Well, think of it; we correlated all 50 attributes 2 x 2, which resulted in 2,450 tests (not considering the correlation of each attribute with itself). Such spurious correlation was quite expectable. The usual threshold below which we consider a relationship significant is p = 0.05, as we will discuss in Chapter 8, Probability Distributions, Covariance, and Correlation. This means that we expect to be wrong once in 20 times. You would be right to suspect that there are other significant correlations in the generated data frame (there should be approximately 125 of them in total). This is the reason why we should always correct the number of tests. In our example, as we performed 2,450 tests, our threshold for significance should be 0.0000204 (0.05 / 2450). This is called the Bonferroni correction.

Spurious correlations are always a possibility in data analysis and this should be kept in mind at all times. A related concept is that of overfitting. Overfitting happens, for instance, when a weak classifier bases its prediction on the noise in data. We will discuss overfitting in the book, particularly when discussing cross-validation in Chapter 14, Cross-validation and Bootstrapping Using Caret and Exporting Predictive Models Using PMML. All the chapters are listed in the following section.

We hope you enjoy reading the book and hope you learn a lot from us!

Chapter 1, Setting GNU R for Predictive Analytics, deals with the setting of R, how to load and install packages, and other basic operations. Only beginners should read this. If you are not a beginner, you will be bored! (Beginners should find the chapter entertaining).

Chapter 2, Visualizing and Manipulating Data Using R, deals with basic visualization functions in R and data manipulation. This chapter also aims to bring beginners up to speed for the rest of the book.

Chapter 3, Data Visualization with Lattice, deals with more advanced visualization functions. The concept of multipanel conditioning plots is presented. These allow you to examine the relationship between attributes as a function of group membership (for example, women versus men). A good working knowledge of R programming is necessary from this point.

Chapter 4, Cluster Analysis, presents the concept of clustering and the different types of clustering algorithms. It shows how to program and use a basic clustering algorithm (k-means) in R. Special attention is given to the description of distance measures and how to select the number of clusters for the analyses.

Chapter 5, Agglomerative Clustering Using hclust(), deals with hierarchical clustering. It shows how to use agglomerative clustering in R and the options to configure the analysis.

Chapter 6, Dimensionality Reduction with Principal Component Analysis, discusses the uses of PCA, notably dimension reduction. How to build a simple PCA algorithm, how to use PCA, and example applications are explored in the chapter.

Chapter 7, Exploring Association Rules with Apriori, focuses on the functioning of the apriori algorithm, how to perform the analyses, and how to interpret the outputs. Among other applications, association rules can be used to discover which products are frequently bought together (marked basket analysis).

Chapter 8, Probability Distributions, Covariance, and Correlation, discusses basic statistics and how they can be useful for prediction. The concepts given in the title are discussed without too much technicality, but formulas are proposed for the mathematically inclined.

Chapter 9, Linear Regression, builds upon the knowledge acquired in the previous chapter to show how to build a regression algorithm, including how to compute the coefficients and p values. The assumptions of linear regression (ordinary least squares) are rapidly discussed. The chapter then focuses on the use (and misuse) of regression.

Chapter 10, Classification with k-Nearest Neighbors and Naïve Bayes, deals with the classification problems of using two of the most popular algorithms. We build our own k-NN algorithm, with which we analyze the famous iris dataset. We also demonstrate how Naïve Bayes works. The chapter also deals with the use of both algorithms.

Chapter 11, Classification Trees, explores classification using no less than five classification tree algorithms: C4.5, C5, CART (classification part), random forests, and conditional inference trees. Entropy, information gain, pruning, bagging, and other important concepts are discussed.

Chapter 12, Multilevel Analyses, deals with the use of nested data. We will briefly discuss the functioning of multilevel regression (with mixed models), and will then focus on the important aspects in the analysis, notably, how to create and compare the models, and how to understand the outputs.

Chapter 13, Text Analytics with R, focuses on the use of some algorithms that we discussed in other chapters, as well as new ones, with the aim of analyzing text. We will start by showing you how to perform text preprocessing, we will explain important concepts, and then jump right into the analysis. We will highlight the importance of testing different algorithms on the same corpus.

Chapter 14, Cross-validation and Bootstrapping Using Caret and Exporting Predictive Models Using PMML, deals with two important aspects, the first is ascertaining the validity of the models and the second is exporting the models for production. Training and testing datasets are used in most chapters. These are minimal requirements, and cross-validation as well as bootstrapping are significant improvements.

Appendix A, Exercises and Solutions, provides the exercises and the solutions for the chapters in the book.

Appendix B, Further Reading and References, it provides the references for the chapters in the book.

All you need for this book is a working installation of R > 3.0 (on any operating system) and an active internet connection.

If you are a statistician, chief information officer, data scientist, ML engineer, ML practitioner, quantitative analyst, or student of machine learning, this is the book for you. You should have basic knowledge of the use of R. Readers without previous experience of programming in R will also be able to use the tools in this book.

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: " Now, open the R script file called helloworld.R."

A block of code is set as follows:

print("Hello world")

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: " The File menu contains functions related to file handling."

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.