In this chapter, we went through a step-by-step process, from big data to a rapid development of fraud detection systems from which we processed data on Spark and then built several models to predict frauds. With this, we then developed rules and scores to help the ABC company prevent frauds.

Specifically, we first selected a supervised machine learning approach with a focus on Random forest and decision trees as per business needs, after we prepared Spark computing and loaded preprocessed data. Second, we worked on feature extraction and selection. Third, we estimated model coefficients. Fourth, we evaluated these estimated models using a confusion matrix and false positive ratios. Then, we interpreted our machine learning results. Finally, we deployed our machine learning results, with a focus on scoring but also used insights to develop rules.

The preceding process is similar to the process of working with small data. However, in dealing with big data, we need parallel computing...