Book Image

Scala Machine Learning Projects

Book Image

Scala Machine Learning Projects

Overview of this book

Machine learning has had a huge impact on academia and industry by turning data into actionable information. Scala has seen a steady rise in adoption over the past few years, especially in the fields of data science and analytics. This book is for data scientists, data engineers, and deep learning enthusiasts who have a background in complex numerical computing and want to know more hands-on machine learning application development. If you're well versed in machine learning concepts and want to expand your knowledge by delving into the practical implementation of these concepts using the power of Scala, then this book is what you need! Through 11 end-to-end projects, you will be acquainted with popular machine learning libraries such as Spark ML, H2O, DeepLearning4j, and MXNet. At the end, you will be able to use numerical computing and functional programming to carry out complex numerical tasks to develop, build, and deploy research or commercial projects in a production-ready environment.

Related Content you might be interested in

Current Title:

Small book image
Scala Machine Learning Projects
Jan, 2018 | 470 pages
Small book image
Java Deep Learning Projects
Md Rezaul Karim
Jun, 2018 | 436 pages
Small book image
Predictive Analytics with TensorFlow
Md Rezaul Karim
Nov, 2017 | 522 pages
Small book image
Deep Learning with TensorFlow
Md Rezaul Karim | Giancarlo Zaccone
Mar, 2018 | 484 pages
Table of Contents (17 chapters)
Title Page
Title Page
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Analyzing Insurance Severity Claims
Analyzing Insurance Severity Claims
Machine learning and learning workflow
Hyperparameter tuning and cross-validation
Analyzing and predicting insurance severity claims
LR for predicting insurance severity claims
GBT regressor for predicting insurance severity claims
Boosting the performance using random forest regressor
Comparative analysis and model deployment
Summary
2
Analyzing and Predicting Telecommunication Churn
Analyzing and Predicting Telecommunication Churn
Why do we perform churn analysis, and how do we do it?
Developing a churn analytics pipeline
LR for churn prediction
SVM for churn prediction
DTs for churn prediction
Random Forest for churn prediction
Selecting the best model for deployment
Summary
3
High Frequency Bitcoin Price Prediction from Historical and Live Data
High Frequency Bitcoin Price Prediction from Historical and Live Data
Bitcoin, cryptocurrency, and online trading
High-level data pipeline of the prototype
Historical and live-price data collection
Model training for prediction
Scala Play web service
Predicting prices and evaluating the model
Demo prediction using Scala Play framework
Summary
4
Population-Scale Clustering and Ethnicity Prediction
Population-Scale Clustering and Ethnicity Prediction
Population scale clustering and geographic ethnicity
1000 Genomes Projects dataset description
Algorithms, tools, and techniques
Configuring programming environment
Data pre-processing and feature engineering
Summary
5
Topic Modeling - A Better Insight into Large-Scale Texts
Topic Modeling - A Better Insight into Large-Scale Texts
Topic modeling and text clustering
Topic modeling with Spark MLlib and Stanford NLP
Other topic models versus the scalability of LDA
Deploying the trained LDA model
Summary
6
Developing Model-based Movie Recommendation Engines
Developing Model-based Movie Recommendation Engines
Recommendation system
Spark-based movie recommendation systems
Selecting and deploying the best model 
Summary
7
Options Trading Using Q-learning and Scala Play Framework
Options Trading Using Q-learning and Scala Play Framework
Reinforcement versus supervised and unsupervised learning
A simple Q-learning implementation
Developing an options trading web app using Q-learning
Summary
8
Clients Subscription Assessment for Bank Telemarketing using Deep Neural Networks
Clients Subscription Assessment for Bank Telemarketing using Deep Neural Networks
Client subscription assessment through telemarketing
Summary
9
Fraud Analytics Using Autoencoders and Anomaly Detection
Fraud Analytics Using Autoencoders and Anomaly Detection
Outlier and anomaly detection
Autoencoders and unsupervised learning
Developing a fraud analytics model
Hyperparameter tuning and feature selection
Summary
10
Human Activity Recognition using Recurrent Neural Networks
Human Activity Recognition using Recurrent Neural Networks
Working with RNNs
Human activity recognition using the LSTM model
Implementing an LSTM model for HAR
Tuning LSTM hyperparameters and GRU
Summary
11
Image Classification using Convolutional Neural Networks
Image Classification using Convolutional Neural Networks
Image classification and drawbacks of DNNs
CNN architecture
Large-scale image classification using CNN
Tuning and optimizing CNN hyperparameters
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

Data pre-processing and feature engineering

I already stated that all the 24 VCF files contribute 820 GB of data. Therefore, I decided to use the genetic variant of chromosome Y only one two make the demonstration clearer. The size is around 160 MB, which is not meant to pose huge computational challenges. You can download all the VCF files as well as the panel file from ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/.

Let us get started. We start by creating SparkSession, the gateway for the Spark application:

val spark:SparkSession = SparkSession
    .builder()
    .appName("PopStrat")
    .master("local[*]")
    .config("spark.sql.warehouse.dir", "C:/Exp/")
    .getOrCreate()

Then let's show Spark the path of both VCF and the panel file:

val genotypeFile = "<path>/ALL.chrY.phase3_integrated_v2a.20130502.genotypes.vcf"
val panelFile = "<path>/integrated_call_samples_v3.20130502.ALL.panel "

We process the panel file using Spark to access the target population data and...

Previous Section
End of Section 6
Next Section