Book Image

Java Data Science Cookbook

By : Rushdi Shams
Book Image

Java Data Science Cookbook

By: Rushdi Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.

Related Content you might be interested in

Current Title:

Small book image
Java Data Science Cookbook
Rushdi Shams
Mar, 2017 | 372 pages
Small book image
Machine Learning in Java
AshishSingh Bhatia | Bostjan Kaluza
Nov, 2018 | 300 pages
Small book image
Java for Data Science
Richard Reese | Jennifer L Reese
Jan, 2017 | 386 pages
Small book image
Mastering Java for Data Science
Alexey Grigorev
Apr, 2017 | 364 pages
Table of Contents (16 chapters)
Java Data Science Cookbook
Java Data Science Cookbook
Credits
Credits
About the Author
About the Author
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Preface
Preface
Free Chapter
1
Obtaining and Cleaning Data
Obtaining and Cleaning Data
Introduction
Retrieving all filenames from hierarchical directories using Java
Retrieving all filenames from hierarchical directories using Apache Commons IO
Reading contents from text files all at once using Java 8
Reading contents from text files all at once using Apache Commons IO
Extracting PDF text using Apache Tika
Cleaning ASCII text files using Regular Expressions
Parsing Comma Separated Value (CSV) Files using Univocity
Parsing Tab Separated Value (TSV) file using Univocity
Parsing XML files using JDOM
Writing JSON files using JSON.simple
Reading JSON files using JSON.simple
Extracting web data from a URL using JSoup
Extracting web data from a website using Selenium Webdriver
Reading table data from a MySQL database
2
Indexing and Searching Data
Indexing and Searching Data
Introduction
Indexing data with Apache Lucene
Searching indexed data with Apache Lucene
3
Analyzing Data Statistically
Analyzing Data Statistically
Introduction
Generating descriptive statistics
Generating summary statistics
Generating summary statistics from multiple distributions
Computing frequency distribution
Counting word frequency in a string
Counting word frequency in a string using Java 8
Computing simple regression
Computing ordinary least squares regression
Computing generalized least squares regression
Calculating covariance of two sets of data points
Calculating Pearson's correlation of two sets of data points
Conducting a paired t-test
Conducting a Chi-square test
Conducting the one-way ANOVA test
Conducting a Kolmogorov-Smirnov test
4
Learning from Data - Part 1
Learning from Data - Part 1
Introduction
Creating and saving an Attribute-Relation File Format (ARFF) file
Cross-validating a machine learning model
Classifying unseen test data
Classifying unseen test data with a filtered classifier
Generating linear regression models
Generating logistic regression models
Clustering data points using the KMeans algorithm
Clustering data from classes
Learning association rules from data
Selecting features/attributes using the low-level method, the filtering method, and the meta-classifier method
5
Learning from Data - Part 2
Learning from Data - Part 2
Introduction
Applying machine learning on data using Java Machine Learning (Java-ML) library
Classifying data points using the Stanford classifier
Classifying data points using Massive Online Analysis (MOA)
Classifying multilabeled data points using Mulan
6
Retrieving Information from Text Data
Retrieving Information from Text Data
Introduction
Detecting tokens (words) using Java
Detecting sentences using Java
Detecting tokens (words) and sentences using OpenNLP
Retrieving lemma, part-of-speech, and recognizing named entities from tokens using Stanford CoreNLP
Measuring text similarity with Cosine Similarity measure using Java 8
Extracting topics from text documents using Mallet
Classifying text documents using Mallet
Classifying text documents using Weka
7
Handling Big Data
Handling Big Data
Introduction
Training an online logistic regression model using Apache Mahout
Applying an online logistic regression model using Apache Mahout
Solving simple text mining problems with Apache Spark
Clustering using KMeans algorithm with MLib
Creating a linear regression model with MLib
Classifying data points with Random Forest model using MLib
8
Learn Deeply from Data
Learn Deeply from Data
Introduction
Creating a Word2vec neural net using Deep Learning for Java (DL4j)
Creating a Deep Belief neural net using Deep Learning for Java (DL4j)
Creating a deep autoencoder using Deep Learning for Java (DL4j)
9
Visualizing Data
Visualizing Data
Introduction
Plotting a 2D sine graph
Plotting histograms
Plotting a bar chart
Plotting box plots or whisker diagrams
Plotting scatter plots
Plotting donut plots
Plotting area graphs

Clustering data points using the KMeans algorithm

In this recipe, we will be using the KMeans algorithm to cluster or group data points of a dataset together.

How to do it...

  1. We will be using the cpu dataset to cluster its data points based on a simple KMeans algorithm. The cpu dataset can be found in the data directory of the installed folder in the Weka directory.

    We will be having two instance variables as in the previous recipes. The first variable will be containing the data points of the cpu dataset, and the second variable will be our Simple KMeans clusterer:

            Instances cpu = null; 
        SimpleKMeans kmeans;

  2. Then, we will be creating a method to load the cpu dataset, and to read its contents. Please note that as clustering is an unsupervised method, we do not need to specify the class attribute of our dataset:

            public void loadArff(String arffInput){ 
          DataSource source = null; 
          try { 
            source = new DataSource(arffInput...
Previous Section
End of Section 9
Next Section