Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Java Data Science Cookbook

By : Shams

Java Data Science Cookbook

By: Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

1. Obtaining and Cleaning Data

1. Obtaining and Cleaning Data

Introduction

Retrieving all filenames from hierarchical directories using Java

Retrieving all filenames from hierarchical directories using Apache Commons IO

Reading contents from text files all at once using Java 8

Reading contents from text files all at once using Apache Commons IO

Extracting PDF text using Apache Tika

Cleaning ASCII text files using Regular Expressions

Parsing Comma Separated Value (CSV) Files using Univocity

Parsing Tab Separated Value (TSV) file using Univocity

Parsing XML files using JDOM

Writing JSON files using JSON.simple

Reading JSON files using JSON.simple

Extracting web data from a URL using JSoup

Extracting web data from a website using Selenium Webdriver

Reading table data from a MySQL database

2. Indexing and Searching Data

2. Indexing and Searching Data

Introduction

Indexing data with Apache Lucene

Searching indexed data with Apache Lucene

3. Analyzing Data Statistically

3. Analyzing Data Statistically

Introduction

Generating descriptive statistics

Generating summary statistics

Generating summary statistics from multiple distributions

Computing frequency distribution

Counting word frequency in a string

Counting word frequency in a string using Java 8

Computing simple regression

Computing ordinary least squares regression

Computing generalized least squares regression

Calculating covariance of two sets of data points

Calculating Pearson's correlation of two sets of data points

Conducting a paired t-test

Conducting a Chi-square test

Conducting the one-way ANOVA test

Conducting a Kolmogorov-Smirnov test

4. Learning from Data - Part 1

4. Learning from Data - Part 1

Introduction

Creating and saving an Attribute-Relation File Format (ARFF) file

Cross-validating a machine learning model

Classifying unseen test data

Classifying unseen test data with a filtered classifier

Generating linear regression models

Generating logistic regression models

Clustering data points using the KMeans algorithm

Clustering data from classes

Learning association rules from data

Selecting features/attributes using the low-level method, the filtering method, and the meta-classifier method

5. Learning from Data - Part 2

5. Learning from Data - Part 2

Introduction

Applying machine learning on data using Java Machine Learning (Java-ML) library

Classifying data points using the Stanford classifier

Classifying data points using Massive Online Analysis (MOA)

Classifying multilabeled data points using Mulan

6. Retrieving Information from Text Data

6. Retrieving Information from Text Data

Introduction

Detecting tokens (words) using Java

Detecting sentences using Java

Detecting tokens (words) and sentences using OpenNLP

Retrieving lemma, part-of-speech, and recognizing named entities from tokens using Stanford CoreNLP

Measuring text similarity with Cosine Similarity measure using Java 8

Extracting topics from text documents using Mallet

Classifying text documents using Mallet

Classifying text documents using Weka

7. Handling Big Data

7. Handling Big Data

Introduction

Training an online logistic regression model using Apache Mahout

Applying an online logistic regression model using Apache Mahout

Solving simple text mining problems with Apache Spark

Clustering using KMeans algorithm with MLib

Creating a linear regression model with MLib

Classifying data points with Random Forest model using MLib

8. Learn Deeply from Data

8. Learn Deeply from Data

Introduction

Creating a Word2vec neural net using Deep Learning for Java (DL4j)

Creating a Deep Belief neural net using Deep Learning for Java (DL4j)

Creating a deep autoencoder using Deep Learning for Java (DL4j)

9. Visualizing Data

9. Visualizing Data

Introduction

Plotting a 2D sine graph

Plotting histograms

Plotting a bar chart

Plotting box plots or whisker diagrams

Plotting scatter plots

Plotting donut plots

Plotting area graphs

Reading contents from text files all at once using Java 8

On many occasions, data scientists have their data in text format. There are many different ways to read text file contents, and they each have their own pros and cons: some of them consume time and memory, while some are fast and do not require much computer memory; some read the text contents all at once, while some read text files line by line. The choice depends on the task at hand and a data scientist's approach to that task.

This recipe demonstrates how to read text file contents all at once using Java 8.

How to do it...

First, create a String object to hold the path and name of the text file you are going to read:
```
        String file = "C:/dummy.txt";  
```
Using the get() method of the Paths class, we get to the path of the file we are trying to read. The parameter for this method is the String object that points to the name of the file. The output of this method is fed to another method named lines(), which is in the Files class. This method reads all lines from a file as a Stream, and therefore, the output of this method is directed to a Stream variable. Because our dummy.txt file contains string data, the generics of the Stream variable is set to String.

The entire process of reading needs a try...catch block for attempts such as reading a file that does not exist or damaged and so on.

The following code segment displays the contents of our dummy.txt file. The stream variable contains the lines of the text file, and therefore, the forEach() method of the variable is used to display each line content:

        try (Stream<String> stream = Files.lines(Paths.get(file))) { 
        stream.forEach(System.out::println); } catch (IOException e) { 
        System.out.println("Error reading " +  file.getAbsolutePath()); 
        }

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Java Data Science Cookbook

Search

Your notes and bookmarks