Book Image

Java Data Science Cookbook

By : Rushdi Shams
Book Image

Java Data Science Cookbook

By: Rushdi Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.
Table of Contents (16 chapters)
Java Data Science Cookbook
About the Author
About the Reviewer
Customer Feedback

Conducting a Kolmogorov-Smirnov test

The Kolmogorov-Smirnov test (or simply KS test) is a test of equality for one-dimensional probability distributions that are continuous in nature. It is one of the popular methods to determine whether two sets of data points differ significantly.

How to do it...

  1. Create a method that takes two different data distributions. We will see if the difference of the two data distributions is significant by using Kolmogorov-Smirnov test:

            public void calculateKs(double[] x, double[] y){ 
  2. One of the key statistics in the test is d-statistic. It is a double value that we will need in order to calculate the p-value of the test:

            double d = TestUtils.kolmogorovSmirnovStatistic(x, y); 
  3. To evaluate the null hypothesis that the values are drawn from a unit normal distribution, use the following code:

            System.out.println(TestUtils.kolmogorovSmirnovTest(x, y, 
  4. Finally, the p-value of the significance test can be found...