Book Image

Java Data Science Cookbook

By : Rushdi Shams
Book Image

Java Data Science Cookbook

By: Rushdi Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.
Table of Contents (16 chapters)
Java Data Science Cookbook
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Retrieving all filenames from hierarchical directories using Java


This recipe (and the following) is for the data scientist who wants to retrieve the file paths and names (for some future analysis) from a complex directory structure that contains numerous directories and files inside a root directory.

Getting ready

In order to perform this recipe, we will require the following:

  1. Create directories within directories (as many layers as you want).

  2. Create text files in some of these directories while leaving some directories empty for more excitement.

How to do it...

  1. We are going to create a static method that takes a File argument, which is eventually the root directory or the directory to start with. The method will return a set of files that are found within this root directory (and in all other subsequent directories):

            public static Set<File> listFiles(File rootDir) {  
    
  2. First, create a HashSet that will contain the file information:

            Set<File> fileSet = new HashSet<File>(); 
    
  3. Once the HashSet is created, we need to check whether the root directory or the directories within it are null. For such cases, we do not need to proceed further:

            if (rootDir == null || rootDir.listFiles() == null){ 
                         return fileSet; 
               } 
    
  4. We consider one directory (or file) from the root directory at a time and check whether we are dealing with a file or with a directory. In the case of a file, we add that to our HashSet. In the case of a directory, we recursively call this method again by sending the path and name of the directory:

            for (File fileOrDir : rootDir.listFiles()) { 
                         if (fileOrDir.isFile()){ 
                           fileSet.add(fileOrDir); 
                         } 
                         else{ 
                           fileSet.addAll(listFiles(fileOrDir)); 
                         } 
                 } 
    
  5. Finally, we return the HashSet to the caller of this method:

           return fileSet; 
              } 
    

    The complete method, with the class and the driver method to run it, is as follows:

    import java.io.File; 
    import java.util.HashSet; 
    import java.util.Set; 
     
    public class TestRecursiveDirectoryTraversal { 
       public static void main(String[] args){ 
          System.out.println(listFiles(new File("Path for root 
              directory")).size()); 
       } 
        
       public static Set<File> listFiles(File rootDir) { 
           Set<File> fileSet = new HashSet<File>(); 
           if(rootDir == null || rootDir.listFiles()==null){ 
               return fileSet; 
           } 
           for (File fileOrDir : rootDir.listFiles()) { 
                 if (fileOrDir.isFile()){ 
                   fileSet.add(fileOrDir); 
                 } 
                 else{ 
                   fileSet.addAll(listFiles(fileOrDir)); 
                 } 
         } 
     
           return fileSet; 
       } 
    } 
    

Note

Note the use of HashSet to store the paths and names of the files. This means that we will not have any duplicates since the Set data structures in Java do not contain duplicate entries.