Book Image

Java Data Science Cookbook

By : Rushdi Shams
Book Image

Java Data Science Cookbook

By: Rushdi Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.
Table of Contents (16 chapters)
Java Data Science Cookbook
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Extracting PDF text using Apache Tika


One of the most difficult file types for parsing and extracting data is PDF. Some PDFs are not even possible to parse because they are password-protected, while some others contain scanned texts and images. This dynamic file type, therefore, sometimes becomes the worst nightmare for data scientists. This recipe demonstrates how to extract text from PDF files using Apache Tika, given that the file is not encrypted or password-protected and contains text that is not scanned.

Getting ready

In order to perform this recipe we will require the following:

  1. Download Apache Tika 1.10 JAR file from http://archive.apache.org/dist/tika/tika-app-1.10.jar, and include it in your Eclipse project as an external Java library.

  2. Have any unlocked PDF file saved as testPDF.pdf on your C: drive.

How to do it...

  1. Create a method named convertPdf(String), which takes the name of the PDF file to be converted as parameter:

            public void convertPDF(String fileName){ 
    
  2. Create an input stream that will contain the PDF data as a stream of bytes:

            InputStream stream = null; 
    
  3. Create a try block as follows:

            try{ 
    
  4. Assign the file to the stream you have just created:

            stream = new FileInputStream(fileName); 
    
  5. There are many different parsers offered in the Apache Tika package. If you do not know which parser you are going to use, or say you have not only PDFs but also other types of documents to get converted, you should use an AutoDetectParser as follows:

            AutoDetectParser parser = new AutoDetectParser(); 
    
  6. Create a handler to handle the body content of the file. Note the -1 as the parameter of the constructor. Usually, Apache Tika is limited to handling files with at most 100,000 characters. The -1 value ensures that this limitation is overlooked by the body handler:

            BodyContentHandler handler = new BodyContentHandler(-1); 
    
  7. Create a metadata object:

            Metadata metadata = new Metadata(); 
    
  8. Call the parser() method of the parser object with all these objects you just created:

            parser.parse(stream, handler, metadata, new ParseContext()); 
    
  9. Use the tostring() method of the handler object to get the body text extracted from the file:

            System.out.println(handler.toString()); 
    
  10. Close the try block and complement it with a catch block and finally block, and close the method as follows:

            }catch (Exception e) { 
                          e.printStackTrace(); 
                     }finally { 
                         if (stream != null) 
                              try { 
                                   stream.close(); 
                              } catch (IOException e) { 
                                System.out.println("Error closing stream"); 
                              } 
                       } 
            } 
    

    The full method with the driver method in a class will be as follows. The method you have just created can be called by sending it the path and the name of the PDF file you need to convert, which is in your C: drive saved as testPDF.pdf:

    import java.io.FileInputStream; 
    import java.io.IOException; 
    import java.io.InputStream; 
    import org.apache.tika.metadata.Metadata; 
    import org.apache.tika.parser.AutoDetectParser; 
    import org.apache.tika.parser.ParseContext; 
    import org.apache.tika.sax.BodyContentHandler; 
     
    public class TestTika { 
         public static void main(String args[]) throws Exception { 
              TestTika tika = new TestTika(); 
              tika.convertPdf("C:/testPDF.pdf"); 
         } 
         public void convertPdf(String fileName){ 
              InputStream stream = null; 
              try { 
                  stream = new FileInputStream(fileName); 
                  AutoDetectParser parser = new AutoDetectParser(); 
                  BodyContentHandler handler = new BodyContentHandler(-1); 
                  Metadata metadata = new Metadata(); 
                  parser.parse(stream, handler, metadata, new 
                      ParseContext()); 
                  System.out.println(handler.toString()); 
              }catch (Exception e) { 
                  e.printStackTrace(); 
              }finally { 
                  if (stream != null) 
                       try { 
                            stream.close(); 
                       } catch (IOException e) { 
                            System.out.println("Error closing stream"); 
                       } 
              } 
         } 
    }