Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Java Data Science Cookbook
  • Table Of Contents Toc
  • Feedback & Rating feedback
Java Data Science Cookbook

Java Data Science Cookbook

By : Shams
close
close
Java Data Science Cookbook

Java Data Science Cookbook

By: Shams

Overview of this book

If you are looking to build data science models that are good for production, Java has come to the rescue. With the aid of strong libraries such as MLlib, Weka, DL4j, and more, you can efficiently perform all the data science tasks you need to. This unique book provides modern recipes to solve your common and not-so-common data science-related problems. We start with recipes to help you obtain, clean, index, and search data. Then you will learn a variety of techniques to analyze, learn from, and retrieve information from data. You will also understand how to handle big data, learn deeply from data, and visualize data. Finally, you will work through unique recipes that solve your problems while taking data science to production, writing distributed data science applications, and much more - things that will come in handy at work.
Table of Contents (10 chapters)
close
close

Extracting PDF text using Apache Tika

One of the most difficult file types for parsing and extracting data is PDF. Some PDFs are not even possible to parse because they are password-protected, while some others contain scanned texts and images. This dynamic file type, therefore, sometimes becomes the worst nightmare for data scientists. This recipe demonstrates how to extract text from PDF files using Apache Tika, given that the file is not encrypted or password-protected and contains text that is not scanned.

Getting ready

In order to perform this recipe we will require the following:

  1. Download Apache Tika 1.10 JAR file from http://archive.apache.org/dist/tika/tika-app-1.10.jar, and include it in your Eclipse project as an external Java library.
  2. Have any unlocked PDF file saved as testPDF.pdf on your C: drive.

How to do it...

  1. Create a method named convertPdf(String), which takes the name of the PDF file to be converted as parameter:
            public void convertPDF(String fileName){ 
    
  2. Create an input stream that will contain the PDF data as a stream of bytes:
            InputStream stream = null; 
    
  3. Create a try block as follows:
            try{ 
    
  4. Assign the file to the stream you have just created:
            stream = new FileInputStream(fileName); 
    
  5. There are many different parsers offered in the Apache Tika package. If you do not know which parser you are going to use, or say you have not only PDFs but also other types of documents to get converted, you should use an AutoDetectParser as follows:
            AutoDetectParser parser = new AutoDetectParser(); 
    
  6. Create a handler to handle the body content of the file. Note the -1 as the parameter of the constructor. Usually, Apache Tika is limited to handling files with at most 100,000 characters. The -1 value ensures that this limitation is overlooked by the body handler:
            BodyContentHandler handler = new BodyContentHandler(-1); 
    
  7. Create a metadata object:
            Metadata metadata = new Metadata(); 
    
  8. Call the parser() method of the parser object with all these objects you just created:
            parser.parse(stream, handler, metadata, new ParseContext()); 
    
  9. Use the tostring() method of the handler object to get the body text extracted from the file:
            System.out.println(handler.toString()); 
    
  10. Close the try block and complement it with a catch block and finally block, and close the method as follows:
            }catch (Exception e) { 
                          e.printStackTrace(); 
                     }finally { 
                         if (stream != null) 
                              try { 
                                   stream.close(); 
                              } catch (IOException e) { 
                                System.out.println("Error closing stream"); 
                              } 
                       } 
            } 
    

    The full method with the driver method in a class will be as follows. The method you have just created can be called by sending it the path and the name of the PDF file you need to convert, which is in your C: drive saved as testPDF.pdf:

    import java.io.FileInputStream; 
    import java.io.IOException; 
    import java.io.InputStream; 
    import org.apache.tika.metadata.Metadata; 
    import org.apache.tika.parser.AutoDetectParser; 
    import org.apache.tika.parser.ParseContext; 
    import org.apache.tika.sax.BodyContentHandler; 
     
    public class TestTika { 
         public static void main(String args[]) throws Exception { 
              TestTika tika = new TestTika(); 
              tika.convertPdf("C:/testPDF.pdf"); 
         } 
         public void convertPdf(String fileName){ 
              InputStream stream = null; 
              try { 
                  stream = new FileInputStream(fileName); 
                  AutoDetectParser parser = new AutoDetectParser(); 
                  BodyContentHandler handler = new BodyContentHandler(-1); 
                  Metadata metadata = new Metadata(); 
                  parser.parse(stream, handler, metadata, new 
                      ParseContext()); 
                  System.out.println(handler.toString()); 
              }catch (Exception e) { 
                  e.printStackTrace(); 
              }finally { 
                  if (stream != null) 
                       try { 
                            stream.close(); 
                       } catch (IOException e) { 
                            System.out.println("Error closing stream"); 
                       } 
              } 
         } 
    } 
    
Visually different images
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Java Data Science Cookbook
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon