Natural Language Processing (NLP) has been used to address a wide range of problems, including support for search engines, summarizing and classifying text for web pages, and incorporating machine learning technologies to solve problems such as speech recognition and query analysis. It has found use wherever documents contain useful information.
NLP is used to enhance the utility and power of applications. It does so by making user input easier and converting text to more usable forms. In essence, NLP processes natural text found in a variety of sources, using a series of core NLP tasks to transform or extract information from the text.
This book focuses on core NLP tasks that will likely be encountered in an NLP application. Each NLP task presented in this book starts with a description of the problem and where it can be used. The issues that make each task difficult are introduced so that you can understand the problem in a better way. This is followed by the use of numerous Java techniques and APIs to support an NLP task.
Chapter 1, Introduction to NLP, explains the importance and uses of NLP. The NLP techniques used in this chapter are explained with simple examples illustrating their use.
Chapter 2, Finding Parts of Text, focuses primarily on tokenization. This is the first step in more advanced NLP tasks. Both core Java and Java NLP tokenization APIs are illustrated.
Chapter 3, Finding Sentences, proves that sentence boundary disambiguation is an important NLP task. This step is a precursor for many other downstream NLP tasks where text elements should not be split across sentence boundaries. This includes ensuring that all phrases are in one sentence and supporting parts of speech analysis.
Chapter 4, Finding People and Things, covers what is commonly referred to as Named Entity Recognition. This task is concerned with identifying people, places, and similar entities in text. This technique is a preliminary step for processing queries and searches.
Chapter 5, Detecting Parts of Speech, shows you how to detect parts of speech, which are grammatical elements of text, such as nouns and verbs. Identifying these elements is a significant step in determining the meaning of text and detecting relationships within text.
Chapter 6, Classifying Texts and Documents, proves that classifying text is useful for tasks such as spam detection and sentiment analysis. The NLP techniques that support this process are investigated and illustrated.
Chapter 7, Using Parser to Extract Relationships, demonstrates parse trees. A parse tree is used for many purposes, including information extraction. It holds information regarding the relationships between these elements. An example implementing a simple query is presented to illustrate this process.
Chapter 8, Combined Approaches, contains techniques for extracting data from various types of documents, such as PDF and Word files. This is followed by an examination of how the previous NLP techniques can be combined into a pipeline to solve larger problems.
Java SDK 7 is used to illustrate the NLP techniques. Various NLP APIs are needed and can be readily downloaded. An IDE is not required but is desirable.
Experienced Java developers who are interested in NLP techniques will find this book useful. No prior exposure to NLP is required.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and explanations of their meanings.
Code words in text are shown as follows: "The keyset method returns a set of all the annotation keys currently held by the Annotation object."
Database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "To demonstrate the use of POI, we will use a file called TestDocument.pdf."
A block of code is set as follows:
for (int index = 0; index < sentences.length; index++) {
String tokens[] = tokenizer.tokenize(sentences[index]);
Span nameSpans[] = nameFinder.find(tokens);
for(Span span : nameSpans) {
list.add("Sentence: " + index
+ " Span: " + span.toString() + " Entity: "
+ tokens[span.getStart()]);
}
}The output of code sequences looks like what is shown here:
Sentence: 0 Span: [0..1) person Entity: Joe Sentence: 0 Span: [7..9) person Entity: Fred Sentence: 2 Span: [0..1) person Entity: Joe
New terms and important words are shown in bold.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.