Book Image

Natural Language Processing with Java Cookbook

By : Richard M. Reese

Book Image

Natural Language Processing with Java Cookbook

By: Richard M. Reese

Overview of this book

Natural Language Processing (NLP) has become one of the prime technologies for processing very large amounts of unstructured data from disparate information sources. This book includes a wide set of recipes and quick methods that solve challenges in text syntax, semantics, and speech tasks. At the beginning of the book, you'll learn important NLP techniques, such as identifying parts of speech, tagging words, and analyzing word semantics. You will learn how to perform lexical analysis and use machine learning techniques to speed up NLP operations. With independent recipes, you will explore techniques for customizing your existing NLP engines/models using Java libraries such as OpenNLP and the Stanford NLP library. You will also learn how to use NLP processing features from cloud-based sources, including Google and Amazon Web Services (AWS). You will master core tasks, such as stemming, lemmatization, part-of-speech tagging, and named entity recognition. You will also learn about sentiment analysis, semantic text similarity, language identification, machine translation, and text summarization. By the end of this book, you will be ready to become a professional NLP expert using a problem-solution approach to analyze any sort of text, sentence, or semantic word.

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Preparing Text for Analysis and Tokenization

Preparing Text for Analysis and Tokenization

Technical requirements

Tokenization using the Java SDK

Tokenization using OpenNLP

Tokenization using maximum entropy

Training a neural network tokenizer for specialized text

Identifying the stem of a word

Training an OpenNLP lemmatization model

Determining the lexical meaning of a word using OpenNLP

Removing stop words using LingPipe

Isolating Sentences within a Document

Isolating Sentences within a Document

Technical requirements

Finding sentences using the Java core API

Performing SBD using the BreakIterator class

Using OpenNLP to perform SBD

Using the Stanford NLP API to perform SBD

Using the LingPipe and chunking to perform SBD

Performing SBD on specialized text

Training a neural network to perform SBD with specialized text

Performing Name Entity Recognition

Performing Name Entity Recognition

Technical requirements

Using regular expressions to find entities

Using chunks with regular expressions to identify entities

Using OpenNLP to find entities in text

Isolating multiple entities types

Using a CRF model to find entities in a document

Using a chunker to find entities

Training a specialized NER model

Detecting POS Using Neural Networks

Detecting POS Using Neural Networks

Technical requirements

Finding POS using tagging

Using a chunker to find POS

Using a tag dictionary

Finding POS using the Penn Treebank

Finding POS from textese

Using a pipeline to perform tagging

Using a hidden Markov model to perform POS

Training a specialized POS model

Performing Text Classification

Performing Text Classification

Technical requirements

Training a maximum entropy model for text classification

Classifying documents using a maximum entropy model

Classifying documents using the Stanford API

Training a model to classify text using LingPipe

Using LingPipe to classify text

Performing sentiment analysis on reviews

Finding Relationships within Text

Finding Relationships within Text

Technical requirements

Displaying parse trees graphically

Using probabilistic context-free grammar to parse text

Using OpenNLP to generate a parse tree

Using the Google NLP API to parse text

Identifying parent-child relationships in text

Finding co-references in a sentence

Language Identification and Translation

Language Identification and Translation

Technical requirements

Detecting the natural language in use using LingPipe

Discovering supported languages using the Google API

Detecting the natural language in use using the Google API

Language translation using Google

Language detection and translation using Amazon AWS

Converting text to speech using the Google Cloud Text-to-Speech API

Converting speech to text using the Google Cloud Speech-to-Text API

Identifying Semantic Similarities within Text

Identifying Semantic Similarities within Text

Technical requirements

Finding the cosine similarity of the text

Finding the distance between text

Finding differences between plaintext instances

Finding hyponyms and antonyms

Common Text Processing and Generation Tasks

Common Text Processing and Generation Tasks

Technical requirements

Generating random numbers

Spell-checking using the LanguageTool API

Checking grammar using the LanguageTool API

Summarizing text in a document

Creating, inverting, and using dictionaries

Extracting Data for Use in NLP Analysis

Extracting Data for Use in NLP Analysis

Technical requirements

Connecting to an HTML page

Extracting text and metadata from an HTML page

Extracting text from a PDF document

Extracting metadata from a PDF document

Extracting text from a Word document

Extracting metadata from a Word document

Extracting text from a spreadsheet

Extracting metadata from a spreadsheet

Creating a Chatbot

Creating a Chatbot

Technical requirements

Creating a simple chatbot using AWS

Creating a bot using AWS Toolkit for Eclipse

Creating a Lambda function

Uploading the Lambda function

Executing a Lambda function from Eclipse

Installation and Configuration

Installation and Configuration

Technical requirements

Getting ready to use the Google Cloud Platform

Configuring Eclipse to use the Google Cloud Platform

Getting ready to use Amazon Web Services

Configuring Eclipse to use Amazon Web Services

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Detecting the natural language in use using the Google API

If the language being processed is unknown, we can use the GCP to determine the language. In this recipe, we will illustrate how to do this using the Translate class's detect method. To illustrate the process, we will use the first paragraph of the English and German Wikipedia web pages for the word language. These are shown next. The following shows the first paragraph of the English Wikipedia web page (https://en.wikipedia.org/wiki/Language):

"Language is a system that consists of the development, acquisition, maintenance and use of complex systems of communication, particularly the human ability to do so; and a language is any specific example of such a system."

The following shows the first paragraph of the German Wikipedia web page (https://de.wikipedia.org/wiki/Language):

"Language ist ein Lied...