Book Image

Natural Language Processing with Java Cookbook

By : Richard M. Reese

Book Image

Natural Language Processing with Java Cookbook

By: Richard M. Reese

Overview of this book

Natural Language Processing (NLP) has become one of the prime technologies for processing very large amounts of unstructured data from disparate information sources. This book includes a wide set of recipes and quick methods that solve challenges in text syntax, semantics, and speech tasks. At the beginning of the book, you'll learn important NLP techniques, such as identifying parts of speech, tagging words, and analyzing word semantics. You will learn how to perform lexical analysis and use machine learning techniques to speed up NLP operations. With independent recipes, you will explore techniques for customizing your existing NLP engines/models using Java libraries such as OpenNLP and the Stanford NLP library. You will also learn how to use NLP processing features from cloud-based sources, including Google and Amazon Web Services (AWS). You will master core tasks, such as stemming, lemmatization, part-of-speech tagging, and named entity recognition. You will also learn about sentiment analysis, semantic text similarity, language identification, machine translation, and text summarization. By the end of this book, you will be ready to become a professional NLP expert using a problem-solution approach to analyze any sort of text, sentence, or semantic word.

Preface

Who this book is for

What this book covers

To get the most out of this book

Free Chapter

Preparing Text for Analysis and Tokenization

Preparing Text for Analysis and Tokenization

Technical requirements

Tokenization using the Java SDK

Tokenization using OpenNLP

Tokenization using maximum entropy

Training a neural network tokenizer for specialized text

Identifying the stem of a word

Training an OpenNLP lemmatization model

Determining the lexical meaning of a word using OpenNLP

Removing stop words using LingPipe

Isolating Sentences within a Document

Isolating Sentences within a Document

Technical requirements

Finding sentences using the Java core API

Performing SBD using the BreakIterator class

Using OpenNLP to perform SBD

Using the Stanford NLP API to perform SBD

Using the LingPipe and chunking to perform SBD

Performing SBD on specialized text

Training a neural network to perform SBD with specialized text

Performing Name Entity Recognition

Performing Name Entity Recognition

Technical requirements

Using regular expressions to find entities

Using chunks with regular expressions to identify entities

Using OpenNLP to find entities in text

Isolating multiple entities types

Using a CRF model to find entities in a document

Using a chunker to find entities

Training a specialized NER model

Detecting POS Using Neural Networks

Detecting POS Using Neural Networks

Technical requirements

Finding POS using tagging

Using a chunker to find POS

Using a tag dictionary

Finding POS using the Penn Treebank

Finding POS from textese

Using a pipeline to perform tagging

Using a hidden Markov model to perform POS

Training a specialized POS model

Performing Text Classification

Performing Text Classification

Technical requirements

Training a maximum entropy model for text classification

Classifying documents using a maximum entropy model

Classifying documents using the Stanford API

Training a model to classify text using LingPipe

Using LingPipe to classify text

Performing sentiment analysis on reviews

Finding Relationships within Text

Finding Relationships within Text

Technical requirements

Displaying parse trees graphically

Using probabilistic context-free grammar to parse text

Using OpenNLP to generate a parse tree

Using the Google NLP API to parse text

Identifying parent-child relationships in text

Finding co-references in a sentence

Language Identification and Translation

Language Identification and Translation

Technical requirements

Detecting the natural language in use using LingPipe

Discovering supported languages using the Google API

Detecting the natural language in use using the Google API

Language translation using Google

Language detection and translation using Amazon AWS

Converting text to speech using the Google Cloud Text-to-Speech API

Converting speech to text using the Google Cloud Speech-to-Text API

Identifying Semantic Similarities within Text

Identifying Semantic Similarities within Text

Technical requirements

Finding the cosine similarity of the text

Finding the distance between text

Finding differences between plaintext instances

Finding hyponyms and antonyms

Common Text Processing and Generation Tasks

Common Text Processing and Generation Tasks

Technical requirements

Generating random numbers

Spell-checking using the LanguageTool API

Checking grammar using the LanguageTool API

Summarizing text in a document

Creating, inverting, and using dictionaries

Extracting Data for Use in NLP Analysis

Extracting Data for Use in NLP Analysis

Technical requirements

Connecting to an HTML page

Extracting text and metadata from an HTML page

Extracting text from a PDF document

Extracting metadata from a PDF document

Extracting text from a Word document

Extracting metadata from a Word document

Extracting text from a spreadsheet

Extracting metadata from a spreadsheet

Creating a Chatbot

Creating a Chatbot

Technical requirements

Creating a simple chatbot using AWS

Creating a bot using AWS Toolkit for Eclipse

Creating a Lambda function

Uploading the Lambda function

Executing a Lambda function from Eclipse

Installation and Configuration

Installation and Configuration

Technical requirements

Getting ready to use the Google Cloud Platform

Configuring Eclipse to use the Google Cloud Platform

Getting ready to use Amazon Web Services

Configuring Eclipse to use Amazon Web Services

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Discovering supported languages using the Google API

The Google Translate API supports a number of languages. In this recipe, we will illustrate how we can programmatically determine which languages are supported. This is useful when we need to determine the two-letter code for a language, which is needed for some translation tasks. These two-letter codes generally conform to the ISO-639-1 identifiers, as documented at https://en.wikipedia.org/wiki/ISO_639-1.

In addition, we may need to be aware of any new languages that are supported. A list of currently-supported languages can be found at https://cloud.google.com/translate/docs/languages.

The Google Cloud Platform (GCP) Translate API supports phrase-based machine translation (PBMT) and neural machine translation (NMT) models. PBMT is a statistical approach that requires bilingual text and tries to predict how words should be...