Book Image

Natural Language Processing with Java - Second Edition

By : Richard M. Reese
Book Image

Natural Language Processing with Java - Second Edition

By: Richard M. Reese

Overview of this book

Natural Language Processing (NLP) allows you to take any sentence and identify patterns, special names, company names, and more. The second edition of Natural Language Processing with Java teaches you how to perform language analysis with the help of Java libraries, while constantly gaining insights from the outcomes. You’ll start by understanding how NLP and its various concepts work. Having got to grips with the basics, you’ll explore important tools and libraries in Java for NLP, such as CoreNLP, OpenNLP, Neuroph, and Mallet. You’ll then start performing NLP on different inputs and tasks, such as tokenization, model training, parts-of-speech and parsing trees. You’ll learn about statistical machine translation, summarization, dialog systems, complex searches, supervised and unsupervised NLP, and more. By the end of this book, you’ll have learned more about NLP, neural networks, and various other trained models in Java for enhancing the performance of NLP applications.

Related Content you might be interested in

Current Title:

Small book image
Natural Language Processing with Java - Second Edition
Richard M. Reese
Jul, 2018 | 318 pages
Small book image
Natural Language Processing with Java Cookbook
Richard Reese
Apr, 2019 | 386 pages
Small book image
Java for Data Science
Richard Reese | Jennifer L Reese
Jan, 2017 | 386 pages
Small book image
Java Data Science Cookbook
Rushdi Shams
Mar, 2017 | 372 pages
Table of Contents (19 chapters)
Title Page
Title Page
Dedication
Dedication
Packt Upsell
Packt Upsell
Contributors
Contributors
Preface
Preface
Free Chapter
1
Introduction to NLP
Introduction to NLP
What is NLP?
Why use NLP?
Why is NLP so hard?
Survey of NLP tools
Deep learning for Java
Overview of text-processing tasks
Understanding NLP models
Preparing data
Summary
2
Finding Parts of Text
Finding Parts of Text
Understanding the parts of text
What is tokenization?
Simple Java tokenizers
NLP tokenizer APIs
Understanding normalization
Summary
3
Finding Sentences
Finding Sentences
The SBD process
What makes SBD difficult?
Understanding the SBD rules of LingPipe's HeuristicSentenceModel class
Simple Java SBDs
Using NLP APIs
Training a sentence-detector model
Summary
4
Finding People and Things
Finding People and Things
Why is NER difficult?
Techniques for name recognition
Using regular expressions for NER
Using NLP APIs
Building a new dataset with the NER annotation tool
Training a model
Summary
5
Detecting Part of Speech
Detecting Part of Speech
The tagging process
Using the NLP APIs
Summary
6
Representing Text with Features
Representing Text with Features
N-grams
Word embedding
GloVe
Word2vec
Dimensionality reduction
Principle component analysis
Distributed stochastic neighbor embedding
Summary
7
Information Retrieval
Information Retrieval
Boolean retrieval
Dictionaries and tolerant retrieval
Vector space model
Scoring and term weighting
Inverse document frequency
TF-IDF weighting
Evaluation of information retrieval systems
Summary
8
Classifying Texts and Documents
Classifying Texts and Documents
How classification is used
Understanding sentiment analysis
Text-classifying techniques
Using APIs to classify text
Summary
9
Topic Modeling
Topic Modeling
What is topic modeling?
The basics of LDA
Topic modeling with MALLET
Summary
10
Using Parsers to Extract Relationships
Using Parsers to Extract Relationships
Relationship types
Understanding parse trees
Using extracted relationships
Extracting relationships
Using NLP APIs
Extracting relationships for a question-answer system
Summary
11
Combined Pipeline
Combined Pipeline
Preparing data
Using boilerpipe to extract text from HTML
Using POI to extract text from Word documents
Using PDFBox to extract text from PDF documents
Using Apache Tika for content analysis and extraction
Pipelines
Using the Stanford pipeline
Using multiple cores with the Stanford pipeline
Creating a pipeline to search text
Summary
12
Creating a Chatbot
Creating a Chatbot
Chatbot architecture
Artificial Linguistic Internet Computer Entity
Summary
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Index
Index

Simple Java tokenizers

There are several Java classes that support simple tokenization; some of them are as follows:

  • Scanner
  • String
  • BreakIterator
  • StreamTokenizer
  • StringTokenizer

Although these classes provide limited support, it is useful to understand how they can be used. For some tasks, these classes will suffice. Why use a more difficult-to-understand and less-efficient approach when a core Java class can do the job? We will cover each of these classes as they support the tokenization process.

The StreamTokenizer and StringTokenizer classes should not be used for new developments. Instead, the String class' split method is usually a better choice. They have been included here in case you come across them and wonder whether they should be used or not.

Using the Scanner class

The Scanner class is used to read data from a text source. This might be standard input or it could be from a file. It provides a simple-to-use technique to support tokenization.

The Scanner class uses whitespace as the default...

Previous Section
End of Section 4
Next Section