Book Image

Natural Language Understanding with Python

By : Deborah A. Dahl
5 (1)
Book Image

Natural Language Understanding with Python

5 (1)
By: Deborah A. Dahl

Overview of this book

Natural Language Understanding facilitates the organization and structuring of language allowing computer systems to effectively process textual information for various practical applications. Natural Language Understanding with Python will help you explore practical techniques for harnessing NLU to create diverse applications. with step-by-step explanations of essential concepts and practical examples, you’ll begin by learning about NLU and its applications. You’ll then explore a wide range of current NLU techniques and their most appropriate use-case. In the process, you’ll be introduced to the most useful Python NLU libraries. Not only will you learn the basics of NLU, you’ll also discover practical issues such as acquiring data, evaluating systems, and deploying NLU applications along with their solutions. The book is a comprehensive guide that’ll help you explore techniques and resources that can be used for different applications in the future. By the end of this book, you’ll be well-versed with the concepts of natural language understanding, deep learning, and large language models (LLMs) for building various AI-based applications.
Table of Contents (21 chapters)
1
Part 1: Getting Started with Natural Language Understanding Technology
4
Part 2:Developing and Testing Natural Language Understanding Systems
16
Part 3: Systems in Action – Applying Natural Language Understanding at Scale

Data exploration

Data exploration, which is sometimes also called exploratory data analysis (EDA), is the process of taking a first look at your data to see what kinds of patterns there are to get an overall perspective on the full dataset. These patterns and overall perspective will help us identify the most appropriate processing approaches. Because some NLU techniques are very computationally intensive, we want to ensure that we don’t waste a lot of time applying a technique that is inappropriate for a particular dataset. Data exploration can help us narrow down the options for techniques at the very beginning of our project. Visualization is a great help in data exploration because it is a quick way to get the big picture of patterns in the data.

The most basic kind of information about a corpus that we would want to explore includes information such as the number of words, the number of distinct words, the average length of documents, and the number of documents in each...