Book Image

Advanced Natural Language Processing with TensorFlow 2

By : Ashish Bansal, Tony Mullen
Book Image

Advanced Natural Language Processing with TensorFlow 2

By: Ashish Bansal, Tony Mullen

Overview of this book

Recently, there have been tremendous advances in NLP, and we are now moving from research labs into practical applications. This book comes with a perfect blend of both the theoretical and practical aspects of trending and complex NLP techniques. The book is focused on innovative applications in the field of NLP, language generation, and dialogue systems. It helps you apply the concepts of pre-processing text using techniques such as tokenization, parts of speech tagging, and lemmatization using popular libraries such as Stanford NLP and SpaCy. You will build Named Entity Recognition (NER) from scratch using Conditional Random Fields and Viterbi Decoding on top of RNNs. The book covers key emerging areas such as generating text for use in sentence completion and text summarization, bridging images and text by generating captions for images, and managing dialogue aspects of chatbots. You will learn how to apply transfer learning and fine-tuning using TensorFlow 2. Further, it covers practical techniques that can simplify the labelling of textual data. The book also has a working code that is adaptable to your use cases for each tech piece. By the end of the book, you will have an advanced knowledge of the tools, techniques and deep learning architecture used to solve complex NLP problems.
Table of Contents (13 chapters)
Other Books You May Enjoy

What this book covers

Chapter 1, Essentials of NLP, provides an overview of various topics in NLP such as tokenization, stemming, lemmatization, POS tagging, vectorization, etc. An overview of common NLP libraries like spaCy, Stanford NLP, and NLTK, with their key capabilities and use cases, will be provided. We will also build a simple classifier for spam.

Chapter 2, Understanding Sentiment in Natural Language with BiLSTMs, covers the NLU use case of sentiment analysis with an overview of Recurrent Neural Networks (RNNs), LSTMs, and BiLSTMs, which are the basic building blocks of modern NLP models. We will also use for efficient use of CPUs and GPUs to speed up data pipelines and model training.

Chapter 3, Named Entity Recognition (NER) with BiLSTMs, CRFs, and Viterbi Decoding, focuses on the key NLU problem of NER, which is a basic building block of task-oriented chatbots. We will build a custom layer for CRFs for improving the accuracy of NER and the Viterbi decoding scheme, which is often applied to a deep model to improve the quality of the output.

Chapter 4, Transfer Learning with BERT, covers a number of important concepts in modern deep NLP such as types of transfer learning, pre-trained embeddings, an overview of Transformers, and BERT and its application in improving the sentiment analysis task introduced in Chapter 2, Understanding Sentiment in Natural Language with BiLSTMs.

Chapter 5, Generating Text with RNNs and GPT-2, focuses on generating text with a custom character-based RNN and improving it with Beam Search. We will also cover the GPT-2 architecture and touch upon GPT-3.

Chapter 6, Text Summarization with Seq2seq Attention and Transformer Networks, takes on the challenging task of abstractive text summarization. BERT and GPT are two halves of the full encoder-decoder model. We put them together to build a seq2seq model for summarizing news articles by generating headlines for them. How ROUGE metrics are used for the evaluation of summarization is also covered.

Chapter 7, Multi-Modal Networks and Image Captioning with ResNets and Transformers, combines computer vision and NLP together to see if a picture is indeed worth a thousand words! We will build a custom Transformer model from scratch and train it to generate captions for images.

Chapter 8, Weakly Supervised Learning for Classification with Snorkel, focuses on a key problem – labeling data. While NLP has a lot of unlabeled data, labeling it is quite an expensive task. This chapter introduces the snorkel library and shows how massive amounts of data can be quickly labeled.

Chapter 9, Building Conversational AI Applications with Deep Learning, combines the various techniques covered throughout the book to show how different types of chatbots, such as question-answering or slot-filling bots, can be built.

Chapter 10, Installation and Setup Instructions for Code, walks through all the instructions required to install and configure a system for running the code supplied with the book.