Understanding Sentiment in Natural Language with BiLSTMs

Natural Language Understanding (NLU) is a significant subfield of Natural Language Processing (NLP). In the last decade, there has been a resurgence of interest in this field with the dramatic success of chatbots such as Amazon's Alexa and Apple's Siri. This chapter will introduce the broad area of NLU and its main applications.

Specific model architectures called Recurrent Neural Networks (RNNs), with special units called Long Short-Term Memory (LSTM) units, have been developed to make the task of understanding natural language easier. LSTMs in NLP are analogous to convolution blocks in computer vision. We will take two examples to build models that can understand natural language. Our first example is understanding the sentiment of movie reviews. This will be the focus of this chapter. The other example is one of the fundamental building blocks of NLU, Named Entity Recognition (NER). That will be the main focus of the next chapter.

Building models capable of understanding sentiments requires the use of Bi-Directional LSTMs (BiLSTMs) in addition to the use of techniques from Chapter 1, Essentials of NLP. Specifically, the following will be covered in this chapter:

• Overview of NLU and its applications
• Overview of RNNs and BiRNNS using LSTMs and BiLSTMS
• Analyzing the sentiment of movie reviews with LSTMs and BiLSTMs
• Using tf.data and the TensorFlow Datasets package to manage the loading of data
• Optimizing the performance of data loading for effective utilization of the CPU and GPU

We will start with a quick overview of NLU and then get right into BiLSTMs.