In an age where more and more is generated, and especially where every individual can post his or her opinion on the internet, the value of automatically analyzing these posts with high accuracy on a large scale is important for businesses and politics. In Chapter 4, Recurrent and Recursive Neural Networks, we've already shown how to apply RNNs with LSTM units to classify short sentences, such as movie reviews. In the following recipe, we will increase the complexity by classifying the sentiments of Twitter messages. We do this by predicting both binary classes and fine-grained classes.
- We start by all the libraries as follows:
from nltk.tokenize import word_tokenize from nltk.stem import WordNetLemmatizer import numpy as np import random import pickle from collections import Counter import tensorflow as tf
- Next, we process the English sentences with the nltk package. We start by defining the functions we need for preprocessing:
lemmatizer = WordNetLemmatizer...