Summary
We started this chapter with a discussion about the KDE and its usefulness in understanding the underlying distribution of data. We proceeded by explaining how to extract tweets from Twitter for a given search string in R. Then, we proceeded to explain the sentiment ming, dictionary, and machine learning approaches. Using a dictionary approach, we calculated the sentiment scores for the tweets. We further explained text pre-processing routines required to prepare the text data. We covered weighting schemes for creating document term matrixes. We discussed the classic tfidf
and the new Delta TFIDF schemes. We created our training set using the Delta TFIDF scheme. Using this training set, we finally built a Naive Bayes KDE classifier to classify tweets based on the sentiment the text carried.
In the next chapter, we will be working on Record Linakage. A master data management technique to do data dedpulication.