-
Book Overview & Buying
-
Table Of Contents
Python Data Science Essentials - Second Edition
By :
This section is not strictly related to machine learning, but it contains some machine learning results in the area of Natural Language Processing. Python has many toolkits to process text data, but the most powerful and complete toolkit is NLTK, the Natural Language Tool Kit.
In the following sections, we'll explore its core functionalities. We will work on the English language; for other languages, you will first need to download the language corpora (note that sometimes languages have no free open source corpora for NLTK).
Refer to the official website of NLTK data, http://www.nltk.org/nltk_data/, to have access to corporas and lexical resources in many languages, ready to work with NLTK.
Tokenization is the action of splitting text into words. Chunking whitespace seems very easy, but it's not, because text contains punctuation and contractions. Let's start with an example:
In: my_text = "The coolest job in the next 10 years will...