Book Image

Natural Language Processing Fundamentals

By : Sohom Ghosh, Dwight Gunning
Book Image

Natural Language Processing Fundamentals

By: Sohom Ghosh, Dwight Gunning

Overview of this book

If NLP hasn't been your forte, Natural Language Processing Fundamentals will make sure you set off to a steady start. This comprehensive guide will show you how to effectively use Python libraries and NLP concepts to solve various problems. You'll be introduced to natural language processing and its applications through examples and exercises. This will be followed by an introduction to the initial stages of solving a problem, which includes problem definition, getting text data, and preparing it for modeling. With exposure to concepts like advanced natural language processing algorithms and visualization techniques, you'll learn how to create applications that can extract information from unstructured data and present it as impactful visuals. Although you will continue to learn NLP-based techniques, the focus will gradually shift to developing useful applications. In these sections, you'll understand how to apply NLP techniques to answer questions as can be used in chatbots. By the end of this book, you'll be able to accomplish a varied range of assignments ranging from identifying the most suitable type of NLP task for solving a problem to using a tool like spacy or gensim for performing sentiment analysis. The book will easily equip you with the knowledge you need to build applications that interpret human language.
Table of Contents (10 chapters)

5. Topic Modeling

Activity 10: Topic Modelling Jeopardy Questions


Let's perform topic modeling on the dataset of Jeopardy questions. Follow these steps to implement this activity:

  1. Open a Jupyter notebook.
  2. Insert a new cell and add the following code to import the pandas library:
    import pandas as pd
    pd.set_option('display.max_colwidth', 800)
  3. To load the Jeopardy CSV file into a pandas DataFrame, insert a new cell and add the following code:
    JEOPARDY_CSV =  'data/jeopardy/Jeopardy.csv'
    questions = pd.read_csv(JEOPARDY_CSV)
  4. The data in the DataFrame is not clean. In order to clean it, we remove records that have missing values in the Question column. Add the following code to do this:
    questions = questions.dropna(subset=['Question'])
  5. Now import the gensim preprocessing utility and use it to preprocess the questions further. Add the following code to do this:
    from gensim.parsing.preprocessing import preprocess_string...