Book Image

Natural Language Processing Fundamentals

By : Sohom Ghosh, Dwight Gunning
Book Image

Natural Language Processing Fundamentals

By: Sohom Ghosh, Dwight Gunning

Overview of this book

If NLP hasn't been your forte, Natural Language Processing Fundamentals will make sure you set off to a steady start. This comprehensive guide will show you how to effectively use Python libraries and NLP concepts to solve various problems. You'll be introduced to natural language processing and its applications through examples and exercises. This will be followed by an introduction to the initial stages of solving a problem, which includes problem definition, getting text data, and preparing it for modeling. With exposure to concepts like advanced natural language processing algorithms and visualization techniques, you'll learn how to create applications that can extract information from unstructured data and present it as impactful visuals. Although you will continue to learn NLP-based techniques, the focus will gradually shift to developing useful applications. In these sections, you'll understand how to apply NLP techniques to answer questions as can be used in chatbots. By the end of this book, you'll be able to accomplish a varied range of assignments ranging from identifying the most suitable type of NLP task for solving a problem to using a tool like spacy or gensim for performing sentiment analysis. The book will easily equip you with the knowledge you need to build applications that interpret human language.
Table of Contents (10 chapters)

Saving and Loading Models

After a model has been built and its performance matches our expectations, we may want to save it for future use. This eliminates the time needed for rebuilding it. Models can be saved in the hard disk by using joblib and pickle.

To deploy saved models, we need to load them from the hard disk to the memory. In the next section, we will solve an exercise based on this to get a better understanding.

Exercise 39: Saving and Loading Models

In this exercise, first we will create a tf-idf representation of sentences. Then, we will save this model on disk. Later, we will load it from the disk. Follow these steps to implement this exercise:

  1. Open a Jupyter notebook.
  2. Insert a new cell and the following code to import the necessary packages:
    import pickle
    from joblib import dump, load
    from sklearn.feature_extraction.text import TfidfVectorizer
  3. Defining a corpus consisting of four sentences, add the following code:
    corpus = [
    'Data Science...