Book Image

Artificial Intelligence with Python

Book Image

Artificial Intelligence with Python

Overview of this book

Artificial Intelligence is becoming increasingly relevant in the modern world. By harnessing the power of algorithms, you can create apps which intelligently interact with the world around you, building intelligent recommender systems, automatic speech recognition systems and more. Starting with AI basics you'll move on to learn how to develop building blocks using data mining techniques. Discover how to make informed decisions about which algorithms to use, and how to apply them to real-world scenarios. This practical book covers a range of topics including predictive analytics and deep learning.
Table of Contents (23 chapters)
Artificial Intelligence with Python
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Extracting the frequency of terms using a Bag of Words model


One of the main goals of text analysis is to convert text into numeric form so that we can use machine learning on it. Let's consider text documents that contain many millions of words. In order to analyze these documents, we need to extract the text and convert it into a form of numeric representation.

Machine learning algorithms need numeric data to work with so that they can analyze the data and extract meaningful information. This is where the Bag of Words model comes into picture. This model extracts a vocabulary from all the words in the documents and builds a model using a document term matrix. This allows us to represent every document as a bag of words. We just keep track of word counts and disregard the grammatical details and the word order.

Let's see what a document-term matrix is all about. A document term matrix is basically a table that gives us counts of various words that occur in the document. So a text document...