6. Social Media Insight Using Naive Bayes | Learning Data Mining with Python

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

Learning Data Mining with Python

By : Robert Layton

3.7 (7)

Learning Data Mining with Python

3.7 (7)

By: Robert Layton

Overview of this book

If you are a programmer who wants to get started with data mining, then this book is for you.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Getting Started with Data Mining

1. Getting Started with Data Mining

Introducing data mining

Using Python and the IPython Notebook

A simple affinity analysis example

A simple classification example

What is classification?

Summary

2. Classifying with scikit-learn Estimators

2. Classifying with scikit-learn Estimators

scikit-learn estimators

Preprocessing using pipelines

Pipelines

Summary

3. Predicting Sports Winners with Decision Trees

3. Predicting Sports Winners with Decision Trees

Loading the dataset

Decision trees

Sports outcome prediction

Random forests

Summary

4. Recommending Movies Using Affinity Analysis

4. Recommending Movies Using Affinity Analysis

Affinity analysis

The movie recommendation problem

The Apriori implementation

Extracting association rules

Summary

5. Extracting Features with Transformers

5. Extracting Features with Transformers

Feature extraction

Feature selection

Feature creation

Creating your own transformer

Summary

6. Social Media Insight Using Naive Bayes

6. Social Media Insight Using Naive Bayes

Disambiguation

Text transformers

Naive Bayes

Application

Summary

7. Discovering Accounts to Follow Using Graph Mining

7. Discovering Accounts to Follow Using Graph Mining

Loading the dataset

Finding subgraphs

Summary

8. Beating CAPTCHAs with Neural Networks

8. Beating CAPTCHAs with Neural Networks

Artificial neural networks

Creating the dataset

Training and classifying

Improving accuracy using a dictionary

Summary

9. Authorship Attribution

9. Authorship Attribution

Attributing documents to authors

Function words

Support vector machines

Character n-grams

Using the Enron dataset

Summary

10. Clustering News Articles

10. Clustering News Articles

Obtaining news articles

Extracting text from arbitrary websites

Grouping news articles

Clustering ensembles

Online learning

Summary

11. Classifying Objects in Images Using Deep Learning

11. Classifying Objects in Images Using Deep Learning

Object classification

Application scenario and goals

Deep neural networks

GPU optimization

Setting up the environment

Application

Summary

12. Working with Big Data

12. Working with Big Data

Big data

Application scenario and goals

MapReduce

Application

Summary

A. Next Steps…

A. Next Steps…

Chapter 1 – Getting Started with Data Mining

Chapter 2 – Classifying with scikit-learn Estimators

Chapter 3: Predicting Sports Winners with Decision Trees

Chapter 4 – Recommending Movies Using Affinity Analysis

Chapter 5 – Extracting Features with Transformers

Chapter 6 – Social Media Insight Using Naive Bayes

Chapter 7 – Discovering Accounts to Follow Using Graph Mining

Chapter 8 – Beating CAPTCHAs with Neural Networks

Chapter 9 – Authorship Attribution

Chapter 10 – Clustering News Articles

Chapter 11: Classifying Objects in Images Using Deep Learning

Chapter 12 – Working with Big Data

More resources

Index

Index

Customer Reviews

3.7 (7)

5 star

28.6%

4 star

28.6%

3 star

28.6%

2 star

14.3%

1 star

0%

Text transformers

Now that we have our dataset, how are we going to perform data mining on it?

Text-based datasets include books, essays, websites, manuscripts, programming code, and other forms of written expression. All of the algorithms we have seen so far deal with numerical or categorical features, so how do we convert our text into a format that the algorithm can deal with?

There are a number of measurements that could be taken. For instance, average word and average sentence length are used to predict the readability of a document. However, there are lots of feature types such as word occurrence which we will now investigate.

Bag-of-words

One of the simplest but highly effective models is to simply count each word in the dataset. We create a matrix, where each row represents a document in our dataset and each column represents a word. The value of the cell is the frequency of that word in the document.

Here's an excerpt from The Lord of the Rings, J.R.R. Tolkien:

Three Rings for the Elven...

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Learning Data Mining with Python

Search

Your notes and bookmarks