Book Image

Practical Data Analysis - Second Edition

By : Hector Cuesta, Dr. Sampath Kumar
Book Image

Practical Data Analysis - Second Edition

By: Hector Cuesta, Dr. Sampath Kumar

Overview of this book

Beyond buzzwords like Big Data or Data Science, there are a great opportunities to innovate in many businesses using data analysis to get data-driven products. Data analysis involves asking many questions about data in order to discover insights and generate value for a product or a service. This book explains the basic data algorithms without the theoretical jargon, and you’ll get hands-on turning data into insights using machine learning techniques. We will perform data-driven innovation processing for several types of data such as text, Images, social network graphs, documents, and time series, showing you how to implement large data processing with MongoDB and Apache Spark.
Table of Contents (21 chapters)
Practical Data Analysis - Second Edition
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface

E-mail subject line tester


Spam is junk e-mail, which is understood as Unsolicited Bulk Email (UBE). E-mails can be blocked before they are delivered to the recipient based on e-mail filter reports. The e-mail filter scans the subject line of e-mails for spam or ham (e-mail that is not spam is often called ham). One of the e-mail filters is the e-mail subject line filter. Over 35 percent of spam mails are detected from the subject line of an e-mail.

An E-mail Subject Line Tester is a simple program that will define whether a certain subject line in an e-mail is spam or not. In this chapter, we will program a Naïve Bayes classifier from scratch. The example will classify whether a subject line is a spam or not with a very simple code by breaking the subject lines into a list of relevant words that will be used as a feature vector in the algorithm. In order to do this, we will use the SpamAssassin public dataset. SpamAssasin includes three categories: spam, easy ham, and hard ham. In this...