Book Image

Building Machine Learning Systems with Python - Second Edition

By : Luis Pedro Coelho, Willi Richert
Book Image

Building Machine Learning Systems with Python - Second Edition

By: Luis Pedro Coelho, Willi Richert

Overview of this book

<p>Using machine learning to gain deeper insights from data is a key skill required by modern application developers and analysts alike. Python is a wonderful language to develop machine learning applications. As a dynamic language, it allows for fast exploration and experimentation. With its excellent collection of open source machine learning libraries you can focus on the task at hand while being able to quickly try out many ideas.</p> <p>This book shows you exactly how to find patterns in your raw data. You will start by brushing up on your Python machine learning knowledge and introducing libraries. You’ll quickly get to grips with serious, real-world projects on datasets, using modeling, creating recommendation systems. Later on, the book covers advanced topics such as topic modeling, basket analysis, and cloud computing. These will extend your abilities and enable you to create large complex systems.</p> <p>With this book, you gain the tools and understanding required to build your own systems, tailored to solve your real-world data analysis problems.</p>
Table of Contents (20 chapters)
Building Machine Learning Systems with Python Second Edition
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Measuring the relatedness of posts


From the machine learning point of view, raw text is useless. Only if we manage to transform it into meaningful numbers, can we then feed it into our machine learning algorithms, such as clustering. This is true for more mundane operations on text such as similarity measurement.

How not to do it

One text similarity measure is the Levenshtein distance, which also goes by the name Edit Distance. Let's say we have two words, "machine" and "mchiene". The similarity between them can be expressed as the minimum set of edits that are necessary to turn one word into the other. In this case, the edit distance will be 2, as we have to add an "a" after the "m" and delete the first "e". This algorithm is, however, quite costly as it is bound by the length of the first word times the length of the second word.

Looking at our posts, we could cheat by treating whole words as characters and performing the edit distance calculation on the word level. Let's say we have two...