With the progress in hardware, our storage capacity has increased now, and because of this, there are many organizations that want to store all types of events for analytical purpose. This is giving birth to a new area of machine learning. The field of machine learning is very complex, and writing those algorithms is not a piece of cake. Apache Mahout provides us with readymade algorithms in the area of machine learning and saves us from the complex task of algorithm implementation.
The intention of this book is to cover clustering algorithms available in Apache Mahout. Whether you have already worked on clustering algorithms using some other tool, or whether you are completely new to this field, this book will help you. So, start reading this book, explore the clustering algorithms in a strong, community-supported, open source, and one of the most popular Apache projects—Apache Mahout.
Chapter 1, Understanding Clustering, explains clustering in general. This chapter will further discuss the different distance matrices and how to calculate them.
Chapter 2, Understanding K-means Clustering, introduces K-means clustering and how Mahout can be used for K-means clustering algorithms.
Chapter 3, Understanding Canopy Clustering, introduces Canopy clustering and its uses in Apache Mahout.
Chapter 4, Understanding the Fuzzy K-means Algorithm Using Mahout, talks about the Fuzzy K-means algorithm and how this algorithm works as a preprocessing step for K-means. We will further discuss how to use Mahout for the Fuzzy K-means algorithm.
Chapter 5, Understanding Model-based Clustering, discusses model-based clustering. This chapter further discusses the topic of modeling using Dirichlet clustering.
Chapter 6, Understanding Streaming K-means, introduces the Streaming K-means algorithm, which is used for streaming data. We will further discuss how Mahout can be used for Streaming K-means.
Chapter 7, Spectral Clustering, introduces spectral clustering and how Mahout has implemented spectral clustering.
Chapter 8, Improving Cluster Quality, covers the steps that should be followed to improve cluster quality once you are ready with your clustering algorithm, in detail. It also discusses what techniques Mahout provides to improve cluster quality.
Chapter 9, Creating a Cluster Model for Production, introduces the techniques that should be followed in a production environment while applying the clustering algorithm.
To use the examples in this book, you should have the following software installed in your system:
Java 1.6 or further
Eclipse
Hadoop
Mahout (we will discuss the installation in Chapter 2, Understanding K-means Clustering)
Maven (depending on how you are installing Mahout)
If you are a data scientist who has some experience with the Hadoop ecosystem and machine learning methods and want to try out clustering on large datasets using Mahout, this book is ideal for you. Knowledge of Java is essential.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Once done, you can test it by typing the command – mahout and this will show you the same screen as shown in preceding figure."
A block of code is set as follows:
generateSamples(500, 1, 1, 3); // 500 samples of sd 3 generateSamples(300, 1, 0, 0.5); //300 sample of sd 0.5 generateSamples(300, 0, 2, 0.1); //300 sample of sd 0.1
Any command-line input or output is written as follows:
bin/mahoutcanopy --input /user/hue/20newsdatavec/tfidf-vectors/ --output /user/hue/canopycentroids --distanceMeasure org.apache.mahout.common.distance.EuclideanDistanceMeasure –t1 1550 --t2 2050--method mapreduce
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Click on the Keys and Access Tokens tab, and you will find ConsumerKey and ConsumerSecret under Application Settings."
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.