Book Image

Raspberry Pi Super Cluster

By : Andrew K. Dennis
Book Image

Raspberry Pi Super Cluster

By: Andrew K. Dennis

Overview of this book

A cluster is a type of parallel/distributed processing system which consists of a collection of interconnected stand-alone computers cooperatively working together. Using Raspberry Pi computers, you can build a two-node parallel computing cluster which enhances performance and availability. This practical, example-oriented guide will teach you how to set up the hardware and operating systems of multiple Raspberry Pi computers to create your own cluster. It will then navigate you through how to install the necessary software to write your own programs such as Hadoop and MPICH before moving on to cover topics such as MapReduce. Throughout this book, you will explore the technology with the help of practical examples and tutorials to help you learn quickly and efficiently. Starting from a pile of hardware, with this book, you will be guided through exciting tutorials that will help you turn your hardware into your own super-computing cluster. You'll start out by learning how to set up your Raspberry Pi cluster's hardware. Following this, you will be taken through how to install the operating system, and you will also be given a taste of what parallel computing is about. With your Raspberry Pi cluster successfully set up, you will then install software such as MPI and Hadoop. Having reviewed some examples and written some programs that explore these two technologies, you will then wrap up with some fun ancillary projects. Finally, you will be provided with useful links to help take your projects to the next step.
Table of Contents (15 chapters)
Raspberry Pi Super Cluster
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

A Hadoop application to calculate Pi


Hadoop comes packaged with a number of example applications. We are of course interested in calculating П program in particular.

The source code for this application can be downloaded from Apache's website at the following URL:

https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/QuasiMonteCarlo.java

The JAR file containing the compiled class can be found on your machine at:

/home/pi/hadoop/hadoop-1.2.1/hadoop-examples-1.2.1.jar

Let's navigate to this directory:

cd ~/hadoop/hadoop-1.2.1

We are now going to run the example. The program takes two inputs: the number of maps and the number of samples. Try running the following demonstration:

hadoop jar hadoop-examples-1.2.1.jar pi 2 4

You should now see something similar to:

Number of Maps  = 2
Samples per Map = 4
Wrote input for Map #0
Wrote input for Map #1
Starting Job
…
Job Finished in 273.167 seconds
Estimated value...