Book Image

Instant Parallel Processing with Gearman

By : John Ewart
Book Image

Instant Parallel Processing with Gearman

By: John Ewart

Overview of this book

Many of today's applications need to be able to process large volumes of data, and vertical scaling has its limits both in terms of prohibitive cost and hardware limits. Gearman is an open source job manager that is well-suited to building horizontally scalable systems, from map-reduce algorithms to simple data processors capable of handling massive amounts of information. Instant Gearman is a practical, hands-on guide to getting started with building an open source job management server system that is built to grow. Learn the basics of building a distributed application that spans multiple components and learn how Gearman fits into building an application that scales from one to hundreds of components that interact to process data. With Gearman, you can build software that scales horizontally as your need for computation increases. Instant Gearman has in-depth examples and a step-by-step approach to building distributed systems, helping you to build systems that are scalable and modular in their approach to processing data. Once you are comfortable with building simple workers and clients, learn how to build a cluster of managers and see how to reduce single-point-of-failure in your architecture. Next, build a simple map-reduce application using Gearman and scale it up from a single instance to multiple parallel processing components.
Table of Contents (7 chapters)

People and places you should get to know


There are a number of available resources to help you to continue explore using Gearman in your application's infrastructure.

Client libraries

There are a number of libraries for various languages. Here are some of the more popular ones:

Another PHP library for Gearman is available in the PEAR repository.

Community

There are a number of community-based resources available on the Internet. The primary method of communication around the Gearman project is the mailing list as well as an IRC channel:

  • Official Google Group: This is a fairly active (1-10 messages per day) mailing list for users and developers of Gearman projects. More information on this can be found at http://groups.google.com/group/gearman.

  • IRC chat room: Available on any FreeNode IRC network server as #gearman. More information on this can be found at irc://irc.freenode.net/#gearman.