Over the course of the last few chapters, we've been progressively moving outward. We started assuming that everything would run on one processor, probably in a single thread. Then we looked at how to structure our program without that assumption, performing different tasks on many threads. Next we tried to speed up processing by getting multiple threads and cores working on the same task. Now we've pulled back about as far as we can, and we're going to look at how to break the work to execute on multiple computers. For large amounts of data, this can be especially useful.

In fact, big data has become more and more common. The definition of big data is a moving target as disk sizes grow, but you work with big data if you have trouble processing it in memory, or even storing it on disk. There's a lot of information locked in, but getting it out can be a real problem. The recipes in this chapter will help address those issues.

The most common way of distributing computations currently...