The fact that distributed systems do not have a single address space means that the usual mechanisms for sharing data between modules (parameters and global variables) are not available. Instead, data movement must be explicitly specified. While IPython provides for the use of a "global" dictionary (which implicitly uses a message passing mechanism similar to what is described in this chapter), more sophisticated communication patterns require more full-featured tools.

In this chapter, we looked at two of these tools: ZeroMQ and MPI. ZeroMQ is a lightweight, socket-like mechanism that has become the basis of IPython's internal architecture. It is easy to use, is efficient, supports many different messaging patterns, and provides support for user-defined patterns. MPI is the workhorse of most HPC applications. It has been in use for a long time and is efficient and thoroughly debugged. The ability to dynamically create processes is an important feature.

Either mechanism is capable of...