The tools
In Chapter 3, Parallelism in Python, we looked at a few standard library modules that can be used to introduce (single-node) parallelism in our applications. We experimented with both the threading
and multiprocessing
modules directly and via the higher-level concurrent.futures
module.
We saw how, for non-distributed, parallel applications, Python offers a really robust foundation. The preceding three modules are complete and included in every modern Python distribution. They have no external dependencies, which makes them quite appealing.
We explored a few third-party Python modules for simple distributed computing in Chapter 4, Distributed Applications – with Celery. These included Celery, Python-RQ, and Pyro. We saw how to use them in our code, and mostly, we saw how simple it is to get up and running with each one of them.
They all require some pieces of infrastructure such as message brokers, databases, or name servers, and so they might or might not be of use in all contexts...