Book Image

Daniel Arbuckle's Mastering Python

By : Daniel Arbuckle
Book Image

Daniel Arbuckle's Mastering Python

By: Daniel Arbuckle

Overview of this book

Daniel Arbuckle's Mastering Python covers the basics of operating in a Python development environment, before moving on to more advanced topics. Daniel presents you with real-world solutions to Python 3.6 and advanced-level concepts, such as reactive programming, microservices, ctypes, and Cython tools. You don't need to be familiar with the Python language to use this book, as Daniel starts with a Python primer. Throughout, Daniel highlights the major aspects of managing your Python development environment, shows you how to handle parallel computation, and helps you to master asynchronous I/O with Python 3.6 to improve performance. Finally, Daniel will teach you the secrets of metaprogramming and unit testing in Python, helping you acquire the perfect skillset to be a Python expert. Daniel will get you up to speed on everything from basic programming practices to high-end tools and techniques, things that will help set you apart as a successful Python programmer.
Table of Contents (13 chapters)

Using the multiprocessing packages

In the previous section, we saw that the concurrent.futures package makes it very simple to farm out computational jobs to worker processes. If the program we need doesn't fit into the send out jobs and collect the results model, we're probably better off working at a somewhat lower level of abstraction.

So, let's now move on to look at another package that helps us handle multiprocess programs that don't fit that model, but the pieces are only partly independent of each other. From time to time, they need to pass information between themselves, not just back to the controlling process. We can't do that with concurrent.futures because it just doesn't fit into the model that concurrent.futures uses to describe parallel processing.

Alternatively, what if we need to be able to cancel a job after a worker process has...