Multiprocessing in Python
Python includes the multiprocessing
(most of the time abbreviated to just mp
) module to support process-level parallelism and the required communication primitives. In our example, we will use the two main classes from this module:
mp.Queue
: A concurrent multi-producer, multi-consumer FIFO (first in, first out) queue with transparent serialization and deserialization of objects placed in the queuemp.Process
: A piece of code that is run in the child process and methods to control it from the parent process
PyTorch provides its own thin wrapper around the multiprocessing
module, which adds the proper handling of tensors and variables on CUDA devices and shared memory. It provides exactly the same functionality as the multiprocessing
module from the standard library, so all you need to do is use import torch.multiprocessing
instead of import multiprocessing
.