Book Image

Python Parallel Programming Cookbook - Second Edition

By : Giancarlo Zaccone
Book Image

Python Parallel Programming Cookbook - Second Edition

By: Giancarlo Zaccone

Overview of this book

<p>Nowadays, it has become extremely important for programmers to understand the link between the software and the parallel nature of their hardware so that their programs run efficiently on computer architectures. Applications based on parallel programming are fast, robust, and easily scalable. </p><p> </p><p>This updated edition features cutting-edge techniques for building effective concurrent applications in Python 3.7. The book introduces parallel programming architectures and covers the fundamental recipes for thread-based and process-based parallelism. You'll learn about mutex, semaphores, locks, queues exploiting the threading, and multiprocessing modules, all of which are basic tools to build parallel applications. Recipes on MPI programming will help you to synchronize processes using the fundamental message passing techniques with mpi4py. Furthermore, you'll get to grips with asynchronous programming and how to use the power of the GPU with PyCUDA and PyOpenCL frameworks. Finally, you'll explore how to design distributed computing systems with Celery and architect Python apps on the cloud using PythonAnywhere, Docker, and serverless applications. </p><p> </p><p>By the end of this book, you will be confident in building concurrent and high-performing applications in Python.</p>
Table of Contents (16 chapters)
Title Page

Introducing Python

Python is a powerful, dynamic, and interpreted programming language that is used in a wide variety of applications. Some of its features are as follows:

  • A clear and readable syntax.
  • A very extensive standard library, where, through additional software modules, we can add data types, functions, and objects.
  • Easy-to-learn rapid development and debugging. Developing Python code in Python can be up to 10 times faster than in C/C++ code. The code can also work as a prototype and then translated into C/C ++.
  • Exception-based error handling.
  • A strong introspection functionality.
  • The richness of documentation and a software community.

Python can be seen as a glue language. Using Python, better applications can be developed because different kinds of coders can work together on a project. For example, when building a scientific application, C/C++ programmers can implement efficient numerical algorithms, while scientists on the same project can write Python programs that test and use those algorithms. Scientists don't have to learn a low-level programming language and C/C++ programmers don't need to understand the science involved.

You can read more about this from

Let's take a look at some examples of very basic code to get an idea of the features of Python.

The following section can be a refresher for most of you. We will use these techniques practically in Chapter 2Thread-Based Parallelism, and Chapter 3, Process-Based Parallelism.

Help functions

The Python interpreter already provides a valid help system. If you want to know how to use an object, then just type help(object).

Let's see, for example, how to use the help function on integer 0:

>>> help(0)
Help on int object:

class int(object)
| int(x=0) -> integer
| int(x, base=10) -> integer
| Convert a number or string to an integer, or return 0 if no
| arguments
are given. If x is a number, return x.__int__(). For
| floating point
numbers, this truncates towards zero.
| If x is not a number or if base is given, then x must be a string,
| bytes, or bytearray instance representing an integer literal in the
| given base. The literal can be preceded by '+' or '-' and be
| surrounded
by whitespace. The base defaults to 10. Valid bases are 0
| and 2-36.

| Base 0 means to interpret the base from the string as an integer
| literal.

>>> int('0b100', base=0)

The description of the int object is followed by a list of methods that are applicable to it. The first five methods are as follows:

 | Methods defined here:
| __abs__(self, /)
| abs(self)
| __add__(self, value, /)
| Return self+value.
| __and__(self, value, /)
| Return self&value.
| __bool__(self, /)
| self != 0
| __ceil__(...)
| Ceiling of an Integral returns itself.

Also useful is dir(object), which lists the methods available for an object:

>>> dir(float)
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

Finally, the relevant documentation for an object is provided by the .__doc__ function, as shown in the following example:

>>> abs.__doc__
'Return the absolute value of the argument.'


Python doesn't adopt statement terminators, and code blocks are specified through indentation. Statements that expect an indentation level must end in a colon (:). This leads to the following:

  • The Python code is clearer and more readable.
  • The program structure always coincides with that of the indentation.
  • The style of indentation is uniform in any listing.

Bad indentation can lead to errors.

The following example shows how to use the if construct:

print("first print")
if condition:
print(“second print”)
print(“third print”)

In this example, we can see the following:

  • The following statements: print("first print"), if condition:, print("third print") have the same indentation level and are always executed.
  • After the if statement, there is a block of code with a higher indentation level, which includes the print ("second print") statement.
  • If the condition of if is true, then the print ("second print") statement is executed.
  • If the condition of if is false, then the print ("second print") statement is not executed.

It is, therefore, very important to pay attention to indentation because it is always evaluated in the program parsing process.


Comments start with the hash sign (#) and are on a single line:

# single line comment

Multi-line strings are used for multi-line comments:

""" first line of a multi-line comment
second line of a multi-line comment."""


Assignments are made with the equals symbol (=). For equality tests, the same amount (==) is used. You can increase and decrease a value using the += and -= operators, followed by an addendum. This works with many types of data, including strings. You can assign and use multiple variables on the same line.

Some examples are as follows:

>>> variable = 3
>>> variable += 2
>>> variable
>>> variable -= 1
>>> variable

>>> _string_ = "Hello"
>>> _string_ += " Parallel Programming CookBook Second Edition!"
>>> print (_string_)
Hello Parallel Programming CookBook Second Edition!

Data types

The most significant structures in Python are lists, tuples, and dictionaries. Sets have been integrated into Python since version 2.5 (the previous versions are available in the sets library):

  • Lists: These are similar to one-dimensional arrays, but you can create lists that contain other lists.
  • Dictionaries: These are arrays that contain key pairs and values (hash tables).
  • Tuples: These are immutable mono-dimensional objects.

Arrays can be of any type, so you can mix variables such as integers and strings into your lists, dictionaries and tuples.

The index of the first object in any type of array is always zero. Negative indexes are allowed and count from the end of the array; -1 indicates the last element of the array:

#let's play with lists
list_1 = [1, ["item_1", "item_1"], ("a", "tuple")]
list_2 = ["item_1", -10000, 5.01]

>>> list_1
[1, ['item_1', 'item_1'], ('a', 'tuple')]

>>> list_2
['item_1', -10000, 5.01]

>>> list_1[2]
('a', 'tuple')

['item_1', 'item_1']

>>> list_2[0]

>>> list_2[-1]

#build a dictionary
dictionary = {"Key 1": "item A", "Key 2": "item B", 3: 1000}
>>> dictionary
{'Key 1': 'item A', 'Key 2': 'item B', 3: 1000}

>>> dictionary["Key 1"]
item A

>>> dictionary["Key 2"]

>>> dictionary[3]

You can get an array range using the colon (:):

list_3 = ["Hello", "Ruvika", "how" , "are" , "you?"] 
>>> list_3[0:6]
['Hello', 'Ruvika', 'how', 'are', 'you?']

>>> list_3[0:1]

>>> list_3[2:6]
['how', 'are', 'you?']


Python strings are indicated using either the single (') or double (") quotation mark and they are allowed to use one notation within a string delimited by the other:

>>> example = "she loves ' giancarlo"
>>> example
"she loves ' giancarlo"

On multiple lines, they are enclosed in triple (or three single) quotation marks (''' multi-line string '''):

>>> _string_='''I am a 
>>> _string_
'I am a \nmulti-line\nstring'

Python also supports Unicode; just use the u "This is a unicode string" syntax :

>>> ustring = u"I am unicode string"
>>> ustring
'I am unicode string'

To enter values in a string, type the % operator and a tuple. Then, each % operator is replaced by a tuple element, from left to right:

>>> print ("My name is %s !" % ('Mr. Wolf'))
My name is Mr. Wolf!

Flow control

Flow control instructions are if, for, and while.

In the next example, we check whether the number is positive, negative, or zero and display the result:

num = 1

if num > 0:
print("Positive number")
elif num == 0:
print("Negative number")

The following code block finds the sum of all the numbers stored in a list, using a for loop:

numbers = [6, 6, 3, 8, -3, 2, 5, 44, 12]
sum = 0
for val in numbers:
sum = sum+val
print("The sum is", sum)

We will execute the while loop to iterate the code until the condition result is true. We will use this loop over the for loop since we are unaware of the number of iterations that will result in the code. In this example, we use while to add natural numbers up to sum = 1+2+3+...+n:

n = 10
# initialize sum and counter
sum = 0
i = 1
while i <= n:
sum = sum + i
i = i+1 # update counter

# print the sum
print("The sum is", sum)

The outputs for the preceding three examples are as follows:

Positive number
The sum is 83
The sum is 55


Python functions are declared with the def keyword:

def my_function():
print("this is a function")

To run a function, use the function name, followed by parentheses, as follows:

>>> my_function()
this is a function

Parameters must be specified after the function name, inside the parentheses:

def my_function(x):
print(x * 1234)

>>> my_function(7)

Multiple parameters must be separated with a comma:

def my_function(x,y):
print(x*5+ 2*y)

>>> my_function(7,9)

Use the equals sign to define a default parameter. If you call the function without the parameter, then the default value will be used:

def my_function(x,y=10):
print(x*5+ 2*y)

>>> my_function(1)

>>> my_function(1,100)

The parameters of a function can be of any type of data (such as string, number, list, and dictionary). Here, the following list, lcitiesis used as a parameter for my_function:

def my_function(cities):
for x in cities:

>>> lcities=["Napoli","Mumbai","Amsterdam"]
>>> my_function(lcities)

Use the return statement to return a value from a function:

def my_function(x,y):
return x*y

>>> my_function(6,29)

Python supports an interesting syntax that allows you to define small, single-line functions on the fly. Derived from the Lisp programming language, these lambda functions can be used wherever a function is required.

An example of a lambda function, functionvar, is shown as follows:

# lambda definition equivalent to def f(x): return x + 1

functionvar = lambda x: x * 5
>>> print(functionvar(10))


Python supports multiple inheritances of classes. Conventionally (not a language rule), private variables and methods are declared by being preceded with two underscores (__). We can assign arbitrary attributes (properties) to the instances of a class, as shown in the following example:

class FirstClass:
common_value = 10
def __init__ (self):
self.my_value = 100
def my_func (self, arg1, arg2):
return self.my_value*arg1*arg2

# Build a first instance
>>> first_instance = FirstClass()
>>> first_instance.my_func(1, 2)

# Build a second instance of FirstClass
>>> second_instance = FirstClass()

#check the common values for both the instances
>>> first_instance.common_value

>>> second_instance.common_value

#Change common_value for the first_instance
>>> first_instance.common_value = 1500
>>> first_instance.common_value

#As you can note the common_value for second_instance is not changed
>>> second_instance.common_value

# SecondClass inherits from FirstClass.
# multiple inheritance is declared as follows:
# class SecondClass (FirstClass1, FirstClass2, FirstClassN)

class SecondClass (FirstClass):
# The "self" argument is passed automatically
# and refers to the class's instance
def __init__ (self, arg1):
self.my_value = 764
print (arg1)

>>> first_instance = SecondClass ("hello PACKT!!!!")
hello PACKT!!!!

>>> first_instance.my_func (1, 2)


Exceptions in Python are managed with try-except blocks (exception_name):

def one_function():
# Division by zero causes one exception
except ZeroDivisionError:
print("Oops, error.")
# There was no exception, we can continue.
# This code is executed when the block
# try..except is already executed and all exceptions
# have been managed, even if a new one occurs
# exception directly in the block.
print("We finished.")

>>> one_function()
Oops, error.
We finished

Importing libraries

External libraries are imported with import [library name]. Alternatively, you can use the from [library name] import [function name] syntax to import a specific function. Here is an example:

import random
randomint = random.randint(1, 101)

>>> print(randomint)

from random import randint
randomint = random.randint(1, 102)

>>> print(randomint)

Managing files

To allow us to interact with the filesystem, Python provides us with the built-in open function. This function can be invoked to open a file and return an object file. The latter allows us to perform various operations on the file, such as reading and writing. When we have finished interacting with the file, we must finally remember to close it by using the file.close method:

>>> f = open ('test.txt', 'w') # open the file for writing
>>> f.write ('first line of file \ n') # write a line in file
>>> f.write ('second line of file \ n') # write another line in file
>>> f.close () # we close the file
>>> f = open ('test.txt') # reopen the file for reading
>>> content = () # read all the contents of the file
>>> print (content)
first line of the file
second line of the file
>>> f.close () # close the file

List comprehensions

List comprehensions are a powerful tool for creating and manipulating lists. They consist of an expression that is followed by a for clause and then followed by zero, or more, if clauses. The syntax for list comprehensions is simply the following:

[expression for item in list]

Then, perform the following:

#list comprehensions using strings
>>> list_comprehension_1 = [ x for x in 'python parallel programming cookbook!' ]
>>> print( list_comprehension_1)

['p', 'y', 't', 'h', 'o', 'n', ' ', 'p', 'a', 'r', 'a', 'l', 'l', 'e', 'l', ' ', 'p', 'r', 'o', 'g', 'r', 'a', 'm', 'm', 'i', 'n', 'g', ' ', 'c', 'o', 'o', 'k', 'b', 'o', 'o', 'k', '!']

#list comprehensions using numbers
>>> l1 = [1,2,3,4,5,6,7,8,9,10]
>>> list_comprehension_2 = [ x*10 for x in l1 ]
>>> print( list_comprehension_2)

[10, 20, 30, 40, 50, 60, 70, 80, 90, 100]

Running Python scripts

To execute a Python script, simply invoke the Python interpreter followed by the script name, in this case, Or, if we are in a different working directory, then use its full address:

> python
From now on, for every invocation of a Python script, we will use the preceding notation; that is, python, followed by, assuming that the directory from which the Python interpreter is launched is the one where the script to be executed resides.

Installing Python packages using pip

pip is a tool that allows us to search, download, and install Python packages found on the Python Package Index, which is a repository that contains tens of thousands of packages written in Python. This also allows us to manage the packages we have already downloaded, allowing us to update or remove them.

Installing pip

pip is already included in Python versions ≥ 3.4 and ≥ 2.7.9. To check whether this tool is already installed, we can run the following command:


 If pip is already installed, then this command will show us the installed version.

Updating pip

It is also recommended to check that the pip version you are using is always up to date. To update it, we can use the following command:

 C:\>pip install -U pip

Using pip

pip supports a series of commands that allow us, among other things, to search, download, install, update, and remove packages.

To install PACKAGE, just run the following command:

C:\>pip install PACKAGE