3. Process-based Parallelism | Python Parallel Programming Cookbook

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Python Parallel Programming Cookbook

By : Giancarlo Zaccone

4.1 (11)

Python Parallel Programming Cookbook

4.1 (11)

By: Giancarlo Zaccone

Overview of this book

This book will teach you parallel programming techniques using examples in Python and will help you explore the many ways in which you can write code that allows more than one process to happen at once. Starting with introducing you to the world of parallel computing, it moves on to cover the fundamentals in Python. This is followed by exploring the thread-based parallelism model using the Python threading module by synchronizing threads and using locks, mutex, semaphores queues, GIL, and the thread pool. Next you will be taught about process-based parallelism where you will synchronize processes using message passing along with learning about the performance of MPI Python Modules. You will then go on to learn the asynchronous parallel programming model using the Python asyncio module along with handling exceptions. Moving on, you will discover distributed computing with Python, and learn how to install a broker, use Celery Python Module, and create a worker. You will understand anche Pycsp, the Scoop framework, and disk modules in Python. Further on, you will learnGPU programming withPython using the PyCUDA module along with evaluating performance limitations.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

1. Getting Started with Parallel Computing and Python

1. Getting Started with Parallel Computing and Python

Introduction

The parallel computing memory architecture

Memory organization

Parallel programming models

How to design a parallel program

How to evaluate the performance of a parallel program

Introducing Python

Python in a parallel world

Introducing processes and threads

Start working with processes in Python

Start working with threads in Python

2. Thread-based Parallelism

2. Thread-based Parallelism

Introduction

Using the Python threading module

How to define a thread

How to determine the current thread

How to use a thread in a subclass

Thread synchronization with Lock and RLock

Thread synchronization with RLock

Thread synchronization with semaphores

Thread synchronization with a condition

Thread synchronization with an event

Using the with statement

Thread communication using a queue

Evaluating the performance of multithread applications

3. Process-based Parallelism

3. Process-based Parallelism

Introduction

How to spawn a process

How to name a process

How to run a process in the background

How to kill a process

How to use a process in a subclass

How to exchange objects between processes

How to synchronize processes

How to manage a state between processes

How to use a process pool

Using the mpi4py Python module

Point-to-point communication

Avoiding deadlock problems

Collective communication using broadcast

Collective communication using scatter

Collective communication using gather

Collective communication using Alltoall

The reduction operation

How to optimize communication

4. Asynchronous Programming

4. Asynchronous Programming

Introduction

Using the concurrent.futures Python modules

Event loop management with Asyncio

Handling coroutines with Asyncio

Task manipulation with Asyncio

Dealing with Asyncio and Futures

5. Distributed Python

5. Distributed Python

Introduction

Using Celery to distribute tasks

How to create a task with Celery

Scientific computing with SCOOP

Handling map functions with SCOOP

Remote Method Invocation with Pyro4

Chaining objects with Pyro4

Developing a client-server application with Pyro4

Communicating sequential processes with PyCSP

Using MapReduce with Disco

A remote procedure call with RPyC

6. GPU Programming with Python

6. GPU Programming with Python

Introduction

Using the PyCUDA module

How to build a PyCUDA application

Understanding the PyCUDA memory model with matrix manipulation

Kernel invocations with GPUArray

Evaluating element-wise expressions with PyCUDA

The MapReduce operation with PyCUDA

GPU programming with NumbaPro

Using GPU-accelerated libraries with NumbaPro

Using the PyOpenCL module

How to build a PyOpenCL application

Evaluating element-wise expressions with PyOpenCl

Testing your GPU application with PyOpenCL

Index

Index

Point-to-point communication

One of the most important features among those provided by MPI is the point-to-point communication, which is a mechanism that enables data transmission between two processes: a process receiver, and process sender.

The Python module mpi4py enables point-to-point communication via two functions:

Comm.Send(data, process_destination): This sends data to the destination process identified by its rank in the communicator group
Comm.Recv(process_source): This receives data from the source process, which is also identified by its rank in the communicator group

The Comm parameter, which stands for communicator, defines the group of processes, that may communicate through message passing:

comm = MPI.COMM_WORLD

How to do it…

In the following example, we show you how to utilize the comm.send and comm.recv directives to exchange messages between different processes:

from mpi4py import MPI

comm=MPI.COMM_WORLD
rank = comm.rank
print("my rank is : " , rank)

if rank==0:
    data...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Python Parallel Programming Cookbook

Search

Your notes and bookmarks