Book Image

Concurrent and Parallel Programming in Python [Video]

By : Maximilian Schallwig
Book Image

Concurrent and Parallel Programming in Python [Video]

By: Maximilian Schallwig

Overview of this book

In a big data project, a plethora of information is retrieved, big numbers are crunched on our machine, or both. If the coding is sequential or synchronous, our application will struggle to execute. Two mechanisms to alleviate such bottlenecks are concurrency and parallelism. In Python, concurrency is represented by threading, whereas multiprocessing achieves parallelism. This course begins with an introduction about potential programming speed bottlenecks and solving them. You will delve into Python concepts and create a Wikipedia Reader, Yahoo Finance Reader, Queues, and Master Scheduler. You will build a multi-threaded program to grab data from the Internet and parse and save them into a local database. Implement multiprocessing in Python, which lets us use multiple CPUs in our code. Learn about threading, multiprocessing, asynchronous wait, locking, multiprocessing queues, Pool Map Multiple Arguments, writing asynchronous programs, and combining async and multiprocessing. Upon completion, we can spread our workload over all cores available on the used machine. We will combine both elements, multiprocessing with asynchronous programming, to maximize benefit and CPU resource usage and minimize the time spent waiting for IO responses. You will create multi-threaded, asynchronous, multi-process programs to make programs run faster. All resources are available at: https://github.com/PacktPublishing/Concurrent-and-Parallel-Programming-in-Python
Table of Contents (3 chapters)
Chapter 3
Asynchronous
Content Locked
Section 5
Creating Asynchronous For Loops
This lesson elaborates on what a for loop does and does not do in an asynchronous program code.