Book Image

Advanced Python Programming - Second Edition

By : Quan Nguyen
Book Image

Advanced Python Programming - Second Edition

By: Quan Nguyen

Overview of this book

Python's powerful capabilities for implementing robust and efficient programs make it one of the most sought-after programming languages. In this book, you'll explore the tools that allow you to improve performance and take your Python programs to the next level. This book starts by examining the built-in as well as external libraries that streamline tasks in the development cycle, such as benchmarking, profiling, and optimizing. You'll then get to grips with using specialized tools such as dedicated libraries and compilers to increase your performance at number-crunching tasks, including training machine learning models. The book covers concurrency, a major solution to making programs more efficient and scalable, and various concurrent programming techniques such as multithreading, multiprocessing, and asynchronous programming. You'll also understand the common problems that cause undesirable behavior in concurrent programs. Finally, you'll work with a wide range of design patterns, including creational, structural, and behavioral patterns that enable you to tackle complex design and architecture challenges, making your programs more robust and maintainable. By the end of the book, you'll be exposed to a wide range of advanced functionalities in Python and be equipped with the practical knowledge needed to apply them to your use cases.
Table of Contents (32 chapters)
1
Section 1: Python-Native and Specialized Optimization
8
Section 2: Concurrency and Parallelism
18
Section 3: Design Patterns in Python

Using multiple processes

The standard multiprocessing module can be used to quickly parallelize simple tasks by spawning several processes while avoiding the GIL problem. Its interface is easy to use and includes several utilities to handle task submission and synchronization.

The Process and Pool classes

You can create a process that runs independently by subclassing multiprocessing.Process. You can extend the __init__ method to initialize resources, and you can write a portion of the code that will be executed in a subprocess by implementing the Process.run method. In the following code snippet, we define a Process class that will wait for 1 second and print its assigned id value:

    import multiprocessing 
    import time 
    class Process(multiprocessing.Process): 
        def __init__(self, id): 
            ...