Book Image

Secret Recipes of the Python Ninja

Book Image

Secret Recipes of the Python Ninja

Overview of this book

This book covers the unexplored secrets of Python, delve into its depths, and uncover its mysteries. You’ll unearth secrets related to the implementation of the standard library, by looking at how modules actually work. You’ll understand the implementation of collections, decimals, and fraction modules. If you haven’t used decorators, coroutines, and generator functions much before, as you make your way through the recipes, you’ll learn what you’ve been missing out on. We’ll cover internal special methods in detail, so you understand what they are and how they can be used to improve the engineering decisions you make. Next, you’ll explore the CPython interpreter, which is a treasure trove of secret hacks that not many programmers are aware of. We’ll take you through the depths of the PyPy project, where you’ll come across several exciting ways that you can improve speed and concurrency. Finally, we’ll take time to explore the PEPs of the latest versions to discover some interesting hacks.
Table of Contents (17 chapters)
Title Page
Copyright and Credits
Packt Upsell
Foreword
Contributors
Preface
Index

Chapter 5. Generators, Coroutines, and Parallel Processing

In this chapter, we will take a look at generators, coroutines, and parallel processing. Specifically, we will cover the following topics:

  • How iteration works in Python
  • Using the itertools module
  • Using generator functions
  • Simulating multithreading with coroutines
  • When to use parallel processing
  • Forking processes
  • How to implement multithreading
  • How to implement multiprocessing

Although the various topics that will be covered in this chapter may seem to have little to do with each other, they do have an effect on one another. First, iteration is the process of walking through a sequence; Python provides several ways to iterate over objects. Generators are functions that generate values in a sequential order, implementing iteration functionality under the hood.

This moves into parallelism. Coroutines use generators to effectively create multiple processes to allow multitasking but it is controlled by the programmer. Multithreading switches processing...