Book Image

Advanced Python Programming - Second Edition

By : Quan Nguyen
Book Image

Advanced Python Programming - Second Edition

By: Quan Nguyen

Overview of this book

Python's powerful capabilities for implementing robust and efficient programs make it one of the most sought-after programming languages. In this book, you'll explore the tools that allow you to improve performance and take your Python programs to the next level. This book starts by examining the built-in as well as external libraries that streamline tasks in the development cycle, such as benchmarking, profiling, and optimizing. You'll then get to grips with using specialized tools such as dedicated libraries and compilers to increase your performance at number-crunching tasks, including training machine learning models. The book covers concurrency, a major solution to making programs more efficient and scalable, and various concurrent programming techniques such as multithreading, multiprocessing, and asynchronous programming. You'll also understand the common problems that cause undesirable behavior in concurrent programs. Finally, you'll work with a wide range of design patterns, including creational, structural, and behavioral patterns that enable you to tackle complex design and architecture challenges, making your programs more robust and maintainable. By the end of the book, you'll be exposed to a wide range of advanced functionalities in Python and be equipped with the practical knowledge needed to apply them to your use cases.
Table of Contents (32 chapters)
1
Section 1: Python-Native and Specialized Optimization
8
Section 2: Concurrency and Parallelism
18
Section 3: Design Patterns in Python

Getting started with Numba

Numba was started in 2012 by Travis Oliphant, the original author of NumPy, as a library for compiling individual Python functions at runtime using the Low-Level Virtual Machine (LLVM) toolchain.

LLVM is a set of tools designed to write compilers. LLVM is language-agnostic and is used to write compilers for a wide range of languages (an important example is the Clang compiler). One of the core aspects of LLVM is the intermediate representation (the LLVM IR), a very low-level, platform-agnostic language-like assembly, that can be compiled to machine code for the specific target platform.

Numba works by inspecting Python functions and compiling them, using LLVM, to the IR. As we saw in the last chapter, speed gains can be obtained when we introduce types for variables and functions. Numba implements clever algorithms to guess the types (this is called type inference) and compiles type-aware versions of the functions for fast execution.

Note that Numba...