Book Image

Advanced Python Programming - Second Edition

By : Quan Nguyen
Book Image

Advanced Python Programming - Second Edition

By: Quan Nguyen

Overview of this book

Python's powerful capabilities for implementing robust and efficient programs make it one of the most sought-after programming languages. In this book, you'll explore the tools that allow you to improve performance and take your Python programs to the next level. This book starts by examining the built-in as well as external libraries that streamline tasks in the development cycle, such as benchmarking, profiling, and optimizing. You'll then get to grips with using specialized tools such as dedicated libraries and compilers to increase your performance at number-crunching tasks, including training machine learning models. The book covers concurrency, a major solution to making programs more efficient and scalable, and various concurrent programming techniques such as multithreading, multiprocessing, and asynchronous programming. You'll also understand the common problems that cause undesirable behavior in concurrent programs. Finally, you'll work with a wide range of design patterns, including creational, structural, and behavioral patterns that enable you to tackle complex design and architecture challenges, making your programs more robust and maintainable. By the end of the book, you'll be exposed to a wide range of advanced functionalities in Python and be equipped with the practical knowledge needed to apply them to your use cases.
Table of Contents (32 chapters)
1
Section 1: Python-Native and Specialized Optimization
8
Section 2: Concurrency and Parallelism
18
Section 3: Design Patterns in Python

Summary

In this chapter, we introduced the basic principles of optimization and applied those principles to a test application. When optimizing an application, the first thing to do is test and identify the bottlenecks in the application. We saw how to write and time a benchmark using the time Unix command, the Python timeit module, and the full-fledged pytest-benchmark package. We learned how to profile our application using cProfile, line_profiler, and memory_profiler, and how to analyze and navigate the profiling data graphically with KCachegrind.

Speed is undoubtedly an important component of any modern software. The techniques we have learned in this chapter will allow you to systematically tackle the problem of making your Python programs more efficient from different angles. Further, we have seen that these tasks can take advantage of Python built-in/native packages and do not require any special external tools.

In the next chapter, we will explore how to improve performance using algorithms and data structures available in the Python standard library. We will cover scaling and sample usage of several data structures, and learn techniques such as caching and memorization. We will also introduce Big O notation, which is a common computer science tool to analyze the running time of algorithms and data structures.