Book Image

Python High Performance, Second Edition - Second Edition

By : Dr. Gabriele Lanaro
Book Image

Python High Performance, Second Edition - Second Edition

By: Dr. Gabriele Lanaro

Overview of this book

Python is a versatile language that has found applications in many industries. The clean syntax, rich standard library, and vast selection of third-party libraries make Python a wildly popular language. Python High Performance is a practical guide that shows how to leverage the power of both native and third-party Python libraries to build robust applications. The book explains how to use various profilers to find performance bottlenecks and apply the correct algorithm to fix them. The reader will learn how to effectively use NumPy and Cython to speed up numerical code. The book explains concepts of concurrent programming and how to implement robust and responsive applications using Reactive programming. Readers will learn how to write code for parallel architectures using Tensorflow and Theano, and use a cluster of computers for large-scale computations using technologies such as Dask and PySpark. By the end of the book, readers will have learned to achieve performance and scale from their Python applications.

Related Content you might be interested in

Current Title:

Small book image
Python High Performance, Second Edition - Second Edition
Dr. Gabriele Lanaro
May, 2017 | 270 pages
Small book image
IPython Interactive Computing and Visualization Cookbook
Cyrille Rossant
Jan, 2018 | 548 pages
Small book image
Python Parallel Programming Cookbook
Giancarlo Zaccone
Sep, 2019 | 370 pages
Small book image
Mastering Python 2E
Rick van Hattem
May, 2022 | 710 pages
Table of Contents (10 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
Benchmarking and Profiling
Benchmarking and Profiling
Designing your application
Writing tests and benchmarks
Better tests and benchmarks with pytest-benchmark
Finding bottlenecks with cProfile
Profile line by line with line_profiler
Optimizing our code
The dis module
Profiling memory usage with memory_profiler
Summary
2
Pure Python Optimizations
Pure Python Optimizations
Useful algorithms and data structures
Caching and memoization
Comprehensions and generators
Summary
3
Fast Array Operations with NumPy and Pandas
Fast Array Operations with NumPy and Pandas
Getting started with NumPy
Rewriting the particle simulator in NumPy
Reaching optimal performance with numexpr
Pandas
Summary
4
C Performance with Cython
C Performance with Cython
Compiling Cython extensions
Adding static types
Sharing declarations
Working with arrays
Particle simulator in Cython
Profiling Cython
Using Cython with Jupyter
Summary
5
Exploring Compilers
Exploring Compilers
Numba
The PyPy project
Other interesting projects
Summary
6
Implementing Concurrency
Implementing Concurrency
Asynchronous programming
The asyncio framework
Reactive programming
Summary
7
Parallel Processing
Parallel Processing
Introduction to parallel programming
Using multiple processes
Parallel Cython with OpenMP
Automatic parallelism
Summary
8
Distributed Processing
Distributed Processing
Introduction to distributed computing
Dask
Using PySpark
Scientific computing with mpi4py
Summary
9
Designing for High Performance
Designing for High Performance
Choosing a suitable strategy
Organizing your source code
Isolation, virtual environments, and containers
Continuous integration
Summary

Designing for High Performance

In the earlier chapters, we learned how to use the vast array of tools available in Python's standard library and third-party packages to assess and improve the performance of Python applications. In this chapter, we will provide some general guidelines on how to approach different kinds of applications as well as illustrate some good practices that are commonly adopted by several Python projects.

In this chapter, we will learn the following:

  • Picking the right performance technique for generic, number crunching, and big data applications
  • Structuring a Python project
  • Isolating Python installations with virtual environments and containerization
  • Setting up continuous integration with Travis CI
Previous Section
End of Section 1
Next Section