Book Image

Python High Performance, Second Edition - Second Edition

By : Dr. Gabriele Lanaro
Book Image

Python High Performance, Second Edition - Second Edition

By: Dr. Gabriele Lanaro

Overview of this book

Python is a versatile language that has found applications in many industries. The clean syntax, rich standard library, and vast selection of third-party libraries make Python a wildly popular language. Python High Performance is a practical guide that shows how to leverage the power of both native and third-party Python libraries to build robust applications. The book explains how to use various profilers to find performance bottlenecks and apply the correct algorithm to fix them. The reader will learn how to effectively use NumPy and Cython to speed up numerical code. The book explains concepts of concurrent programming and how to implement robust and responsive applications using Reactive programming. Readers will learn how to write code for parallel architectures using Tensorflow and Theano, and use a cluster of computers for large-scale computations using technologies such as Dask and PySpark. By the end of the book, readers will have learned to achieve performance and scale from their Python applications.

Related Content you might be interested in

Current Title:

Small book image
Python High Performance, Second Edition - Second Edition
Dr. Gabriele Lanaro
May, 2017 | 270 pages
Small book image
IPython Interactive Computing and Visualization Cookbook
Cyrille Rossant
Jan, 2018 | 548 pages
Small book image
Python Parallel Programming Cookbook
Giancarlo Zaccone
Sep, 2019 | 370 pages
Small book image
Mastering Python 2E
Rick van Hattem
May, 2022 | 710 pages
Table of Contents (10 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
Benchmarking and Profiling
Benchmarking and Profiling
Designing your application
Writing tests and benchmarks
Better tests and benchmarks with pytest-benchmark
Finding bottlenecks with cProfile
Profile line by line with line_profiler
Optimizing our code
The dis module
Profiling memory usage with memory_profiler
Summary
2
Pure Python Optimizations
Pure Python Optimizations
Useful algorithms and data structures
Caching and memoization
Comprehensions and generators
Summary
3
Fast Array Operations with NumPy and Pandas
Fast Array Operations with NumPy and Pandas
Getting started with NumPy
Rewriting the particle simulator in NumPy
Reaching optimal performance with numexpr
Pandas
Summary
4
C Performance with Cython
C Performance with Cython
Compiling Cython extensions
Adding static types
Sharing declarations
Working with arrays
Particle simulator in Cython
Profiling Cython
Using Cython with Jupyter
Summary
5
Exploring Compilers
Exploring Compilers
Numba
The PyPy project
Other interesting projects
Summary
6
Implementing Concurrency
Implementing Concurrency
Asynchronous programming
The asyncio framework
Reactive programming
Summary
7
Parallel Processing
Parallel Processing
Introduction to parallel programming
Using multiple processes
Parallel Cython with OpenMP
Automatic parallelism
Summary
8
Distributed Processing
Distributed Processing
Introduction to distributed computing
Dask
Using PySpark
Scientific computing with mpi4py
Summary
9
Designing for High Performance
Designing for High Performance
Choosing a suitable strategy
Organizing your source code
Isolation, virtual environments, and containers
Continuous integration
Summary

Fast Array Operations with NumPy and Pandas

NumPy is the de facto standard for scientific computing in Python. It extends Python with a flexible multidimensional array that allows fast and concise mathematical calculations.

NumPy provides common data structures and algorithms designed to express complex mathematical operations using a concise syntax. The multidimensional array, numpy.ndarray, is internally based on C arrays. Apart from the performance benefits, this choice allows NumPy code to easily interface with the existing C and FORTRAN routines; NumPy is helpful in bridging the gap between Python and the legacy code written using those languages.

In this chapter, we will learn how to create and manipulate NumPy arrays. We will also explore the NumPy broadcasting feature used to rewrite complex mathematical expressions in an efficient and succinct manner.

Pandas is a tool that relies heavily on NumPy and provides...

Previous Section
End of Section 1
Next Section