The idea of this book came to me from the nice people at Packt Publishing. They wanted someone who could delve into the intricacies of high performance in Python and everything related to this subject, be it profiling, the available tools (such as profilers and other performance enhancement techniques), or even alternatives to the standard Python implementation.
Having said that, I welcome you to Mastering Python High Performance. In this book, we'll cover everything related to performance improvements. Knowledge about the subject is not strictly required (although it won't hurt), but knowledge of the Python programming language is required, especially in some of the Python-specific chapters.
We'll start by going through the basics of what profiling is, how it fits into the development cycle, and the benefits related to including this practice in it. Afterwards, we'll move on to the core tools required to get the job done (profilers and visual profilers). Then, we will take a look at a set of optimization techniques and finally arrive at a fully practical chapter that will provide a real-life optimization example.
Chapter 1, Profiling 101, provides information about the art of profiling to those who are not aware of it.
Chapter 2, The Profilers, tells you how to use the core tools that will be mentioned throughout the book.
Chapter 3, Going Visual – GUIs to Help Understand Profiler Output, covers how to use the pyprof2calltree and RunSnakeRun tools. It also helps the developer to understand the output of cProfile with different visualization techniques.
Chapter 4, Optimize Everything, talks about the basic process of optimization and a set of good/recommended practices that every Python developer should follow before considering other options.
Chapter 5, Multithreading versus Multiprocessing, discusses multithreading and multiprocessing and explains how and when to apply them.
Chapter 6, Generic Optimization Options, describes and shows you how to install and use Cython and PyPy in order to improve code performance.
Chapter 7, Lightning Fast Number Crunching with Numba, Parakeet, and pandas, talks about tools that help optimize Python scripts that deal with numbers. These specific tools (Numba, Parakeet, and pandas) help make number crunching faster.
Chapter 8, Putting It All into Practice, provides a practical example of profilers, finds its bottlenecks, and removes them using the tools and techniques mentioned in this book. To conclude, we'll compare the results of using each technique.
Your system must have the following software before executing the code mentioned in this book:
Python 2.7
Line profiler 1.0b2
Kcachegrind 0.7.4
RunSnakeRun 2.0.4
Numba 0.17
The latest version of Parakeet
pandas 0.15.2
Since the topics tackled in this book cover everything related to profiling and optimizing the Python code, Python developers at all levels will benefit from this book.
The only essential requirement is to have some basic knowledge of the Python programing language.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "We can print/gather the information we deem relevant inside the PROFILER function."
A block of code is set as follows:
import sys
def profiler(frame, event, arg):
print 'PROFILER: %r %r' % (event, arg)
sys.setprofile(profiler)When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
Traceback (most recent call last):
File "cprof-test1.py", line 7, in <module>
runRe() ...
File "/usr/lib/python2.7/cProfile.py", line 140, in runctx
exec cmd in globals, locals
File "<string>", line 1, in <module>
NameError: name 're' is not defined Any command-line input or output is written as follows:
$ sudo apt-get install python-dev libxml2-dev libxslt-dev
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Again, with the Callee Map selected for the first function call, we can see the entire map of our script."
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from: https://www.packtpub.com/sites/default/files/downloads/9300OS_GraphicBundle.pdf.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.