Just like any infant needs to learn how to crawl before running 100 mts with obstacles in under 12 seconds, programmers need to understand the basics of profiling before trying to master that art. So, before we start delving into the mysteries of performance optimization and profiling on Python programs, we need to have a clear understanding of the basics.
Once you know the basics, you'll be able to learn about the tools and techniques. So, to start us off, this chapter will cover everything you need to know about profiling but were too afraid to ask. In this chapter we will do the following things:
We will provide a clear definition of what profiling is and the different profiling techniques.
We will explain the importance of profiling in the development cycle, because profiling is not something you do only once and then forget about it. Profiling should be an integral part of the development process, just like writing tests is.
We will cover things we can profile. We'll go over the different types of resources we'll be able to measure and how they'll help us find our problems.
We will discuss the risk of premature optimization, that is, why optimizing before profiling is generally a bad idea.
You will learn about running time complexity. Understanding profiling techniques is one step into successful optimization, but we also need to understand how to measure the complexity of an algorithm in order to understand whether we need to improve it or not.
We will also look at good practices. Finally, we'll go over some good practices to keep in mind when starting the profiling process of your project.