Skip main navigation

£199.99 £139.99 for one year of Unlimited learning. Offer ends on 28 February 2023 at 23:59 (UTC). T&Cs apply

Find out more

Using cProfile

In this video Dr. Jussi Enkovaara gives a brief example of finding performance bottlenecks with cProfile tool.
Let’s see how to use the cProfile with a simple example. In the most basic use case one can measure the profile of the whole program with the cProfile by starting to python under the cProfile module with -m cProfile After that with -o we can provide the name of the file where we want to store the profiling results. In this case I choose just profile.dat, it can be any file. And finally we give the name of the file of our program we want to execute. So, we can see that it executes just as normally and this nice figure of Mandelbrot fractal appears in the screen, and we can see also printed result from the internal timer.
Now, in order to investigate the profile we use the pstats module. So we can start python with the module pstats and provide just the name of the file where the results were stored. This brings up now the interactive profile statistics browser. With help we can see information about all the possible commands that are available here. For example, we can look stats command which is used to printing out statistics and we can just say there stats and then print out then functions and modules. As you can see, the output is a bit messy here so we in many cases want to remote full path names of the modules here with the strip command.
Also, the functions normally appear in a random order so we typically want to sort them according to some specified key. By typing sort alone, we can see all the possible keys that are available. Here we want to see how much time is spent in each function so we sort according the time which measures the time spent in the function itself. If we now print out stats with 10 again that shows us 10 most time-consuming functions in the program.
What you appear to see here is that as first actually something which has spent almost 10 seconds, is main loop of some TKinter and this is actually now due to the fact that it took me some time to actually measure or close te graphical window appearing there. Normally we are not actually that interested at how long time the graphical window appears there so what we want to do is to disable the printing out or plotting out the figure before continuing with the profile. So, let’s exit the profiler and edit briefly our main file here , and comment out the plotting from this one.
It’s a relatively common that there are some parts of the program that you are not interested in measuring, for example related to input and output which you might want to disable. We can now continue and run the program again under the cProfile just same as before and now the graphical window doesn’t appear and our previous profile got overwritten. If we now start again the interactive browser here, strip the path names there, sort according to time and print out the ten most time-consuming functions, we can see that now it’s actually the computational part that shows up here. Function kernel in the module is definitely the most time-consuming function.
It has been called 40,000 times and in total it has spent about 0.9 seconds. The other time we can see here, cumulative time, measures also the time spent in the children of the function, so if we sort according to cumtime and print now ten most time-consuming functions in this respect we can see that the first there is the main program which then calls all others and for example the compute_mandel function here itself it takes very little time, 0.04 seconds, but because the kernel is called within that function the cumulative time includes also time spent in this kernel.
cProfile provides easy way to find out which are really the most time-consuming parts of the program and where you should focus your optimization efforts in future.

cProfile is powerful performance analysis tool included in the Python standard library.

In this video we walk through the main steps when investigating performance of Python programs with cProfile. As an example, we use a simple program that calculated Mandelbrot fractal.

This article is from the free online

Python in High Performance Computing

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education