NumPy arrays
Learn what high-performance arrays and matrices are provided in the NumPy library to dramatically speed up the runtime of code.
Hi, and welcome back to the course. So the main theme this week has been learning about time complexity, learning about the data structures of Python, and what that means about the time complexity of algorithms and operations you can perform on them. We also saw a cheat sheet to help us look at that. We also saw a cheat sheet of more general data structures and algorithms. But one thing we need to talk about that falls under the list of how can we write faster code is NumPy arrays. This is a little different question than what we talked about.
This is more about having a Python library - or Python package - which is NumPy that uses compiled code, code that’s written in C and C++. Maybe there’s some Fortran in there. I don’t know. But it’s compiled code to process math on arrays like vectors and matrices. And this is so much faster than what you would do in Python, because it’s compiled. So Python is an interpreted language, and it’s always going to be slower than a language like C. So if you have a Python package like NumPy that uses underlying code compiled in C on your computer, it will be much faster. And if you are working with data, you will definitely want to use NumPy.
Now, you might be using NumPy indirectly, and that’s fine. What I mean by that is pandas that we talk about in the data analysis course, scikit-learn that we also talk about in the machine learning with Python course, these are all built on top of NumPy. TensorFlow uses NumPy. Dask uses NumPy. Almost any scientific computing Python package will be built on top of NumPy for this very reason, because it is so much faster. And you want to be using that. So I think - I know most of the students are learning Python for some level of data analysis, whether it’s statistical analysis or machine learning or just basic processing of data. So you will definitely want to take advantage of NumPy.
And again, that could be you’re using pandas. And so you’re not directly using NumPy, but pandas is built on top of NumPy. So you really are. But since we’re in this theme of how can we just run faster code, we need to learn about NumPy. It’s really important. And I have an applied math background. So this really resonates with me because in mathematics - and if you come from a Matlab background, which is a different programming language, you’re always trying to - you’re essentially trying to avoid for loops by writing things as vector math. And that’ll make a little more sense, actually, in our first example here. So let’s go ahead and dive into our first example.
So here we have a function. This is from a previous lecture. And it will convert a list of temperatures in Celsius to Fahrenheit. So here let’s run the cell. Actually, we have to run this first cell to import the libraries we’ll use. But let’s run the cell to define the function. Now let’s create a list that we’re pretending is full of Celsius temperatures, and let’s run our function with the timeit magic command. And there we go, so 202 microseconds. OK, that’s good. Now let’s convert that list to a NumPy array. And to do that, you can just pass the list to the array method of the NumPy module here that we imported.
And by the way, when you import NumPy, just as with pandas, if you’ve taken that course, how you import it as pd, NumPy is almost always imported as np. You don’t have to. This is an optional alias. But almost always, you’ll see it imported as np. So that’s what we did here. And so we create this NumPy array called temps_c_array. And I wanted to go ahead and just print the type and then print the first 10 elements. And you can index it a lot like a list, and it prints a lot like a list. So it’s a NumPy array. It happens to be a one-dimensional array because we passed a list to it.
And a list only has one dimension, the length. That’s it. But now we have this NumPy array. And so now, instead of having to loop through each item like we did in our function, you have to loop through each item to operate on it. You can’t operate on the entire list at once. With a NumPy array, we can. We can just multiply every element in that array by 9 over 5 and then add 32. So this is just the formula to convert Celsius to Fahrenheit. And we can apply that to the entire vector, and it will apply that formula to every element at once. So we run that with our timeit magic command.
And you see it comes back, and it’s much faster, right? Only eight microseconds, so much, much faster. It’s maybe about 25 times faster when you use a NumPy array. Now, this may not seem - it’s still very fast, right? This was still very fast. But what if you’re not just doing one list of 1,000, but what if you’re doing 1,000 lists for 1,000 points on the globe, and you have 10 years of data? So now it’s a lot more data. You do not want to use nested for loops, where you would loop over each location of the 1,000 locations and then each temperature for each location. That would be very bad. We don’t want to do that.
We want to use a NumPy array. This will be much faster.
And then here I’m just running the functions again. So we can print the results, and you can see they’re the same. Now, when you print a list, it prints the commas. NumPy prints a little differently - no commas. And if the decimal is a 0, it just won’t print it. But otherwise, it’s the exact same. All right, so that was a good introduction. I should say it was a motivation for why we want to use NumPy. You can see that it can be much faster. Now, in the next lectures, we’ll learn some of the basics of NumPy. And NumPy, we could spend probably hours on NumPy. It’s a very well-developed Python package, kind of like scikit-learn or pandas.
So we won’t be able to explore it all this week, not even close. But we can explore some of the basics to get you comfortable with the basics, show some examples, and then if you go to numpy.org, you will find documentation of tutorials. So you can go deeper into NumPy if you think it’d be a good fit for your work. All right, excellent job, and I’ll see you in the next lecture.
Throughout this week you have learned about time and space complexities, the data structures of Python, and the implications for time complexities of algorithms and operations that you can perform on these structures.
Another topic to add to our list for writing faster code is NumPy arrays. The NumPy library provides high-performance arrays and matrices that we can use to dramatically speed up the runtime of our code. NumPy uses underlying compiled code (C/C++) to achieve this.
Processing data in a NumPy array is much faster than processing it in a Python list. If you find yourself processing lists of numbers, you should always ask yourself if you could model it as an array and model the processing steps vector/matrix math.
In this video, we will use an example to learn how NumPy can allow you to run faster code.
If you are interested in learning about the NumPy package in more detail, feel free to explore the NumPy docs and tutorials.
Follow along
The file used in this video is Numpy Arrays To The Rescue!.ipynb. Please download this file from the Downloads section below.Make sure you are able to access it, in order to follow along with the video.
From the examples shown in this step, you got a good idea of why you can use NumPy, and in the next step we will explore some more basics of NumPy.
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
