Python List vs Numpy Arrays
In the given code which we have just initializes an array with size 10^8 size with Python list and Numpy arrays. Run the code and see the output 👀.
import random
from datetime import datetime
start_time = datetime.now()
python_list = []
for i in range(10**8):
value = random.randint(1, 10)
python_list.append(random.randint(1, 10))
end_time = datetime.now()
diff = end_time - start_time
print(f"Python List takes {diff} time.")
start_time = datetime.now()
numpy_array = np.ndarray(shape=(1,10**8), dtype=int)
for i in range(10**8):
value = random.randint(1, 10)
numpy_array[0][i] = value
end_time = datetime.now()
diff = end_time - start_time
print(f"Numpy List takes {diff} time.")Ok let me give you the output I get after running this.
Python List takes 0:01:57.168245 time.
Numpy List takes 0:01:34.931801 time.There is almost 23s diff in both even though it just creates a basic list with values. Have you ever wonder why Numpy is fast over default Python List ⏩. In this post we will break down the reason for this.
Python List
Python List is used to store a list of items in Python, it is mutable and we can store any type of data in a given list. List provides many default functions and can be very useful for dealing with lists of items in your application.
Couple of list functions,
- It is Mutable.
- It is Iterable.
- It can store any data type as an item, even another list as an item.
- It allows duplicates.
- It maintains order of elements.
I have a full article on How Python List works? Do check it out also.
Coming back to main topic, as data type and size is not fixed the Python needs to do many operations and checks to maintain these functions.
Python List provides the insertion in O(1) but as list size increases the operation to increase the default size of list is increased the time on execution. So even though List provides many useful and dynamic functions, at some time speed will be an issue with huge data size.
Also as data type is not fixed The Python has to do many operations on runtime before doing operation on list 🌚.
Numpy Array
Numpy is The fundamental package for scientific computing with Python. Indeed it uses almost all high computation tasks in Python. It provides a default way to work with numbers with efficacy. From working on 1-d data to n-d data with all types of computations.
I have a full article on How does a Numpy array work? Do check it out also.
The array in Numpy is called ndarray. As the name suggests it will be an n-dimensional array. Can be mapped with Matrix in normal terms.
One the main thing we can notice is that it is homogeneous data types with fixed size. Same as the C array if you remember.Because the size of the array is fixed with the same data types of all items it gives Numpy to buy pass many default validation and calculator Python list do.we can assume Numpy array as array in C.
Advantages Numpy has over Python list because of fixed size 🐱.
- The size of the array is fixed which allows memory allocation fast.
- The data type is the same which allows the Numpy array to map directly with the C array.
- Overall operations become fast as size and data type is fixed.
Comparison:
Numpy Array fixed size and data type allows Numpy to perform operation way faster then Python List.
Let’s breakdown why Numpy take less time then Python list in our case:
- First when you create an array with size 10^8, Numpy allocates memory at time on array initialization, but where else Python lists allocation memory as an item appended on it.
- Added/Update values at index x in Numpy just update values at that index where else in Python list it creates new allocation and maps the point at list index.
- The Numpy array size is fixed but Python lists create a list when the limit is reached whenever we append values one by one.
And there are many other advantages with Numpy arrays over List when It comes to doing operations on fixed data.Thus Numpy is used in many high computation tasks.
Python list can be used when your data size is not fixed and the data type of all items is not the same. Based on the use case you can decide what you need to use.
TLDR;
Sometimes you have to be more flexible and sometimes you have to be more strict in life based on use cases, same as Python List and Numpy.Bye 👋 …..
