Introduction
Memoryview is a powerful feature in Python that allows you to access the internal memory of object arrays. It provides you with direct access to the memory buffer of an array, which is useful when dealing with large datasets or working with data that is stored in a non-standard way. With Memoryview, you can manipulate data directly without the need for extra copies or conversions. In this tutorial, we will explore what Memoryview is, how it works, and how you can use it in your Python programs to optimize performance. Let's dive in!
Table of Contents :
- What is Buffer protocol in Python
- What is Memoryview in Python
- Why do we need memoryview
Before understanding memoryview in python, we'll have to study a concept called buffer protocol.
What is Buffer protocol in Python
- In normal scenario, while using an object for different operations, Python creates an intermediate copy of the object.
- When dealing with huge datasets, this process uses lot of memory and execution time.
- to overcome this problem, buffer protocol can be used for processes involving huge data sets and scientific computations.
- Buffer protocol allows access to the internal data of an object. This internal data is a memory array (also known as buffer).
- In the buffer protocol
- Python objects expose raw byte arrays to other Python objects.
- in other words one object exposes its internal data (buffers) to other objects.
- Multiple objects can use and manipulate views of same data buffers directly, without the need to make multiple copies of data.
- This technique is very useful when dealing with large datasets and scientific computing.
- Buffer protocol in python can be used through our normal code but can be accessed only at the C-API level.
- This is where memoryview comes into play.
What is Memoryview in Python
- A memory view is a method of utilizing the buffer protocol through normal python code.
- The memoryview creates a view of the internal data of an object without the need to make a copy of object.
- In Python bytes and bytearray support the buffer protocol.
- Hence memoryview can be created on top of bytes and bytearray objects.
- Memoryview objects can make use of indexing much like lists and tuples.
- Code Sample :
obj = bytearray('Kolledge','utf-8')
m = memoryview(obj)
print("Data type of m : ", type(m))
print(m[0])
print(bytes(m[0:3]))
# Output
# Data type of m : <class 'memoryview'>
# 75
# b'Kol'
Why do we need memoryview
- In Python when some action is performed on an object (e.g. function call, slicing array, sorting list etc.), Python interpreter first creates a copy of the object.
- In case of large data (e.g. binary data of an image, video etc.) needless copies of huge chunks of data are created.
- This process results in huge loss of time and memory.
- By using buffer protocol, we can access and modify this large data without copying it.
- Memoryview can utilize buffer protocol to access this large data without making a copy of it.
- Thus memoryview decreases the execution time and saves memory.
Prev. Tutorial : Bytes and ByteArray Data type
Next Tutorial : Bool Data type