Multiprocessing in Python

Multiprocessing is a technique that allows a program to divide its workload across multiple cores or processors, enabling it to execute multiple tasks in parallel and significantly speeding up the processing time. In this tutorial, we will explore the basics of multiprocessing in Python, understand why it is important, and learn how to use the multiprocessing module to write efficient and scalable multi-core programs. Whether you're a beginner or a seasoned Python developer, this tutorial will help you unlock the true potential of Python's multiprocessing capabilities.

Introduction to Multiprocessing in Python :

• Multiprocessing is a way of achieving parallelism by running multiple processes at the same time.
• Python provides a built-in  multiprocessing  module to support multiprocessing.

What are I/O Bound Tasks :

• I/O bound tasks are tasks that involve a lot of input/output operations such as reading or writing to files or databases.
• Multiprocessing can be useful for I/O bound tasks as it allows the CPU to perform other operations while waiting for the I/O to complete.

What are CPU Bound Tasks :

• CPU bound tasks are tasks that involve a lot of CPU computation such as numerical calculations or image processing.
• Multiprocessing can be useful for CPU bound tasks as it allows the workload to be split across multiple CPUs to increase speed and efficiency.

How to use Multiprocessing in Python :

• Here's an example of how multiprocessing can be used to speed up a CPU bound task of calculating the square of numbers.
• Code Sample :

import multiprocessing

def calc_square(numbers):
   for n in numbers:
       print('Square:', n*n)

if __name__ == '__main__':
   numbers = [1, 2, 3, 4, 5]
   
   p = multiprocessing.Process(target=calc_square, args=(numbers,))
   p.start()
   p.join()
   
   
   

How to Use Multiprocessing Module in python :

• The  multiprocessing  module provides several classes and functions to support multiprocessing.
• Code Sample :

import multiprocessing

def worker(num):
   """worker function"""
   print('Worker:', num)
   return
if __name__ == '__main__':
   jobs = []
   for i in range(5):
       p = multiprocessing.Process(target=worker, args=(i,))
       jobs.append(p)
       p.start()
       
       
       

Python Multiprocessing Practical Example 

• Here's a practical example of how multiprocessing can be used to speed up a CPU bound task of calculating the pages of a large PDF file.
• Code Sample :

import multiprocessing
import PyPDF2

def count_pages(filename):
   with open(filename, 'rb') as file:
       pdf_reader = PyPDF2.PdfFileReader(file)
       return pdf_reader.getNumPages()

if __name__ == '__main__':
   filenames = ['file1.pdf', 'file2.pdf', 'file3.pdf']
   
   with multiprocessing.Pool() as pool:
       results = pool.map(count_pages, filenames)
   
   for filename, pages in zip(filenames, results):
       print(f'{filename}: {pages} pages')