Entering into the World of Concurrency with Python

In this tutorial, we will explore concurrency in Python. We'll discuss Threads and Processes and how they're similar and different. You'll also learn about Multi-threading, Multi-processing, Asynchronous Programming, and Concurrency in general in Python.

Many tutorials discuss these concepts, but they can be challenging to grasp due to the lack of clear examples. I will guide you through the concepts and delve into the world of Concurrency in Python, providing straightforward explanations for better understanding.

Table of Contents

• Sequential Programming
• What Are Threads?
• What Are Processes?
• Threads vs Processes
• What is Multi-Threading?
• What is Multi-Processing?
• What is Asynchronous Programming?
• What is Concurrency?
• Global Interpreter Lock (GIL)
• How to Remove the Limitations of GIL

Sequential Programming

When you begin learning programming languages like Python or Java, your program usually runs in a sequence, starting from the top and going down.

a = 25
b = 30

def add(a, b):
    print("Addition value: ", a+b)

def sub(a, b):
    print("Subtraction Value: ", a-b)

add(a,b)
sub(a,b)

What Are Threads?

When you run your program, the Python interpreter creates a small program internally and instructs it to start running. This small program, created by the interpreter, is called a Thread. A thread is a small program that performs a certain task.

What Are Processes?

Imagine you own a company and have many tasks to do. To handle them, you will begin hiring people and assigning a task to each person.

Each person works independently on their task. They have their own set of tools and resources, and if one person finishes their task, it doesn't directly affect what the others are doing.

In the world of computers, each person doing a task is like a separate process. Processes are independent, have their own memory space, and don't directly share resources.

Threads vs Processes

What is Multi-Threading?

Life is easier when you have a lot of workers to complete all the necessary tasks, isn't it? Similarly, if you have multiple threads, they help you divide all your program tasks into smaller parts.

import threading

add_thread = threading.Thread(target=add, args=(a, b))
sub_thread = threading.Thread(target=sub, args=(a, b))
add_thread.start()
sub_thread.start()
add_thread.join()
sub_thread.join()

What is Multi-Processing?

From the name itself, you may be able to understand that multiprocessing means running multiple processes separately without sharing resources directly.

from multiprocessing import Process

add_process = Process(target=add, args=(a, b))
sub_process = Process(target=sub, args=(a, b))
add_process.start()
sub_process.start()
add_process.join()
sub_process.join()

What is Asynchronous Programming?

Asynchronous programming is a specific approach to managing tasks that involves waiting for external operations to complete. It allows a program to continue executing other tasks while waiting for these operations to finish, rather than blocking until they complete.

This way of managing the tasks to execute simultaneously when one task is waiting for external resources is called Asynchronous programming.

What is Concurrency?

Global Interpreter Lock (GIL)

GIL is a rule that allows only one thread at a time to run the Python bytecode. Global Interpreter Lock is a lock that protects access to Python objects, preventing multiple threads from executing Python bytecode simultaneously in a single process.

import time

def count_up():
    count = 0
    for i in range(100000000):
        count = count + i

def count_down():
    count = 0
    for i in range(100000000):
        count = count + i

if __name__ == "__main__":
    start_time = time.time()
    count_up()
    count_down()
    end_time = time.time()
    print(f"Time taken: {end_time - start_time} seconds")

So how do you remove the limitations of GIL?

1. Use another flavor of Python

Python standard implementation is Cpython. To avoid GIL, we can use Jython (Python developed using Java), IronPython (Python developed using .NET), or PyPy (Python developed using Python).

2. Use the multiprocessing module

Multiprocessing is a module that helps you take advantage of your multiple CPU cores for running your program in separate processes. Each process has its own CPU core, memory, resources, and interpreter.

import multiprocessing
import time

if __name__ == "__main__":
    start_time = time.time()
    process1 = multiprocessing.Process(target=count_up)
    process2 = multiprocessing.Process(target=count_down)
    process1.start()
    process2.start()
    process1.join()
    process2.join()
    end_time = time.time()
    print(f"Time taken: {end_time - start_time} seconds")

3. Implement asynchronous programming!

Using the asyncio module of Python, you can implement the concept of Asynchronous Programming. This module is mainly used in I/O operations and allows you to write non-blocking code.

Summary

Until then, your friend here, Hemachandra, is signing off…