Distinguishing between Concurrency, Parallelism, Asynchronous Programming and Multi-threading
Cover Photo by Martin Sanchez on Unsplash
Introduction
Hello Everyone!
So, I have always wondered how applications handle multiple client requests simultaneously. My curiosity led me to study the following concepts:
- Concurrency
- Asynchronous Programming
- Parallelism
- Multi-threading
During my study, I realized that many people often confuse these concepts. This article aims to simplify these concepts for newbies to programming.
Having said the lot, let’s jump right into it.
Concurrency
Concurrency refers to the coordination and management of multiple tasks happening at the same time. The concurrently running tasks can be parts of the same program, methods of the same program or even different programs. The need for more processing speed makes concurrency an important concept in computing.
Concurrency is visible in everyday computing, for instance:
- You can use a word processor program and at the same time listen to music on the same computer.
- You can run multiple instances of the same application on our computer at the same time.
- Multimedia streaming applications allow you to download a file and consume it at the same time.
The increased demand for computers that support multiple processors has facilitated concurrent programming. Parallelism makes concurrency possible in a multi-processor system. Concurrency is possible in a single processor system via Context-Switching. Context switching is a system whereby the processor assigns resources to tasks based on two factors:
- time slice
- task priority
Parallelism
Parallelism in computing refers to a paradigm where programs are designed to run concurrently on more than one processor. The process of parallelism is described as follows:
- A problem is broken into discrete tasks that can be solved concurrently
- Each task is further broken down into a series of instructions
- Instructions from each task execute simultaneously on different processors
- A system that coordinates message sharing and synchronization between processes is employed.
The increased usage of multicore processors and GPUs have facilitated parallel processing. Parallelism is utilized heavily in Game Development. GPUs work together with CPUs to increase the throughput of data and the number of concurrent calculations within an application.
image source: hpc.llnl.gov/sites/default/files/parallelPr..
Asynchronous Programming
In other to understand the concept of asynchronous programming, you should first understand the concept of synchronous programming.
In synchronous programming, the processor executes tasks in a sequence. The currently running task must first finish before the processor picks up the next task for execution. In synchronous programming, tasks execute in a blocking fashion(the execution of the next task is blocked until the current task is finished).
In asynchronous programming, the processor executes tasks simultaneously. The processor does not wait for a task to complete execution before it picks up another task. This fashion of simultaneous execution is referred to as non-blocking.
In a computer program, experts advise that a function should be asynchronous when it performs a lengthy operation. A function is designed for asynchrony by including a callback function when invoking them. The asynchronous function executes the callback when it has completed execution as a way of notifying the program about its completion. The execution of the asynchronous function does not block the execution of other functions.
Asynchronous programs can be single-threaded or multi-threaded. So in asynchronicity, you see dependencies between several parts of a single process
image source: linkedin.com/pulse/asynchronous-calls-sprin..
Multi-threading
A thread is a unit of work or a task. An operation can be split to run on multiple threads. Multi-threading refers to the concurrent/ parallel execution of multiple threads.
In single-processor systems, Multi-threading is achieved via context-switching.
In multi-processor systems, Multi-threading is achieved via parallelism.
image source: baeldung.com/cs/async-vs-multi-threading
A basic example of multi-threading is downloading two files from two different tabs in a web browser. Each tab uses a new thread to download the requested file. No tab waits for the other one to finish, the downloads run concurrently.
Multi-threading is breaking down tasks into threads that can be executed by different processors. Asynchronous programs can be multi-threaded or can be single-threaded.
A program that runs in parallel is also multi-threaded since parallel programs are designed in such a way that they can be broken down into discrete threads that run over multiple processors in a multi-processor system
To solidify the concepts, I find the following analogies quite useful:
Parallelism is like hiring two chefs( 2 processors). One cooks rice (one process) using a set of resources(stove and pots) and the other prepares stew (another process) using another set of resources.
Single-threaded asynchronicity is like setting to cook fried rice. You place the rice on fire to cook and while the rice is cooking, you slice your vegetables and make a sauce. After the rice is cooked, you mix the rice and the sauce
Multi-threaded asynchronicity is like you setting out to cook fried rice with your brother. You place the rice on fire to cook while he slices the vegetables. You make a sauce while he sets the table for the meal. (All these three tasks involved in the process of cooking rice are interdependent. One process does not wait for the other to finish). When the rice is cooked, you proceed to mix it with the sauce.
I look forward to hearing from you.
Always code with ❤️
I hope that you enjoyed the tutorials and learnt something new too. You can follow me on Twitter on @ehizman. This will help me write more valuable content.
Special shout-outs to @nomso_okere and Mr Debola Olowose for helping me with reviews and edits.