£199.99 £139.99 for one year of Unlimited learning. Offer ends on 14 November 2022 at 23:59 (UTC). T&Cs apply

Find out more
Execution and data model in MPI
Skip main navigation

Execution and data model in MPI

In MPI, a parallel program is executed as a set of independent processes. Watch Jussi Enkovaara explain the execution and data model in MPI.
11.6
Let’s look briefly into execution and data model in MPI. In MPI parallel program is executed as a set of independent, identical process. That is, when program is launched for example with mpirun, the runtime creates N copies of the program which all execute the same program code. These processes are often called MPI tasks. How the N copies of the program can then perform different tasks? The MPI runtime assigns each process an unique id number which in MPI terminology is called rank.
54
The ranks start always from zero and go up to N-1. Based on the rank, the different processes can, for example, branch into different path of the program execution. Identical processes means thus that all the processes execute the same if statement, however, how they branch then depends on the rank. As the MPI tasks are process, they have their own memory space which are distinct from the other processes. Thus, the MPI tasks cannot directly access the data of the other tasks. In MPI, as the name message passing suggests, the tasks exchange data by sending and receiving messages. In the normal use case, the communication is two sided, there is always both sending and receiving involded, and sends and receives need to match.
108.3
To summarize, in MPI one launches multiple processes which all execute the same program code. Based on the rank, these multiple processes can perform different tasks. Processes communicate by sending and receiving messages.

In MPI, a parallel program is executed as a set of independent processes.

In this video we illustrate in more detail the execution and data model in MPI.

This article is from the free online

Python in High Performance Computing

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education