Overview of MPI shared memory
The following list shows the key issues for using MPI shared memory:
- Split main communicator into shared memory islands
MPI_Comm_split_type
- Define a shared memory window on each island
MPI_Win_allocate_shared
- Result (by default):
contiguous array, directly accessible by all processes of the island
- Accesses and synchronization
- Normal assignments and expressions
- No
MPI_Put/Get!
- Normal MPI one-sided synchronization, e.g.,
MPI_Win_fence
MPI_Comm_split_type into shared memory islands, because only the processes that have physical shared memory together can produce a shared memory window.
MPI_Win_allocate_shared. Of course, only the processes that are together in a shared memory island can call this routine collectively. It behaves like MPI_Win_allocate from Steps 1.10 and 2.1, but by default, you get all your windows combined to one (long) contiguous array. There is another method where the array is not contiguous, and typically uses several physical memories within a cc-NUMA node.
MPI_Put or MPI_Get here is absolutely unnecessary. In fact, what your Fortran or C compiler produces is significantly faster than if you call a library routine like MPI_Put or MPI_Get.
MPI_Win_fence.
Caution:
- Memory may already be completely pinned to the physical memory of the process with rank 0, i.e. the first touch rule (as in OpenMP) does not apply!
- First touch rule: a memory page is pinned to the physical memory of the processor that first writes a byte into the page.
At this point the first warning: with your Win_allocate_shared you will typically get a memory that may already be completely pinned in the hardware. This means that the affinity of a memory chunk to a given process will not be determined by the first access to it (i.e. the so-called first-touch rule, which is typically valid with OpenMP, does not apply here).
One-Sided Communication and the MPI Shared Memory Interface
One-Sided Communication and the MPI Shared Memory Interface
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
