Skip main navigation

Restrictions for MPI shared memory

In this article we discuss restrictions for MPI shared memory, for example limited size or number of MPI shared memory windows.
© HLRS

The following caution is discussed with members of the MPI forum:

Caution: On some systems
  • the number of shared memory windows, and
  • the total size of shared memory windows may be limited.
Some operating systems may provide options, e.g.,
  • at job launch, or
  • MPI process start,
to enlarge restricting defaults.
In effect, the number of bytes you can get for shared memory windows and the number of shared memory windows you can get at all, can be limited. On some operating systems there are methods for you to extend these limitations. The first limitation you have is that the number of window handles is limited, for example, to around 2043. Despite this fact, in principle, every MPI library has a method to increase this number by an environment variable before you call mpirun. So, if you need 7.000 window handles this should be possible. The second limitation you have is that only a part of the memory may be available for shared memory:
If MPI shared memory support is based on POSIX shared memory:
  • Shared memory windows are located in memory-mapped /dev/shm or /run/shm
  • Default: 25% or 50% of the physical memory,
  • Root may change size, e.g., with:
    mount -o remount,size=6G /dev/shm

In most cases, if you have a POSIX shared memory system, your available shared memory is mapped to /dev/shm or /run/shm. A system administrator can change this size at any time with a remount. Typically, your administrator will give you 25 or 50% of the memory and the rest won’t work. In current supercomputing system it may be possible to increase this percentage up to 90%.

© HLRS
This article is from the free online

One-Sided Communication and the MPI Shared Memory Interface

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education