New interfaces for musical expression: NIME

What does the term, NIME, mean?

The term NIME refers to “new interfaces for musical expression”. The term is derived from the conference and community with the same name (NIME).

Common among practitioners in the NIME community is that of designing, building, performing and evaluating new musical instruments and other types of musical interfaces.

Musical Instruments

It is difficult to come up with a clear definition of a musical instrument, particularly if one considers all sorts of musical instruments. One common denominator, however, is that an instrument functions as a mediator between action and sound.

As such, any pair of objects and actions that are used in music could be considered a musical instrument.

Digital Musical Instruments

Many new interfaces for musical expression are digital musical instruments (DMIs), involving the use of digital technology in the mediation between actions and sound. This can be seen as a subset of electronic musical instruments, which embeds also analogue, electronic instruments such as the famous MOOG synthesizers.

Three essential parts of a digital instrument

A model capturing three essential parts of the term digital musical instrument is shown in the figure below. The controller is the physical device that the user interacts with. A controller in itself will not produce any sound, however, so the sound engine is where sound generation takes place.

This may be a physical hardware unit or a piece of software. The mapping is a set of rules for how the sound engine should respond when the user interacts with the controller.

There exist numerous digital musical instruments. Many commercial digital musical instruments are based on a piano-like interface, with a controller, mapping, and sound engine built into the same physical device.

Although piano-based instruments are hardly considered new interfaces for musical expression any longer, they are nevertheless digital musical instruments.

^{Nord Stage (Photo by Ben Garney, CC BY 2.0 license)}

The Hand by Michel Waisvisz is an example of a “classic” digital musical instrument not inspired by traditional musical instruments. The controller uses various sensors to capture hand, arm and finger movements.

The sound engine is also separated from the controller, which makes it possible to create many different types of mappings between movement and sound. See The Hands in action in this youtube video.

^{The Hands (Photo by Luka Ivanovic, CC BY-SA 2.0 license)}

NIME as a research field

One of the main areas of interest to NIME researchers is the development of new modes of interaction:

• How can musical sound be controlled in intuitive and interesting manners?
• How does the way we control musical sound shape our perception of the sound?
• How can new technologies be used to enhance our musical experiences?

At the University of Oslo, we have been interested in exploring different ways of creating NIMEs based on our research on embodied music cognition. Here we have to use the results from cognitive experiments when designing mappings between movement and sound.

Similarly, we have explored how NIMEs may be used in experiments. One such example is that of the “instrument” MuMYO.

MuMYO

The MuMYO instrument used in the previous video uses the commercially available MYO armband as the sensor. The MYO consists of eight electromyography (EMG) sensors that measure muscle tension, in addition to an inertial measurement unit measuring motion and rotation.

Using a machine-learning algorithm to extract useful features from the muscle sensing data, the device is able to detect hand postures such as waving in or out, making a fist, or spreading the fingers.

The possibility to capture muscle tension data is interesting when it comes to developing new musical instruments, as the effort put into tightening the muscles may be accessed more directly than when only measuring motion.

References

• E.R. Miranda, and M.M. Wanderley. New digital musical instruments: control and interaction beyond the keyboard. Middleton, WI: AR Editions, Inc., 2006.
• K. Nymoen, M.R. Haugen, A.R. Jensenius (2015). “MuMYO – Evaluating and Exploring the MYO Armband for Musical Interaction” In Proceedings of the International Conference on New Interfaces for Musical Expression, Louisiana State University, Baton Rouge, LA, 2015.