New interfaces for musical expression: NIME

As we learned in the previous video, the term NIME refers to “new interfaces for musical expression”. The term is derived from the conference and community with the same name (NIME).

Common among practitioners in the NIME community is that of designing, building, performing and evaluating new musical instruments and other types of musical interfaces.

Musical Instruments

It is difficult to come up with a clear definition of a musical instrument, particularly if one considers all sorts of musical instruments. One common denominator, however, is that an instrument functions as a mediator between action and sound.

Illustration: Three boxes where the action box is pointing to the instrument box, and the instrument box is pointing to the sound box

As such, any pair of objects and actions that are used in music could be considered a musical instrument.

Digital Musical Instruments

Many new interfaces for musical expression are digital musical instruments (DMIs), involving the use of digital technology in the mediation between actions and sound. This can be seen as a subset of electronic musical instruments, which embeds also analogue, electronic instruments such as the famous MOOG synthesizers.

A model capturing three essential parts of the term digital musical instrument is shown in the figure below. The controller is the physical device that the user interacts with. A controller in itself will not produce any sound, however, so the sound engine is where sound generation takes place. This may be a physical hardware unit or a piece of software. The mapping is a set of rules for how the sound engine should respond when the user interacts with the controller.

controller-mapping-soundengine

There exist numerous digital musical instruments. Many commercial digital musical instruments are based on a piano-like interface, with a controller, mapping, and sound engine built into the same physical device. Although piano-based instruments are hardly considered new interfaces for musical expression any longer, they are nevertheless digital musical instruments.

Image of a Nord Stage-piano Nord Stage (Photo by Ben Garney, CC BY 2.0 license)

The Hands by Michel Waisvisz is an example of a “classic” digital musical instrument not inspired by traditional musical instruments. The controller uses various sensors to capture hand, arm and finger movements. The sound engine is also separated from the controller, which makes it possible to create many different types of mappings between movement and sound. See The Hands in action in this youtube video.

The Hands The Hands (Photo by Luka Ivanovic, CC BY-SA 2.0 license)

NIME as a research field

One of the main areas of interest to NIME researchers is the development of new modes of interaction:

  • How can musical sound be controlled in intuitive and interesting manners?
  • How does the way we control musical sound shape our perception of the sound?
  • How can new technologies be used to enhance our musical experiences?

At the University of Oslo we have been interested in exploring different ways of creating NIMEs based on our research on embodied music cognition. Here we have use the results from cognitive experiments when designing mappings between movement and sound. Similarly, we have explored how NIMEs may be used in experiments. One such example is that of the “instrument” MuMYO.

MuMYO

The MuMYO instrument used in the previous video uses the commercially available MYO armband as the sensor. The MYO consists of eight electromyography (EMG) sensors which measure muscle tension, in addition to an inertial measurement unit measuring motion and rotation.

Using a machine learning algorithm to extract useful features from the muscle sensing data, the device is able to detect hand postures such as waving in or out, making a fist, or spreading the fingers. The possibility to capture muscle tension data is interesting when it comes to developing new musical instruments, as the effort put into tightening the muscles may be accessed more directly than when only measuring motion.

References

Share this article:

This article is from the free online course:

Music Moves: Why Does Music Make You Move?

University of Oslo

Get a taste of this course

Find out what this course is like by previewing some of the course steps before you join: