Skip main navigation

Creating music with motion and muscles

In this video, you’ll see Mari play with the prototype instrument MuMYO.
4.8
We’ve been looking at many different types of technologies now, for capturing human motion. Or even muscles, and muscle sensing, and other types of data. And these are used for the analysis, and for more scientific purposes. But in our group here at the University of Oslo, and also around the world, many people are using the same type of technologies to create music. And one word that describes this and that’s often used, is that NIME, which stands for New Interfaces for Musical Expression. And here one of the ideas is that we use technology to create sounds and music, but this also requires that we work with the mappings from them. Different types of motion to sound.
53.8
And Christian, you have a sensor here now, can you just tell a little bit about how you have been approaching this, when you’ve been trying to make this into a musical instrument. Sure, I’ve been looking at this arm band called the MYO, which has eight sensors that captures the muscle tension of the lower arm, in addition to a gyroscope and accelerometer. Which allows you to extract the orientation of the device. And I’ve been using this for a musical instrument to control sound. That’s already there, sort of process with sound effects. And also to trigger sounds like piano sounds, or drum sounds. [MYO DEVICE PLAYING]
203.6
And it’s quite effective in how it works, with low latency, and very high– something right, on the data. Yeah, and how is this connected to the– kind of studies you have doing on sounds and actions? How is this related to working– I mean, the other way around really? Now what is interesting is when you know how people relate movement to sound, you can use that knowledge to map the data from this device into a synthesiser. For instance, if you were to control some sort of tone, going up and down, maybe you would want to use this armband, going up and down to control that pitch.
246.4
So if you know that people relate pitch to vertical movement, up and down, maybe you want to use that information and let this movement be mapped to pitch on your synthesiser. And one thing that many people often are a little bit confused about, is this– when we were talking about technologies, that you have either you can make an instrument or you can make more of a musical device. What is your– what is your take on this? I guess that’s a big debate, but I don’t see why we should confine musical instruments to very narrow things. I would say, if you use this to control the DJ mix, it’s still a musical instrument.
288.1
It’s not some sort of musical device, a musical instrument can be many things. So one of the things that we see then, when we work with music technology for creating new music, is that it is possible then to use knowledge from our scientific studies. Also for Creative Studies, and what we also learned is that it’s possible to use really any type of technology to create new musical instruments.

So far, we have looked at how we can use technologies for analysing muscle activity and human motion.

At the University of Oslo, we also research how it is possible to create music using various motion capture devices. In this video, you’ll see Mari play with the prototype instrument MuMYO. This instrument was developed by our former RITMO colleague Kristian Nymoen and is an example of a new interface for musical expression. The instrument is based on capturing both motion and muscle data from a Myo armband. Unfortunately, this armband is not commercially available any longer, but other products will hopefully soon appear on the market.

This article is from the free online

Motion Capture: The Art of Studying Human Activity

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education