Creating music with motion and muscles

We’ve been looking at many different types of technologies now, for capturing human motion. Or even muscles, and muscle sensing, and other types of data. And these are used for the analysis, and for more scientific purposes. But in our group here at the University of Oslo, and also around the world, many people are using the same type of technologies to create music. And one word that describes this and that’s often used, is that NIME, which stands for New Interfaces for Musical Expression. And here one of the ideas is that we use technology to create sounds and music, but this also requires that we work with the mappings from them. Different types of motion to sound.

And Christian, you have a sensor here now, can you just tell a little bit about how you have been approaching this, when you’ve been trying to make this into a musical instrument. Sure, I’ve been looking at this arm band called the MYO, which has eight sensors that captures the muscle tension of the lower arm, in addition to a gyroscope and accelerometer. Which allows you to extract the orientation of the device. And I’ve been using this for a musical instrument to control sound. That’s already there, sort of process with sound effects. And also to trigger sounds like piano sounds, or drum sounds. [MYO DEVICE PLAYING]

And it’s quite effective in how it works, with low latency, and very high– something right, on the data. Yeah, and how is this connected to the– kind of studies you have doing on sounds and actions? How is this related to working– I mean, the other way around really? Now what is interesting is when you know how people relate movement to sound, you can use that knowledge to map the data from this device into a synthesiser. For instance, if you were to control some sort of tone, going up and down, maybe you would want to use this armband, going up and down to control that pitch.

So if you know that people relate pitch to vertical movement, up and down, maybe you want to use that information and let this movement be mapped to pitch on your synthesiser. And one thing that many people often are a little bit confused about, is this– when we were talking about technologies, that you have either you can make an instrument or you can make more of a musical device. What is your– what is your take on this? I guess that’s a big debate, but I don’t see why we should confine musical instruments to very narrow things. I would say, if you use this to control the DJ mix, it’s still a musical instrument.

It’s not some sort of musical device, a musical instrument can be many things. So one of the things that we see then, when we work with music technology for creating new music, is that it is possible then to use knowledge from our scientific studies. Also for Creative Studies, and what we also learned is that it’s possible to use really any type of technology to create new musical instruments.