Studying music perception in the motion capture laboratory

So far, we have been looking at some of the cognitive foundations and the more theoretical parts of how we perceive sounds and actions and music and motion. But as music researchers, we actually study these things also in a laboratory context. So Kristian, I know that you have been working in the lab and running experiments on sounds and actions. Could you tell a little bit about this? Sure. I’ve been studying how people move to short sounds, presenting them with simple sounds– with, for instance, changing pitch and spectral centroid and dynamic envelope, and then asking them to move to the sound. So people have moved in various ways, and we’ve found quite a few things that are consistent, even between people.

And what type of things? The most obvious, I would say, is pitch and verticality. Almost everybody would trace an ascending pitch with going up, and the same vice versa, going down. Spectral centroid also– if you have a sound that goes like whoossst, people would trace that also upwards. So it’s in the frequency domain and the verticality is a strong connection. And things like the dynamic envelope, which would typically identify sounds that are impulsive– say something like [CLAP], which has a strong ascending curve first and then falling slowly, as opposed to something that’s a continuous sound with continuous energy put into the sound, where people would trace the sound with a different type of movement.

For instance, these impulsive sounds are typically traced with an impulsive action, and the continuous sounds with some continuous action like pushing the sound through space, for instance. And one of the very interesting things here is if you have two opposing ideas in the same sound– for instance, ascending pitch and then descending spectral envelope, people will get confused, trying– often starting following one of these curves, and then in the middle not being able to decide whether to continue upwards or to start going downwards, for instance. And is this the same for all your subjects? I mean also independent of the musical background or training they have? Mostly, this is quite consistent among both musical experts and novices, yes. Interesting.

This is for simple sounds that are maybe not really music itself. Do you have any ideas on the musical aspect here? If you present music to people, how does this make people move? Yeah, I’ve been doing some research where we’ve been looking at the same type of connection from music to more large-scale motion. And playing, for example, musical excerpts to people and asking them to move freely or to play an air instrument, like air guitar or air drums or a piano or even conduct, just to kind of follow contours in the music. And it’s very interesting to see how, through the body motion, we can understand more about the cognitive processes of people.

For example, when people are asked to play air piano, we would assume that they know kind of the size of a piano and also how it’s laid out and both the attacks and onsets of tones, but also, for example, the frequency range of this. And the interesting thing is that even people without any musical training at all managed to do this very well really. So in a way, people are more musical than they believe, I think. But based on this, then, there are some obvious connections then, both between sounds and actions and also music and motion.

And this may actually be a very interesting thing to try out at home, to take a few musical excerpts and try to imagine how you would move to these excerpts. And then try to actually move to them. And also ask a friend to move and look at how is the movement in relation to various types of sonic and musical features. Try yourself and see.