Motion capture of folk musicians and dancers

We have been looking at how we can do motion capture of musicians and also of perceivers. But some people are actually doing it even more complicated than that, and trying to combine doing motion capture of many people at the same time. And Mari, you’re one of them. Tell me a little bit about your studies. OK. Yeah, so I have done recordings of both musicians and dancers in folk music genres. I recorded them simultaneously. Wow. But that must be quite complex, because you have then people moving around, and sitting still. Well, how did it go?

Yeah, so we did a lot of planning, and to plan how to place the cameras and stuff like that, because as you said, the dancers are really moving around. And in particular, one of the styles that I’ve been recording is the Norwegian folk music style called Telespringar. There you have it looking at couples dancing together. That must be very difficult. Yeah. And also, it’s a challenge because they dance very close to one another. So, you really have to really think about where to place the markers to prevent occlusion, but also how to place the cameras. But then the musicians were still sitting quite still.

But you managed then to do them both, one musician sitting still, and dancers moving together, and spinning around. And also in a fairly large space. How big of spaces were you working in? Yeah. It wasn’t that big, actually. But it was big enough for the dancers to move about. And then in terms of putting on the markers, I guess if you have markers on the front for example, they would be covered up. Or how did you handle that? No, that actually worked quite well. Of course, I had some occlusion when there’s so much interaction between the dancers, but it actually worked quite well.

And in addition to the markers that are put on the joints that I was actually interested in, I also had quite a few control markers so I could differentiate between the two dancers. And you’ve been doing studies in the lab, but you’ve also been doing some studies outside of the lab. And what do you feel are the most important differences between doing something inside of the lab and outside? Of course, in the lab it’s more controlled and you have control over the placement of the camera. So, when you go somewhere, especially if you don’t necessarily know them, the room, and how to place the cameras, that could be challenging.

But it’s also nice to be able to go where the people are, that they don’t have to come into the lab. That you can actually just visit them. And maybe that also makes them more comfortable. And I remember once you came back from a field trip like this, then you also talked about the floor, and some of the challenges of the floor. What was that about? Yeah. Because some floors are actually like bouncing, and I had tripods. So I had to make sure that the tripods were far away from the dancers, because if they were close then the tripods would start to sway. And that’s not good. Because then actually the cameras that started moving, not the people only.

But then I guess it would be better to hang the cameras, but it may not have been possible. That was not possible in that case, but I think that’s the best if that’s possible. And if you can show me something on the screen here. Let’s have a look. So there we have some of the dancers. Here is from the recording that we were talking about, with the one fiddler and the dance couple. And here in this representation I actually removed the control markers, because it’s for identifying the markers but I don’t need them. So control marker, what do you mean by that? It’s, for example, in this dance they hold them, they have their arms like this all the time.

And I need to know if this is the female dancer, or the male dancer. So, then I make sure that the distance between the joint markers and the control markers, that is between these markers, that they are different in the two arms. You can use that afterwards then to identify which one is which. But you don’t use it then in the analysis. No. It’s only for making that model. Yes. Are there any other tricks that you have learned from doing these type of studies? Well, it’s the camera placement. As I said the dancers are moving quite about, but the fiddler is sitting still.

So, I had some cameras that was only for the fiddler, and then I had more cameras for the dancers. So, that is also a trick to kind of make a separate box for the fiddler. And what about the sound recording during a setup like this? How did you handle that? Yeah, so I did a separate sound recording. That can be a bit challenging, and it’s more to do some synchronisation. And how did you do the synchronisation in there? I actually used a clapboard with markers on. Yeah. Because the nice thing with then having the markers on is that you can see then the clap in the motion capture as well, and get that spike. Exactly.

That’s a smart way of doing it. Cool, so then you see there are many different things to learn from this kind of use cases and also take a look at some of Mari’s papers as well that I’ll link up here in the text below.