Laura’s research on musicians.

Do musicians move their bodies differently when playing alone than when playing together with other people? Research suggests that they do move differently.

The presence of a co-performer can prompt musicians to move more, move more predictably, and use directed communicative visual signals, and periods of coordinated motion (where performer’s movements are mutually influential) can emerge.

My research looks at how the social and attention demands of a musical performance affect how musicians move when they play. I also look at how these differences in body motion affect musicians’ internal experiences during the performance and their relationships with their co-performers and the music. In parallel, I consider how audiences’ perceptions of performance quality and expressivity relate to the musicians’ interactive body motion.

Set-up

Sometimes I choose to run experiments with music ensembles outside of our motion capture lab. Usually, this is because I want to maximize ecological validity, which is difficult to achieve in a controlled lab space. For example, I might choose to have the ensembles play in a concert space with a stage; or, I may bring a motion capture setup to a location with instruments that are difficult to move (such as grand pianos).

We have a portable 8-camera OptiTrack system at RITMO that we can use to capture body motion in real-world settings. The cameras can be placed on tripods or fixed to scaffolding. In a recent experiment with duo pianists, four cameras were attached to scaffolding on the ceiling of a musicology lecture hall. Two of the four cameras were directed to face each pianist. Four additional cameras were placed on tripods, two behind each pianist. This camera setup allowed us to capture the pianists’ head, back, arm, and hand motion. In the second picture below, the position of the four cameras is highlighted in green.

In another recent capture session (a “trial run” experiment that we also live-streamed), we recorded a string quartet as they performed on a small stage. All eight cameras were placed on scaffolding around the stage, pointing from various angles towards the musicians.

We aimed to capture the musicians’ head, back, and arm motion and the motion of their bows. In the end, we were only partially successful. The musicians’ head and bow motion was well-captured; however, their arms and backs were captured less successfully. Poor capture of arm motion was mostly attributable to instruments covering up the markers. Poor capture of back motion (especially for the 2nd violinist and violist) resulted from our non-optimal camera placement.

If we were to re-run the capture, it would be better to place a couple of cameras on tripods behind the ensemble to capture their back markers. We would also need to re-evaluate where the arm markers should be placed so that they are in sight of the cameras.

For both of these experiment settings, I used a frame rate of 120 Hz. This is the highest frame rate that our OptiTrack Flex cameras are capable of.

Analysis

After post-processing—including marker labelling, gap filling, and smoothing, my first step is to calculate velocity and acceleration (the first and second derivatives of position) for each marker in 3D space. This is an important step because I am primarily interested in how musicians’ motion changes over time rather than in the specific trajectories that it takes.

Next, I usually carry out some analyses on the quantity of motion that the musicians’ produced while playing under different conditions. Quantity of motion can be calculated as the sum of distances that a marker moved in 3D space per unit of time (e.g., per second). Musicians move more when they want to play with exaggerated expression than when they want to constrain their expressivity. Their quantity of motion also depends on how well they know the music they are playing.

I also look at how similar the musicians were in the timing of their expressive, communicative movements. For this analysis, I focus on parts of the body that are used more for expression than sound production (e.g., the head, upper back, or elbows). This analysis might involve calculating cross-correlations between the acceleration curves of two musicians’ head or backmarkers. Coordination between musicians in body motion has been shown to relate to leader and follower roles.

It may also be useful for me to look at some of the motion involved in sound production. The fluency of sound-producing motion can be measured as the jerk, the third derivative of position. The jerk describes the rate of acceleration change and can be contrasted to smoothness. So motion that is high in smoothness is low in jerk. Fluency in musicians’ motion can vary depending on their skill and familiarity with the music and may also vary according to their performance conditions. Their motion may be more jerky when they struggle to coordinate with an ensemble partner than when coordination is more successful.

Conclusions

My research on piano duo and string quartet playing has revealed some interesting effects. My analyses show that musicians’ body motion changes depending on their focus of attention, and they move more when attending to each other than when focusing on score-reading. In exploring the relationship between body motion and musical structure, I have found that musicians move more and more variably during expressively intense parts of a piece than during less intense parts. Together, findings like these help to clarify how musicians relate to the music and each other through their bodies.

References

• Bishop, L., Gonzalez Sanchez, V., Laeng, B., Jensenius, A. R., & Høffding, S. (in review). Move like everyone is watching: Social context affects head motion and gaze in string quartet performance.