The ways that musicians synchronise their performances is delicate matter of gestures, gaze, body movement and sequencing;. When that is conducted over even the best of broadband connections, complications can arise. I’m delighted that Sam Duffy, who is both a musician and well versed in interaction analysis, can tell us something about the interrelationship between the two in a time when most of us are still coming to terms with online interaction.

COVID-19 has had a devastating impact on the Performing Arts. Professional performers and composers have had their future income stream wiped out overnight (and for the foreseeable future). Students are struggling to finish their graded year-end recitals or oversee recordings of their work as they were originally imagined. Amateur and community choirs and orchestras can no longer get together at a time when the social aspect would be a valuable support. Members of professional ensembles cannot maintain their repertoire, or work on new material together to perform once restrictions are lifted. 

In response, many music education organisations have used online tuition to maintain the instrumental teaching relationships that are so crucial to developing musicians. This has meant a steep learning curve for some, who may not have had the opportunity to understand the way that conversation and musical interaction changes when it is carried out online through video technology (Duffy et al., 2012). Ensembles rehearsing and playing together in real time face an additional challenge in the form of latency and delay (Duffy, 2020). However, technical issues are not the only consideration for remote rehearsing and performing.   

Musical Collaboration

Self-directed ensembles (i.e. ones with no conductor), such as quartets, often rehearse in a configuration that enables them to have maximum visibility of each other. They can see each other’s non-verbal cues, as well as directional cues from the nominated leader. In performance, this might be adjusted for venue layout and visual access for the audience, but visual access will still be maximised for the quartet members. 

In a saxophone quartet, for example, the soprano saxophone generally leads. This means they coordinate pauses, entrances and endings through playing techniques, gaze and gesture with their body and instrument. There is already a body of work on how players lead an ensemble (for example, King, 2006; Seddon & Biasutti, 2009) so I will not go into depth on this here. What is important is that this non-verbal interaction is significantly disrupted when a rehearsal takes place online, using conventional video technology (Duffy, 2015, Chapter 14; Duffy & Healey, 2012; Duffy et al., 2012). For example, how does the soprano gesture to one player specifically when everyone else is arranged on one flat screen? From the perspective of the other players, how does everyone else know which player the soprano was directing the gesturing towards? The order of players in the viewing gallery may differ for each member, even if they select the same viewing preference – there is no common ground in relative positioning or proximity.

The image below is from an edited online production rather than a real time interaction, but it illustrates the problem. It also shows the difficulty of splitting gaze between the music and the screen at the same time – something achieved through peripheral monitoring in a same-room interaction (Duffy & Healey, 2017).

Other challenges include the effect of viewing on a desktop computer or laptop screen. The composite image of the four players on screen is small, the screen size reduces the image of the participant to significantly less than life-size, which reduces the efficiency of non-verbal communication such as gesture and gaze (Cooperstock, 2005). It is also far more difficult to peripherally monitor each other for non-verbal cues on a small, flat screen.

Implications

There are many projects looking at minimising latency, but even if this can be reduced to a tolerable delay for performing ensembles (which is a big ‘if’), self-directed ensembles face other challenges when they cannot be in the same room together – the ability to coordinate their performance through non-verbal interaction and use of space.

This is, of course, true for ordinary conversationalists struggling with pauses, overlap and gaps; but it is difficult particularly difficult for those who are concentrating on making music. Coordination when there is lag and latency in the system isn’t an insurmountable problem, but it does need to be taken into account by players experimenting with ways to maintain vital musical relationships whilst social distancing restricts their ability to play together in the same room. 

Sam Duffy is Centre Manager for the RNCM Centre for Practice & Research in Music & Science, PRiSM. Read more about her work on remote music interaction in her blog here: https://musicinteractiononline.wordpress.com. This article is an extract from the post Real-time online music performance – fact or fiction?  

References

Cooperstock, J. (2005). Interacting in shared reality. HCI International 2005, 1–7.

Duffy, S. (2015). Shaping Musical Performance Through Conversation (Queen Mary University of London).

Duffy, S. (2020). Real-time online music performance – fact or fiction? Retrieved August 12, 2020, from https://musicinteractiononline.wordpress.com/2020/06/18/real-time-online-music-performance-fact-or-fiction/

Duffy, S., & Healey, P. G. T. (2012). Spatial Co-ordination in Music Tuition. In N. Miyake, D. Peebles, & R. P. Cooper (Eds.), Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 1512–1517).

Duffy, S., & Healey, P. G. T. (2017). A New Medium for Remote Music Tuition. Journal of Music, Technology and Education, 10(1), 5–29.

Duffy, S., Williams, D., Stevens, T., Kegel, I., Jansen, J., Cesar, P., & Healey, P. G. T. (2012). Remote Music Tuition. Proceedings of 9th Sound and Music Computing Conference (SMC ’12), 333–338.

King, E. C. (2006). The roles of student musicians in quartet rehearsals. Psychology of Music, 34(2), 262–282.

Seddon, F., & Biasutti, M. (2009). A comparison of modes of communication between members of a string quartet and a jazz sextet. Psychology of Music, 37(4), 395–415.