An application kit for networked dance performance

Abstract

A component-based distributed system, SoDA enables networked dance practice and computer-assisted choreography for virtual interaction between human and non-human performers (including third-party software such as Wekinator, INScore, MaxMSP, etc). Using SoDA, participants in a networked rehearsal receive aural and visual cues in the form of sounds, verbal cues, and visual displays that correlate to specific actions or movements. Cues regarding relative positioning and movement quality (direction, weight, speed and flow as per Laban movement theory) as well as pre-composed sequences or routines are communicated via webcam, microphone input, or shared video playback. Through such cues, the system fosters co-creation in a networked performance space as developed by the ongoing artistic research project “[Social D[ist]ancing: Development of a networked artistic practice out of confinement](SoDA - An application kit for networked dance performance · NIME 2021)” at the University of Music and Performing Arts Vienna, which is working to leverage ML and integrate computer capabilities within networked music and dance practices.

The SoDA kit consists of a set of applications that communicate through a custom message broker application (AMEX) in a remote server. SoDA works parallel to video conferencing applications (Jitsy, Zoom, etc) to annotate, document and enhance the performer’s experience. A SoDA Node exchanges and translates messages between SoDA components and third-party applications such as INScore (GRAME) or Wekinator (Rebecca Fiebrink). An optional SoDA Point of View (PoV) component monitors temporal communication parameters –latency, clock offset– relatively to a designated ‘central observer’, adding artificial delays where necessary to fine tune the perceived synchrony of actions at that particular node.

SoDA Nodes exchange messages via TCP protocol across the network, but communication between the Nodes and other SoDA components as well as third-party software —typically running alongside each other in the same machine— uses Open Sound Control (OSC) over UDP. This network design provides flexibility to the SoDA mesh, while having the advantage of being extremely scalable – the specifically conceived AMEX application running in the server allows for a large number of SoDA nodes to interact seamlessly in real time.

Additional SoDA components include SoD4L, a MaxForLive device that serves as a bridge between Ableton Live and the SoDA mesh, and SoDATA, a standalone application used for sequence recall and choreography annotation in different human-readable formats.

Using the SoDA kit enables dance practitioners to harness the immediacy and scalability of computer systems for artistic experimentation, and allows them to tap into the potential of AI in a networked, distributed environment.

Credits

Adrián Artacho is currently a PhD candidate at the University of Music and Performing Arts of Vienna, researching the use of technology to enhance performance capabilities. He is also an active performer of live electronics either solo or in different configurations. As a composer, his interest in cross-media projects and dance in particular has led him to regularly collaborate with choreographers and to become founder of the dance companies Tanz.Labor.Labyrinth and SyncLab Tanzkollektiv. (adrian@neuesatelier.org)

Oscar Medina Duarte has many years of experience working on technological projects in the safety-critical domain (aviation, railway, …). His great interest in the performing arts brought him to an active role at NeuesAtelier where he is active as a technologist and software developer. As a technologist, his main interest is on the effects of the pervasive tecnification of society and the role of performing arts in a post-pandemic world. (oscar@neuesatelier.org)

Links

Here is a link to a video showcasing the use of the SoDA system by a team of artists-researchers in the context of the research project “Social D[ist]ancing: Development of a networked artistic practice out of confinement” at the University of Music and Performing Arts Vienna.

Here is the project public repository, including instructions for the installation and use of the Soda kit:

https://bitbucket.org/AdrianArtacho/soda-node/

Acknowledgements

The authors would like to thank Hanne Pilgrim, Mariama Diagne, Benedikt Berner, Katharina Püschel, Dalma Sarnyai, Magdalena Eidenhammer, Maximilian Resch and Maria Solberger for their participation in the evaluation and testing of the applications. It was their invaluable feedback that ultimately informed the design of the software. This work is supported by the Artistic Research Center and the Research Support department of the University of Music and Performing Arts Vienna.

Modules

Name	Description
sod4l	SoDA for Live (Max For Live device)
soda-node	Basic SoDA network unit
soda-pov	SoDA - point of view (latency management)
sodapink
sodata
soundcues	Aural cues
videogrid	Video collage tool
zoomkeeper	Zoom Videoconference tool