46: Hearing the Middle Ages: Playing with and Contextualising Acoustical Heritage and Historical Soundscapes Research

46: Hearing the Middle Ages: Playing with and Contextualising Acoustical Heritage and Historical Soundscapes Research

Mariana Lopez, @Mariana_J_Lopez, University of York I am a Senior Lecturer in Sound Production and Post Production at the Department of Theatre, Film, Television and Interactive Media at the University of York. This short article explores the challenges and opportunities of using digital games to present sound heritage research.

Although I will be drawing from my work on sound in the ‘York Mystery Plays’ medieval drama cycle, and the creation of the interactive web experience 'The Soundscapes of the York Mystery Plays', which can be visited here, the findings and lessons learnt are widely applicable. Acoustical heritage and historical soundscapes work are not new but the challenges of presenting findings in a transparent and engaging manner to non-expert audiences has received limited attention. But what do I mean by 'transparent'? Acknowledging the unknowns and limitations in sound heritage work. Unknowns are linked to (1) the way history often relies on fragmented sources that may make several interpretations possible. (2) The fact that acoustical work is limited by the acoustical theories and techniques utilised, for example, the difficulty in knowing the acoustic properties of historical surface materials, the determination of sound source and listener positions and the geometric acoustic theory used by computer models (Vorländer 2013; Álvarez-Morales, López and Álvarez-Corbacho 2020). Moreover, and very importantly, acoustical work is limited by the assumptions made by researchers, which include which spaces are deemed ‘worthy’ of study. (3) The importance of understanding the cultural context of listening experiences: we cannot ‘hear’ as our ancestors did, the senses are time and space specific (Smith 2007). Understanding context is key, but acoustical heritage work tends to leave this aside. Whereas linear sound recreations, such as those at museums, often present one version of the past, the non-linear nature of digital games allows for the exploration of different possible sonic pasts. By presenting different options, we welcome uncertainty and reflection (Lopez 2020). 'The Soundscapes of the York Mystery Plays' was designed to explore how to present the uncertainty and multiplicity of sonic renditions by inviting users to change the performance setting of the plays based on historical findings and possible scenarios (see Figure 46.1).

Figure 46.1: The Soundscapes of the York Mystery Plays interface. Visual design by Oswin Wan.

75

Scenarios include some of the most commonly discussed unknowns regarding performances of the York Mystery Plays: how the wagons utilised were actually used for performances; how they were positioned in the streets of York; and where performers and audiences were located (see Figure 46.2). The available scenarios also allow users to explore the impact of audiences on the acoustics of the performance spaces, by including scenarios with and without audiences. Furthermore, different listening positions can be selected, which are then visually indicated in red. As the user selects the performance settings, different acoustical renditions (auralisations) are played. They are the result of thorough studies based on acoustic measurements in York and computer models, combined with recordings of extracts from the plays.

Figure 46.2: Different possible performance settings available to users in The Soundscapes of the York Mystery Plays interface. Visual design by Oswin Wan. In addition to changing the acoustical setting, the user can also mix in sounds that are likely to have been present during the performances in the medieval streets of York. Thus, creating richer soundscapes and allowing for further exploration. Said sounds include wagon wheels, audiencerelated sounds, animals, bells and sound effects connected to the weather. The user can explore multiple aural experiences, all plausible, all the product of in-depth research, some differences very noticeable and others subtler. Very importantly, through this process the user can engage playfully with the sounds of the Mystery Plays. The interface allows for the exploration of historical acoustics and soundscapes while also placing the listening experience in the users’ hands by allowing them to create customised aural experiences. The interface, therefore, refuses to settle for one aural version of the York Mystery Plays, embracing multiplicity. A challenge though was, and is, how to contextualise the different aural experiences presented without overcrowding the interface. This was partly addressed by following Jenkins’ concept of ‘Environmental Storytelling’ (2004), seeking to use visual design to aid the narrative. Work included an avatar communicating through text on an illuminated manuscript, who introduces the plays to the user by reading ‘Dear friend, The Lord Mayor has issued the billets, we are bringing forth the wagons for the York Mystery Plays, will you join us in Stonegate?’ A manicule with the text ‘Take me there!’ was also included at the bottom of the illuminated manuscript. By clicking on it, the user is then led to a map of medieval York indicating the site of the performances (the site used for the simulations, Stonegate). Once within the main page of the interface, the background images change to indicate the performance setting chosen by the user.

76

‘The Soundscapes of the York Mystery Plays’ borrows elements of its visual design and interaction mechanics from digital RPG and strategy games, but it is not a game, it is a historical simulation, as there are no goals/challenges that need solving and no rewards (Salen and Zimmerman 2003). Feedback collected indicated users enjoyed combining sounds and exploring the acoustics; they also praised the high quality of the audio and enjoyed the atmospheric nature of the experience, as well as the visual design. However, more was felt could be done to help users fully grasp the meaning of sounds and settings, and the research behind them. Further contextualisation would increase awareness of the historical context, relevance of the sounds, and rationale behind the choices made. Future work will explore how further contextualisation could be provided without cluttering the interface and without relying on overly long text extracts and without requiring the user to delve into obscure menus. An interesting strategy is used in the 1990s game 'Cosmology of Kyoto' (Softedge 1993), featuring a Reference tab available throughout, and intrinsic to the experience. As the user progresses, the information available through the Reference tab changes.

‘The Soundscapes of the York Mystery Plays’ is the first online interface to engage playfully with sound heritage while also articulating a critical view of the static approach of acoustical heritage experiences, which risk presenting a rigid version of the sonic past. Techniques and reflections will hopefully invite researchers on sound heritage to consider playful and transparent approaches to presenting research outcomes. At the same time, I hope researchers in the field of historical games are encouraged to widen their focus beyond visual elements.

References:

Álvarez-Morales, Lidia, López, Mariana and Álvarez-Corbacho, Ángel “The acoustic environment of York Minster’s Chapter House.” Acoustics 2:1 (2020). Jenkins, Henry “Game Design as Narrative Architecture.” In First Person: New Media as Story, Performance, Game, edited by Noah Wardrip-Fruin and Pat Harrigan. Cambridge: MIT Press, 2004.

Lopez, Mariana “Heritage Soundscapes: contexts and ethics of curatorial expression.” In The St. Thomas Way and the Medieval March of Wales: Exploring place, heritage, pilgrimage, edited by Catherine A.M. Clarke, 103-120. Leeds: Arc Humanities Press, 2020. Salen, Katie and Zimmerman, Eric. Rules of Play: Game Design Fundamentals. MIT Press, 2003. Smith, Mark. Sensory History. Oxford: Bloomsbury, 2007. Softedge, Cosmology of Kyoto. 1993.

Vorländer, Michael “Computer simulations in room acoustics: Concepts and uncertainties” Journal of Acoustical Society of America 133 (2013).

77