Architects of Realities (Arts x Tech Lab 2021 Compendium) by Joanne Ho

ARCHITECTS OF REALITIES CHERILYN WOO

KHAIRUL KAMSANI

NELSON CHIA

THONG PEI QIN

JOANNE HO TIMOTHI LIM

ARCHITECTS OF REALITIES A participant of NAC’s Art x Tech Lab 2021

Collaborators (In alphabetical order) Cherilyn Woo Joanne Ho Khairul Kamsani Nelson Chia Thong Pei Qin Timothi Lim

With gratitude to Nine Years Theatre The Doodle People Acting students at LASALLE College of Arts NUS CUTE

Supported by

This compendium was compiled in January of 2022.

CONTENT 1 Architects of Realities Initial Intents Collaborators

33 Glossary of terms

49 Sessions Timeline Session 1 - 3 Reflections Session 4 - 8

151 Final Reflections

177 Looking forward

iii

ARCHITECTS OF REALITIES

WHAT IS ARCHITECTS OF REALITIES? In “The Empty Space”, Peter Brook noted: “I can take any empty space and call it a bare stage. A man walks across this empty space, whilst someone else is watching him, and this is all that is needed for an act of theatre to be engaged.” By definition, and referring back to Brook’s statement on the basic spatial elements of a performance space, theatron is a space for seeing. With our project proposal, “Architects of Realities”, we wish to redefine the way theatre is typically constructed for viewing and being viewed, while keeping the rest of the ingredients which constitute theatre performance constant. By integrating digital technology, the project ultimately seeks to reimagine the shared space between performer and spectator and examine its impact on the act of viewing and being viewed. “Architects of Realities” is a pilot program for an intermedial performance] dramaturgy lab exploring how directors research with actors in live performance, using immersive digital technology in three levels of overlaying extensions of spaces: (1) mixed reality (projection mapping); (2) augmented reality (visuals on live feed of events captured by a digital screen); and (3) virtual reality. This program experiments with the future of art/performance space by investigating the collision of bodies and technology through the existing methodologies and practices of actor-training. The experiments involve overlapping multiple virtual spaces on the physical space and the body to question the duality between live/real (phenomenological) and fictional (semiotic). By observing the cycle of influence and response between the actor’s body and technology in a series of eight (8) Practice-as-Research sessions, we hope to discover elements that culminate in a few alternate methods of immersing the actors and audience into an experience of multiple realities, and a feedback loop connecting them with the technology employed. Is there a way to bring performance to unconventional theatre spaces, real or virtual?

INITIAL INTENTS Each participant in this research process delves into their own individual intents, motivations and reflections later on in our compendium, this section aims to consolidate our intent as a collective, the Architects of Realities. Whether our background may be in the performing or visual arts, our respective practices deal with appearance, representation, presence, creation and perception of an external world. How does a painting or computer-generated image reflect the world around the artwork itself? Or even with other pieces of art in its contemporary, past, or future? How does a physical gesture or monologue performed by an actor comment on our perceptions of our own lives as audiences during, before or after the fact? Even before embarking on this Arts X Tech initiative by the National Arts Council Singapore, each member of our team has already encountered the task of navigating, arranging, framing and presenting a ‘version’ of reality for audiences as a piece of art. Therefore, Architects of Realities is not exactly a new venture but instead an extension of our practices into an even larger playing field of multiple practices, mediums, contexts, and questions. As far as we are aware, our research is the first of its kind in Singapore; theatre companies create intermedial theatre work using emerging technology, Virtual Reality technology companies create experiences using hardware and software at their disposal and development and Projection Mapping artists/collectives/companies create installations for specific galleries, events or personal experiments but in our studio, our goals were never to create, but to ‘simply’ allow our range of tools to coexist and interact with each other in the same time and space- to create new times and spaces. Early on in our journey, we agreed that an outcome driven process would have been counterproductive to our experimental research, instead, we opted for an exploratory driven process in which we playfully ‘jammed’ with our tools (theatre making, projection mapping and/or VR) to experience the product that appears. We would then share our personal findings with each other and take notes of which variables may or may not have contributed to the resulting experience. Overall, we

were immersed, curious, overwhelmed, excited and hopeful in each step we took in navigating and mapping the layers of realities that may exist in a single time and space. This compendium of our research is designed to take you on the journey that we undertook as a co-researcher, going under the hood of our research questions, methodologies, and findings. Certainly to experience the experiments yourself would yield more tacit knowledge for you as it did with us, but we will have to look forward to the future in which we can run these exploratory workshops for the public!

Initial Intents

COLLABORATORS

Cherilyn Woo Joanne Ho Khairul Kamsani Nelson Chia Thong Pei Qin Timothi Lim 7

CHERILYN WOO Freelance Theatre Stage Director and Writer Associate Director @ Nine Years Theatre Cherilyn.woo@gmail.com Www.cherilynwoo.com

Cherilyn is a Freelance Theatre Director and Writer based in Singapore. Director Credits: People (Lasalle), The Sound Inside (SRT), Oedipus (NYT), Dearest S (SWF), Note For Note (Arts House), Fantastic Mr Fox (SRT, Associate Director), Faust/Us ( NYT), Il Combattimento di Tancredi e Corinda (YST), La Boheme (Singapore Symphony Orchestra), Nightingale(SRT, Associate Director), The Woman Before, The Bus Stop,Abstraction, The Chairs. From 2014 - 2016, she was Resident at the Singapore Repertory Theatre’s Directing Residency. She is currently Associate with Nine Years Theatre and one half of the collective Issy x Cher. As a practitioner, Cherilyn has varied interests which range from adapting classics to incorporating technology as a character in her work. Most recently, she’s been exploring how robotics can be an integral form of expression in storytelling experiences. One of her recent works (A Collection of Things) with Issy x Cher included a self guided audio tour around Serangoon Gardens that was hosted entirely through a telegram bot. She also created an intimate theatrical experience on a telegram bot for the Singapore Writers Festival with writer Gong Xin, called Dearest S. One of her greatest inspirations in life is rap music. Her most recent work with rap music can be seen in her collaboration with Patch and Punnet titled, Lil’ Shake Mixtape.

A bout Cheril yn Woo

Cheril yn’s Initial Inquiries

CONTEXT As a Stage director, one of my first access points to conceptualising the production is the examination of space. Before considering the various physical venues and logistical limitations, I begin my process on a “blank canvas.” A canvas suggest merely two dimensions, but my canvas expands that into spherical canvas, I step into the world and start creating the world of the play. “Space must become a medium of exchange, not simply defined as an absence of body.”

Skipping forward past the designing process, a world is configured for a narrative to exist in, and now as a director I start playing with dolls in a model box testing out walking routes and composition frames for pivotal scenes in the piece. This is done as a counter measure to ensure that the world is completed. In my practice, I am a firm believer that the idea of space which leads to the conceptualisation of the world in which the narrative takes place is incomplete without the bodies that inhabit the space. Body and space share a co-dependent relationship in which the audience must read both in order to form a narrative. For me, motion of the body in relation to the elements of the space of the world conveys action and emotion vital to the audience to perceive a narrative. Bodies move past themselves, entering a space that is always receiving them to communicate and commingle in the physicality of the world.1 In the process of making directorial decisions about how the bodies should move in the given world that is conceptualised there are some key frameworks in which I consider. One of the frameworks used is Anne Bogart and Tina Landau’s Viewpoints, in which nine “physical” viewpoints are constructed to study the body in space.

Page 15, Heidegger Among the Sculptors, Body, Space, and the Art of Dwelling, Andrew Mitchell

RESEARCH INVESTIGATION

Through analysing and observe this I’m curious to know what the reflections would be for direc-

Technological advancements have gifted the contemporary theatre new spatial possibilities, not

tors. My interest in these reflections are on a couple of levels:

only will the reading of the space be recalibrated, but the process in which one works with bodies in space

INTERNAL DIRECTOR PROCESS:

goes through a process of adaptation and application

•

as well.

How does one juggle dealing with many different spaces to make the world cohesive?

• The key question that started my interest in being a part of this project is: In creating a perfor-

What are the structural changes in the methodology of working through rehearsal?

•

What psychological or emotional variations does

mance where there are mixed realities that consti-

one notice when working with bodies traversing

tute the world of the play, how does the director’s

between physical and virtual spaces?

perception of “bodies” in “space” change and how

•

What’s useful in creating a bridge between phys-

does it shape the process of “performance” mak-

ical space and virtual space, to produce a cohe-

ing?

sive world for the performance? Notice I put bodies, space and performance

in quotes. Even now as I am articulating the question

DIRECTOR TO ACTOR COMMUNICATION: •

and considering the levels of realities this project is holding, bodies and space vary from each level. The

fiction? •

term performance could refer to a live experience, an encounter or even a screening. The question above

What is needed to communicate to the actor before introducing a virtual world?

•

led me to decide that one possible pathway to explore that would be to investigate my own process of

What aids the actors process in maintaining their

What new vocabulary is useful in the actor-director relationship?

•

space and bodies in performance as a director.

If the fiction work of the actor is visualised through a virtual space, how does a director aid in keeping the fiction alive and working the imaginative

I’m interested in finding out how my process

muscle? Is the virtual space a useful aid

in approaching this element of staging gets adapted, applied or translated when working in a virtual real-

DIRECTOR TO TECHNOLOGIST

ity and when there are mixed realities of the perfor-

COLLABORATION:

mance stage. What I am intrigued by is the following:

•

How the bodies in space perform, namely how does the physical actor perform when in a virtual

•

Is there a potential for a system to be formulated to aid in effective communication.

•

How does one juggle the language of the tech-

space, and how does the virtual body perform in

nologist and translate that into actionable direc-

the given space.

tions for the actor?

What is the performativity of the space. Name-

•

What are some key structural elements in con-

ly how does the space perform that informs the

structing the world that is useful to create a con-

body or the directorial choices of where to place

ducive world for the work to take place in?

the body and plan the movement of bodies. Cheril yn’s Initial Inquiries

•

What are the intersections of language used be-

tween directors and technologist that aid in effec-

HYPOTHESIS

tive communication?

I don’t know how to write a hypothesis, but I have a bunch of questions.

(Please note that the above are questions to consider, not all might be answered in the given time-frame, or

•

How can theatre directors and technologists find

more might arise, but this is part of the process and

a common language when in the rehearsal pro-

investigation, to see what questions are raised)

cess? •

How does the hierarchical system of communica-

Bearing in mind the constrains of time and my lack of

tion, if in the case that in a conventional set up the

knowledge in the technological field, through further

director is the veto of all decisions, change when

exploration of the group I recognise that I’ll need to

working with technology and different levels of

select key questions and narrow the field of interest.

realities.

It is my hope that through processing and reflect-

•

In a process where technology becomes perfor-

ing personal experiences, and listening to my team

mative, and to a certain extent a performer itself,

mates experiences, I can extrapolate the understand-

how can a technologist and director direct it in

ings gained and connect it with wider cultural experi-

a similar and intersecting language? And by that

ences, beyond the world of theatre.

extension, direct a live human performer in an intersecting language? •

What are some of the ways of creating cohesiveness between realities, how can the team of creators as a whole identify patterns and links that help the audience experience?

WORKSHOP DESIGN To pick a scene from a well known play that everyone knows, and work to create a short performance from that. 1.

Creating a concept: Coming up with a concept and type of performance where this scene can take place

2. Design: Working together to create worlds where different realms of realities can come together in this scene. Creating the world of this piece visually, using performers and design. 3. Performers: Defining the different laws and limitations of the realities and how that affects performance. Setting their intentions and objectives in the space itself.

JOANNE HO Jo Ho, Artist Sl.joanneho@gmail.com Www.j-o-h-o.com

Jo is an artist working with digital and physical spaces that question the role of presence, agency, and creation of meaning and value in our increasingly automated world, where views of “machine as tool” and “machine as creator” are being challenged. Her recent work explores perceptions of digital media today and attempts to reveal the tangibility beneath digital works in today’s world of rapid content production. Most of Jo’s works are collaborative efforts where the decision-making in the visual direction of each work alternates between Jo and frameworks of computer generation. She works with emerging digital technologies to practice a range of modes in space-making, some of which include speculative fiction and visuals using machine learning, virtual reality, and large-scale projections, creating works that live between the built and the natural environment, amorphous and discrete forms, and hand-crafted and computer-generated images. Jo’s work often reflects on the ways in which we perceive and learn from patterns from our surroundings and explores the persisting question of being human in an age of ubiquitous autonomous machines.

A bout Joanne Ho

Joanne’s Initial Inquiries

My artistic journey started very young. As a result of certain childhood events, I started to nurture an active imagination to fill the space of my mind with extraordinary places and stories while having little choice to change my environment. The expansion of space beyond its physical boundaries has always been something I’ve been interested in exploring, whether through spatial experiences and relationships that are affected by people and objects or through crafting the boundaries of physical screens (walls, windows, ceilings, floors, etc). I pursued Architecture early in my career hoping to understand the ways in which meaningful spaces can be created and expanded.

In a recent interview I did with a student who was conducting research on placemaking in Singapore through the use of projection mapping, I was asked what placemaking meant to me. Initially, I thought of the word “home,” a word which has a slippery meaning when it comes to my life. To add some context, along with having built extraordinary worlds in my imagination outside the bounds of physical space, I’ve also had many spatial and cultural (and general environmental) transformations in my life from moving “homes” multiple times. A singular, stationary place with heightened sentimentality feels alien to me. Nowhere is home. But home is the city in which I’m currently residing... Everywhere is home. But placemaking--something that I’ve thought about for a long time, though not in this particular term--was what I was attempting to understand in my education and in architectural practice. Placemaking was not just making a home, where “home” as an idea already seemed to have all the baggage of specific familial and cultural rituals. A summary of what I said to my interviewer: To me, part of placemaking does have a nostalgic element to it - infusing a place with sentiment and meaning from a personal experience and relationship to a certain place. However, the process of placemaking is also framed by a culmination of environmental elements--the atmosphere, by the sounds, by specific architectural buildings, or it can even be made by a row of trees by a route in the park you frequent. With the amount of places I’ve lived in, there were some

places I was bound to revisit years after I had left.

Then four of us met each other at the Art x

Once visiting my past “homes,” places that held sig-

Tech networking event, which I was very curious

nificant memories to me were transformed and/or

about. I was just starting to expand my network and

replaced entirely by new objects. Places that were

community of artists in Singapore since I was quite

imbued with so much feeling and sentimentality no

new to the scene. What initially interested me in col-

longer exist in our physical world at this particular

laborating with theater directors and dramaturgs was

time. However, it still lives on in my mind. Even cer-

the idea of learning how to create imaginations of

tain songs that I listened to during the time in my life

spaces and stories while thinking about the shared

in a place can trigger the memories more vividly than

space and relationships between the stage, actors,

if I were to be physically present in these places now.

audiences, words and time to create an immersive experience of a place. I’ve always been interested in

With each time I get a shock from the non-

performance art as well. However, I would like to un-

alignment of a place made in my mind to its now

derstand more deeply the way stories are crafted in

changed physical location, the more I come to real-

the field of theatre.

ize that places exist nonlinearly in the mind and in memories, while spaces in the external world exist in

There was one artist I really admired when I

linear slices of time in which your physical body is

started doing new media art not too long ago ( just

located. Places are ephemeral in the physical world,

2018!)-- Es Devlin. She was working with Bjarke In-

but live on in our memories, where the strongest are

gels, who I also admired as an architect when I first

encoded because of its association with emotion.

started architecture. When looking at her body of

Being present in constant change is how we make

work, I was quite curious about how she started to

meaning while moving through the world. The way

craft space and places - with theater set design!

we placemake is the way we create memories, and the strongest memories we hold are created because of its ties to our emotions.

After working with all four theatre directors in our team in our third session and learning about the ways in which to think about moving through space

I began materializing these thoughts when I

and time (by walking, by performing actions, by per-

moved back “home” to Singapore during COVID-19

forming words), I was very intrigued by the wealth

when I was displaced from Berlin, where I was still

of knowledge that I haven’t tapped into about craft-

technically living. I experimented with creating visu-

ing spaces and making a place. Through the lens of

als with machine learning for the first time as a result

placemaking, I feel like there is much to explore for

of curiosity, boredom, and a lot of time. I realized that

me in terms of transforming the space of the stage,

this process of collecting data, having the machine

actor and audience using the mediums I’ve been

learn its patterns and creating new images from the

working with for the past three years: virtual environ-

ones I had given it was very much like the way we

ments, interactive visuals, emerging technology and

create memories. Machine learning was, after all,

light as object.

modeled after the way we think our brain learns. This was included in my project (RE)ROOTING that was projected on the facade at the National Gallery this year. Joanne’s Initial Inquiries

WORKSHOP IDEAS: How do we experience space using the resources of our own homes? •

Remote set-up for viewing

•

360 camera in the middle of the stage, where actors play around the camera

•

The experience can be viewed in 360 on the computer screen

•

Wherever the viewer looks, the space of the stage lights up. The more viewers look at that space, the stronger that area will fill with light

KHAIRUL KAMSANI Theatre maker, acting coach and co-artistic director of Bold Moment Theatre & Actor Training. Contact@khairulkamsani.com Www.khairulkamsani.com

Khairul Kamsani is a theatre director, researcher and acting teacher on PhD scholarship at De Montfort University conducting practice-based research on Actor Training with Virtual Reality (2021-2024). A graduate from the MA Actor Training & Coaching program at the Royal Central School of Speech & Drama (2018) after an Assistant Director Residency at Singapore Repertory Theatre (2016-2017). His work as a director, playwright, producer, designer, dramaturg and teacher over the past decade has been across Singapore, Hong Kong and the United Kingdom. His current thesis Cybernetic Psychophysical

Actor Training, has been presented at Borderlines VIII: Rejecting, Tracing, Embracing Performance Legacies (2021) by De Montfort University, Leicester, and will be at the upcoming Har-

nessing a 21st Century approach to Performing Arts: Technology, Practice, Education and Research (2022) by Hong Kong Academy of Performing Arts. Upcoming productions include directing Constellations (2022) at the Arts House Playden and Sita of Troy (2022) at Aliwal Arts Centre. Sita of Troy is supported by the National Arts Council SEP Grant and will be the debut production for a new theatre company, Bold Moment that he is co-artistic director with Kristina Pakhomova. Khairul will be principal instructor in the Meisner, Viewpoints and Michael Chekhov techniques as well as developing a new acting program, Acting in the

Metaverse.

A bout K hairul Kamsani

K hairul ’s Initial Inquiries

My overlapping artistic practices can be viewed as a Venn diagram with my self between two major distinctions- Theatre and Martial Arts, and within these there are more specific fields of focus; I direct plays, dramaturg scripts, teach acting, research and perform PhD research under my theatre practices and study and train in Muay Thai, Boxing, Brazillian Jiu Jitsu, wrestling, bodybuilding, Yoga and MMA under my Martial Arts practices. This current endeavour with the Architects of Realities would seem to fall under my Theatre Practices umbrella but it is also inevitable that the intersections of knowledge within me will inform and affect the participation and gained experience from this research group.

The impetus for the collective revealed itself from a mutual curiosity towards possibilities and potentialities that our collective expertise could present through their intersections. Even as four theatre directors alone, there is a range of approaches, experiences, and knowledges within each of us that contribute to ‘the theatre practice’ that appears within this collective. The two creative technologists too, offer a range of knowledge, expertise and experience to the fold; there would already be plenty to explore within the intersections of Virtual Reality and Projection Mapping. The task that we have set for ourselves is truly unprecedented for us and the effort to align each of our practices in a single space (literal or figurative) is taking time and considered communications- this could essentially be our project in a sentence, before considering the documentation and articulation process we have set for ourselves. From the outset of this research project, it was asked of us during our proposal interview whether I would and how I could discern this research differently and separately from my PhD project which share similar points of interest, theory and approaches to practice-as-research. Knowing where my PhD currently sits and the trajectory it has, the variance between both projects are clearly distinct for me though understandably of such a similar vein to each other from outside perspectives that it could be mistaken to be of the same subject matter. A major task for me then within this project is to be able to draw the boundaries between both re-

search projects in order to identify clear approaches

in any ‘session time’ would be my acting teaching

between ‘Actor Training with VR’ and ‘Theatremaking

practice: a practice I would describe as “a process of

processes with VR and Projection Mapping’ given

cultivating a considered practice of ‘being’ and ‘doing’

that the latter includes elements of the former under

of a performer”. With the layers of technological reali-

its scope.

ties introduced, my focus would expand to “a process of cultivating a considered practice of ‘being’ and ‘do-

Another large element of Theatremaking in-

ing of a performer through a multitude of spaces, ac-

volves work with actors: bodies in space. Therefore

tions and selves (virtual and corporeal)”; how then do

within the Architects of Realities, it becomes imper-

I perform my teaching and cultivating? What learning

ative to consider and examine the approaches and

outcomes for the participants should be in focus?

conditions of performers’ ontologies enmeshed with

Which/what approaches do I apply or am presented

various technologies (low and high). Given that with-

with given the inter-medial continuums at play? An

in my Actor Training practice section of my Venn dia-

important aspect of this project is to not only attempt

gram, psychophysical training (Martial Arts) will find

to answer these questions but also capture the phe-

its engagement with VR and Projection Mapping;

nomenological process of us doing and experiencing

the question arising pertaining to this practice-as-re-

these experiments, it is from there can we review and

search process is whether I can or should seek an

revise for the next ‘session time’ each time we enter

approach and/or knowledge under the Architects

the playspace(s), to be articulated and reflected in

of Realities scope divorced from approaches and

these complementary writings. As we move ahead,

knowledge currently within and to be gained in my

I look forward to guiding my peers on writing for

PhD research. This would directly affect my engage-

our ‘prototype’, a compendium/guidebook towards

ment with the technologies and bodies in the space.

collaborative process work between the performing

Our project seeks to investigate the engagements (or

arts and creative technologists. At this time, I have

collisions) of ‘realities’ across the corporeal, Mixed,

to consider the level of academic rigor in place for

Augmented and Virtual states of being/perception

our inquiries and reflections; how could (and whether

through practice-as-research; given that I am cur-

I should) I introduce and weave in theoretical refer-

rently undergoing a practice-as-research (PAR) PhD

ence texts for my peers? Considering Robin Nelson’s

at De Montfort University, it seems to naturally fall to

notion of ‘praxis’ that we are undertaking, perhaps the

me within our collective to navigate the PaR elements

theory begins from their first articulation?

of our project- from the planning, execution and articulation. Therefore, in fact while I am keenly interested in the interplay between Arts and Technology in our project, I feel that my practice within this project is actually involving the structuring, provocations, editing and guiding the articulations of my peers as they undergo their explorations (individually and as a collective). Returning to the relationship of my practical practices within the Architects of Realities, my focus K hairul ’s Initial Inquiries

A bout Nelson Chia

NELSON CHIA Co-Founder and Artistic Director, Nine Years Theatre. Director, Performer, Writer, Educator in Theatre. Nelson@nineyearstheatre.com Www.nineyearstheatre.com

Nelson is an actor, director, writer and theatre educator. He has been an active player on the Singapore stage for almost three decades, taking on numerous major and leading roles in English and Mandarin productions. He was the first local artist to be commissioned for a consecutive three years to present works at the Esplanade’s Huayi Festival, a two-time Best Actor (2011 and 2013) winner and fourtime Best Director (2014, 2015, 2018 and 2019) winner at the ST Life! Theatre Awards. In 2017, he was commissioned by the Singapore International Festival of Arts to open the festival with Art Studio, a stage adaptation of the novel by local award-winning author Yeng Pway Ngon. Nelson is the co-founder and artistic director of Nine Years Theatre (NYT). Within a few years, NYT has grown to become one of the key players in the scene. Through his artistic direction, NYT has managed to not only rejuvenate the Singapore Mandarin theatre scene with their work, but raised awareness of actor training among the artistic community. Nelson has been training regularly in the “Suzuki Method of Actor Training” and “Viewpoints” since 2008. He had studied these methods with the SITI Company in New York and the Suzuki Company of Toga in Japan, and had performed at the Toga Summer Festival.

THE PLACE OF VR / AR / MR IN FICTION WORK Research Question: How can we employ VR / AR / MR technologies to help invoke abstract imagination in Fiction Work?

Introduction to Fiction Work Since 2017, I have been leading actors from the Nine Years

Theatre Ensemble in a research project on actor training methodology. The research is title Body Paradox: The NYT Actors’ Work. It is essentially a series of approaches to rehearsal and performance. Fiction Work is a segment of the research that focuses on the use of “Fiction” in helping actors create performance. 2. Definition of Fiction Fiction is used to describe the realm of story, narrative, character, objects, space and time that belong within the performance, and that is distinguished from the reality that the audience members find themselves in. 3. Prepatory Exercises for Fiction Work One key element necessary for the Fiction Work is the imagination of the actors. In order to facilitate the process of imagination, some exercises are used to help actors prepare the condition of their bodies and minds. The preparatory work is divided into stages and actors move from stage 1 to 4 in steps before they go into the Fiction Work. • Stage 1 - Everyday Body and Mind In this stage, actors recognise the characteristics of an everyday body and mind, in order to distinguish them from subsequent stages that follow.

Nelson’s Initial Inquiries

• Stage 2 - State of Wakefulness

4. Using the Fiction Work

In this stage, actors awaken their senses – the

In executing the Fiction Work, actors select

5 natural senses of sight, hearing, touch, smell

an image that is related to or inspired by something

and taste, and an additional sense of inner res-

significant in the text, context, action or situation of

onance with the space and time that the actors

the scene. The actors then call upon their imagination

are in and with one another. The feeling of this

to invoke that image, or to be precise, the sensation of

state is a body and mind that is open and recep-

that image. It is important that actors do not begin to

tive, with a sensibility so sharp that every sen-

visualise their images as solely a realistic representa-

sation around the actors seem a little too much,

tion, instead they should focus on the abstract sensa-

but not unbearable. What is also important to

tion associated with the encounter of that image.

achieve in this state is a reciprocal response to the sensations, that is free and impulsive. This is

The aim of using the Fiction is to respond to

a state where the actors are awake and alive in

the fiction, constantly and continuously. And to allow

their bodies and mind.

the moment-to-moment experience and response to the Fiction affect the actors’ bodies, mind, breath and

• Stage 3 - Composed State In this stage, the actors begin to collect the

journey as they perform the text and the scene. 5. Three Types of Fiction

energy and breath that resulted from the sensations felt, and gather them in the core of their

Following that, actors are able to work on the Fiction

bodies. The aim is to come into a collected state

in 3 ways.

– a more composed being that is no less awake and alive as before. Another meaning of the

• External Fiction

word “compose” is to create, and hence it is in this stage that the actors begin to use the Fiction

Where the Fiction is outside of the actor – for

Work as an approach to creating performance.

example, the encounter with (the abstract sensation) of a flickering candle – how does it feel to

• Stage 4 - Heightened State

see that light? Is it fragile? Is it precarious? How far away is the candle? Are you attracted to it? Is

In this stage, on top of being alive in a Fiction, the actors allow the emergence of an inner life that

it warm? What does it make you do? How does it make you react?

forms the basis of characterisation. This stage is of a more advance nature and since the Fiction

• Environmental Fiction

Work resides mostly in stage 3, I will therefore not go into detail about this stage in this writing.

Where the Fiction is imagined as an environment that surrounds the actors – for example, (the abstract sensation of being in) a misty forest – what is that sensation on the skin? Is it damp? Is the forest cold? Is it dark? Does it give

you a sense of loss? Are you afraid? How is it

HYPOTHESIS

affecting your physicality, your breathing, your muscular tension and your breath?

What is the experience of using VR/AR/MR technologies to help invoke abstract imagination in actors

• Embodied Fiction Where the actors are the Fiction – for exam-

during Fiction Work. 4 questions to help guide the assessment and reflec-

ple, you are a melting, dying candle. What is the

tion of the experiment:

sensation of your skin and flesh melting? How

•

does it feel to be dragged to the ground? How is the heat of the flame making you feel? How is

for the abstract imagination? •

it affecting your physicality, your breathing, your muscular tension and your breath?

Do the technologies act to provide interactions Do the technologies impede the experience of abstract imagination?

•

How does the actor’s body and mind react to what the technologies are offering?

6. Difficulty

•

What does the experiment do to the notion of Fiction?

One of the challenges in using Fiction Work is that it relies on the imagination of the actors. In other

DESIGN OF THE EXPERIMENT

words, its effectiveness may be limited to the range and power of the individual actor’s imagination, es-

•

Select a dramatic passage / text .

pecially when it is required that the images incline to-

•

Extract two Fictions from the passage / text.

wards the abstract sensation rather than the realistic

•

Start with the first Fiction - Design the technolo-

representation of the Fiction.

gies to offer interaction to the actor in 3 types of the Fiction Work --

Conclusion

• External • Environmental

The aim of this exploration is to find out how

• Embodiment.

the VR / AR / MR technologies can be engaged to

•

Make observations guided by the 4 questions.

help inspire and invoke the actors’ imagination of ab-

•

If time permits, extract another Fiction from the

stract sensations in the practice of Fiction Work.

same passage / text. •

Repeat the experiment.

•

Make observations guided by the 4 questions.

•

Compare and contrast the experiences of the two Fictions.

Nelson’s Initial Inquiries

A bout Thong P ei Qin

THONG PEI QIN Theatre Practitioner (Director & Educator) Thongpeiqin@gmail.com Thongpeiqin.wix.com/livingtheatre/

Pei Qin is an independent Singaporean theatre director who is passionate about devising original works of critical social relevance. She graduated from the University of Essex (M.A. Distinctions in Theatre Directing) and the National University of Singapore (B.A. Honours in Theatre Studies) and has devised multi-disciplinary works on experimental platforms, often bringing performances out of conventional theatre spaces. She strongly believes that theatre should be made by and shared with everyone, and in her own highly collaborative working style, pushes for accessible equitable working ethos and creative processes to excavate original Singaporean stories, movement and site-specific work. Her theatre-making encourages the staging of new writing, and rediscovering innovative forms of cross-disciplinary research and creations across the physical, visual, verbal, textual and technology. Her selected recent directing credits are “Mine” (2020), “SUPER MERMAID” (2019), “BITTEN: return to our roots” (2018), “招：When The Cold Wind Blows” (2018) and “Step Outta Line” (2018). She was the Associate Artist of The Theatre Practice, Associate Director of Saga Seed Theatre, and is currently a creative-in-residency with Centre 42, and a director-in-residency with W!LD RICE. She is also currently a part-time theatre lecturer at NIE/ NTU, LASALLE, and has lectured at NUS and NAFA.

Her directing portfolio can be found at (thongpeiqin.wix.com/livingth-

Pei Qin is excited to call NAC’s inaugural ArtxTech Lab her

eatre) and her official FB page at

playground to experiment with her fellow technologist-artist

(www.facebook.com/thongpeiqin.

collaborators on how to direct in virtual reality.

director).

MIMICKING AND MANIPULATING THE REALITIES OF TIME & SPACE My personal on-going artistic pursuit and investigations: As a theatre director, I am interested in devising and choreographing repeatable movements/ gestures of the performers’ bodies, with text. I do this with learnt Biomechanics principles and ways to create movement sentences and larger movement scores, fused with the traditional art forms such as Bharata Natyam mudras and codified margams. With my collaborators, I devise ways of creating patterned movements that can be systemised/ codified.

REFLECTIONS ON ART XTECH LAB WORKSHOPS #1 & #2 After attending both Timothi and Joanne’s first two sharing sessions of our ArtxTech Lab, a couple of pertinent questions arose for me: 1.

What is the merit in attaining machine-like accuracy in performance? (E.g. Ensemble precision in timing and form, the plasticity of the performer’s body on stage à la Meyerhold’s biomechanics1, etc.)

2. Why is there a fascination and hence constant pursuit in theatre, to realise the imagination of the human body working like clockwork or as machines/ inanimate objects? (E.g. “Frankenstein”, “Oiwa” by The Finger Players, etc.) My own research interest would be to seek an understanding of the tension between animate “liveness” and inanimate “liveless-ness” when pursuing accuracy in repeatable movements or mimicry in performance. Whilst there are advantages to precision and coordination in a well-oiled machine (be it individual performer’s bodies or in an ensemble) and also beauty in the morbidity of having dead 1

P ei Qin’s Initial Inquiries

Refer to additional reading material on Meyerhold’s Biomechanics below

objects appear lifelike and vice versa, there are also

ensemble working as a team, and of the entire play.

disadvantages of life being restricted by methods/ rules/ technical precision/ technology hence appear-

A simple exercise for an actor to test on a reg-

ing too dead, or being “killed off”. The latter is usually

ular basis would be “What does 1 minute feel like to

observed in rehearsals when actors/ directors give

you?” Without relying on counting or the clock, the

feedback about not having room for surprises, impro-

actor could choose to sit in a meditative style, or pace

visations, creativity or life.

about a contained space, finding ways to imagine the duration of 1 minute (e.g. Seeing him/herself going

My curiosity is to find or strike a balance

through a usual routine that is 1-minute long). This

between these two extremes in actor training pro-

exercise was documented in our video recording on

cesses, with the assistance of technology such as

2nd July.

projection mapping, AR/MR/VR. I am also interested

The discoveries from this exercise led me to

in finding out how technology could help to improve

ponder deeper on the concept of “phenomenology”

actors’ movement training such as establishing

in theatre. The “liveness” of theatre is fundamental in

and breaking patterns and habits.

incorporating the real as part of the fiction, uniting both the body and mind. It is the process within psy-

As an actor is often trained to establish space and time in both the actual theatre space, and in the

chophysical training that generates ‘presence’ which is the actor’s own.

fiction of the storytelling, what technology can do as an additional layer to this shared make-belief world

The actor in his or her “material, physiological

would be to mimic and manipulate the realities of

facticity,” inserts a “fundamental and intrusive actual-

time & space. This could destabilise/ warp both the

ity into the field of dramatic representation... A root-

actor’s and audience’s sense of time and space, or

edness in the biological present that always, to some

help enhance their imaginations.

extent, escapes transformation into the virtual realm” (Garner).

PEI QIN’S SHARING FOR WORKSHOP #3

On the self-givenness quality of time on stage:

AND FURTHER REFLECTIONS “Clearly, there is something about a working Exercise 1

clock on stage that is minimally disturbing to an audience. But it has less to do with time per se than with

As a director, it is important to train the actor’s sense of time.

our awareness that theatrical time is being measured by a real clock—an instrument that is visibly obeying its own laws of behaviour” (Bert O’ States, 30).

An actor can be viewed as an athlete, aiming to replicate exact moments/ footholds show af-

On the duality of the real and the fictional on stage:

ter show, down to the most minute detail such as an intake of breath or a pause after a specific line,

“There is a tension between the actors’ phe-

or following a particular movement. This is for the

nomenal body, their bodily being-in-the-world, and

maintaining of consistency and rhythm of the whole

the use of that body as a sign to portray a character. ...

We will describe this tension as occurring between the

SUGGESTION FOR FUTURE EXPERIMENTS

‘phenomenal body’ and the ‘semiotic body’” (Fisch-

ON OUR ART XTECH LAB

er-Litche, 26). 1. I am interested in playing with that duality in our own “Architects of Realities” ArtxTech Lab, using the disturbances or provocations that technology can bring.

Repeated movement tracking with “laggy” projection mapping (track for accuracy/ patterns of what came before)

2. Movement choreography with technology as a fellow player: jam with certain rules with technology to create a movement-music score together (inspired by Joanne’s “agar-agar” experiment, but substituting the food items with actors’

Exercise 2

bodies, keeping in mind components of “terrains/ spatial relationships” and “timing” when testing

On training the actor’s observation skills

or creating movements)

and level of accuracy in mimicry. HYPOTHESIS & DESIGN FOR NEXT WORKThis was an actor training exercise modified

SHOP (10 AUG)

based on Stanislavski’s principles which my GITIS Russian professors did with me daily when I was train-

How can technology (project mapping/ AR/

ing there. (It trained our sensibilities of: “as if for the

VR) jam as a fellow “performer” with the move-

first time”, “give and take”, “listening”, “the moment be-

ments/ bodies of actor-devisers to create a move-

fore”, etc.) Basically an ensemble of actors takes turns

ment-sound score together? I posit that a live/ im-

to perform and watch a repeatable gesture done in

mediate jamming is possible, as a hypothesis.

the performance area, observing and remembering the gesture done to the most minute of detail. The

Further, I am interested in testing for the

objective is for each actor to take turns entering the

efficacy/ effects of technology responding live to

performance space to mimic/ replicate the exact pre-

the actors’ movements and director’s prompts, such

ceding gesture, and adding on one more gesture of

as whether “kinaesthetic response” is possible, and

his/ her own after. This exercise was documented in

whether technology can enhance the jamming pro-

our video recording on 2nd July.

cess of creating movement scores.

This is a kind of training that is systematic and

sequential, and enables actors to hone their craft of portraying emotions in a specific, controlled and replicable way, while training observation skills.

How can technology transform the way an actor interacts with locales (an imagined fiction where events happen in) and location (actual architecture of a given venue)? Such interactions will also inform how an actor moves within the space (real or imagined).

P ei Qin’s Initial Inquiries

A bout Timothi L im

TIMOTHI LIM Technical Game Designer, AR Experience Engineer Creative Director (Augmented Reality) at The Doodle People. Timothiellim@gmail.com Www.timothiellim.com

As a Creative Director, Timothi leads The Doodle People to develop AR effects and standalone apps to empower brand awareness, inspire clients, and support AR R&D for their business. As an individual, Timothi is an interactive media developer who seeks to connect generations through the union of physical and digital lifestyles. Timothi has developed interactive media works for a wide range of platforms from physical card games to the Hololens and Magic Leap headsets. His work has received awards and has been showcased at IndieCade, E3, and GDC. In his time as an interactive media developer, Timothi has worked with a wide range of firms including Grab Games, Sony Santa Monica, Riot Games, Marvel Entertainment, and Seagate Technologies. About The Doodle People The Doodle People LLP is a Singapore-based studio specialising in cutting-edge interactive and digital media. Founded in 2014, The Doodle People has been engaged by various organisations for their communications and branding needs. The studio’s strength is its ability to attract, captivate, engage, and empower their audiences, so applying their skills to performance arts was a natural progression. Find out more: www.thedoodlepeople.com

Build the Impossible - Creative Technology Develops Global Connections on a Generational Scale and Fosters United Worldviews that multiplies innovation through increased Trust and Communication The world that I grew up in was a crucible of miracles. The happenings and circumstances that allowed me to even exist boggles the mind and defies logic. In most of my alternate timelines, I’m pretty sure I would have died at childbirth, but yet here I am. In my mother’s womb, choked by the umbilical cord that sustained me, I stopped breathing. Through medical advances, modern understanding, and technology, I was saved via emergency c-section. Birthed in the small country of Singapore, I grew up to realize that the place I called home was a miracle as well, another impossibility that somehow exists. Before the 1950s, the idea that a small independent country without natural resources could forge its own path and become a developed country was unthinkable, practically impossible. I learnt from recorded history that throughout our very short human timeline, we have as a species continuously achieved remarkable feats previously deemed impossible or unthinkable and that to me is the purpose of human life. I believe that the purpose of human life is to build the impossible through a compounding challenge or drive built by the previous generations. We as humans will perish in the now but what we have developed will live on through the generations to come. With technology, the effort of our present will be multiplied into the future. The dreams we cannot achieve now will, in time, always be achievable by future generations. Today I build with tomorrow’s technology so that the future can inspire generations. Through my work, I seek to empower, redefine, and build a united worldview that is born of a multi-generational understanding fostered by shared connections strengthened through technology. Shared connections developed by experiences allow us humans, beyond race, gender, and other definitions, to break through the notion of the “Other” and find common ground; increased commonalities with each other as individuals. Common ground is ideal for fostering trust and communication which leads to dialogue, discussions, and most importantly collaboration. Collaboration is the

Timothi ’s Initial Inquiries

cornerstone of innovation and the driver for building

To help improve the understanding of how

the impossible. Not only should we collaborate with

connection leads to future innovation, I have created

our own generation but with those who came before

a simplistic function:

and will come in the future. Together, regardless of race, identity, age, language or religion, we will build the impossible of the present by uniting the future with creative experiences developed to connect. Collaboration is connection, and that makes connection the pivotal keyword to explore in this entire articulation. Connection to me is defined as awareness and understanding. Awareness is the conscious state of acknowledging an existence of a concept or object. Understanding is an accepted rea-

Where,

son, process, or answer to the what, how, and why of a concept or object. Both awareness and under-

= Time

standing exist on an individual level which is then

= Number of unique individuals

tempered, altered and even mutated, by a community

= Generations divided in 10 years. From 0 - 1

level of awareness and understanding driven by en-

(e.g 10-20 year olds = 1 generation = 0.1)

vironmental and socio political factors. With technol-

= Trust Quotient

ogy, especially social media, the community level of

= Communication Quotient

awareness and understanding, the communal con-

= Impossibility Quotient

nection, is multiplied. Yet, the individual connection,

= Future Potential

the awareness and understanding of our own internal connections or worldview, is rarely multiplied by technology. To me, the exploration, learning, and sharing of Arts and Culture leads well into self-reflection and

The above function above is by no means

thus is an effectively viable avenue for empowering

mathematically accurate. The function exists to help

individual connections through technology. The spe-

align our understanding of future potential. To me fu-

cific use of technology is to utilize the affordances of

ture potential of any given work can be described as

technology to develop creative experiences that en-

the total time multiplied by the number of unique in-

able individuals to connect to each other, especially

dividuals that have engaged with the work. Multiply-

to different individuals they would never meet, and

ing that by the generations that engage in the work as

thus through that connection create a self-reflection

a decimal point from 0 - 1 will then show the impact

on their own worldview. Meeting, sharing, conversing,

of the work as a number. The result of ((TxN)G) must

and ultimately learning from another individual who

then be multiplied by (Trust x Communication), both

is wholly different enables an individual to self-reflect

of which are currently arbitrary numbers which have

and thus progresses the possibility for them to con-

yet to receive full thought. In some sense, trust can be

nect to others, and fosters collaboration to exist.

calculated by polling or by recognition, some method must be devised. Nonetheless, human trust in any

given work is vital to the future potential of the work.

is the opportunity, and with it brings change. Change

Trust that this work is authentic, speaks the truth, or

can be an avalanche, violent and quick, or it can be

expresses with well intentions. Communication can

river eroding rock, calm and steady.

be calculated by the number of people who understand or acknowledge a point of the work, as an ex-

I believe in the impossible, in our future po-

ample. Together Trust (Tr) and Communication (Co)

tential to change despite the perceived odds that hu-

are multiplied to create Change Potential.

mankind will most likely destroy itself or the environment it lives in. Perhaps we will be a blip in the eons

Change potential is what happens when a

of life the galaxy will create, yet, let our light shine

large number of generations and individuals trust and

the brightest even in the darkness. We will one day

communicate about a given point or body of work.

return to nothing, so let us be the greatest version of

Change Potential ((TxN)G)(Tr x Co) is then divided by

ourselves.

the impossibility quotient which is an individually or communal arbitrary number that describes either the hope or despair, the possibility of success. Logical, emotional, irrational, rational, it does not matter. Can belief be calculated? If it is, then it is liminal, fleeting. The impossible is a belief and the number changes at every moment. In the end, when Change potential is divided by the Impossible Quotient, I believe is where Future Potential lies. Future Potential, in some views is both an indicator of the Hope or Despair to come. The function above is not a proof. The function describes what I believe to be my own vision or calculation of Future potential. One perspective it provides is a simple mathematical model to show how Hope and Despair manifests for me. Future Potential can be negative, it can be very small, it can be very large, it is always changing, ever shifting, it can be used to describe one work or all works since time began. Overall through this, I seek to express that the future lies in our belief, in our visions, in our self. We are the arbiters of our own fate, and the choice to collaborate together always belongs to us. We can choose to rule, be ruled, we can choose to create or choose to destroy. Choice happens even in silence. All potential energy will one day be converted to kinetic energy when given the opportunity. Choice Timothi ’s Initial Inquiries

GLOSSARY OF TERMS

GLOSSARY OF TERMS Disclaimer: These definitions are as we interpret these terms, as used within our discussions during the lab. We are open for further discussions on any of these.

0-9 360 Camera A camera that captures images and video in 360-degrees around the camera’s lens.

A Actant Something which an action can be acted upon e.g. A prop that can also be seen as an actor or character Acting The craft of practicing “being” and “doing” either as self in the real or as a character in an imagined circumstance, or both Actor Conventionally known to be a person who practices the artistic craft of acting (see above) but in recent times can be used to describe any person who performs a physical action (see Schechner’s performance theory) Actual/Physical Reality Actual Reality is used as an agreed term to describe the shared physical world that our corporeal bodies inhabit, as such, actual reality can be clarified to be our tangible Physical Reality. This term helps to create delineation from the intangible virtual spaces or virtual reality.

Architects of Realities (AoR) A group of artists, theatre directors, dramaturgs and technologists who formed a research group in 2020 as an initiative of National Arts Council Singapore’s Art x Tech Lab. Consists of (in alphabetical order) Cherilyn Woo, Joanne Ho, Khairul Kamsani, Nelson Chia, Thong Pei Qin & Timothi Lim. Augmented Reality Augmented Reality or AR is used to describe a physical reality that has been overlaid with virtual objects, e.g a digital tennis ball floating in the air above a table but the tennis ball does not interact with the physical world. Augmented Reality is quite explicit in that it does not encompass the interaction between the virtual and physical, instead a virtual object is rendered visually in our reality but does not understand the context of our reality, therefore to the virtual object, our physical reality does not exist. Augmented Reality is great for superimposing a virtual reality onto our physical reality, i.e Projection-Mapping or using a phone camera, thus allowing us to view virtual objects in relation to the physical world, e.g to gauge scale. In our work, Augmented Reality is used to overlay virtual imagery to create different environments, sensations, or to offer a window into the virtual world. Azure Kinect Developed by Microsoft, the Azure Kinect Depth Camera is used in our sessions as a markerless motion capture devices. The Azure Kinect camera uses machine learning software to track the human form and thus translate movement to the virtual world. While the markerless tracking is not as “clean” as a motion capture suit, the camera itself is highly portable, has little setup time, and does not need anyone to wear a motion capture suit. As such, the Azure Kinect is very useful for prototyping and the data capture can only get better with larger investments or better setups. Thus the Azure Kinect can validate at a very low cost if motion capture fits the project and if a more expensive setup is viable. Remember the entire lab was also designed to be accessible and affordable to setup, this is why we chose the Azure Kinect over other solutions.

B G lossar y of Terms

Barrier (in VR) Virtual barrier that is designed to limit movement or actions. A virtual barrier was created in the virtual space to prevent the virtual avatars from falling outside the space. While the virtual barrier was intended to serve a functional effect similar to how a railing would prevent a person from falling, it was interacted with by the players and thus included in their performance, such as becoming a representation of an invisible cage or a technological barrier that physical actions could not penetrate. Butoh A form of Japanese Dance Theatre.

C C.G. Character A digital avatar created by computer graphics, where its movements are mapped and animated manually or driven by motion of a person in real space that is using a motion capture device like the Rokoko suit or motion sensor device like Azure Kinect. Continuum An interdependent transference of information between two entities that is procedurally generated Control A constant in an experiment that does not change, as a means to measure the results of repeated experiments against one another Corporeal Of the flesh; relating to a person’s body

D Digital Viewpoints A technique, ideated by AoR, that thinks about movement and gestures between bodies through digital technology and frames, developed from viewpoints, an existing technique in theatre and dance.

E Emotional Contagion A term from Fischer-Litche. Emotional contagion is a form of social contagion involving the spontaneous spread of emotions and related behaviors. Such emotional convergence can happen from one person to another, or in a larger group. Emotions can be shared across individuals in many different ways both implicitly or explicitly. Environment The surroundings or conditions in which a person or living organism lives or operates. The setting or conditions in which a particular activity is carried on. Environmental theatre A term coined by Richard Schechner. Environmental theater, sometimes called site-specific theatre, is a style of theatrical performance which strives for greater authenticity and audience engagement by staging its action in appropriate real-world settings rather than in traditional theater spaces. It transcends the boundary or fourth wall which exists between audience and performer. A secondary definition of environmental theater is used to describe any production that attempts to immerse the audience in the performance by bringing the action off the stage area. For example, some acting may happen in aisles. In the case of a black box theater acting platforms may even be built between audience section. Sometimes a performer will talk to, or otherwise involve an audience member in a scene. This can be a real audience member, as in interactive theatre, or an actor planted to appear as an audience member. A subset of this is promenade theatre, in which there is little or no seating for audience members, who watch the action happening among them and may follow the performers around the space. Environs The area surrounding a place

G lossar y of Terms

F Fiction Fiction is used to describe the realm of story, narrative, character, objects, space and time that belong within the performance, and that is distinguished from the reality that the audience members find themselves in. Fiction Work Part of a research project on actor training methodology by Nine Years Theatre. The bigger research is titled Body Paradox: The NYT Actors’ Work. It is essentially a series of approaches to rehearsal and performance. Fiction Work is a segment of the research that focuses on the use of “Fiction” in helping actors create performance. Frame Way of seeing, categorization of a perspective Frame-rate Speed of screen refresh rate, which sets the smoothness of the image on the screen.

G Generator A point of information/sensory output Glitch A temporary halt to a working system or machine. God’s Eye View An omniscient perspective, in which the players or characters immersed in a scene or virtual reality, cannot view themselves.

H Hermetically-sealed This usually means airtight (as in the lid of a container). In the discus-

sion of theatre, the film and television mediums are described as hermetically-sealed from the audience with the presence of a screen as a separation of the performance showcased, as opposed to the liveness and immediacy of the theatre viewing experience with porous borders between audience and performers. Hybrid Reality Sometimes used to describe Mixed Reality. However, to my knowledge, the term has also been used colloquially to describe a reality during COVID-19 where ‘hybrid’ situations were formed, creating situations that allow digital and physical reality to collide. For example, a hybrid event would have a real world stage and audience but also camera equipment that would stream the recording of the event in real-time to a digital audience connected via the Internet. Hybrid Reality can then be described as the realization that there is now a ‘new-normal’ or a version of reality where physical actions and connections are limited and some part of the whole reality is viewed through a digital medium such as a video call.

I Immersion Deep mental involvement in an activity, experience or an imagined fiction. Interactive Anything that requires two or more parties to influence each other is interactive. Interactive Technology is the influence of technology on a human and vice-versa. For example, we regularly use our smartphones to influence the online digital world, e.g a platform like Instagram. As such, the smartphone device is an interactive device and the act of swiping, typing, or tapping, is interactive. Cyborg Theory or Ludology, the study of games, is very much good reading and includes interactivity in its academic studies.

G lossar y of Terms

K Kick Something the pulls the performer out of the fictional world into something more grounded and rooted like their physical body (from Christopher Nolan’s Inception) Kinaesthetic Awareness of a person’s physical position and body movements.

L Latency Delay between an action by a physical action and the output from the technology or from on technological output to another. Live-ness The descriptor of evaluating the presence of an other, whether physically co-present or tele-present

M Machine Learning Machine learning algorithms are models designed to learn from patterns within a dataset that then generates new data from what its learned. Meisner Sanford Meisner was a member of the Group Theatre which was the first American theatre company to learn of Stanislavsky’s System. Meisner developed ‘The Meisner Technique’ at the Neighborhood Playhouse after leaving the Group Theatre over disputes over Stanislavsky’s key tenets Michael Chekhov Nephew of playwright Anton Chekhov who both worked extensively with Stanislavsky. Michael Chekhov developed and contributed extensively into concepts of psychophysical actor training in his time.

Mist (a design feature in our VR world) Our work had a ‘puppet master’ that was a human who interfaced with the virtual world through a keyboard and mouse connected to a desktop, allowing the human to control an invisible virtual avatar that had administrative properties - e.g changing the world / switching the level. As such, a particle system, which was rendered similar to real world mist, was added on to the invisible avatar to provide it a visual form while viewed in the virtual world. Initially this was kept as a debugging feature but was kept to elicit different reactions. Mixed Reality Mixed Reality or MR, also known as Extended Reality or XR, is the intersection between AR and VR, where both sides take into context our physical reality, e.g the digital world understands that there is a table and allows the digital object to interact / acknowledge the presence of a physical reality object. As such, in MR a virtual tennis ball can be dropped onto a real life table and the virtual ball will bounce as if it were a real ball. In our work, Mixed Reality is created through the use of Azure Kinect cameras that enable body motion and tracking, thus allowing us to convert physical body movement into digital data, allowing your physical body to interact with the virtual world or control a virtual avatar. Movements of frames See Frame-rate.

N NPC A non-player character in a game that is not controlled by ones that are playing the game. Usually driven by an A.I. Algorithm.

O P

G lossar y of Terms

Phantom Limb A residual sensation of a limb that has been amputated and creates chronic pain for the amputee. Treatments like mirror therapy have seen some success, where a mirror is placed in front of the amputee to mirror their existing limb to create a visual “virtual” limb where there is none. This helps phantom limb pain by associating the painless movement of the existing limb to the one that has been amputated. Phenomenology It is a live presence, a given-ness in a real authentic self and quality an actor or actant brings into a space, and usually keeps to its own laws in the theatre, defying the cooperation of creating illusions, fictions or manipulating realities. A philosophical stance first introduced by Edmund Husserl which posited that a singular consciousness can be aware of the perceivable world but only ever subjectively. Followed by Martin Heidegger who developed the principles into understanding how the singular consciousness engages with ‘the world’ and that a personal subjective reality exists among every other personal subjective reality, questioning any possibility of an objective reality. Maurice Merleau-Ponty proposed in Phenomenology of Perception (1965) the ways that knowledge of consciousness and the reality of the world is bound by our perceptive faculties. Presence The concept of co-sharing an experience with an other being Projection Mapping Projection mapping is a technology that uses video projectors to shine light on any surface that isn’t simply a flat screen. It has become more popular as a way to create visually immersive environments, transforming a space because of its flexibility in turning anything into a visual surface. Psychophysical A concept which views the mind and the body to be of an inseparable unified being that experiences being in world as a whole unit

Q 43

R Receiver / Reactor The interdependent sender and receiver within a continuum of information Richard Schechner Key researcher and theories into the study of Performance Theory since the 1960’s. Defined performance to encapsulate ‘being’ and ‘doing’ (among other descriptors) across the daily, the ritual and the theatrical Rokoko Suit A great motion capture suit that doesn’t break budgets and provides good recordings / data! The Rokoko suit has been on my personal and business radar for some time. As with all motion capture technology, a monetary investment is needed before getting started. The Rokoko Suit is affordable (by motion capture technology standards) and provides a great setup to scale from. Entry level setups would be around SGD $3500 and a full studio setup would be around $12000+. One of the key elements that limits my own business side from setting up Mocap in the studio is a lack of space. However dance studios, theater rehearsal studios or the stage itself offer a great abundance of rooms to set up motion capture. In addition, I believe motion capture is perfect in theater rehearsal environments as actors/actresses can wear the suit and a technologist can gather data from their movements and even directors can analyze these movements. Overall, I believe motion capture is a great technology that bridges the gap between physical and digital realities and thus allows technologists and artists (particularly theater) to connect better.

S Stanislavski Regarded as a central proponent of psychological realism in the practice of acting and theatre making, his work influenced many practitioners in developing a wide range of approaches towards acting that are available today

G lossar y of Terms

T Theatre of Cruelty Commonly associated with Antonin Artaud, a French dramatist. It is a form of Theatre aimed at shocking the audience and their senses, often using images of violence. Theatron A place for seeing/ viewing (in Greek) Totum A compass/an anchor to help the performer in VR navigate what is virtual and what is real, what is fictional and what is physical (from Christopher Nolan’s Inception) TouchDesigner A software that uses visual coding to create interactive experiences and is used widely for multiple types of outputs, one of which includes projection mapping.

U Unity Game Engine that is regularly used by many video game development companies and groups to develop interactive software. The Unity game engine is very accessible, affordable, and applicable to connecting the multiple technologies used in this lab. Another viable option would be Unreal Engine. The reason a video game development engine was chosen is that it allows for real-time interaction with 3D objects and is able to take in a wide variety of workflows, making the engine suitable for art sector related projects with multiple skilled disciplines, e.g acting, directing, lighting, sound, etc Unreal Engine A real-time 3D creation tool that is used for realistic visuals and immersive experiences. Originally used by game developers and creators, now it is expanding into virtual production use in film and commercial video production.

V Variable A real-time 3D creation tool that is used for realistic visuals and immersive experiences. Originally used by game developers and creators, now it is expanding into virtual production use in film and commercial video production. Viewpoints A technique in theatre and dance that thinks about movement, gesture and creative space. First developed by Mary Overlie and then adapted by Anne Bogart and Tina Landau. Virtual Avatar A visually designed representation of the player in digital reality Virtual Reality Virtual Reality or VR is used to describe anything that is not physically existing, e.g playing a match of virtual tennis in a fantastical world where the floor is lava or where dragons are real. In theory an idea is virtual, however for the purpose of the work, we use Virtual Reality to refer to the digital world created by computer graphics and viewed through monitor screens or virtual reality headsets such as the Oculus Quest. In 2021, our interaction with virtual reality is limited by physical input devices such as controllers but there may be a time in the future when we are able to interface directly with the virtual world. Future interfaces may incorporate our full being or mind, e.g our senses, into a connection with the digital world, thus enabling us to interact with virtual objects as if they were tangible objects (at least perhaps to our mind).

X Y Z

G lossar y of Terms

III

SESSIONS

February 27, 2021

June 22, 2021

July 02, 2021

Collaborators meet at

Architects of Realities

Art x Tech Lab Briefing

Session #1

Session #3

June 04, 2021

July 01, 2021

NAC Art x Tech Lab

Architects of Realities

Commences

Session #2

TIMELINE

August 30, 2021

September 26, 2021

Architects of Realities

Session #5

Session #7

August 10, 2021

September 13, 2021

October 04, 2021

Architects of Realities

Session #4

Session #6

Session #8

SESSION #1 June 22, 2021 // The Doodle People Headquarters

Timothi Lim presents VR / AR Technologies

TIMOTHI LIM PRESENTS VR / AR TECHNOLOGIES Created a Google Slides presentation with pictures and descriptions that could form the basis for discussion or alignment regarding Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) technology and how that can connect to theater or artistic processes. The presentation introduced the three different technologies via a what, how, why strategy and aimed at aligning the entire team on the current use cases and potential for the technology. Most importantly, the terms would now be known by the entire team and thus we could build upon our understood conventions of the technology itself without confusion. A key element of the presentation as well was snippets of “creative technologist mindsets” that I snuck into the presentation. This mindset blurbs like a slide with the title “Empathy warrants connection” allows the team to peer into how I view the technology and its use case as an expression of our human condition. For example, here I believe that Virtual Reality as a medium is viable to create instances of empathy as VR players are highly immersed into the virtual world and thus feel a greater connection to virtual objects in comparison to AR or MR. Session # 1

SESSION #2 July 01, 2021 // Nine Years Theatre

Joanne Ho presents Projection Mapping

JOANNE HO PRESENTS PROJECTION MAPPING This was an exploratory session where we played around with the technologies of projection mapping. Projection mapping is a technology that uses video projectors to shine light on any surface that isn’t simply a flat screen. It has become more popular as a way to create visually immersive environments, transforming a space because of its flexibility in turning anything into a visual surface. These surfaces can range from built sets, to buildings, and even to landscapes.

Session # 2

First I demonstrated a few projects that I have done with projection mapping, some of which I mapped myself, and some of which I had the privilege of working with professional technologists. Then, I had prepared a few TouchDesigner (visual coding software) patches for us to play with. It required a camera, a projector and some bodies in space. The first one we played with was a simple interactive experience, where if you stepped within the white box, the white box would turn off, and the other side of the screen would turn “on”, and vice versa. The second experience was a projection of the bodies in front of the screen but with a lag. These were some experiments that I made to quickly show them the possibilities of using projection mapping with performing bodies in space.

SESSION #3 July 02, 2021 // Nine Years Theatre

Cherilyn Woo, Khairul Kamsani, Nelson Chia & Thong Pei Qin present performance methodologies

CHERILYN PRESENTS SPACE What I’m interested in looking at is performativity in space and the performability of space. The process of performing in spaces and how the space performs for us to perceive. Now that we’re looking at different levels of reality, how does the methodology of working with space and performers change as we transit between each level.

Space is more than just it’s architectural elements, it’s also about how people behave and move. The framing of the space with a performance essentially requires three things. Space - The stage A Person - The Performer Someone watching - The audience

Session #3

Theatre Venue: There’s a certain performativity the audience does because the physical space performs a certain way. How does this change if I am a virtual audience? Or maybe if the venue is virtual, how does one influence the physical audience? The Stage: Often in theatre and as a director, we focus on building the world of the play. This usually means a couple of things: The colour, the feel, the textures and the essential elements. It’s not just about the physical elements of the world but the sensorial and philosophical elements of it. This world is a space that connects performer and audience. And that’s this idea of a commune. During a live theatre performance where mixed media technology isn’t used, the space is manipulated by space, lighting, design, sound. Even the actors can help. Where they are placed, how they stand, how they move across the space. These are all ideas of how it can create the idea of space. Theatre exist in the idea that the power of the imagination will be so strong to manifest the same world. It will be interesting to investigate how to bridge the two worlds (Physical and virtual) even closer. Exploring the idea of perception of depth not only aesthetically, but philosophically. How do we inspire a world for the performers where there can be different realities?

When we enter the virtual realm, what happens to the performative body, how does it help signify a different world or the same world? For the language between directors and actors, the physical world and the virtual world are both performative spaces, they are both the “world of the play” hence they are one unified entity. A unified entity with different limitations. But in the performance is it meant to feel like a cohesive world? How can we come up with a system of working between directors, designers and actors to codify different levels of reality and vocabulary.

Session #3

PEI QIN ON TIME This was an introductory sharing session on guiding everyone in discussions and two main exercises to gain more awareness of Space and Time. As a theatre-maker who enables an actor to create a textual and movement score, it is important for the actor to train their sense of time. Seeing the actor as an athlete and replicating exact moments and timings show after show, we questioned the pros and cons in pursuing accuracy and precision in the actor’s delivery of live performance, and discussed the fascination in imagining the human body as a machine. Inspired by our artist-technologists’ previous sharings on tracking live presence with technology, I introduced the concept of “Phenomenology”. It is a live presence, a given-ness in a real authentic self and quality an actor or actant brings into a space, and usually keeps to its own laws in the theatre, defying the cooperation of creating illusions, fictions or manipulating realities. I am interested in playing with that duality of an actor’s actual presence (with all their given qualities) and make-belief (embodying a role or character). In the rest of our lab, I proposed investigating the tension between what is imagined or fictional, and what actually is physically present on stage, when navigating multiple layers of realities.

NELSON ON FICTIONWORK Since 2017, I have been leading actors from the Nine Years Theatre Ensemble in a research project on actor training methodology. The research is title Body Paradox: The NYT Actors’ Work. It is essentially a series of approaches to rehearsal and performance. Fiction Work is a segment of the research that focuses on the use of “Fiction” in helping actors create performance.

KHAIRUL ON MEISNER The Meisner Acting Technique was developed by American theatre practitioner Sanford Meisner from 1935 and is still used today in leading acting schools and professional training studios. The core of the technique is to focus on the participants’ scene partner and pursue authentic “behaviour under imaginary circumstances”. This approach was one of the techniques we selected to work with in our research as it draws upon the environment of the user to generate physical action and even text; the projected, virtual and augmented realities would then provide stimuli for experiment participants to procedurally generate movement and speech. Session #3

Given the short on-boarding period for the workshop participants to learn the technique, I taught the basics with some skips in the actual process required to learn and retain the technique. Moving forward we started most sessions with a warm up with some of the Meisner exercises as a way to stretch the body, mind and voice. The main exercise we worked up towards is the ‘Level 2 : Physical Observations Repetition’, in which pairs would take turns verbally calling out physical actions performed by their partner in that very moment- from a nod, leg shake, laugh, fist clench or even a breath, the objective is to ‘make real’ what is observable by verbally calling attention to it. Later we attempted to apply the same principle and process to participants, ‘making real’ the virtual environments that they would find themselves immersed in. Our video documentation can demonstrate the level of success (or not) that we found in applying the Meisner Technique among various layers of realities.

SESSION #4 August 11, 2021 // Nine Years Theatre

Jam Sessions

ITINERARY 6:00 - 7:00

Jo & Tim Tech Setup

7:00 - 7:30

Khai Meisner Technique (generate procedural text and gestures)

7:30 - 8:00

Orientation (to blueprint of what/how to jam, everyone gets used

to viewpoints kinaesthetics jamming, 4 actors to demo)

8:00 - 8:30

Play 1 (Jo’s sound and projection tech as a fellow player first)

8:30 - 8:45

Talk

8:45 - 9:15

Play 2 (Tim’s VR tech joins in, with pre-designed elements)

9:15 - 9:30

Talk

9:30 - 10:00

Play 3 (Tim takes a more active role in being Puppet Master here

with his VR tech to manipulate Avatar)

10:00 - 10:30

Wrap up, pack up

Session #4

TRANSCRIPTIONS OF OUR SESSION TALKS AFTER PLAY 1 JH:

I saw that you (NC, CW, PQ) were inter-reacting with the tech,

but I wanted a more deliberate spatial arrangement. Maybe we can move around the boxes, maybe change some of the ways you look at other people in the space. You guys really interacted with the sound. The sound really added a lot. It felt spatial because of the echo. TL:

There was still very much focus on the visuals. At one point there

was 2 Pei Qin on the box, that changed up the monotony of it. KK:

I feel that there’s still a lot of structures holding the actors back.

So I want to know how they feel about the amount of vocabulary to play, and also where their focus is. And I know that we did restrict their focus. And I think my question then is, would be to ask the actors if we can focus on that as much as we have, and then move to focusing on themselves, and then with another person playing with different layers of reality? I think we’ve taken a good first step, I think in establishing the fourth perCW

Cherilyn Woo

son or the fifth, actually, there’s like 11 other people that know what these

Joanne Ho

technologies, right. So to be able to listen to them, but also be able to

Khairul Kamsani

ignore them, and not to overpower them. The audience or whoever can

Nelson Chia

just watch that as a backdrop in silence. Because I think his audience

Thong Pei Qin

members we can choose also right, where do we want to focus? So I

Timothi Lim

think that is what I would like to see next.

NC:

What is interesting to me is that after what Jo

was doing in real time. So it fed as a loop because then

suggested, that we focus on three things, but then the

she copied onto the movement. So it was real (PQ and

three things are actually... Us. What I see on the screen, is

CW as physical bodies). Real (CW reacting to projection

essentially, what is PQ is doing. She affected the screen,

PQ on one side). But lag (physical body to PQ projection

the dust thing, and then so I was interacting with the

on the other side). Yeah, but then it became one loop. So I

dusting. And so, my image becomes something that

actually got confused. Like, which side was lagging for like

Cherilyn is describing, and PQ was looking at Cherilyn

a split second. So for me that was quite like mind blowing.

in a way. So that three realities, like I said, three levels of

Yeah

reality became connected. So, I see the tech coming in first as a intermediary, but I can’t see PQ, maybe PQ in

NC:

I didn’t know she did something to the speed, I

Australia, for example, but then I sense her. So, she is re-

couldn’t see. So you not only took the sound and played it

acting real time. And I see it as a dust form or some other

back as a kind of action, but you also the speed.

images. And hence forth for the two of your experiences. So one is that the tech becomes the kind of intermediary

JH:

I was gonna leave it, but Khai came and whis-

but then when we speak something we have a sound,

pered in my ear.

and then it was picked up, and then you decide to playback, then that tech, it’s not just as an intermediary. But

PQ:

So I would say that tech manipulated time.

JH:

Yes

CW:

And it was quite visceral actually

little bit. But I think something interesting is happening. We

PQ:

Because I was really real time all the way. And nel-

could think about redesigning the setting up or changing

son is real time as well. I started laughing because I kept

the projection or whatever. But yeah, the levels are certain-

this, I was supposed to just focus on Cherilyn. Right, but I

ly coming out, but what does it mean?

could see what Cherilyn was seeing. Which is me lagging.

the Tech has a different vocab. I have recorded sound playback, but it comes in and then it affected all three of us, we could choose to react. So yeah, I don’t know how to describe, the 3? 4 levels, whatever is happening down there. Yeah, but I agree with Khai, I think we held back a

And, Cherilyn was stalling a lot and she was glitching as a CW:

For me, I found that my viewfinder was only in that

real person.

small box. So my world so to speak, was just that rectangle for quite a while, but then about as I was trying to get

CW:

Because I had like an out of body experience. I was

acclimatized to that so like how do I react with this? How

seeing two (versions of PQ) and PQ was live commenting

do I find things to react and respond to it? Then the turn-

me. What she was commenting was what I was doing in

ing point for me was when you switch the reaction times

the past. And I was reacting to what was happening to her

on both sides. And then suddenly for me there were four

in the past. So the feeling was actually pretty visceral. So I

people to look at. Right. So that’s one screen. But also I

knew when she said I was glitching. I was literally glitching

was very aware that they are outside. So there was stuff on

because my body was overwhelmed.

the outside, which is this sound and movement in the air that I can feel that I can react to so that’s one reality, then

KK:

Did you try to re-orientate your present? Like

I have my window. But then as in the window, there are

where am I?

two different realities for me. And so, there was actually one point where I know what speed PQ, who was the

CW:

Yeah, and where’s my playing field now? Cause I

closest to me, is really moving at which is actually on

know I have two screens, but I’m like, which time zone do I

one side of the screen, but the other side of the screen

respond to?

is a different speed. And that was one moment where I felt like, I was stuck in a time space warp. Because I was

PQ:

copying her(PQ). And she was she was narrating what I

of time. So that changed my reaction. Because for me

Session #4

Then when the sound came in... Echo is a layer

is always real time. So then I was also focusing on CW

type? Or are we doing the realistic type but realistic also

glitching. And then her echo kept going on. So I respond-

can be warped?

ed to echo. So I just copy your Echo. \ So that it will be prolonged, but I tried to stop. Cause Nelson kept break-

KK:

Nelson was Cherilyn’s Totem. I think for me it’s

ing out of the form, you tried other rules. So I’m inspired

like – who is who’s totem. What is going to ground them

by that and tried different forms of echo, so I will try as

in that reality. What is my anchor to be in the delayed

speedy a sharper sound. Yeah, but I’m still actually live

space, what is my anchor in the dust space what is my

commenting on CW. So I’m playing with sound and try-

anchor in the physical space, and I think maybe that

ing to break that. I felt that Cherilyn was trapped.

might be an opportunity to guide actors? This is your totem; this is the reality you can play with. But then as

CW:

The thing that actually pulled me out was Nelson

a director, you can kind of map out what is the mise en

verbalizing. Because I know that for like, when you(PQ)

scene. What is the audience going to see based on your

are commenting on me, it was different because I feel

coordination of totems and displayers of reality?

that it’s part of the world that I’m trying to observe. But for me now, NC’s voice is totally out of that world. And I

JH:

But like it’s like now when realities are overlap-

know for sure that is reality. What I was experiencing in

ping, like how people are trying to make real space like

the Time Warp reality, I then was like okay, if that’s my

with virtual reality, whatever people are trying to make

only reality then i feed of that which is what then that

real space in virtual reality. It’s quite relevant. I mean, I’m

means to be inspired to play with the sound because that

just talking about real world and how applications are

was my anchor.

being made, how do we ground ourselves. There are so many different realities are being made and the news,

JH:

That’s so interesting because you’re (NC) the

media and whatever

voice of reality for CW. But you’re actually you’re just looking at a wall of like what PQ affecting on the screen

PQ:

but then you’re(PQ) actually the one who is affecting

grounding right? I felt like, because the next one we go

physically a lot of it but you’re not even focusing on the

for is mediums. So we as actors, we need to ground our-

tech which is like another grounds for reality but no one’s

selves on something and the directors can help us with

actually watching you but like you’re saying things be-

that. Jo was also directing us as well. You were feeding

cause you know you’re affecting things but since you’re

us things, so you actually self-directed. Which ties back

only looking at the wall, the way that you’re saying things

to our director-actor process. So as a director later, for JH

is not deliberately affecting anything.

and TE, you can also use us as mediums as well as actors.

PQ:

JH:

NC did affect me though. Because he would react

So when we mention totem which is sort of like

Khairul was also directing me. I was watching you

back to the dust and I saw it coming towards to me. Al-

guys playing space. But obviously, I can’t read your minds.

though my focus was supposed to be just on CW. Because

But like the movements that you guys were making. I also

I sense that my body is moving the screen. And I was in the

couldn’t really read like, I couldn’t read that you (CW)were

middle of the two of them.

glitching. But you (PQ) obviously read that she was glitching. And like the way that you (all) reacted, it’s like, quite

NC:

Actually, the image that I’m looking at is the most

interesting. Like listening to all three of you discuss what

abstract. I keep hearing the both of you (CW and PQ) that

you were thinking. Because it really is like a whole world

has nothing to do with me. It’s just me and the dust. That’s

that is in your head.

why in my text it became something else. Like just rising colliding and just describing what is lowering growing just

NC:

Khai mentioned totem, which is from inception.

use a description like this. Which brings me to articulate

The totem is a tool for us to ascertain whether the guy is in

that these different levels of reality, could also have dif-

a dream or not. But I just want to point out that in such a

ferent layers of expression like are we doing the abstract

situation, it’s interesting that I thought the technology just

now... I said, PQ, you could be in Australia right now. And

PQ:

You know, I felt like to me watching, it felt like you

I could be doing a live performance with you, if you are

predicted her next movement

transmitted through a kind of images, real time with what

CW:

you are doing... But how do I know that it’s really you? Jo

but she(PQ) had her(JH) delayed voice to the speaker.

She (JH) was speaking the narration in real time,

can just switch to a video that she has just designed. And all the while I thought I was dancing with you.

JH:

But I was actually delaying what I was seeing, too,

because I was waiting for my delayed voice to finish. And KK:

Lets talk a little bit about what we would like to

then I would say things that I was already thinking before.

talk about in our next play session.

Because I can’t say things while hearing my voice back.

JH:

NC:

As actors in the space, like, and also directors as

Maybe you should be wearing headphones with

well, like, how do you think the space could be rearranged?

music. But to me it felt, like, you(JH) predicted what CW

And maybe to like? And is the narrative that you created

did. And Nelson was very accurate with what JH was say-

for yourself that reacted to the tech? Is it like, interesting

ing. I felt JH and NC were super in sync, but CW was really

enough to do it again? Or like maybe you switch roles?

doing her own thing.

KK:

CW:

So are you doing movements based off her audio?

NC:

Yeah

PQ:

And the aim is to get the both of you (CW and NC)

I think the next thing is to place Tim’s VR in the

space. And that would rearrange who’s in the space and what is each of your totems. Do you(TE) have a preference of where you’d like to place an actor? TL:

Probably in the center. In VR we can set up that

to be the same thing.

they can walk around the boxes. But in the center they’ll be able to move around the space.

NC:

It won’t be totally the same, I will have a delay.

JH:

And I will always be delayed because I’m watching

For the next play, Anyone who is in the headset

will hear a sound that is different from the other two people

you and I have to think of the words to say. And then I’m

in the space. The space will continue to play the sound we

waiting toward my delayed voice to finish talking so I can

had before.

talk. So I actually was like, triple Delayed.

TL:

There’s three layers of sound, there’s Jo’s delayed

sound, Khai’s premade sound, and the VR has it’s own

I was observing the virtual world. When you said

Cherilyn was punching the air, she was actually dancing.

sound. JH: PQ:

We can still keep the structure from the previous

It looked like she was punching something really

frantic?

play. Where we each focus on one person

AFTER PLAY 2

PQ:

(To TE) what was it like being a manipulator?

TL:

I wasn’t manipulating much. The figure inside was

a kind of cloud that was human shaped. PQ:

Okay, so what I observed was actually Jo wasn’t

And so like, some things I can do is jump over or run

really describing what Cherilyn was doing. You (JH) were

around Cherilyn, but I can’t really impact her, but it’s her

predicting her (CW) movements.

choice whether she wants to interact with me. But most the time i was pretty far away just like observing and using

JH:

I was delayed because of my delayed voice. I kept

delaying because I kept hearing my delayed voice.

Session #4

the camera.

JH:

Yeah, but then describing motions constantly was

NC:

We can think about deepening what we have

like quite. It was really difficult. I don’t know if it’s because I

found, but also adding on or refining on. Maybe the spatial

never talk about motion.

arrangement wasn’t that ideal. I feel what we found now we haven’t fully –

PQ:

It felt smooth to me. JH:

JH:

(to NC) how did you feel being directed by me?

This is the first time we’re really playing with di-

rection, which is so great. Like two different types of like, playing sessions. But so many ideas are different.

NC:

It wasn’t too bad. I knew I couldn’t be very specific.

Because you say punching in the air, is it here or here. I

PQ:

just kind of went.

wonder if Khai’s soundscape affected Nelson?

JH:

NC:

It’s so difficult to be really specific.

I have a question. Khai’s soundscape came in. I

It definitely did. Also, with the narration. The nar-

ration took on another layer with the sound as well. But I KK:

How did you feel Cherilyn?

thought I’m more comfortable in the space moving with the narration and all those things so like I kind of felt like

CW:

I knew I was kind of isolated in a different world.

I’m more in the space and more at ease.

The first phase for me was kind of adapting to the world. At that first part, I was quite fascinated by all the different

PQ:

It felt like it blended together somewhat. The nar-

things and that can be going for quite a while. And after

ration and the sound scape. Did Jo hear that?

that, I was like, Oh, actually, I got nothing else to do. So then I had to create my own, I guess kind of fiction in a way

JH:

I could but I was so focused on like saying words

in which point, I decided to pick up that objects and start

to describe the actions actually the sound scape was in

creating shapes. Then after that, actually, the man was in-

affecting me at all because I like totally blocked it out. Like

troduced. And he was starting off quite far away. For me

the only thing that was distracting was hearing my own

playing off that I was like, Oh, this, there’s another entity in

voice because I recognize it. And like it was mirroring me

this world. Let me try and get his attention. I’ll try and con-

so I kept - I was also like glitching a little bit because I was

nect. And then after a while, this other person didn’t quite

like, trying to say something and then I was waiting for my-

interact with me in the way that I thought I was interacting.

self to finish and yeah,

And then I did have a dance, I did try and dance, but also the same, like he didn’t join in the party. As he was playing

NC:

The soundscape was crescendoing. But what ac-

with the men trying to like jump over me, And it came quite

tually happened as a performer, we naturally latch on it as

close. I played off that in a sense of I felt quite intimidated.

an anchor, the same way Cherilyn latches on to the man.

Then, this man became something that was a threat to me.

That’s what happens to performers. Even sometimes fur-

So then that’s when I became more aggressive because I

niture anchors as. So in the design, you may want to think

was trying to defend myself. So it was like the fiction kind

about what is the anchoring element that comes in the

of slowly came out of trying to find a way to continue being

middle or already in the beginning or changes. Actors tend

in this world, because I knew I had to still continue to play

to want to anchor things. Even in Meisner.

and be moving. And I couldn’t have anything outside that could help me . I could hear Jo narrating, but it was just

KK:

I just realized that we didn’t use fully the tasks that

murmurs. There’s nothing for me to kind of latch on to. So

we wanted to bring in like the place, events. ( Refer to Di-

then I had to latch on to the man.

rection for session #4 above) So I’m just curious like if we did it again and we had that fiction, that event and that

PQ:

Building off that, what if in the next session we de-

place, what would it bring?

signed something that could interact more with the person in the VR.

NC:

I was reminded of the Birthing when I was interre-

JH:

The totem is an interesting element that it’s like

acting with the dust.

that that’s being introduced.

JH:

PQ:

It’s a discovery today.

before. And then like the surreal landscape. I mean, like vi-

JH:

I really like that idea because like when we’re deal-

sually it’s the thematically surreal but like, what we’re doing

ing with multi layered realities Yeah, how does one incur-

is already surreal.

How does one like focus... When you’re in like, different-

I think that in a sense like this multiple realities

is already built into, like what we were trying to do even

when you’re really aware that like multiple layers of realities CW:

Actually yeah. So I first realized that I’m in this

are around you?

world and I thought that this world was limitless. Until I hit the walls and then I realized how small this space is

PQ:

What was your (JH) totem?

gressive. I actually thought some of the same space as this

JH:

I was really in a hole. I didn’t eeve3n hear KK’s

man(avatar in VR) but I slowly discovered the limits of it

soundscape. I’ve heard it when I first came in. And then I

then I’m like, why am I blocked? But this man is moving all

was deliberately blocked out immediately because, I was

around me.

so focused on my work and the tasks and like I didn’t even

and that actually influenced me to become a bit more ag-

look at Nelson on the screen. I just looked at you (CW). KK:

What would your ideas be for next time (Session

5)?

And like the lines, your actions, were my totem and then like I was constantly being distracted by my own voice. And on top of that, I was like, struggling to get the words

TL:

I think we can try one without the man perhaps.

out of my head

We can project what’s being done inside the VR and you can layer stuff over that as well.

KK:

If we’re going to use totem from inception, what is

our kick? The kick is what they use to get back into reality. PQ:

But then if you take away the man, then she has

no more anchor.

JH:

My voice was my kick, it kicked me out all the time.

TL:

It could be other stuff, like a dinosaur or a cat.

KK:

CW did you have a kick?

CW:

Because I knew the man is somebody from the

CW:

Actually, before the whole thing started, her touch

real world. So that’s why I latched on to it as an anchor. If

was my kick. Cause she touched me once during the false

maybe my soundscape was the same soundscape as theirs

start. When we ended, I could actually hear KK say end.

that can also be my anchor. I guess I’m trying to bond with the real world. Because the VR set is quite isolating.

NC:

My kick was actually seeing you (CW). That’s why

I half turned my eyes and turned around which you(PQ) JH:

So like anything that seems like it’s being affected

by someone in the real world could be a totem. CW:

That’s what I naturally latch on to.

TE:

The headset can feed the ambient sounds the mi-

crophone picks up on. PQ:

So TE was CW’s totem, and then JH was NC’s to-

tem, and then KK was also NC’s totem. Because JH’s voice and KK’s sound came in.

Session #4

thought I was in a trance.

REFLECTIONS At key milestones of our research project, we have designated points of self-reflection in writing for several reasons. To mark our own milestones as researchers in our relevant fields and towards aspects that we are less familiar with, to make a statement about what we might have newly discovered and even new questions that may have arisen. While our in-session group discussions after each ‘live exercise session’ allows us to think as a group on our next steps, discoveries and questions, a personal reflective text has allowed us to think deeper into our own personal crafts, beliefs, and theories of the related fields we are working with. After each milestone reflection, we read each other’s reflections to gain insight into individual personal growth within the team. Perhaps in doing so this might highlight new ideas in ourselves in discerning out next inquiry, objective and methodology. Key Milestone Points: -

Pre-research inquiry

Mid-term sharing reflection and inquiry

Post-Sessions reflection and future inquiry As each team member embarked on this process at different lev-

els of knowledge, experience, and inquiry of the related aspects of our research (theatre, directing, technology, virtuality, visual art or architecture), we have decided to allow the reflections not remain as separate personal narratives of each individual. The following reflections have not been curated for a singular group-voice or narrative, each reflection is the words and expressions of each individual member of the team directly of their own journeys, explorations, challenges and discoveries through the Architects of Realities.

Cherilyn Woo’s Reflections

ELEMENTS PRESENT Physical Space •

Three rectangular boxes in the space as physical tangible objects.

•

Three human bodies as well.

•

Architecture of the room

Virtual Space •

Projection of “dust” that we could interact with (refer to picture above)

•

Different landscapes projected on the smaller boxes

•

Projection of the physical space on two different sides of the taller box. One of which was in a different time speed. These projections moved in accordance to how the bodies were moving in the physical tangible space

•

There was also the landscape and “being” in the VR headset

Mid-sessions reflections : Cheril yn Woo

Throughout our session, the spaces remained

myself. It was as though when Time did not move in

as is, and what we varied was sound and time. Going

a way I was used to, my body didn’t know how to re-

into the session, my perception of reality was linked

act. The only thing I could hold on to was identifying

to the idea of space. Physical space to me was a tan-

what was real to me, which was the physical bodies

gible immovable space, virtual space to me was in-

of the others in the playing space with me, and their

tangible and movable. What I found extremely fasci-

voices which I could hear in real life. I had to be in

nating was that Time could design space as well.

tuned with my own senses to know where my baseline of reality was. I found that extremely important as

PLAYING WITH TIME

a performer. To focus on breath, the feeling of hearing something in the same space and not the speakers.

Where we manipulated time:

In that sense how I perceived reality as a performer

•

Sound: There was live narration that was hap-

was not by sight, but my touch and hearing. It’s inter-

pening, and there was a time lag on when it came

esting to note that when the realms of reality multiply,

through the speakers

it’s almost as if the eyes are not the one to rely on, but

Projection: There was a projection of the physical

your other senses.

•

space and different time speeds on the different projects created the sense that these projections

BEING IN THE VIRTUAL SPACE

could be portals as if one could traverse different times.

I was lucky enough to be in the virtual space. IN that sense I was locked in a completely different

The sensation of using time as an element of

space. Even though consciously I knew I was in a

space was visceral. It felt immersive and tangible as a

physical space, my sense of sight and hearing were

performer. Suddenly it was like I was in a space where

locked in a space that was not coherent with the

Time was non-linear. Time was a circle in which I

space my skin was sensing. I had no choice but to

could run both clockwise and anticlockwise. Surpris-

lean into it.

ingly this cause a sudden paralyzing sensation within

For safety of myself, there was boundaries

MOVING FORWARD

drawn within the Virtual space. This week felt like an induction into the posSomehow the virtual space although not

sibilities of space. My findings were all drawn to ex-

“real” felt more focused that the reality of the first jam

ploring the landscapes and design of the space itself.

where we had different projection son the boxes. Per-

That was dizzying at times. The senses are over-

haps it was easier because I only had a single target

whelmed with information when there is more than

introduced slowly. First there was me in the space,

one reality to juggle. Reality in itself is hard enough

I was given the time to get to the know the space

to juggle, even harder when there’s more than 2 re-

within the virtual, to be orientated on the laws and

alties to juggle. I wonder of labelling and signposting

boundaries of the space. What could I touch, where

the realities for the performer would help prepare the

could I move, what was the sound of the space in

body to be immersed in it. IT felt as though I was just

relation to the image I was receiving. Then after that

getting used to it.

a “being” was introduced, which was Tim as an avatar in the virtual space interacting off me, but he was controlling his avatar through the computer. Then he became my performer target. His avatar felt free because his avatar could jump over me and run in spaces that were not accessible to me. (They were not accessible, because of the boundary drawn in the virtual space to protect me from bumping into walls or other bodies in space) I could explore the relationship with him. Through that it helped to build my fiction that perhaps I was a character trapped in a space perhaps set up by Tim’s avatar. Suddenly the avatar turned form something I was trying make contact with to someone who was my captor. He became an enemy, someone I needed to get rid of. This helped to build an entire fiction as a performer which then led me to be even more immersed in a fiction. I went from exploring the space as a performer, into a character in narrative. It was as though even though I was in a virtual space, it seemed like an easier access into a fiction of a character than being in the real space where my body was still trying to process the spaces and realities in front of me.

Mid-sessions reflections : Cheril yn Woo

Joanne Ho’s Reflections SESSION NOTES We used the Meisner technique for Play 1, with only the projection mapping of interactive particles on the wall and realized afterwards that there was too much freedom of what everyone is reacting to, since the actors kept facing the wall and were too focused on the particles. We felt the technology was hindering the performance and fiction within the actor’s minds. I felt like we needed more dynamic reactions, and the images of the skies as “portals” as per our SPACE theme of surreal and magical were not as meaningful as projecting live video captures. So for Play 2, I suggested for Nelson to react to the particles, Pei Qin to affect these particles (in the view of the Azure Kinect) while reacting to Cherilyn, and Cherilyn to watch two video captures of Pei Qin being projected onto the props (rectangular blocks).

In the midst of Play 2, though it was not part of our plan, Khairul suggested that one of Pei Qin’s virtual copies that was projected on the prop be delayed, which seemed to create more content for Cherilyn to react to. Then, Khairul suggested for me to introduce sound capture of the space and then output these sound clips as their echoed versions through the speakers. Once the sound was introduced along with Khairul’s music in the background, it created more actions from the actors like playing with the spoken word by elongating syllables or raising/lowering the pitch of their voice. I really enjoyed this moment when the soundscape became a warping unseen element that affected the atmosphere and the “jamming” feel of this play session, as we added on elements as we were actively feeling out what was happening on the stage. It also helped to have a second eye that gave me direction while I was focused on the tech. In the discussion of Play 2, which contained many revelations on playing with time in the actor’s mind and fiction of the space. At this point, I felt like we were getting at important discoveries with our experiments where technology was actively affecting the actors’ performances.

In Play 3, we continued using the Meisner

was a build up of the last, which is a more organic

technique in Play 1 and the “chain of reactions” for

method to develop more ideas and definitely less rig-

actors in Play 2. We introduced Tim’s VR headset

id like following a set of rules to begin with. I would be

where he had created an environment in which Cher-

interested in taking our reflections and themes that

ilyn would inhabit, and only she could hear and see

emerge from each session to elaborate on in future

what is within this space. I was assigned the role of

sessions, (e.g. Glitch movements, inhabiting multiple

narrator, where I narrated Cherilyn’s action as it was

spacetimes, human-machine continuum, representa-

seen on a screen (an outline of her figure using the

tions and boundaries of realities.)

Azure Kinect), sounded out by speakers with a slight delay. Nelson then tried to act out my narration. We

I’m extremely intrigued at how we can use the

realized afterwards that while Nelson reacted in time

human-machine continuum to understand inhabiting

to my narrations, my narration didn’t match up to

multiple spacetimes, sparked by Cherilyn’s glitch in

Cherilyn’s live actions. I reflected upon the distraction

Play 2 and Nelson’s and Khairul’s mention of the To-

I felt when I heard the echoes of my own voice, since

tem and the Kick, which could also be included as

I kept waiting for the echoes to finish before starting

elements of the topic in representations and bound-

to narrate again. So the narration sounded out into

aries of realities. In a world where time is thought of

space was, in fact, doubly-delayed. I thought this was

linearly, and space is thought of as a place in which

another interesting effect that technology had on the

one is presently inhabiting, technology can provide

minds of the performers, especially when I was also

windows for opportunities to sense other times and

an actor. I finally got to get a glimpse of how exactly

spaces. It excites me to think that this type of actor

being “in the moment” and creating a fiction with a

training can also allow any layman to understand in-

focus in mind was so important to actor training.

habiting the multiplicities and layers of different times and spaces. I believe this is quite important as we

SESSION REFLECTIONS

are increasingly, in our daily lives, inhabiting virtual spaces that bring together multiple spaces when we

I learned so much from this session, includ-

interact with other “bodies” online.

ing what actor training could provide as a learning opportunity for someone who had never thought

Moving forward, along with creating multiple

about the process of acting on a deeper level. I’m also

levels of spaces using visuals, I believe we need to

learning a lot from the vocabularies that directors use

play more with the sound aspect of our sessions, as

when describing their experience on stage in our dis-

it is quite an effective way to develop a fictional story.

cussions, hoping that by the end of this lab I’ll be able

In this session, I’ve also realized that not only can the

to understand and utilize some of this vocabulary to

technologist be the “puppet master,” the tech itself

describe elements in my own practice as my art re-

can be the director. Roles can be interchangeable if

lates to space, movement and fiction.

we’re already thinking of technology as a player in the space of the stage.

Though we had planned to incorporate the three elements to each session, I felt like they were just starting points to work with for our first jamming session using tech and actors on stage. Each session Mid-sessions reflections : Joanne Ho

INQUIRIES FOR NEXT SESSION •

Actors actively create their own reality (“I sense danger”; “I sense a moment of calm”). Actors as directors, technologists/environment as puppets

•

More sound as direction. Could we use a live or past recording of another space (maybe even one of our past narrations, or past discussions) as an element in our play?

•

Playing with a two room set-up. Rooms can also be created using technology (like how Cherilyn was in her own isolated space)

•

How can we design human-machine continuums to help people feel and learn certain types of movements and interactions?

•

How exactly does the additional introduction of technology hinder or aid in the process of actor training?

•

More deliberate experimentations with Totems and Kicks

THOUGHTS •

Possible to transcribe our discussions and include them in our documentation booklet? It’s interesting to watch but we can also highlight snippets of important revelations in text to use as reference

•

Calling out topics that emerge from each session and elaborating more on those as inquiries for future sessions

•

Glossary of terms we have created during the process of our sessions (like the topics we’re highlighting as what we deem important)

•

Glossary of existing terms - like describe what “Meisner Technique” or “projection mapping” are

Khairul Kamsani’s Reflections This reflection journal entry (22/08/2021) is after four Workshop meets, the first two being technical presentations, especially for the directors/actors, by Timothi and Joanne of the potential technologies that could be at play for our explorations. The third being a session in which the directors of the group shared with each other, and importantly, the creative technologists an introduction to our practice and enquiry as a theatre director on this project. While we might not have ‘produced’ any technology or ‘creative work’ within the first three sessions, I have found that those were vital in the communicative process for this research if we were to achieve as we have in the fourth session. Considering the basis of this collaborative research project, I believe that we have achieved our goals so far in aligning our interests, positioning our capabilities and limitations to employ fruitful explorative play and discussions.

WORKSHOP 4 PREPARATION In our original proposal to each other, we had proposed an overarching workshop structure that, after sharing our own practices and interests within the first 3 introductory workshops, we would be able to assign the next 6 workshops to be led by a member of the group. This democratic division of workshop time seemed to be a fair method in ensuring that each of us had an equal amount of time to explore our own research enquiry while the others supported the ‘leader’ of that session. By the end of the entire research programme, we would have had six specific individual research enquiries, methodologies and outcomes to report with. In the period between Workshop 3 and 4, I felt that there was a lack of direction if (for example) I were to be the first person to ‘lead’ the fourth workshop with my own inquiry and methodology; I felt that I would have been ‘jumping the gun’ if I had a workshop to lead that had the technology and ‘actors’ at play, to apply these elements in a specific fashion to (or attempt to) answer a very specific enquiry.

Mid-sessions reflections : K hairul Kamsani

To alleviate this, I suggested to the team that

The collective was in agreement with this

since we were to document this process, it might

structure and pairing when we met on Zoom to re-

help us to pen our current individual impressions and

fine a technical direction for the upcoming fourth

inquiries and share it in hopes of finding similar inter-

workshop. That discussion revealed that we had only

ests of inquiry. It turned out that while we managed

begun to highlight key elements towards exploring

to share our interests and capabilities in the first three

‘communication between physical and virtual bodies’.

workshops verbally, it was when we wrote in further

Each pairing represented a ‘pillar’ of performance; Jo-

detail (a +-1000 reflection) could we access specific

anne and Cherilyn had an interest in ‘space’, Pei Qin

interests of our team members. We were also able

and Khairul inspected ‘movement’ while Nelson and

to lay out the ideas, notions and questions in front of

Timothi investigated ‘fiction of self’. To provide struc-

us and organize it into a coherent through line which

ture to the fourth workshop, we assigned each pair

hinted potential directions of process that we could

a ‘building block’ for Joanne and Timothi to create

take. (Please see Annex for detailed reflections/inqui-

in the virtual space. Team ‘space’ would offer a lo-

ries)

cation/environment/atmosphere, team ‘movement’ offers a movement vocabulary and team ‘fiction of Having read through our reflections and in-

self’ to provide an event/given circumstance. In hind-

quiries, attempting to narrow them down I managed

sight perhaps Timothi and Nelson could have offered

to pair them based on common interests.

a character instead of an event though it is rather a moot point given that our Jams did not seem to in-

I proposed that under our collective research inquiry, we could be subset into 3 research inquiries

clude the notion of a ‘birthing’ event that they had proposed.

that have their own investigation within any same moment while we might be experimenting in situ. I

‘MOVEMENT ’ & ‘ TEXT ’ METHODOLOGY

proposed:

DEVELOPMENT

AoRs Research Inquiry: Investigate opportu-

Pei Qin and I tasked ourselves to offering

nities and limitations that AR/MR/VR and Projection

guidance in the aspect of Movements that we could

Mapping technologies present to theatre making

perform in the Jam. Where would we draw the move-

processes, namely-

ment vocabulary from? Should/could we pre-determine what and how our bodies move in the space

A) the actor to a perception of ‘the impossible other’;

when interacting with generative technology? We

the fictional and actual selves [Nelson & Timothi]

started with a discussion on Meyerhold’s Biomechanics as an option, potentially teaching our co-re-

B) immaterial and material space(s) for performing

searchers the ‘etudes’ from Biomechanics for them to

(corporeal and virtual) bodies [Joanne & Cherilyn]

perform in the Jam while in situ with technology and various levels of realities. However, we realized that if

C) an actors’ communion between physical perfor-

we were to do that, each performed ‘etude’ would be

mance to a simultaneous virtual performance [Pei Qin

a solitary performance in situ with the technology at

& Khairul]

play (also considering that we would have to 1) brush up on the etudes and practice it ourselves and then

2) teach it to the team, this on its own takes a con-

and the outcome in the post-Jam feedback/discus-

siderable amount of time); the interaction that a per-

sion documentation. To conclude this reflection, I will

former of an ‘etude’ would be an engagement with the

posit next steps to take for Workshop 5.

beginning to the end of the ‘etude’, focusing on their internal/external movement rather than their body

In summary of our verbal reflections at the

in engagement with the technology around them.

end of workshop 4 and in preparation for the Mid-

Thus, we decided on applying a ‘procedural move-

Point Sharing, our collective has realized that even to

ment methodology’ which means that any movement

apply ‘character fiction’ and ‘place/locale’ in the ex-

generated by the ‘actor’ within the jam is performed

perimental workshop is a step ahead beyond what

live, organically and in engagement with ‘an other’.

we were required to engage with. Cherilyn, Nelson,

As an experimental process, I hypothesized that ap-

Joanne and Pei Qin engaged with the generative

plying the generative nature of text from the Meisner

technologies through procedural utterances without

Technique with the Viewpoints lens towards genera-

a layer of imaginary character or location. The ‘unre-

tive physical movement could have been the answer

al/fictional’ characters, circumstances and locations

towards incorporating ‘text’ of the body within the

created by the virtual spaces ‘appeared’ on their own

Jam. The notion of ‘text’ had to be reshaped among

to them. Do we need to supply ‘character’ and ‘loca-

us as its deconstructed understanding of ‘text’, which

tion/circumstance’ for these experiments? Given the

makes it inclusive beyond ‘words on a page’ or ‘vo-

results so far, how do we adjust our expectations of

calized utterances’. For us, ‘physical movement of the

Workshop five and design for it (technically, physical-

body’ and ‘vocalized utterances’ was text in its most

ly and conceptually)? I would propose that we need

basic form. We also included the contextual meaning

to continue jamming with different layers of realities,

of these physical and vocal utterances within the Jam.

positions of engagement as our ‘selves’ to further de-

If we were working with participants unfamiliar with

velop out notions of ‘totems’ and ‘kicks’ (discussed

Viewpoints, we would have had to teach the frame-

in the video documentation) for performers and

work and vocabulary that would give rise to physi-

technologists to have a mutual point of engagement

cal text, but since the bodies in space were already

when developing a ‘jam space’.

attuned to “Viewpointing”, we only had to teach the Meisner Technique to them as a pre-workshop requisite in order to give rise to vocal utterances as much as physical utterances. WORKSHOP 4 AND 5 SETUP As part of the Setup on the day of Workshop 4, I taught the Meisner Technique in more detail (having done a preliminary session of it within Workshop 3) now knowing that it is a key element to provide ‘vocal utterance’ ‘text’ from a procedural process alongside ‘physical utterances’. Please see the video documentation Annex for more detail on the setup Mid-sessions reflections : K hairul Kamsani

Nelson Chia’s Reflections STRUCTURE AND RESPONSE Before we start on our first “jam” session on the 10th of August, the group came up with a series of pointers such as space, circumstances, event, etc. To serve as a structure to anchor our exploration. I welcomed the idea at that point as I felt a structure will help us understand the exploration better. However, not long after we went into the actual exploration, I was pleasantly surprised to experience how the body and mind would react quite organically to the technical elements that were introduced into the space, to the extent where some of the pre-conceived pointers would give way to more authentic responses that emerged from the interaction between the performers and the technical elements. We realised this had to do largely with the fact that most of these technical elements were also generated “organically” by the tech artists as they interacted with the performers, directed them, or joined in as an “actor”. – this helped us see that what we had designed and experienced was in line with our initial hunch to create a set-up where we might inquire on how performers would react to the existence of different levels of realities.

LEVELS OF REALITIES The different levels of realities presented during the exploration were the most interesting thing for me in that session – seeing how different bodies were living their experiences in the space, all unique and organic, but always inter-connected as one influenced the other. Nevertheless, that also brought about another interesting question: If you were a performer, how would you know what you were experiencing was real (or in real-time)? Since everything that was fed to a performer in real-time could be switched up with something pre-recorded, how could one be certain of the reality that one was in. In other words, the exploration of realities had included the question of ‘What is real and how do we know that?” Or perhaps, there is an opportunity to re-think what does the notion of “Reality” mean to a performer.

TOTEM: OUR ANCHOR OF REALIT Y In the exploration, we also found that performers tend to search for something in the experience that seemed “certain” – something more familiar, that provided a context, a sense of space, time or direction, something that could possibly serve as an anchor. This was in line with a performer’s instinct while in performance. KICK : THAT WHICH BREAKS THE ILLUSION OF REALIT Y While an anchor or “Totem” kept the reality for the performers, a “Kick” – something that distracted the performers – would break their experience. This in turn brought about the notion of “Reality being an Illusion” – a paradox of performance! So, a performing living in the idea of “reality being an illusion” can also be said to be experiencing (inversely) a “Fictional Reality” A further question that we might ask is, “How do we understand the reality of illusion or fictional reality?” FURTHER QUESTION OF INQUIRY How can a director/ player influence the performance via levels of realities? If this influence can be extended to a spectator, is that where our notion of “audience” will come in? What is the audiences’ experience of Fictional Reality?

Mid-sessions reflections : Nelson Chia

Thong Pei Qin’s Reflections ORIGINAL PLAN & INTENTION Under “movement”, Khairul and myself adapted the Viewpoints Lanes exercise and merged it with some of the basic principles of the Meisner technique to generate “procedural” movement vocabulary - aiding actors’ improvisation of giving/ responding to stimulus live in the space. The 5 movement text vocabulary assigned was Clap, Jump, Lie Down, Start/Stop and Walk, from Point A to Point B within the space. This set of movement vocabulary/ content was fixed and assigned as a general rule/ strategy of approach, and meant to guide the actors to “don’t think, just do” in the creation of movement sentences within the flow. The other pairs set the other parameters of our experiment (e.g. Event: the act of giving birth, and Location: surreal Escheresque landscapes). Together with the movement vocabularies, these were meant to be the constants or our controls that would give shape to what we were testing for, or our Totems/ Anchors. I see them as aspects that the actors could instinctively hold on to/ rely on when in the flow dealing with the various elements thrown at them to respond to in the moment, all demanding their attention. What did not work as expected was that these movements were not immediately instinctive enough to the actors (who were doing them for the first time in this context), for these actions to achieve the intended effect of “don’t think just do”. They did not come as naturally enough as Khairul and myself would have liked it for the purposes of this experiment, and in essence, the actors still had to “think before doing”. To counter this, actors would need more time to be prepped and “trained” in such actions and get used to them (almost like wiring into the muscle memory). It isn’t as “procedural” as generating Meisner’s text with the effect of stating facts as observed quite fluidly and easily. As a result, the actors started to break out of what was assigned or prescribed in terms of movement when that did not work for them, or proved to be challenging to hold on to. They started to allow their bodies and senses to respond more organically to what

was happening or given out in their surrounding en-

bing onto something that was constantly not with-

virons, to see what would happen instead. That was a

in my control, fleeting, or slipping away. When the

pleasant surprise as more possibilities of improvisa-

movements did not work instinctively, I found my an-

tions sprung up with new inspirations.

chor to be my assigned duty of focusing on Cherilyn whom I was tagged to, and verbally reporting

Things that worked for us in the following 2

what she was doing constantly. I also discovered that

jams include the discoveries of:

as actors, we independently find our own Totems to

•

Totem and Kick

grasp on to (hence different from everyone else and

•

Splitting up actors by assigning each one to only

independently decided rather than a group decision

respond to one layer of reality (instead of all being

made), when not assigned with a collective under-

in the same layer, or within overlapping layers all

standing on what control would remain unchanged

at the same time)

for us to focus on.

•

Tagging one actor to another one in a loop, for observation and response to an assigned “buddy”

Therefore, moving forward, we could design and agree on a set of Controls that could work for

These separations scaffolded our experiences (for

everyone and for the purposes of the experiment, and

both the observers and the do-ers), and provided us

try to stick to them more before switching if they do

with a more systematic approach.

not work or prove to be too restrictive. This means we try not to let too many Variables take over or mud-

This leads me back to the question of what we are testing for. For a “controlled experiment” to

dle our intended Controls, if not we may lose sight or sense of what we are testing for.

work and prove itself, there needs to be very clear demarcations of parameters and boundaries (what

Some questions/ thoughts I had in relation

can/ cannot be shifted), and of what Variables and

to the deviation from our original plan and agreed

Controls there are. These need to be made clearer,

prompts from the blueprint:

in order for repeated testing to happen and results

•

On organic jamming: very quickly, the players

derived to be analysed fairly. If our Control isn’t con-

broke away from the mode we set up to test with-

stant, we wouldn’t be able to compare results upon

in - question is: have we tested with our initial

repetition.

plan enough before we broke away? If we did not give it enough “chance”, why? How did the initial

For me, I wanted to find out if we could grasp

plan not work for our purposes of jamming? Was

onto and create something tangible, present (live

there a hunch that something else could work

in the moment), corporeal/ visceral and affecting,

better for something we were really interested in

within a time and space that was abstract and ma-

as a group?

nipulated to come across as “unreliable/ unreal/

•

constantly shifting” to the actor. Hence initially to me, I had intended for the prescribed set of move-

Separation of layers did help with our overall clarity of thinking and observation

•

There was a sense that there was too much hap-

ment “rules” to be what the actors could hold on to,

pening too soon. Suggest for not so many over-

unchangeably. Without an anchor, as an actor being

lapping realities or elements to be added on too

in the flow (me in Jam 1), it would feel as if I am grab-

quickly from one jam to the next, or happening at

Mid-sessions reflections : Thong P ei Qin

the same time, so we can focus on and look out

digging deeper into the condition and experience of

for what exactly we’re observing (e.g. Khairul’s

being stuck in a “jet lag”/ “time warp” and how

Meisner Levels are introduced one by one, or 1, 2,

that actor dealing with that condition in Jam 1 (e.g.

3 game - from 1, 2, 3 to add clap, to add stomp, to

Glitching or trying to grasp multiple realities) affects

add standing or sitting) - an increase in complica-

the others in the loop. Only after a couple more tries

tion and cognitive dissonance, while maintaining

of this experiment do we then layer on the audi-

a basic structure.

ence’s presence and ways of viewing into the same

• E.g. Sound/ music underscoring is another

type of jam.

layer all together, and yet another element or stimulation for the actor to deal with. Suggest

Or, in the case of Jam 2 (with VR), to further

adding in only if necessary? So that actor can

refine what we have started out with in Session #4,

focus on moving in response to just visual

here are some ideas for Session #5:

stimuli in the VR world, or in relation to just

•

verbal narration? •

Khairul’s idea: microphone input and avatar movement could be something to work towards

Suggest for observers to also be assigned

(means avatar wearing VR goggles can now

what/ who they should be observing, and pen

move around the space and not remain sta-

down their reflections on the spot on laptops/

tionary)

shared google doc? (A different kind of note-tak-

•

In Session #4, everyone else but Tim will not be

ing to save on the need for more transcribing after

able to see what the Avatar sees. They can only

reflections are shared verbally?)

react to the Avatar’s response to what Tim feeds him/her. In Session #5, maybe Tim can project

RECAP OF JAM 1 AND JAM 2

for everyone else to see what the Avatar sees through his/her VR goggles. This means ev-

Jam 1 and Jam 2 have varied quite far in terms

eryone has a chance to react to that too. (We

of basic structure and ingredients. We were curious

can reveal what actors and audiences can/can-

about jamming with VR equipment, hence we did not

not see, layer by layer, to test out the different

structure Jam 2 as an extension of Jam 1 to build on

types of viewing/being viewed experiences.)

one idea/ exploration further. We simply wanted to

•

Reacting to audience reaction as another lay-

test out different things, in the interest of time. We

er of reality / element of the continuum (e.g. Jo

did keep to the tagging of one actor to another in a

narrating what/ how audience is responding) -

loop, and having each exist in one layer of reality from

perhaps in Session #6 onwards, what if the au-

one jam to the next. Suggest for us to keep to that in

dience can interact with actors or participate

future jams too.

more actively? (Building up from first layer of having audience simply watch passively)

However, for one jam to be a repetition of the preceding one, and to build off the previous experi-

ADDITIONAL IDEAS FOR WHAT AND HOW

ment, there could be more articulation and clarity

TO JAM/ COLLABORATE IN SESSION #5:

on what we would want to find out further - e.g. Not so broad as whether technology hinders or enhances actor’s sensibilities, but maybe specifically

•

Swapping roles, swapping “director-ship” (keep the roles constant though, but switching different

people around in these same roles) • •

Does use of tech interfere with/ enhance the fo-

Glitching in a jet lag/time lag? •

PQ’s experience being an actor inside: influenced

cus of, and ability to make belief for the actor?

by Nelson’s organic free-flowing movements

What are the shorthands we have created and

when responding to projected particles as well.

shared with one another, that we’d need our audience or general public to catch up on? (Thinking

Kinesthetic response for Jam 2:

ahead for compendium)

•

PQ’s experience being an observer: Nelson reacting to a delayed response of Jo’s voice. But, how

WHAT DO THESE MEAN FOR ACTOR TRAIN-

much of it was his response to Jo, how much of it

ING?

was his own interpretation of the other factors in the space - maybe he could sense Cherilyn’s

On Phenomenology and Live Presence/Corpore-

physical corporeality right next to him although

ality

he could not see her, and actually deviated from just listening to Jo’s voice for instruction or stim-

• • •

How can elements and creators untrained in the-

ulus? Was he listening to other sound effects

atre and performance jam with us too?

(echoes, delays, white noise, speakers crackling,

Not acting or not knowing how to act is ironically

Cherilyn’s footsteps, etc) rather than focusing on

“good” acting?

the words and content of what Jo was saying?

Phenomenology: elements that operate based on

Eventually her words can become a blur.

their own laws (i.e time and lags), and us looking

•

Organic movements generated from playing with

out for the most authentic and unpretentious el-

VR headset and responding to that virtual reali-

ements:

ty as impulses could be extrapolated into move-

•

I.e. Technology is not ironic, it is sincere

ment sequences? Could be a way to generate/

•

I.e. Actors do not pre-suppose/ pre-plan/

elicit unpretentious and surprising movements,

think before they do, they simply respond

without pre-planning or choreography. (Play the

truthfully in the moment

ninja slicing game?) What if Nelson could see

What does this mean for actors trained in

Cherilyn the next time and mimic or respond

acting and making believe with different fic-

to her movements? What if he can interact

tions? (Fictional truth)

with her within her space, or intersect within

•

it? What if he can intentionally “kick” her out On Designing Movement An actors’ communion between physical performance to a simultaneous virtual performance.

with his presence? For further discussion and design of experiment (Session #5) in terms of Movement •

How to design movement for intentionally/ pur-

Kinesthetic response for Jam 1:

posefully? (A set of vocabulary? No need to be

•

Each actor affected by the other - the person s/

rigid in the type of movements actors can use -

he is tagged on, or even the other person in the

e.g. Run, lie down, jump, clap etc)

space that s/he isn’t tagged to, in a loop •

Glitching as a movement sequence inspiration?

Mid-sessions reflections : Thong P ei Qin

•

Or rather, how to not design movement but allow the room (time and space) for it to emerge

organically? How to observe the impulse and effects of these generated movements then? On Designing Audience Layer •

How to bring in Audience? (Too soon?)

•

How to direct the viewing of the audience - if there’re too many layers happening at the same time, the audience might be confused. As creators of signs/ semiotics on stage, what signs do we enhance and direct the audience’s attention to look/ focus at?

•

Any meaningful signs we can start to pick out and organise together to form some form of cohesion?

•

If we choose not to “direct” or guide their viewing experience - why?

•

Think more about the aforementioned audience interaction/ level of participation (increasing with each repeated test?)

•

Sensor on the audience (audience feedback loop) - they don’t need to over-perform or preplan

Timothi Limi’s Reflections Note - Original File was lost, thus this is a rewrite from memory. To me, the creative process is a winding road that twists and turns, sometimes veering off-course towards disaster or revelation. This is very true for the ArtXTech lab which has to grapple with a short time frame for creative innovation and with input from the technology front where scientific methods or business strategy are regularly applied. A key opposition to the creative process would be needing the guarantee of successful outcomes and viable market ready products at the end of the process. Creativity is a risk, as much a risk as anything in life, and for some, that risk is too much to stomach, but for me risk is needed in creativity, just as much as design limitations or boundaries are needed. There is a madness to the process that is only revealed once the process has been done, not before. Sometimes, as part of being a business owner, it feels as if I have to guarantee success before the idea has even coalesced, however here with the Architects of Realities team, I can jump right into the process again, and through the act of doing, create innovation even if it doesn’t meet the visions of the stakeholders. Here we have the freedom to express, to collaborate, and through our interactions, actually create a new process to which best fits our intent of innovating an ArtXTech directors/ actors lab. The Architects of Reality team is highly skilled and to share our knowledge, we created presentations / exercises to help align everyone with our mindsets, expertise, and worldviews. I really enjoyed this part of the lab as it was self-motivated and very much needed. Each of us could rightly create a lab by ourselves but only together could we innovate an entirely new perspective or arrangement for the lab. Most importantly for me, and perhaps for the team, is the acknowledgement that even though we might be similar in some of our practices or industry, e.g Joanne Ho and I are both technologists but our processes and outcome are entirely different - we made a point to understand the differences. Understanding our differences and strengths allowed our individuality

Mid-sessions reflections : Timothi L im

and x-factors to shine, leading to new connections or opportunities for ideas to shine. From this connection and understanding, I finally managed to figure out how to connect multiple digital mediums together - PC, VR, AR, and Motion Capture, could now be combined together in one virtual world. To connect four different digital mediums together is mind-boggling but without this lab, I wouldn’t have been able to create the algorithm to do so. Perhaps in the long-term I would have stumbled upon some inspiration, but here in this act, how we connected through sharing differences and connecting on similarities allowed me to create a linker script that connected the four key digital mediums together through the Unity Game engine. Trust the process. Don’t fear the “what if”, that’s just a distraction from the real problem - which is many teams now trusting each other or hyper-focusing on the end goal to the detriment of forgetting that the team is made up of humans. Wish not for what we don’t have and take pride in what we have. Something about the Architects of Reality team and the ArtXTech lab itself feels fully natural, as if it is the perfect locus for innovation to arise. I am fully invested in the process and believe that while we don’t have a fully concrete plan at this moment, by the end of the lab, we will have a process or a compendium that consolidates all of our learnings here. I hope through our burgeoning effort, in this short time frame for the lab, the greatest learning would be the trust and innovation space we have cultivated together.

SESSION #5 August 30, 2021 // Nine Years Theatre

Jam Sessions

ITINERARY 6:00 - 7:30

Jo & Tim Tech Setup

7:30 - 8:00

Khai Meisner Level 3 Technique

(Psychological / opinions)

8:10 - 8:15

Play 1 (Projection tech as player)

8:20 - 8:30

Talk

8:40 - 8:45

Play 2 (Projection tech as player)

8:50 - 9:00

Talk

9:10 - 9:15

Play 3 (Projection tech as player)

9:20 - 9:30

Talk

9:30 - 9:35

Play 4 (Projection tech as player)

9:40 - 9:50

Talk

10:00 - 10:05

Play 5 (VR)

10:10 - 10:20

Talk

10:25 - 10:30

Play 6 (VR)

10:30 - 10:40

Talk

10:40 - 11:00

Wrap up, pack up

Session #5

TRANSCRIPTIONS OF OUR SESSION TALKS AFTER PLAY 1 KK:

How was that for you guys? Is there anything you noticed?

JH:

I noticed that personally I was trying to figure out what question

you guys were thinking of in your heads. So it was like looking at what you were seeing, versus what you were doing and I was trying to imagine.

Cherilyn Woo

Joanne Ho

Khairul Kamsani

Nelson Chia

TL:

Thong Pei Qin

down, it was like a height thing that I noticed. I especially liked when you

Timothi Lim

Because the last session I didn’t know where to start. I was just watching, now I was trying to imagine what’s going on in your head. There was a part where Cherilyn went up and you (PQ) went

(PQ) were laying out on the floor.

101

JH:

And then I was also watching you (CW) and I was

like oh she’s glitching again.

you would catch glimpses of body parts of yours. Or even your head, like your full head, but sometimes it’s just half of your ponytail or whatever. But this one on the block that’s

PQ:

When?

facing this direction is more of an overview of your body. I like that play. It’s kinda like the portal idea.

JH:

When you were on the floor, especially where you

were in a position like that (lies on the floor). And you total-

CW:

It really did seem like a portal, actually, yea.

TL:

They’re different cameras.

JH:

They are different cameras, yea.

PQ:

I also tried not to look at Cherilyn. I thought that

ly were paused. PQ:

Because my legs came out. That one is later (lag-

ging in time), right? The projection on the red block. It’s later. So my feed was real time and you were probably just getting delays

I really should be in my own zone, to have more control. I CW:

So in terms of responding kinesthetically, that one

kept on focus, then I realized that I’ll try one part where I

was the easiest (points to the farthest right projection on

just turn around, then I could see myself, then I could see

the wall).

my effect on Cherilyn. So that one is a bit of a play cheat. When I turned around and saw what I was doing, I just

PQ:

That one is live, right? (Points to the red box pro-

jection)

wanted to try. Then I realized that when I move my body up and down, Cherilyn over that side could touch me. So we were interacting. But I could only do that because there’s a

JH:

That one is live, yea.

viewfinder for me here. It’s like through a two-dimensions that we were interacting with each other.

CW:

Yea, and then probably this is next (redbox), and

then you (PQ) and then that (the wall projection on the

JH:

In the physical space you’re separated, but in this

left). So this one I tried really hard to find in space, but I

flattened view, you’re actually living on the same plane, so

often got lost. So it was between these two (red box and

you can interact with each other on that plane. I like that.

right projection). And I think when I was glitching because I couldn’t see. So instead of focusing on three, then now I focus just on one which is what I can hear or what I can

AFTER PLAY 2

directly see (PQ). but I was trying hard not to look at you (PQ). So the only points that I found quite engaging was

JH:

It was interesting when you guys were playing to-

if you were there (projection on the right) or if you were

gether over there (right projection) with the feet and the

doing something here (red box). But what i found quite in-

hands. It reminded me of shadow puppets.

teresting was there was a point you moved off from there (right projection), but you went here (red box), and then

KK:

You were playing with which?

PQ:

The huge zoom in.

KK:

I was playing with this one (left).

PQ:

Then you saw me inside.

but enters another one. And that could be something we

KK:

I saw you in this one (left), but you were connect-

could play with. Because the one on the right on the wall

ing with me with that one (right).

slowly you moved and then you weren’t there (right projection), but you were here (red box), so it was as if you left one space and went into another space. So that was quite interesting fiction to play with. But other than that it was quite hard to get a fiction. JH:

I like that though, when Pei Qin leaves the space

is close-up. So there’s only certain moments in time where

Session #5

PQ:

Correct.

KK:

I was playing with her. I think that was the beck

and call, whether I’m not going to do a gator, she pushed PQ:

This version is different from just now. Because

me, so there’s a play there. But I wouldn’t say it’s one-sided.

just now I was only focused on myself. But now Khai is

It’s a two-way communication. But I wanted to move for

making a more conscious effort to play with different ele-

her (PQ) and to do things and have something happen.

ments while in this role, so when I was there (right), I knew you were playing with me here (left projection)( already.

TL:

We can try next time putting a camera on the

That’s why I could play with you more there. So there was

head. I saw you (PQ) try to grab him. Cause Khai can’t real-

more communication - connection. When I was in this role,

ly see your response, because it’s a voyeur, audience kind

I was the generator.

of view. But a first-person kind of view.

KK:

You were playing with yourself.

JH:

I love that idea. That’s another layer.

PQ:

Then later on I played a little game being in Cheri-

TL:

Because we have webcams that are really small

lyn’s position. Because just now you all said that I, person A,

that can attach to the head. Because you were trying to

had all the different windows, so then I really wanted to see

grab him, it gave me the idea. We can try that. We just

what happens if he disappears and then he returns again.

need to watch out for the cables, but the cables add a very

And what body part returns. Then I thought that maybe it’s

cybernetic element.

his eyes that I wanted to catch. At first it was just the whole head. Because sometimes, depending on how khai turns,

JH:

This whole time - it’s so interesting how people

your (Khai) eyes might be hidden. And then even over here

are trying to also be in the same space virtually, and see

you were doing this (cover eyes with hands). So I was like,

yourself in the same space as other people. So VR worlds,

in real life can I also catch your eyes? But it’s not true, be-

where you can see the hands and bodies of other people in

cause you might turn away from me, even in real life. Then

space. Even haptic feedback - like holding something, and

finally I can catch your eyes here (right).

someone miles away is vibrating it. Not like that. But yes, that also, but. This has always been an initiative of love.

JH:

Like catch it with your hands? CW:

PQ:

Like a view (makes a triangle with hands) I tried to

Including not only the lives of artists, but long-dis-

tance relationships.

catch a body part but then I decided it was the eyes. JH:

JH:

Exactly.

PQ:

I realized when I did this (triangle gesture), actual-

Did you try to move Khai’s head (with hands)?

“Look at me!”

ly I could have borrowed an equipment. Like what if person PQ:

At first that was just an extension of what I found

2 (on the right) could have a camera. I wanted to capture a

there (right). Because when Khai did - we interacted with

feature. Could I have used my GoPro on me or taken a mini

our body parts - I thought I could move him. As an im-

cam to zoom into the face. Just because I wanted to use

age I could move him. But after that because he was do-

tech to capture it.

ing things on his own already and trying things out. Then I thought - let me try and catch your (Khai) eyes.

JH:

Yea, like having a phone and having it streamed.

Or even a webcam that’s showing what you’re looking at, JH:

When you (Khai) were watching that screen, did

or what your face is facing.

you feel like Pei Qin was trying to move you? Because the only way that you can only see PQ in this set-up is that

PQ:

I’m not sure how you can change this, maybe you

screen (left). So were you actively looking at that (left)?

can enlarge the area (on the left), so that Khai wouldn’t disappear. Like what if you (JH) have control, you can expand it. So he (KK) doesn’t disappear from it.

103

JH:

Because this is such a close-up (right) -

KK:

Yea.

PQ:

The body looks like it left.

JH:

Yea.

JH:

Yea, it looks like you (KK) left the space.

PQ:

It felt like there was a fiction already.

PQ:

So what if the technologist can play with us? That

JH:

It played out within a frame. So I felt like you (CW)

means, you all can change all these.

were playing with this frame (left projection) and you (KK) were playing with that frame (right projection) literally. Be-

JH:

Like make it track.

cause you were putting your hands against the boundaries of the light. And the way that the light was projecting

PQ:

You can track him.

Cherilyn’s shape was overlaying Khai. It was touching his skin. Like when Khai was moving forward and backwards, I could see the limit of the boundary of the light on Khai’s

AFTER PLAY 3

body. Then Khai was moving in and out. I don’t know if you were doing it consciously.

KK:

I’ll start back with what was happening at the end

with me. It was interesting how I felt Cherilyn’s presence on

KK:

Was it that you saw the projection on my body?

JH:

Yea. Because you were right in front of the wall.

that’s why I turned around and I put my back against the

KK:

I totally forgot about that. I thought that if I covered

wall. And I still could feel that she’s there. I felt that prox-

that and I shadowed that one (right projection). I couldn’t

imity disappear when I walked away. So when I walked

see her, but actually you could see her on me.

a projected image. Even with the fact that she’s not next to me. But when she engulfs me, I was like oh there’s a Cherilyn there. And I was like if I turn around, she’s still there, so

away to see if I could still feel her presence there. I couldn’t feel it when I went to the side and I turned around. And

JH & PQ: Yes.

then I looked at the real Cherilyn. It’s interesting how even though I know that that’s the real Cherilyn over there (left), I

PQ:

So it was intimate. So you were hugging.

though there’s an intermedial one there (redbox), that one

JH:

Yea, so when you did the hand thing, and you

was a lot closer to me. That one that I was interacting with

stood up, Cherilyn literally engulfed Khai’s physical form. I

was (right projection) the realest Cherilyn.

was just like (gasp). That was really intense. That moment

still feel this presence (right projection) a lot stronger. Even

was so intense for me. JH:

Do you think it’s because Cherilyn’s version on the

wall was reacting to you (KK), is that what made it real?

CW:

I think the first part we were still dropping into

the session. Because I was also trying to navigate. For KK:

I mean because we did spend time reacting and

me, I was reacting off live bodies. But then after a while I

interacting with each other, and then it came to a point

realized that the screen (left) was quite engaging. I think

where I just wanted to observe her. I was just trying to ob-

he was reacting to me on that screen. So it was the only

serve what was happening. She disappeared, then she ap-

mode of our communication. After a while I wasn’t mov-

peared again. I think that just reinforced that reality. That

ing for my own physical body. I was moving for my pro-

this Cherilyn is the real one. That’s why I wanted to play

jection to him. So I was puppeteering my own self in a

with that, with her presence - whether - will she still be

projected world, which I thought was quite interesting.

here if I turn around.

Because previously I was only in the space live, whereas in this one I felt like I actually entered another layer that

PQ:

Did anyone feel that it was quite intimate?

Session #5

was projected in order to communicate with a project-

ed version of himself but also with his live body. But it

CW:

For the two times I did it, I didn’t really care.

projection), through there (projection of her projected

PQ:

Because if they care about you, they will be play-

body on the left projection), there (right projected), come

ing to you. They will open up towards this direction, and

out to here (where Khai’s live body was standing). So I

they didn’t.

felt like one loop. Here (present body), through there (left

actually managed to forget about him (Khai). TL: TL:

So this is the second time doing this with the pro-

Earlier when you did yours the first time, you were

facing towards us.

jection mapping. Because you did it twice, were there differences in the positions from the first time?

PQ:

I had my own focus, I had my own target.

CW:

JH:

I agree it was quite intimate. There was a lot of -

the other side (right stage), in some way or manner, wheth-

PQ:

-overlap. There was some emotion. It felt like there

er it’s physically or through projection. Because that was

was some emotion. Whereas from the first one, it was emo-

my difference. That I was reacting off the projection, but

tionless. I was the generator. I’m like a machine. Later on it

she (PQ) was by herself. So it didn’t matter what I did on

became more and more collected.

I think for me it makes a difference if the person

here (left stage) is intentionally reacting to the person on

the projection. KK: JH:

I thought about that when I teach Meisner about

I think how you (PQ) started out was how we were

how emotions arise. So when I teach acting, I tell the ac-

doing it in the last session. But actually to close the contin-

tors all the time. Don’t aim for emotion - don’t tell myself

uum, it’s more meaningful when everyone has someone to

i need to be angry. What you need to do is focus on your

react to, rather than being a generator. Because we’re not

partner - and what is happening? What are they doing,

machines. To close the loop, it has to be closed. And to be

and what is your opinion on that situation? And this en-

a generator is really difficult. Because you need something

tire situation. So the scene appears to you. Then the emo-

to start with. Like even with machine learning, you have to

tion will appear, which becomes what is organic acting.

have a dataset.

Authentic reality of the situation. I think that’s what happened today, through a digital loop. Because we didn’t

PQ:

That’s why in the end it became a bit too mean-

choose what we wanted to do. We just observed what

ingless. There was a switch to look at that (projection)

was happening, then we were playing. So the warm up

and I got fed back into the continuum. I ended up looking

that we were doing we’re actually doing it in an abstract

there (left projection). And then I was interacting more with

way, and we had an opinion about what was happening

Cherilyn.

all the time. And then we played with that and then it just appeared. I would say that this is one way it would work

JH:

This is really interesting because I think that as

with technology. That’s why I’m very interested in how

directors and performers and dramaturgs, I feel like the

we can use this to imitate how we generate this continu-

way that you guys analyze presence and space is super

um between people in remote spaces.

important to understand presence in space virtually. Because we’re talking about virtual presence, and interacting

PQ:

between virtual bodies and presence. I’m so into this. I feel

rounds?

Do we still remember totum or kicks in these three

like we can write a whole book. KK: TL:

When you were there, did you feel the audience at

all?

I think before we go into that, generator is anoth-

er term. Generator and continuum. What is the generator and what is the continuum. Can you be both at the same time? How do you move from being a generator to a con-

KK:

No, I didn’t really care about it at all.

tinuum and a receiver?

105

JH:

I think we can talk about generators in two differ-

nologists haven’t really played with us in this set-up yet.

ent terms. Whether you are solitary in the way you gener-

I mean you designed the environment, but we haven’t

ate, or you are generating for someone else.

opened up the opportunity for you to jam as a player yet.

KK:

But then that’s a continuum already.

Are you saying that we are going to get them up

here now? Let’s do it. JH:

That’s true.

KK:

Because when you play with someone else, that

AFTER PLAY 4

means you have something from them, right? It’s not possible to just play something for someone out of nothing.

JH:

It’s really interesting because I didn’t know there

was another angle here (behind the redbox) where I could JH:

Right. I think that it’s never out of nothing, though.

see myself. So actually I was also reacting to that. I could see Tim through that as well.

PQ:

You know last time when we did the one with Nel-

son was here (left stage) interacting with the particles, I

PQ:

thought that he was a generator.

screen to another level.

JH:

CW:

In that sense then, if you’re the generator, does

that mean you’re not reacting to any body?

We saw hide and seek. Tim took the disappearing

Tim intentionally concealed. Which I think was dif-

ferent from ours (KK & CW’s play 3), which was trying to reveal, but for Tim, it was conceal.

CW:

Or maybe you are the epicenter. JH:

Wait, what does that mean?

CW:

So he was basically trying to cover you.

line) or linear radially.

JH:

Oh, yes!

PQ:

With his shadow at first, then it became his body.

JH:

Like you (CW) were the epicenter when you were

in the VR headset and I was narrating your moves. So it was linear, because it’s either linear in this fashion (in a

Although Nelson did react to the particles, it’s not

like he generated for himself. He had particles to help him.

His real body to cover you. But didn’t succeed. I think you

And when me and Cherilyn were more in sync on the other

sensed it, so you started to jump out of the way, so he

side that time, then she was in a loop. Then my movement

couldn’t do anything.

caused the particle to move and then Nelson reacted back to the particle. But he didn’t react to humans, but it was

JH:

more particles.

and in there (right projection).

JH:

PQ:

But Cherilyn also, in the VR headset, had the envi-

I was trying to hide myself in here (left projection)

How did you hide? You hid behind this block phys-

ronment.

ically, then visually, you went out.

CW:

JH:

Yea, and the man.

I was trying to figure out where to focus. If I was

tired of trying to hide, I would go back to viewing myself. JH:

The infamous hostile man.

Looking at my tattoo, looking at myself, moving. And trying to disappear in this (the red block) and appear somewhere

CW:

My captor.

else, then disappear in that (left projection), and appear there (red box). Like jumping places.

PQ:

So I had another question, which is that the techPQ:

Session #5

Is that your totem? Like you had a rule.

JH:

I was trying to escape in different ways. And at

KK:

I did that, too, myself.

because my black - I could see it engulfing you, and then I

PQ:

You all had the rhythm, like you established a

thought about Cherilyn’s tactic. I don’t have black hair but

rhythm. One of those tools that Khai was saying to you. I

I have a black shirt. I was trying to take over the whole

don’t think we said rhythm though.

the end I tried to conceal you (Tim). I was on my tippy toes

screen by concealing. TL: TL:

I had that part with the shadow. Like the black of

Definitely I was connecting to the whole thing

when I started to play with my shadow, which ironically

your blouse conceals my shadow.

isn’t really me. I mean, I could control it.

JH:

KK:

At one point you were on the floor. I was like, did

you give up on trying to conceal me?

I did the same thing I played with my shadow

sometimes. Like why am I playing with it? Is it me? But it’s my shadow.

PQ: TL: KK:

But his feet stuck out. JH:

Actually all of these are shadows.

TL:

It’s interesting there’s a philosophy like Plato’s

I was seeing like oh my feet can do things too! The reason why I didn’t want to stop you guys was

because it felt like you were very in your body. I could see

cave. Because I really like how the light can allow you to do that in this space.

that you were so focused on the task and what you’re exploring that - for you went on your tippy toes and you were

JH:

Even though I was playing with three surfaces, it

moving very exactly on what your body was doing. Very

felt like the world was so big. There’s so many angles. It felt

precise. There’s an irony in that. That is because you’re so

expansive.

focused outside there, that you’re so focused in here. Because you’re like how can i do it “perfectly” or in the right way so that i can achieve my objective. That’s why I didn’t want to stop you. And then the more you were into it, the more he was into it there. And then there was a very strong play , a very strong story there and we all could see it. So what do you think happened for you to get there? Were you at the same state of physical awareness before at the beginning or towards the end? TL:

I think towards the end.

JH:

Same. I think this is the first time we were in the

projected space with multiple versions of us. And I was like what do I focus on? I think what helped me was the frame. How do I transition from out of frame this way, to out of frame that way. And I was playing around with dodging your (Tim) concealment of my image and then repeating it. So because there was a mirrored image of myself, it was much easier to be exact, because I could see myself in a frame. I could see that I was swinging symmetrically and kept doing it until it was a pattern.

107

SESSION #6 September 13, 2021 // Nine Years Theatre

Jam Sessions

109

ITINERARY 6:30 - 7:30

Jo & Tim Tech Setup

7:30 - 8:00

Orientation (to blueprint of what/how to jam)

8:25 - 8:35

Play 1 (VR)

8:35 - 8:45

Talk

8:50 - 8:55

Play 2 (VR)

9:00 - 9:25

Talk

9:30 - 9:35

Play 3 (VR)

9:35 - 10:00

Talk

10:15 - 10:20

Play 4 (VR)

10:20 - 10:25

Talk

10:25 - 10:45

Wrap up, pack up

Session #6

TRANSCRIPTIONS OF OUR SESSION TALKS AFTER PLAY 1 KK:

Alright, how was it? What did you see Pei Qin?

PQ:

Giant. That I assumed was Cherilyn and it’s correct. And I just

tried to, at first mimic her actions. And then later on I tried to interact with the giant. I tried to touch her. Then imagine myself being trampled on by you (CW). So I used my body to react to you. CW:

Oh, I see. Okay.

PQ:

As in I imagined an avatar trampling on me.

KK:

Did you feel that you touched her?

PQ:

At first couldn’t, because there was a barrier right. So I wanted to

touch her giant legs, so I reached out of the barrier. Then later on I don’t CW

Cherilyn Woo

Joanne Ho

Khairul Kamsani

Nelson Chia

Thong Pei Qin

Timothi Lim

know why, you (CW) could step over me, then I adapted my body to you (CW). (To Khai) Then I felt like I was touching her. KK:

That intentionality of I’m trying to touch you vs. I can feel you,

is there a difference? Compared to earlier when you tried to touch but couldn’t, but she stepped on you as if to touch you, what was the shift?

111

PQ:

There was a fiction. Because Cherilyn wouldn’t

CW:

Yeaaah. I think at the beginning I was definitely

know that I’m trying to be touched. So she wouldn’t in-

chasing after her. Then I tried to make it more organic with

tentionally touch me. But once the (CW’s) avatar stepped

my avatar body, then I failed. Then I tried to go back a little

into my world, past the barrier, and I felt like “oh now I can

bit to play with the physical-physical. Then that failed. So I

merge my own body with her (avatar)”, then I imagined that

was kind of at a loss.

I was touched or she was being touched. And it felt more real, it felt like I could play with her more.

PQ:

We did touch twice right? (CW: Yes.) And I was

like “ow”. “Ow” like it was an accident, rather than it was KK:

Okay. Cherilyn, do you have any comments about

purposeful.

touching her in the VR space. Also how it would have or not have helped work in the physical space?

JH:

That is so... Soon as you said you were trying to

touch, like you (CW) was trying to touch Pei Qin with your CW:

I think in the beginning I was trying to get as close

avatar body, in that sense your head was in VR as well.

to her as possible. But then I found that a bit futile. Cause I wanted to interact with her (PQ) more than just purely find

CW:

Yeah, yeah.

CW:

And I could only know my VR coordinates through

an opportunity to almost touch her. KK:

You mean in your physical body or...

her (PQ).

CW:

Physical body. But then after that I realised that,

Jo:

Wow, that is so...(Metaphysical?)

knowingly to each other], then I tried to get on the same

KK:

So you started moving your body because you

plane, but then I realised that how I was moving was prob-

wanted to control this body. But you were moving your

ably not how I was imagining it. So my projected version

body to control that body.

oh, she’s mimicking my avatar body [PQ and Jo nodding

of the VR world was not accurate. Which means I was figuring out my orientation of how can I then use my avatar

PQ:

Come in Khai (to show the camera his physical

to touch her (PQ). But I didn’t know that I actually did. The

demonstration) (Khai enters and demonstrates.)

whole time I thought actually you were responding to the mist. (PQ and Nelson gasped in surprise.) (Collective “orh-

KK (to CW):

hh”.)

So you were moving your body not to control this body (CW’s own body). But to control her (PQ’s) body.

KK:

So Cherilyn, you didn’t know that you didn’t touch

PQ at all in virtual space?

CW:

Yes.

CW:

JH:

(to CW) But the VR body was something you

No. The only time I thought she noticed me was

when she was mimicking me. But then I thought you were

(CW) were imagining, you couldn’t even see.

just distracted by the mist. CW: PQ:

I was actually chasing you and trying to merge my

Yeah. I mean I had a brief idea what it was (gestur-

ing to the computer screen), but I really didn’t know how it

body with you.

was.

KK (to CW):

NC:

But you know from the spectator POV, I was

Can you tell us a bit more about your physical

watching this (computer screen), and Jo was controlling

body touching a physical Pei Qin? Cause it looked like a

this. I was watching this (computer) and I was watching

lot of times you were almost going to touch her but you

that (live bodies). It became very interesting. So in the be-

decided not to?

ginning I was okay, I think you guys were trying to figure

Session #6

out the touch thing, (to PQ) which I agree should be more

never be accurate (CW: Yeap.) Because the scale is so big

prolonged, rather than you knocking into something. So it

(her as a giant).

became very interesting for me is when Cherilyn tried to bend down, but Pei Qin was doing this (demonstrated a

JH:

Yes!

ceiling), so when I looked at here (the PC), Cherilyn’s ava-

CW:

And also the plane is different. I was turning one

tar was actually over Pei Qin’s...

way, but my avatar was turning the other way.

CW:

PQ:

position of PQ lying on her back with all fours facing the

Oh that was intended. Because I saw her trying to

reach, and I thought if I’m taller, then maybe I can (demon-

So scale and direction are both very difficult for

her to grasp.

strate a bowing action), but I don’t know whether I succeeded.

NC:

Unless both characters have a common reference.

Like Azure Kinect captures a physical thing which is also NC:

You succeeded. But then when I watched the two

inside the virtual world. (PQ agrees.)

monitors, it’s a totally different story. So it’s a totally different dance that I see there (live), and I see here (the PC).

TL:

My favourite moment was when Pei Qin threw a

So basically they are connected, I know they are connect-

huge ball and it actually went through...

ed, but they look like two different performances. Then the next question is, what if Cherilyn could see what I saw (on

JH:

It went through Cherilyn’s avatar and into the sky.

the PC).

PQ (to CW who couldn’t see it): It’s like you got shot by some wave...

JH:

Ya, like if it were projected somehow. KK:

Jo, how did you feel?

could you tell us what narrative you could pull out of seeing

JH:

It was so fun for me. It was like watching a really

this and that, as an audience member?

inefficient fight? (Everyone laughs.) Because no one was

KK:

Like if it were mapped... (Everyone nods). Nelson,

reacting to me. You (CW) couldn’t see me. And you (PQ) NC:

Ok, instinctual response. The physical perfor-

weren’t listening to me, I was just a camera and space. I

mance feels more...Tender? Because they are of the same

was just looking at your “fight” from many different angles.

size, and Cherilyn is like this (demonstrates her bending

And in that way I’m the driver for what you were seeing on

downwards and reaching out towards the floor) and Pei

the screen, as you were comparing to the real life bodies.

Qin is like this (demonstrates her with her back on the floor

But for me, because I could see you guys in the periphery

reaching up towards the ceiling). It’s like two creatures

of my vision, I tried not to focus but I could not focus on just

or two babies (laughs). But over here (PC) it’s a different

the screen because I wanted to see a comparison. It was

narrative, where one is towering over the other. Because

like, quite funny. Cause you (PQ) were doing things to the

through the mediation, her size/ height, she could also

body, but you were totally unaware, like you were throwing

look very different. Like if she were designed to look like

blocks through the giant, and in the beginning I tried to

a lion or something. (Tim and CW play act and hold up

create a narrative, like maybe I was protecting Pei Qin’s

“lion paws”.) So my narrative changes depending on where

body from the giant. So I kept circling around, making big-

I look. What if we have the whole thing back projected on

ger circles, like I kept coming back to your avatar. But then

the screen (on the wall), and then I see different physical

that, it just didn’t seem like it was going to work because

bodies in front of it.

you (CW) couldn’t see it, and you (PQ) were not going to respond to it, so I felt like I was a ghost.

PQ:

I just would like to say that she (CW) was trying to

imagine herself (touching) but she couldn’t, I think maybe

PQ:

Cause we didn’t know that we started. We were

it’s because of scale, because try as she might, she will

waiting for Khai to join us.

113

KK:

I think they had said “begin”.

PQ:

Or was it Jo who was lagging? Because by the

time she processed what was happening, it’s gone. PQ:

The sound was too loud, so I didn’t know that we

had “roles” - that Jo was on the mist and I was supposed to

JH:

Right. PQ: Yeah.

there was someone behind it, I would have taken note/

TL:

So you can see on the computer how big things

interacted more intentionally. (Note about setting up rules

are. I have the view here.

take note of it or what. I thought it was random, so if I knew

of engagement and the activity properly)

AFTER PLAY 2

KK:

So the cylinders are like, little things.

TL:

Little things, yeah. So by the time it gets to the

end, like towards there, you notice it late, I think? DependPQ:

Were you the mist?

ing which perspective you are looking at, wait I can show you too.

JH:

I was the mist! It was interesting. I felt like I was

breaking everyone’s fiction, because I kept seeing what

PQ:

In the VR world Khai, do you notice that the giant

was happening, because I had a God’s view. I was like

was very close to you? Like, she’s here. (Gestures directly

“watch out, there’s a ball, they are throwing a ball at you.”

above her head)

So you guys both know, I’m in there somehow, and that I can see you (CW), and that I am communicating what’s

KK:

Not really. Also because I felt she was glitching so

happening in the VR world to you (CW). So I felt like the

much, I kind of like, disregarded her already. (Laughs)

disrupter. I felt so invasive, like I was kicking you guys all out at the same time. Every time they’re not looking at

TL:

Noooo. (Laughs)

KK:

Cause I was like, there’s nothing to...If I make up a

you (CW), you’re right in front of them. I felt (...) quick, like (demonstrated a flustered response “herrr!”).

fiction, I’m directed to this, it’s constantly breaking, I was PQ:

Did Cherilyn respond then, to her (Jo’s) warnings?

like, “no, I want to stay in something.”

CW: Yes, in a way I then depended my fiction on her (Jo nodded). Because I chose not to have my own fiction, and

CW:

she fed me the fiction.

woman.

PQ:

KK:

It was a bit laggy though, because when she (Jo)

But you know at one point you (Khai) were the

I heard it, then I was like “oh, am I?” Then I started

said “throw”, the things (objects in the VR world) were al-

doing things (demonstrating motions) to control the wom-

ready thrown. So the giant lady could not avoid it anymore.

an, but it didn’t seem like I was doing that.

JH:

It was five seconds later. There’s quite a lot.

(to Khai) You were. I felt like, there were so many

elements...(PQ: too many), there’s like also touch (PQ: corKK:

Is it live from here (points to computer monitor), or

rect, like Cherilyn), which was why Cherilyn couldn’t push

thrown live at... (Points at Jo)?

you guys away.

TL:

PQ:

No, I think it’s just general latency. I also think it’s

Correct. Which was fun! Which was like, then she

also the angle. Because you (PQ) would have thrown it and

(CW) could control something, then didn’t have to rely on

you can see it go, cause the giant’s so big, like if you notice

Jo.

the ball... CW: KK:

Oh ya, how big is the cylinder, as opposed to the

giant? (Tim fiddles with the mouse to show on the screen)

Session #6

Mostly so that I could steal my identity back

(laughs).

JH:

YAH! (Everyone laughs/ claps at the epiphany)

NC:

(points to Khai and PQ) Both of you tend to after-

wards focus on each other right? And disregard Cherilyn’s KK:

Oh was that why you moved me?

avatar. (Khai and PQ nod). Is it because of size as well? It’s hard to...

PQ:

She moved us, cause she already knew we were

constantly next to each other, because of her (Jo’s) clue. So

KK:

Ya, I feel that if we want to pay attention to her, we

she just saw our actions physically, then she just taichi-ed

have to do this (look up). Rather than we can just do this

us away lah. So that worked, for physical touch. But other

(demonstrates moving about on the same lower plane with

than that there were too many factors at play that I didn’t

PQ).

know what to focus on. CW:

How about when I was on the ground?

PQ:

Oh! We could step on you!

Khai:

Huh?

and when she lets go, I’m like “ok, now I’m back” (moves

PQ:

She was on the ground, then I did this (demon-

back to original spot). It’s kind of like when you guys play

strates quick repetitive stamping with her feet).

KK:

I definitely felt it was okay to get pushed or

pulled by Cherilyn, because when at the moment when there was somebody touching me or pulling me, I know I am not in that VR space anymore, I can feel my physical body being transported (moves to somewhere else),

with the game (...inaudible...), Ok probably not. Because you can pick up people (demonstrates picking up mini

NC:

people with a pinch of his fingers) and throw them in the

ground?

How come you could see that she was on the

water. (Everyone: Ohhhh! Nice.) PQ:

I saw her on the ground, her avatar.

world, did you know you were dancing, or?

KK:

I can’t quite remember...

CW:

NC:

But she’s still quite big?

PQ:

She’s big, but she’s on the ground. And she’s close

TL:

The last bit, when you all were dancing in the VR

I knew because I could see her (PQ). But I didn’t

know how close or where my avatar was, in relation to her. PQ:

So it felt like doing basic performance improv,

to me, so I could do this (step on her). But I had no feet.

because I had to come back to my physical performer’s body with Cherilyn, and respond to her body kinaesthet-

KK:

I must have been doing my own thing, cause I

ically. So in this world (gestures to a smaller VR world in

don’t remember...

front of her), I knew I could not rely on my sense of sight, cause my sense of sight is tricking me, cause I’m actually,

CW:

No leh, I remember you reacted to me, cause you

my sense of sight is with Khai, or with the mist, but my

said, “Giant lady on the ground! Giant lady’s falling through

real body has to focus on reacting to Cherilyn. So I’m jug-

the ground!”

gling two fictions, or two worlds at the same time. Cause I know I have to be responsible with her (CW) you see, I

PQ:

Then I “yaaayy”.

cannot...They were supporting each other (Khai and Jo), I

JH:

What if the giant was like the same size as you

cannot let go of this (CW and PQ). So I was multitasking.

guys?

JH:

KK:

Then it wouldn’t be a giant. (Laughs)

JH:

Yes...That is a fact. Thank you very much for the

So when you were multi-tasking, were you glitch-

ing? PQ:

Yup!

clarification. (Laughs) We have a scholar in the house. (Everyone claps and laughs. Khai rolls on the floor.)

115

PQ:

You can now be the giant lady on the floor. (Stamps

on him)

JH:

Yah, cause I’m trying to reduce all the factors. There

are just so many things going on, there’s touch, there’s (PQ: scale), there’s me announcing what I’m watching which is

JH:

Do you want a certificate? (Laughs)

like the God Eye’s view, and then there’s you guys (Khai and PQ) can hear me, before I think you (PQ and Khai)

NC:

Were you guys distracted by Jo’s narration?

couldn’t hear anyone at all (PQ nods), and then you (CW) could hear me but you could also see them, so there are

PQ:

Yes.

like so many different factors that affected everything, so it felt like quite...

KK:

Yes. Actually it didn’t distract me, it kind of helped

me. Because I didn’t know what was happening outside of

NC:

I mean, hearing from what you guys are saying,

me.

I do get a sense that you guys (Khai and PQ) connected within the same VR space, you are from the same space

NC:

But the narration is mostly to Cherilyn right?

right, then you sort of see Cherilyn as someone not from the same space, I don’t know how much of that is pre-knowl-

JH:

Yes.

edge. And how much of that is because you guys are in a VR world? I don’t know what you’re supposed to sense?

NC:

Because you were facing away from them right? PQ:

It’s just because she’s (CW) so big.

them at all.

NC:

So it’s the size?

PQ:

So it’s like (both PQ and Khai demonstrate craning

JH:

I was only talking to Cherilyn. I wasn’t speaking to

Because we (Khai and PQ) know that our secret

was being spied on, so we were like, “aiya, not fun already”.

their necks and looking up)

NC:

JH:

Oh, is that the sense?

And also because she’s (CW) not responding “cor-

rectly” (accurately), she’s glitching, she’s not really touchKK:

I felt like, I want to do it more! Cause they were

ing you, it’s not real response. She’s responding to her im-

commenting on us giving things to one another, so I went

age of herself as an avatar responding to you guys. So it’s

(demonstrate the passing of things to each other more rig-

like a level separation.

orously). NC: JH:

Cause they also had a tone. I’m like, “now they are

And that happens to the mist as well right? Be-

cause I hear that people in the VR world don’t respond to

just passing things to each other”.

the mist very much.

KK:

Ya.

JH:

For me, the mist for me, it didn’t really matter any-

Yeah, right! “Are we entertaining? This IS enter-

taining.” (While demonstrating the passing of things to each other)

more. I felt like I was the eyes for Cherilyn. I felt like that. PQ:

I’m like, “so she (CW) now knows what we are do-

ing”, so she came over to taichi our arms. So she is privy

NC:

So in a way the performers deal, although in a VR

already, no longer secret world. Then she succeeded lah,

world, is they still behave like a real human being, although

to break in.

they relate to same size, same shape, more reality than not right? (Everyone: yes, um.) Time to pronounce (an end?)

KK:

Yah. Let’s go back to the trying to reduce things.

Session #6

(Laughs)

PQ:

Because can you imagine we were so fascinated

JH and CW: No.

throwing things through Cherilyn... (Everyone laughs, Khai: yeah!) We were like “wow, we have the power to do that

CW:

She will just tell me my location.

JH:

(nods) And I will tell her if she was getting closer

leh.” KK:

Oh Cherilyn - you couldn’t pick up the giant ball

right? (CW shakes head)

or when she was in your space. And if she was very far away, I’d say “you’re facing them, but they are not really noticing you, they are passing objects to each other.” Ya, so

PQ:

Her rules are different.

in that sense I found like I’m mimicking you. Then I would try to direct you (CW). And you two (Khai and PQ) were

TL:

Oh yea, I was going to allow her to just whack the

like ants.

balls, but I didn’t know how much we can go so...I just haven’t tried it yet. I’m gonna put that in.

KK:

NPCs. (Everyone laughs, “yah”) Ya, Cherilyn felt

very far away. NC:

What if Cherilyn was in the middle of the (Nine

Years’ theatre studio) space, and they (Khai and PQ)

PQ:

Distance wise?

KK:

Not really, actual space the distance was the same

couldn’t hear you all (Jo and CW). Maybe you (Jo) talk through an earphone.

but I don’t know whether it’s because I know that Cherilyn JH:

Or turn on your music. And then you’re (CW) here

is further away, and...(Thinking)

(middle of the space), and you (CW) won’t get affected. JH: CW:

Yeah. (Recording abruptly cut off)

(to Tim) There is a barrier. Like she (CW) couldn’t

get too... TL:

Too close? (Jo nods).

KK:

Is that an actual barrier?

TL:

No. I didn’t change the setting. Just that this

AFTER PLAY 3 PQ: JH:

How did you (Jo) feel? It was more focused. I felt I was really helping

(points to Azure Connect) calculates the distance from

Cherilyn out. (CW laughs and “yeah”). I was the medium

here to there (Tim points from the end of Azure Connect to

between her world and yours (PQ’s). I felt like I was on

the other end of the room).

Cherilyn’s side. Like when Cherilyn was making really aggressive movements, this was the only way that you guys

CW:

would notice her? (Laughs)

not be detected.

PQ:

TL:

Cause it will be like, “Eh? There’s a foot on my

head!” (Laughs)

Basically when I tried to get even closer, I would

Yeah, so one metre in the virtual world, when you

travel one metre, but this (Azure Connect camera) doesn’t affect the distance you are facing, about 2.5m for a hu-

JH:

Then I will be like, “You’re stopping them! Stop

man? So if you get too close you will lose (Tim gestures the

them, stop them!” (Supporting Cherilyn) Like I was on

camera lens’ capturing of the full body), because the body

Cherilyn’s side. Cause I wanted her to also be noticed by

is huge right, so humans have to be about 2.5m away from

you guys (PQ and Khai).

the camera. And where the camera is, it is basically point zero, so you will have to try and get as close to the camera

KK:

Were you (Jo) telling her what to do?

as possible, so I can change the (inaudible - nu?) Speed, which we are going to try out later, it moves at ten times the distance.

117

KK:

Ya, so that Cherilyn can be all around us right? (In-

KK:

Oh ya!

CW:

She (Jo) was my only sense.

dicating the perimeter of the space around us, in the virtual realm) TL:

Yah.

PQ:

She was your informant.

KK:

Yup, I know that it’s all part of it, cause I know that

CW:

Yah. She was my sense of touch, I guess.

corner of the room).

JH:

And sight. I was your eyes in the world.

TL:

NC:

Up till this point right, as a spectator, I could

she sometimes is stuck in that place right (gestures to a

That’s right.

only access the eventual expression (gestures to PQ and PQ:

But I knew that she could still step past the barrier

above us, so her foot could come down.

Khai), from all of you like what are you doing (demonstrates some physical gestures from PQ and Khai), as a movement or whatever, or what Cherilyn is doing right.

TL:

There’s no actual barrier, that’s a technological

barrier? (Laughs)

But without the knowledge of the set-up, I think up to this point lah, it doesn’t quite matter to me who’s real, who’s in what, in a way. It’s a bit like what I said in the be-

JH:

Ya, it was frustrating cause I wanted you (CW) to

ginning, like sometimes when we watch anything now,

have more impact into your (PQ and Khai’s) space in the

we don’t really care whether it’s live or it’s pre-recorded,

VR world. But because I could see everything that was

and technology in movies progress so much that the ren-

happening spatially in the VR world, I was like, “why isn’t

dering of the character can be so human-like right, you

she (CW) moving closer? She is facing the right direction,

also don’t care whether it’s a live character performing

but no matter how far she, you (to CW), move forward, you

or whether it’s a CG character. CG characters are also

just didn’t have a trace in their space.

life-like what. I’m also thinking now, for a spectator, what are

KK:

She still didn’t feel like one of us.

they watching? So when we are developing this thing also as layers of realities, are these realities actually

CW:

Ya.

more important to, for the performer? For them to create, to rehearse or create in a very different way, or is it

PQ:

Just more able to affect us this time round.

relevant or does it have any significance for the viewers? Is it a tool, or some knowledge for the creators? More

KK:

That’s (inaudible - nodded in agreement)...but I

than for the spectators?

think what we should aim for in the next round is to get

I mean, they are just questions lah.

Cherilyn to be with us. To be on the “same team”, even

Is this some... New ways that artists could cre-

though she’s Giant.

ate? Even when apart, maybe? Some choreography?

PQ:

JH:

We could pass things to her? Cause I wanted to

I was thinking about... This is in the same vein as

pass things to you (CW), and see if you could at least bat

your thought. When we were doing the VR jam sessions,

them away?

like you know how when we did the multiple projections, we could fill/ feel the story as the spectator, we could see it

KK:

(to TL) Can we? (TL thinks)

all, and we could see when you (CW and KK) were playing with the shadows on the wall, and when you (TL) were

CW:

Without seeing, it’s hard. Cause I have no other

senses.

playing with my projection, there was a story of me being uh... Of you (TL) trying to conceal my projections, and I was trying to hide. And these were motions that we could

Session #6

read. But then, in these VR sessions, we’ve been trying to,

interact, maybe by sound, or by the cursor, or by the azure

I think like because we are fine-tuning the factors... I’m also

connect or some projections, so that I’m also creating the

wondering like, what are the movements or language that

work with the artist. That could then be the spectator’s ex-

comes out of this? Because... Is it punching? It seems like a

perience that they are having with the piece.

lot of it in VR, because as I narrate, there is a lot of this motion (demonstrates two arms with clenched fists pumping

PQ:

Then there will be more satisfaction, cause they

into the air).

know the rules of how to engage with this “game”.

I’m trying to figure out if there is something else we could introduce, like, maybe (to TL) there are not just

NC:

So they are the co-creator and that thing is being

objects to be thrown. Because there are objects that can

recorded or filmed, for the other participants so that they

be passed around, the movements are already scripted out

co-create the work. So that set-up is basically an installa-

for you (demonstrates two arms passing things forwards

tion.

and backwards), because that’s how you can interact with the objects.

PQ:

It’s like those kind of interactive museum lah. It’s

better than watching something passively or watching a KK:

Do you have an interaction in mind?

final expression or product, then you don’t know what’s been going on. Cause Cherilyn could be pre-programmed

JH:

I’m not really sure. Maybe you guys can help me

out here...

what. Like she doesn’t have to be “live”. Maybe we (Khai and PQ) are live. And we could have just rehearsed our movements according to what a pre-programmed Cheri-

PQ:

Or pull something out, like a creature out, of your

pocket?

lyn does, then that is very staged, and it doesn’t really matter whether it is in the moment or not, to the spectator. So if the spectator knows that “oh, okay, these are the rules”,

JH:

In those projection mapping sessions, there was

like this is how this is supposed to be “played”, or these

kinesthetic response to the fiction that you are creating.

are the limitations that these actors are working with, then

And also response to the environment. So you are creating

they also appreciate the creation of it, then there’s that kind

these yourself, with your body.

of satisfaction or understanding on the same plane as the performers.

PQ:

It’s organic, and also human formed. It’s live, it’s

alive. But just now Nelson was also saying, can the... Be-

KK:

I have an idea...

it be? And we still can’t see certain body parts. That’s the

JH:

I just had a kind of idea. As soon as you (Nelson)

main difference right? In our first projection exercise and

said musicians, I was like (snaps her fingers). But I will de-

this (VR experiment). Cause we are no longer, so... Human.

scribe it after you (Khai) share your thoughts.

JH:

KK:

cause now we are looking at avatars. So how lifelike can

And the movements become super exaggerated...

I think of things that have a more permanent ef-

fect on the space, and also things that can be reversed, NC:

If I move back one step, also to your (JH’s) point,

or things that can have interactivity with other people, for

what if the design of the set-up itself right, is the art piece

example like a ladder, or a button, or a steering wheel, let’s

right, the design of the set-up is the work, so as a specta-

say Cherilyn pulls a lever, and the ceiling starts coming

tor, I’m actually just looking at this theatre set-up, like the

down, like in a temple, and then like you (PQ) have to do

performance is of this people, like in a show I can see the

something, maybe solve a puzzle, so that it (the descent

musicians sometimes, like they are exposed. What if let’s

of the ceiling) stops happening (PQ: like we crawl out of

say there is a dancer or performer in there (gestures to-

the tunnel. Khai: yes we crawl out) and then water starts

wards imaginary VR space) - this performer’s performance

to rise, and then for us we’ll go “oh my god the water is

will be affected by this other artist, organically. But also as

starting to rise, and we need to do something in this space

a spectator I’m provided with a space, and I could also then

to make it stop happening”.

119

JH:

Right.

for today right? It is still an experiment. We can afford to keep playing as we approach... Well not many sessions left

TL:

So this is the fiction in this world. If it were de-

(laughs), as we approach, coming up with something lah!

signed to have narrative in that sense. TL: JH:

We have PC connected to VR, connected to the

Right. I had a thought because VR is visual. It is vi-

Azure Connect, and then AR, and we can close the loop

sually and audio-ly immersive. And whatever motions that

with the projection. Right now we are working with a lot of

you have is a direct response to the visuals. And I’m like,

information and we are opening up new environments for

what if, we use VR but as a way to connect the players in

us? And I love how all the energy and new ideas come in,

the world. But maybe it has only one environment that is

what if we did this? And it inspires more people to use this,

not interactable? So there’s no set motions at all. But you

our guests right, because we provide them with the ability

are reacting to sound, or music. To create a fiction for you.

to experience it. It’s actually quite awesome to see Cherilyn

So like, you are responding to sound, in another environ-

interact with that (points to Azure Connect) and interact

ment. It’s not interactable, you are just in it. But you are

with their world (PQ and Khai), and we can see Khai doing

moving to the fiction you are creating for yourself as a re-

this (demonstrates waving his arms up in the air) doing his

action to the sound. As for the third person, I don’t know... I

own thing, punching in the air.

just thought about this too, so it’s not really coherent. (Everyone laughs.)

The sad part for me is also how isolating it feels too, I mean for Cherilyn, you can add more people, that’s

But the point is, VR is so focused on being so vi-

definitely possible depending on what your avatar is,

sually immersive, it already creates a vision for you. I mean

maybe two or three people dancing, but when it was

it’s ok if you want to use it for that, but for me it’s also ok to

like a lone dancer facing this (Azure Connect camera),

use it as just another world, where you can communicate

there’s no one else in the room but you and a computer,

with someone else and then introduce sound to create the

it feels quite sad. Tik Tok for me is also quite sad, you can

fiction.

have millions of viewers and followers.

PQ:

By the way, as we are just discovering now what

PQ:

The question is whether she (CW) can have any

we are doing, along the way, just to connect both your

real impact on someone else? (That exists outside of this

ideas together right (KK and JH’s), for Khai’s idea, you

VR universe). Because if it’s Tik Tok, then she’s really like,

already know that “ok, press this lever, this might come

hermetically sealed up right? She is just beyond a screen

down” and you kind of know what you are in for, there’s an

or a frame of the Tik Tok roll, or the TV. But let’s say if

objective right, there’s a stake. So that’s how we create a

she can affect us, that’s what we were saying right, then

world of fiction for theatre. And we know “oh we need to

she will have some real impact and some real stake, like

crawl out” we have an intention like “get out of this place

when she steps on us, we really feel like, “shit, we are

alive.” Like just now, our aim was to see how much Cherilyn

going to die.”

can interact and affect our world. So as long as we know what we are doing, then the fiction can be stronger.

TL:

Like we could add some gestures, or some jumps

in, that’s what we could do. TL:

So now Cherilyn is massive, but what if the VR

player is massive? It’s like a rubrics cube we could move

CW:

around.

day right, like trying to look/ get in, but then “tak boleh” (Recording paused)

(can’t). PQ:

AFTER PLAY 4

I think that’s been the tall woman narrative for the

That’s why, then she really sad. (CW: So sad.) And

needs some help (gestures to Tim), but we haven’t really gone there yet.

NC:

Anyway we don’t have to find something for now,

Session #6

TL:

I think Nelson said very well in the first round, and

“I’m back in my mother’s womb.” It’s just like, you lose

Pei Qin and Cherilyn were very near each other, and you

your body. For VR headsets, well, your body’s still there.

all were behind the VR headset, it showed you something

And then, immersiveness in the planetarium is also like,

else, but as a performance, it didn’t really capture digital

you’re also in a theatre setting, I don’t know lah. (CW:

reality, like you noticed that Cherilyn was present physical-

There’s more moonlight?)

ly here, for me I found that quite powerful in the sense that there is actual isolation, or separation of mind and body in

JH:

Yes like more moonlight, 360 (degrees). (JH, TL

that sense, and the technology itself, which is VR, can take

and CW gesture above their heads miming a roof over

the mind and maybe spirit out of the physical location. And

their eyes)

have the human body just as a shell and trying to interact... This is like how people are actually like that, the body’s on

PQ:

It’s even more, we are merging with film already,

the train but the mind and soul’s completely not there. I

meaning the zoom-in and all that goes even more intimate.

don’t know, does this exist in theatre?

So when you’re saying just using the eyes, just visuals, this time the immersiveness of VR is even more, it helps your

PQ:

Time and space don’t need to be on the same

plane.

eyes more, to look at things. That’s why we say, the Ariana Grande thing, it’s like, you don’t even need to buy $400 ticket. You sit in front to get closer to her. Now everyone,

TL:

Ya, so as an actor it’s quite tough right? For me it’s

like, once the VR goggle is up, the body is going to be a

who has access to this technology, is VIP. You can even “climb on her”.

husk, and the mind and soul is somewhere else. KK: NC:

Talking about immersive experiences reminds me

In a way, VR to me, the action is to immerse. When

of, probably the most immersive theatre experience is not

I put the goggles on you right, I’m trying to immerse you

Punch Drunk, it was Julius Caesar at a British theatre, be-

in some other time and space in a way. I think in movies,

cause they included us in the narrative. So the first thing

in cinema and theatre, they have been trying to do this in

when you walk in right, what you see is a band playing, like

another way right. I mean when movie or theatre darken

you are in a club or a bar.

the auditorium, they are trying to put this big goggle onto you basically, with surround sound speakers, and the ac-

JH:

Where is this?

right. The earlier theatre tradition, without even the tech-

KK:

British Theatre, London. So you come in, and

nology right, they put the audience there, the audience

there’s a band playing, they are a top band. And they are

can see the other audience, they are already immersed

playing songs against Julius Caesar. So you are there at

in a way, so it’s not dis-similar. But what is it that the VR

their concert, and at first you are audience at a concert,

technology now... Tim you mentioned - isolate. So it’s not

and then when they start the show properly, to get Julius

community immersion, but now our immersion is quite

Caesar to enter, there are secret service agents who come

isolated (demonstrates placing rounded hands to eyes

in and they go past you “eh move move move move!” And

to mime goggles). But yet it can be connected to avatars

push you all to the side, to make like a pathway for like

and things like that. So how do we use this as an advan-

“Donald Trump” to enter. So we became the people who

tage? (Jo: Yaaa...) In creative ways.

go to those Trump rallies, and these type of events, and

tors are also trying to immerse you in their fiction as well

we become part of the narrative. Another example is, in JH:

So interesting... I’ve never thought of immersion,

the second act when things get very messy in terms of the

or an element of immersion as, taking away the body. Be-

fighting, they have people coming in with guns and SWAT

cause like as you said, the giant goggle as the theatre,

uniforms, and they actually moved us as hostages out of

every time you’re in the theatre you sink into the chair,

this “war zone”, so that was the best way they included us

and you’re in the dark, you lose your body and only have

in this narrative, I think. So how do you do that with VR?

your eyes. And every time I go to the theatre I always say,

Cause VR does that in different ways. And Punch Drunk

121

does it in very different ways. The actions that you choose

are so upset, and (Nelson: tensed up) and you feel that

to do, immerses you in the narrative. Compared to the Ju-

trauma, because your body, seeing that action makes your

lius Caesar one which is they move to you. So there are

body feel the same sensation.

these three ways, how do we play with that and negotiate that? NC:

Right before that, I was just thinking that, be-

cause Jo said “losing the body”. I was thinking, one im-

JH:

Ohhhh.

PQ:

Mirroring. Nelson: Dance also (demonstrates

some dancing moves).

mersive experience that we don’t lose our body, in fact we activate our body, is club scenes, dance, concerts,

PQ:

It’s called the emotional contagion.

KK:

I only know it as mirror neurons.

PQ:

Ya, it’s related.

JH:

I’m collaborating with someone who does exper-

carnivals. Where you immerse, you don’t lose your body. Actually it activates your body. So that kind of experience right, is more akin to projections, where sounds and lights are projected, onto you, to activate you. This is different from our experiments with projections and our VR ones. Maybe that is kind of two different kinds of theatre?

imenting with music, and he performed this piece, it was PQ:

Because this one (points to Khai), you are expect-

four minutes of guttural screaming, so by the end of the

ed to perform what, the Julius Caesar one. Because the

scream, he sounds like he’s gonna throw up. It’s sounds

lights and sounds are on you, and you see everybody else,

funny, but it was so intense, the whole time, as soon as he

they are unwittingly implicated into the action. Not that you

did that, I could see him suffering. Since he’s screaming.

wanted it, but they co-opted you, sort of, into it. And then

So for the first minute I was shocked. But for the next three

they shine light, everything, on you. Then you become, of

minutes I was like crying, I couldn’t stop crying. (Khai: He’s

course, an actor, unwittingly.

suffering.) Because he’s suffering and I could feel it.

KK:

PQ:

Every time you did an act, you knew that your body

Theatre of Cruelty right. Artaud or Grotowski tried

had a place there. There was a character placed upon you,

to do that. They are theatre-makers who kind of used this,

as soon as they made it real for me.

assault of senses, to the point where you feel “assaulted”.

PQ:

JH:

What is it called?

CW:

Artaud.

or cries, you will also laugh or cry. The whole body is ac-

PQ:

Antonin Artaud, Theatre of Cruelty. Although we

tivated, and then it will affect the audience all around you.

don’t want to be throwing theories or terms around, be-

So that is not “losing your body” lah, in a way.

cause you can interpret that in your own way lah.

KK:

NC:

Yup, although what Jo said about being in the

darkness, and you lose your body right, actually let’s say if something very emotional happens on stage, because there’s always a feedback loop right, so if someone laughs

You know about mirrors neurons?

Artaud is difficult to understand, because he

doesn’t write it well. He was mad. His articulation was like JH:

No.

all over the place.

PQ:

Yes.

PQ:

He was in an asylum. But we only learn about him

through his works lah. KK:

So it’s part of your brain that activates the sensa-

tion of feeling and doing something that you observe. So that’s why in theatre right, when you see an actor, and they

Session #6

NC:

Yeah. Also he doesn’t create a lot of work (laughs).

KK:

He just talk only.

NC:

Can the people wearing the VR goggles appear

on the projection as avatars? Like full avatars? PQ:

Jet of Blood. TL:

Full as in, with legs as a (inaudible). With head and

CW:

He did Marat Sade right?

hands yes, you can’t track the rest of the body.

JH:

(to Khai) Maybe that’s his work. Himself. His life.

KK:

Until we get to the rococo suit (laughs).

CW:

Copied from Butoh.

TL:

So normally you look at the legless body.

PQ:

Yah, Butoh as well.

JH:

How did you (CW) feel this time?

JH:

What’s Butoh?

CW:

I felt a little closer in control, and better able to in-

teract with them. I felt that I was more able to interact with CW:

The Japanese dance performance. (Demon-

strates) Ok I totally butchered it (laughs).

them in the real world, and that they were more connected to me, in the real world. But in the virtual world, I think I’m still navigating my own limbs. Cause it’s kind of the first

JH:

Ohhh, yes yes yes!

time I see myself? And it’s operating on different laws then I am. So it’s getting used to that while trying to get used to

KK:

Do we have another play session?

their narrative.

JH:

We can have projection?

JH:

I like this idea that there’s a new body that you

have to control, and you’re trying to navigate this body PQ:

To close the loop?

to interact with other bodies, that some other people are trying to control. It’s quite funny, it’s like touching phan-

Everyone: Ya. Let’s try.

tom limb to phantom limb. I think this is what we are trying to do with other people on the Internet? Like through words and through bodies, through representations of

AFTER PLAY 5

ourselves, avatars, words, voice memos. But it’s all within the limitations of the technology that we are using.

NC:

So just now I was in the VR world, although Tim

So you (CW) were glitching cause this (points to Azure

did not make me invisible but I just stopped doing lah, and

Connect) doesn’t really map your body movements that

I went to the side just as an observer and thinking. I was

well, and then you guys (PQ and KK) don’t have legs.

the link between the two of you (CW and PQ), so it felt like

(Laughs) So it’s the limit of the technology that hinders

what Khai was describing, like bringing into this Caesar’s

this connection process.

play, so up close and looking at you guys interacting with this giant, but as an observer. So that experience was rec-

CW:

Ya I think I was asking Khai straight up if he could

reated lah.

see me? And then he was saying “ya, I’m leaning on your leg”, was quite funny. (Everyone laughs). So he basically

JH:

It felt more inclusive this time. Cherilyn felt like, you

was trying to tell me by touch. It’s like up to my knee of my

guys were playing with her too. You were like, try to catch

leg. Then he double-checked with Pei Qin (laughs harder).

this thing we were trying to throw at you, even though you were not really in there. Ya, a lot more connected.

PQ:

(re-enacts) “Yes, you are leaning on her leg.”

PQ:

JH:

It’s like you were in between her legs (laughs).

Cause we know she (CW) can see us now. So we

naturally talked across the space to her, like she’s part of the team lah.

123

PQ:

Oh yeah (laughs). But then I realised that he was

leaning on her leg.

JH:

Yes! Mirror therapy, where it’s like if I don’t have

my arm here (retracts her left arm), but I have this arm (extends her right arm), so there’s a mirror her (mimes the

JH:

Yeaaah.

space in front of her) and I’m looking at the mirror (image) of my right arm. And then I do this (flexes her left arm), I

CW:

In which case I was still unsure because I had no

indication of it.

think it has something to do with touch as well. Like someone touches your (right arm with your left, as reflected in the mirror), and then you see it, and..

KK:

( jokes) You didn’t feel me leaning on your leg? KK:

CW:

NO. (Laughs) Oh well, but I tried to through the

visual lah.

Ya, and you start to feel it on the other side (that

isn’t reflected) as well. And then they will cut it off (gestures to the left arm). And they will really cut it off! (Jo laughs incredulously). No! The reason why people don’t feel pain

PQ:

So you saw him?

here (gestures to his left arm) is because the body doesn’t realise that there is no arm there.

CW:

I could see, but you guys were a cluster, like at my

calf. I guess you (PQ) can see more clearly like (CW and

JH:

Cause it’s like muscle memory?

perspective of a tiny person around a giant calf.) But for me

KK:

I suppose your body’s like “there’s supposed to be

it’s just like, they are a (tiny) cluster at my calf, like I can’t tell

an arm here, but there’s no arm here, so that’s why it’s gon-

what you were trying to do.

na hurt, that’s why it’s painful.” Then this mirror therapist

PQ demonstrate a tilting up of the head to look up, from the

goes, “but hey look, there’s nothing there.” PQ:

(mimics Gulliver’s travels little people) Like poke

those mini arrows at you (your leg).

JH:

Yeah, but it’s so interesting that your mind has

the memory of touch. Even if the arm isn’t really there JH:

That’s so funny.

anymore. But how can we create memories of things that never existed once?

TL:

You can’t actually feel them there? (PQ: No you

can’t. Laughs).

CW:

And I guess that’s interesting to investigate, in

terms of having a remote player. Or in times of covid, CW:

But maybe I was on a closer scale to them. Then

maybe, in terms of my visual, it would activate my sense of

where you rehearse in some other room, where you can’t feel the sensation of some other person.

touch, and I can imagine the sense of touch. PQ: TL:

We can actually try scaling them down so they

could reach out to you.

What if you all are not avatars of humans any-

more, and you all are just... An octopus? Then how do you feel, your phantom limbs? (Extra legs that are not actually there.) Because you are not experiencing it as a

JH:

That would be crazy if you imagined touch. Cause

human.

you know how, speaking of phantom limbs, you know how people get rid of their pain?

CW:

Actually yah, cause this one (Azure Connect) I

can’t really move my fingers. When I tried to wave, but all CW:

Oh they sand their prosthetic. (Everyone: Huh?)

it came out was (demonstrates a broken wrist flopping up and down).

JH:

That is one option. PQ:

KK:

You’re talking about the mirror therapy thing right?

Session #6

Yes, and you can’t clench (your fist) right? So

you are sub-human what, like some alien.

CW:

Yeah. So in a way I had to relearn how to move.

KK:

Oh, which reminds me of Michael Chekhov’s

work. So the imaginary body is how actors use to imagJH:

I love that you said octopus, because I was read-

ine how, they are bigger than they are. Or smaller, or they

ing some text on Presence. I forgot the name of the writ-

lost their limb, or one leg is shorter than the other, so

er. But she is a theorist on Cybernetics. (To Khai: I think

they can play different characters. So this, it might be in-

you might know her.) I think her name is like... Ugh, I will

teresting to experiment with what you (Jo) talked about,

figure out and tell you later. But her text is about em-

compared to how he (Chekhov) did it, and in VR. How

bodying non-humanoid forms in VR, and for the octopus,

they all connect to each other.

if you can imagine training your arms to be in different positions, like if you see it in VR, you train your arm to

JH:

How did he do it?

reaching out far behind her), and then you keep seeing

KK:

I can do it with you guys later, next time. But

it (this arm in this position) right? And then you add on

also, when we are talking about physics, maybe we could

or something? To the point where you have eight legs.

talk about... You could make out avatars, like move much

Maybe you could have muscle memory of the arms here,

faster than we actually do, so that when we do like this

here, here, here (gestures to eight possible positions of

right (demonstrates a slap), we are actually way fast. So

eight long legs fanned out around her body). And then

that memory of that experience of that movement, can be

you can start to feel like you are the body of the octopus.

translated into physical acting. Which is actually what I

be in this position (demonstrates stretching one arm out,

have been trying to do in my own paper, but now I see the PQ:

This is very relevant. Muscle memory is what we

physics of the space.

do in rehearsals too. Actually I’m interested in Nelson’s experience because he just put on the white VR goggles

JH:

One of my friends in Berlin, he made a game, and

for the first time. And as an observer I felt like you were

it’s a light saber game, but it’s like the faster you swipe, the

relearning, like “wow, so fascinated” (holds up both her

slower your VR thing goes. The faster you swipe, it’s like

hands as if seeing them for the first time), like a baby

lagging. It goes really slow, and then it goes faster.

right. PQ: NC:

Yeah, it’s a bit like having another body lah. So

You mean like the light saber object has actual

weight to it?

whatever your expression is, seen through your own eyes, to another kind of projection in a way lah. And also,

JH:

It’s not the weight but how it looks, like how fast it

objects in the VR world are weightless. So that kind of

goes, like visually, in the VR headset. And it was weird. But

disrupt your sense of the world, I know we can key in the

it’s interesting, because you are trying to match the speed

physics lah.

of the light saber.

But you pick up a big giant ball, is the same as picking up a small thing. And you tend to forget... We are

KK:

You just reminded me of a game in VR...

his feet grounded on the floor).

CW:

Is it the music one?

PQ:

KK:

No. “Super Hot” - do you (Tim) know it?

Tim:

Yeah, “Super Hot”. The one where time only moves

very used to our own sense of gravity already (points to

Oh yah, and the three of you are very close (in VR),

but when you remove the goggles, you are actually very far apart. It disrupts our sense of proximity.

when you move? Yeah. I have it, you can try it out later. NC:

So I’m thinking how these VR variables can...

Like as you (Jo) say, how these octopus hands can be

KK:

It’s a really interesting game, one of the best sell-

dramatised, how these variables can affect the way we

ing VR games.

relearn certain things so that we as performers, we create things that we don’t normally create?

125

PQ:

What does that mean? Time only moves when

you move? TL:

Because there’s a... When you move your head like

this (tilts it to his left side), time will move with you (gesticulates to the empty space in front of him to mimic time moving?) KK:

So it’s game where there are a lot of players to

shoot. So if you are not moving, no one moves. So if you start going sideways and shooting, everything starts going very fast. (Demonstrates) CW:

Can we try?

PQ:

Is there a technology where, when you want to do

something (demonstrate a whack), the handle/ controller actually vibrates? KK:

Yes, Tim is showing something. (Tim plays a video

on the computer, a trailer of the game) PQ:

Let’s play this, like a warm-up.

KK:

Yeah, as a warm-up will be interesting. Shall we

move on to talk about design for the next session? PQ:

We need another zoom meeting to consolidate

another tech direction right? JH:

Yeap.

NC:

We need to digest this also.

PQ:

Yes, we need to digest also.

Session #6

127

SESSION #7 October 04, 2021 // Nine Years Theatre

Jam Sessions

129

ITINERARY 7:00 - 7:30

Orientation

8:00 - 8:30

Fictionwork with Nelson Chia / Orientation

8:30 - 9:00

Play 1 (VR with AoR)

9:05 - 9:10

Play 2 (VR with LASALLE Students)

9:10 - 9:40

Talk

9:45 - 9:50

Play 3 (VR with AoR)

9:35 - 10:00

Talk

10:00 - 10:05

Play 4 (VR with LASALLE Students)

10:05 - 10:15

Talk

10:20 - 10:30

Wrap up, pack up

Session # 7

TRANSCRIPTIONS OF OUR SESSION TALKS FICTIONWORK WITH NELSON CHIA CW:

We were exploring the idea of that we could create a shared fic-

tion for the performance...Because like we are all working on the fiction in our individual minds. So he [points at Timothi Lim setting up t he VR Headset] - has created landscapes that narrate a certain type of fiction, and he has done that in the VR World. She [gesturing towards Joanne Ho] usually does the projection stuff [Video mapping projection]. So what’s going to happen is that there will be three people in the VR world and CW

Cherilyn Woo

Joanne Ho

Khairul Kamsani

Nelson Chia

Thong Pei Qin

Timothi Lim

there will be one person outside. So one person is in the physical body space playing with the projection. So all will be sharing the same world. So you’ll see people just navigating their way like this (points to Timothi in the headset) but they can’t really interact with me. And then there’s going to be one person there detected by these... PQ:

the Azure Kinects

131

CW:

...things that detect the body and feed it back in to

the VR world. (Students exclaim in awe)

so all the spaces of the body are open up - open palm, open chest, open abdomen and we are ready to receive.

So then there’s a closed feedback loop in the

Then I’ll take them through the five senses. Start with just

sense that it’s not two separate worlds, we try to meld the

seeing, seeing things, and take the time to see things. And

two worlds. So to explain more about fiction...[Points to

keep moving the body. See and then move. Sometimes

NC]

I’m just here and then they realize there is space. So seeing, and then we add hearing. (Snaps fingers) So hearing,

NC:

So I have been working with directors and actors

and then we add touch (touches the ground and body).

on this thing I call “Fiction Work”. It’s still a work in progress

Not just your hand but perhaps the back of your leg or

so I will not claim that it has a very full fledged pedagogical

even the clothes on your body. So touch and then smell

thing going on with it ...yet. But then I thought there might

and taste, the smaller senses. So in a way we go through

be some things...And now I have a better understanding

the five senses and then when we have time, I’ll mix the

of Meisner because of Khai (KK). And I thought it is not

senses together. So if you’re seeing something and hearing

very different from what Khai has been saying, it is a very

something at the same time, what is the sensation. So if I’m

different approach but it (Fiction work) addresses a very

feeling something soft and I’m looking at the hard wall. If

important question. If you say something or you do some-

I’m wearing something soft and starting at the white hard

thing, where does it come from? So if I point at you and

wall, what does that feel like. I really want the actor and

say “you have a grey shirt”, it comes from what I see and

actress to a state where they are overwhelmed by senses,

I do something and it comes from what you do to me. So

and I get them to move their body with all these senses,

in a way it is to say the “fiction work” I’m imagining is also

and breathe and move and move. It gets vigorous until I

trying to get actors to be in the moment; to be honest ba-

calm them down and ask them to go into a Composed

sically. It has to firstly equip the actors and performer with

state where they can start composing in a way. And that

certain things. So I’ll give you a sense of it. Could you stand

is where the fiction work comes in. That was all prepared

up? (Students stand up)

to get to a Wakeful state, and now all of their senses are awake. The little things like colors, sound, texture, touch,

NC:

We’re standing up just to get a sense. It starts

smell, even your own sweat is all affecting you because

with some calibration work where there are four states of

you are so wakeful. You can hear your own breathing and

the body. There is the everyday state, which is where we

all these things. Then comes fiction work, there are three

are now. Like Khai said, you come in as a person from the

types. You can sit down now. (Students sit down)

street, and then you do some exercises, and you being, you

There are three types of fiction work. So I remem-

become an actor in training. Same thing here, you come

ber your character today [points at one of the students], so

in as an everyday body and then I’ll take the actor/actress

you’re on the Brooklyn Bridge in New York right?

through a series of exercises that get very vigorous at the end. But we’re not going to go there today. What happens

Student:

Yes

is that the exercises go up in step and that opens up the actors / actresses senses. So I usually start by recognizing

NC:

I’m just taking that as an example. So your charac-

the center and I will ask the actor/ actress to start to sway.

ter will say “this is a crazy place”. Maybe just that, how will

You can join along with me {Students start to sway with

I want to say it? I could say it in a hundred ways but which

Nelson).

one? Which one to choose? If I’m in a rehearsal and I work So the whole exercise I keep stressing that don’t

with fiction. One is where I see something, a fiction that is

stress the leg. Even if you’re going slow or fast. So the cen-

outside. So I’m on the bridge and what do I see? If I’m in

ter is like a ball, it keeps swaying, so we keep it in motion.

rehearsal I’ll ask “what do you see exactly, just tell me one

And then I’ll take them through and speed it up. And then

thing”. It could be lights maybe. “I’m on the Brooklyn bridge

I’ll take them through the senses where I’ll ask them to

and I look down and there’s lights, I really see the lights”.

open up their palms and feet and leg. The first thing would

What kind of lights? What color? How bright? How many?

be to open up the joints of the body. Eye socket, nostrils,

Flashing? And you work with that fiction, it’s outside of you,

Session # 7

and I want them to express. It can be non-realistic. It can be

are uncomfortable, please do let us know. So we’ll play

abstract. Imagine the light hitting you. Is it warm, is it harsh.

first, and feel free to walk around and check out the be-

Is it crazy? “Yes, this is a crazy warmth”. So you’re working

hind-the-scenes. The plan is that after we experience it,

with lights, that means you can work with darkness as well.

you’ll experience it as well.

“This. Is. A. Crazy. Place”. So the fiction changes you, so the sensations makes you breathe, stand, make you move,

Student: Do you want observations?

stand, fall, contract and expand differently, and then you say your line, sing, or express. So one type of fiction. The second type is the environment. I’m on the bridge again, okay, it doesn’t matter what I see, it is what

CW:

Yes, so normally we play and then we get obser-

vations and that is a free world. So we appreciate your observations too.

I feel. How hot or cold? How cold? Like all tense up cold? Windy cold? Hard to breathe? Like how the cold air goes into your windpipe? Short breath, long breath? Do you feel

AFTER PLAY 1

like walking? Sometimes it gets so cold I don’t feel like walking. So the environment, the skin, the touch, the wind-

CW:

I felt when the world came on, it was quite awe-

pipe and even your eyes. How you speak your mind and

some. As in it was really quite like the.. The feeling of sud-

the feeling of the environment. Maybe it’s noisy or warm.

denly seeing yourself in water. Even thought it wasn’t total-

“This is a crazy place”.

ly realistic, it does feel immersive.

The third type is that you are the fiction. You are the fiction. So what if I am the water, the dark water of the

JH:

Yea, you guys were all like “woah”

the water, I am stable, and I go with it, and I breathe. “This

PQ:

It felt destabilizing. Like “WOAH”. Because the first

is a crazy place...” It’s a different way of saying it. It’s an ap-

world is all pitch dark and there was only the objects. So

proach to rehearsal that changes the performance. So later

we were quite used to them and we had a sense of space.

when we see what is inside the VR world, you can think

But when the immersive water world came on, it felt so

about what Khai said, and I can point at the red things over

vast. And even the ground changed. So when we stepped

there. I may not know how to start, I can just point at the

on the ground, everything felt off balance. Just the scale of

red things I see. Perhaps I point and say honestly what is

the world and the distance felt further. So then it was very

that until I am comfortable. Then I let it affect me, and then

destabilizing and I was imbalance while walking

Brooklyn bridge. So I extract away, in rehearsal, what if I am

maybe I breathe with it or I say a line, and then maybe I dance with it. Maybe I don’t want to speak a line, and so I

NC:

I caught myself, while I was in the underwa-

move. So there are language, tools, and ways of which we

ter world, kind of immediately behaving in a way that I

can rehearse and change the performance. It is something

thought I should behave when I am in the water. But a

we can do if we want to. Right, if not we can just admire the

part of me was thinking, am I reacting to the environ-

place (Students laugh).

ment. I was telling Pei Qin, we started walking in a different way, like one sand the foot may sink in, and I started moving slowly as if there was water resistance. Could

ORIENTATION

that be a preconceived notion of how to behave? It’s just a curious question that made me think. One of my chal-

(Addressing the students)

lenges in doing fiction work, it is limited to the actor’s power of imagination. So some actors are quite skilled

KK:

Meisner is how you interact with other people. So

and can go very far, then express it out, but some actors

if we see the way we work between different viewpoints in

do not. It’s only as good as the actor’s imagination. So

VR. So we can do Fictional Work, and we can use Meisner.

I’m wondering if the VR transport allows everyone the

So feel free to choose which one and try it for yourself.

same view, so it aligns the preconceived idea. I’m not

CW:

drawing a conclusion but I’m comparing the experiences

We have to document the process, if any of you

133

between. Is the VR technology helpful or not to an actor?

PQ:

Yea, I thought when we talk about moving under-

That is the question.

water, it was on the floor for a while, and I didn’t know what the story was, I just felt something there. I felt the level of

KK:

I felt out of it. The world. Especially the lag and the

the ground affected the movement.

avatar. It made me spend more time getting used to the world before I felt a part of it. I kept noticing the beings of

KK:

the space. I could not, or I did not notice what they were

tions?

Okay, shall we move on to the student’s observa-

doing. That’s what I think. Student 1 (black dress): JH:

Yea, as an observer, it felt like the world really cre-

Okay, I think while observing you guys that when

ated a set expected movements. Because Pei Qin was do-

the space changed into the underwater world, you could

ing the swimming motion and then a motion that we learn

see that it changed your body where you became either

when we gesture about swimming. That’s a motion that we

super confident or super reserved. It’s like the first reaction

learn when we try to symbolize that we’re walking on sand.

was to concave first and then find your way out whereas others embraced the world. So it’s just interesting to see

PQ:

Did the rest do it?

the initial impulse and how it developed to get acquainted to the space around you. So do we talk about what we felt?

NC:

I did KK:

CW:

We’ll stick with what we observed.

I was too busy being a jellyfish. (Group laughs) I was trying really hard.

Student 2 (Cyan t-shirt): The first thing I noticed was to interact with the

PQ:

I saw that you were looking face down and float-

ing, while Nelson and I were walking around.

environment around you rather than with each other. Even when you’re moving around the space, the interactions with each other seem accidental. It was very interesting

CW:

Oh that was because I had drawn the boundary

to watch. I’m going to draw parallel with our experience.

too small so I had to make do with the virtual space I had.

For ours, we relied on each other first. But with you guys, where it seemed more comfortable in the VR space, the

PQ:

Oh no I didn’t know that! Was it set on purpose?

first instinct is to interact with the space rather than the fellow co-workers.

CW:

No, it glitched and that was the space I was left Pei Qin: So any interaction was accidental? Just like with

JH:

That is so sad!

me and Cherilyn bump into each other?

PQ:

So Sad!

Student 2: Yea, it did seem that way. Especially when the

KK:

So can I just say, because of what Jo said, so Pei

world first came on. Everyone’s first instinct was to interact

Qin, you were doing the motion of the ocean. That’s all I

with the space, and everyone dispersed. And then every-

want to say

one came back together at some point, but it was mostly you two (Pei Qin and Cherilyn) because you were bumping

PQ:

Okay!

each other and came together.

JH:

That’s a quotable moment...It’s so cheesy though

JH:

I love that. What I actually really like is when the

bodies are colliding together in space. Like at one point KK:

That’s the point

you three were colliding and at a certain point, you were all facing each other or facing the same way, and I felt like

Session # 7

the physical presence that was triggered. Then the bod-

different corners of the room and you call out or talk to find

ies gravitated naturally together, which I thought was re-

each other and connect.

ally beautiful. What your bodies and minds were before, it felt very separate before this. So that was why when you

PQ:

It’s almost like how they shot Mandalorian, they

bumped into each other, it felt like the connection was so

project the real world onto the set. And then the actors

special after seeing everyone sort of fuck off and do their

don’t need to use their imagination anymore and they are

own thing. Then suddenly your physical presence, because

just in the setting and they can fight as if they are on the

of the bump, and then you’re like “oh my gosh”, it’s an an-

real set. Linking back to what Nelson said about fiction.

chor into the physical world, and then I don’t know, every

Let’s say if we only did this experiment with just one ac-

time people bumped into each other, it felt special.

tor inside, then it’s easier for the actor to go “what a crazy world” because this actor doesn’t need to juggle with

KK:

For our group or..

interactions. Maybe for the first trial, we’re just exploring. In the second trial, because you all (gestures towards the

JH:

Both! Like as a spectator, it’s when you watch

students), you did three levels of Meisner, that method sur-

people stare off into random directions but then when you

faced a lot more. So I’m just thinking of the application of

bump into each other, it feels meaningful.

that in actor training and in the real world

PQ:

KK:

What would happen if we can actually see one an-

Any burning thoughts before we go on?

other truthful represented in terms of scale and distance... Be it in the physical or virtual world. As it felt very disori-

Student 3 (Yellow T-shirt):

enting when Nelson feels near but he’s actually far. So I’m

It feels like a very intimate kind of training when it

wondering if it’s possible to scaffold it a bit more. Perhaps

comes to VR world. I’m skeptical when it comes to the next

we don’t talk to each other first? So we don’t have a sense

set of Ensemble training. I feel VR might not be the best

of what’s happening outside. And then we really focus, say

and my suspicion was confirmed when I put on the head-

we see a face and then we approach, focusing on the vir-

set. There’s an overeliance on visual to the point when my

tual world first. So that we form an anchor and then can

body doesn’t want to move. It feels better when I’m just in

experience the kicks when we interact with each other. So

a black box and I imagine I’m in the sea. With VR it forces

when we bump into each other, it is a kick that knocks us

you to use your visual which is not for everyone.

out of the individual experiences we are immersed in. NC: JH:

I suppose it can be both where you get kicked out

Immersiveness of VR and even cinema kinds of

deactivates your body, unless it’s a club (Group laughs).

of the virtual world and then get anchored to the physical world. Now that I think about it, the technology we use

KK:

Alright shall we move on to the next world?

today is all about connection and how we try to connect. One of the reasons I don’t really use VR in my art is that it is very isolating. I think VR can be a tool that we could use... What were you saying before... (Points at Cherilyn)

AFTER PLAY 2

CW:

KK:

The physical world interaction is disorienting... Dis-

So ask the same question you asked just now

associating in the virtual world when linked to the physical world

Student 4 (Gray Sweatshirt): Was the touching and the soundmaking on pur-

JH:

And it made me think of ways in which we could

pose

use VR to mimic environments that we can’t mimic without a lot of money. So perhaps if you fill it with fog and you use

NC:

No. It was something that we touched upon brief-

the voice element to find each other. So perhaps you are in

ly, we wanted to explore when someone was in the VR world, what would happen if there was touch. It didn’t quite

135

work for us but for you guys, it seemed that you were in the

you subconsciously ask your mind “what if we combined?”

middle of it, touching. How was it?

A sort of subconscious desire that helps justify the merge. Does it sound like the suspension of disbelief, like how we

Student 4:

watch theatre? Like when we sit in theatre, we can con-

Well it was my first time trying VR in my life. I was just enjoying it and then woah, touch. Then every touch felt

stantly be kicked out of the world created in theatre but our mind doesn’t have this choice issue

like it was taking me out of the world. But then as it kept happening, it felt like somehow it integrated. So I knew

JH:

For me, when you were describing your expe-

they were people and they were bound to do it, and I felt

rience of merging the physical and the digital, the things

them around me. So somehow the immersive experience I

you were looking at were not matching up to what you

had and the physical experience my body was experienc-

were thinking. I thought where have I felt that before? It

ing, it was starting to merge, but then the headset ran out

was when I was drunk, and it happens during vertigo and

of power before I got any further.

also a dark room. Whenever there is a mismatch with the vision and physical and somehow you’re trying to connect

NC:

What does merging mean?

the two, and your brain is trying to do that naturally. But the VR headset is so visual and there’s not enough physics in

Student 4:

the visuals to help the brain connect the two.

So in the VR world, a lot of it is visuals, then what my body is feeling is the touch, the contact, the sounds are

NC:

from my physical body. So at first they were separate, and

in asking actors to play with two contrasting senses. There

so as I was immersed, every touch and sound takes me

was an intention to make them drunk, to make them unfa-

out, but at one point when I knew they were doing that, I

miliar, because I believed that somewhere around the core

thought it was part of the experiment. But when I changed

of performance, that there is this instability of being, That

the idea of that in my head, it started merging and what

we’re talking about this but we’re talking about something,

happened in my physical body, the touch, starts to relay

that we’re talking to the audience but actually want to talk

over to the visual. It started to merge and so instead of

elsewhere. So perhaps VR is helping us or could help us

taking me out, it felt like a part of it.

understand that bit more.

KK:

JH:

Was there a justification for the touch in the virtual

space?

That reminds me. Maybe that’s why I’m interested

I think VR is very powerful but we have to craft

it in ways that are very deliberate. If we’re talking about mismatch of visuals to physical, then the world has to be

Student 4: KK:

At first no.

So about the merging, how did you justify the

touch with what you were seeing?

designed for that. It’s not just... I don’t know PQ:

When we had a brief with Tim, when we talked to

him about his design. Tim designed it to be hyper-real and immersive and to help us believe we are in the world, like

Student 4:

in the sea. When we talk about when people are drunk, ac-

That’s where the imagination comes in. I feel a

tually they are in the flow, or when in performance, actors

touch here, and i look around, but in VR I don’t see anyone,

are in the flow, and they are very present. And you believe

and so I imagine there is someone there. So then imagina-

whatever you’re stepping on, when you are in the floor, you

tion comes in.

make believe that you make a bigger step to make the step, even if it is very near, and we believe in it. So the question

NC:

So it seems at the beginning, the touch is the kick

then is make belief. And I agree with Jo, then do we want

and takes your out of the Virtual Reality. So it seems your

the design to be hyper-real or do we want to disrupt. So

mind is asking “which one do you want to process?” Do

what are we experimenting on. Are we creating this to-

you want to process the real or virtual world, and then later

wards a purpose? But the actor has to work really hard to

Session # 7

believe and then make the audience believe the world they

PQ:

are in.

made me feel hot

JH:

Right, you know how we’ve been stripping down

JH:

Yes! The shadow was bright and the shadow it

That’s funny, I have a project in Unity right now but

layers of technology, first it’s projection mapping, then its

the character you’re playing in first person is just a pill, so

XR, AR ,and then VR. I wonder in VR if we could do some

it’s just a round thing, so when I stand up against the wall,

stripping as well. First the plane and then you imagine the

I’m just this round thing, just a ball

floor, and then its like walls or the sky. Then we build on and see exactly what elements. I mean that’s for future

PQ:

Imagine yourself as an octopus!

JH:

Yes! Shadow work in VR?! It’s like you... If...

CW:

Can you make a physical shadow? Like Venom?

workshops because we only have one workshop left. TL: JH:

Plus a few extra but that one is designing spectatorship. I believe

Like the darker you go, you become the shadow?

there’s so much capability in VR, it’s just it’s literally a whole world and you can do anything. So how do you design in a

JH:

Oh my gosh both! That would be cool because

space where you can do anything.

like...Whenever you look in a mirror, you expect to see yourself, and when you walk in the sun, you expect to see your

Student 4: Right now, what I’m seeing, VR can be applied like information that can be fed to an actor’s subcon-

own silhouette. So to embody the fiction of another form, could be cool to use a shadow to play with the imagination of the embodiment, say like an octopus or a dinosaur.

scious or unconscious that you can tap into during devising. So like right now before I try the thing, and I was

TL:

So you can also use the PC characters and when

told to behave in a jungle, I would draw on my own ex-

they are invisible, they can cast shadows but you can’t see

perience and think of Jurassic park, but after VR I have

the bodies and you don’t know where it’s coming from

information that I can directly feed off on. So how I think about this VR thing is like getting information to feed my

KK:

I was thinking of VR and how they release us from

brain during devising.

our own-selves, and how they can help us play with an image of ourselves. There are so many things that can open

NC:

It’s definitely something that could be quite im-

when you play with yourself in a body that is not yours. To

mediately useful. Sometimes that when we tell actors

do that in VR, different body, and perhaps physics behaves

something, you know, a fire scene, but everyone’s fire is

differently. There’s an opportunity there.

different. Some of the fire is big or small and that makes you think, why are the actors not seeing the same fire?

JH:

Yea what are the ways we recognize ourselves.

So immediately, VR can give us consistency of everyone

Perhaps shadows?

seeing the same thing and that can be technically useful. KK:

Next session we were thinking about the imagi-

TL:

Something like cognitive alignment in that sense?

nary body right?

NC:

Yea, we can all agree on the same scale of things

PQ:

The octopus concept thing?

KK:

Yes. I think we should have a mirror

PQ:

So are you looking at yourself or are you looking at

Pei Qin: I was quite amazed at the meadow and that shadow NC:

The seabed as well

something else, and how do you meet with your imaginaJH:

The shadow of your character?

tion.

137

stantly reminded that my body is still in the real world KK:

The first time when I was there, trying to relate to

physics because I can’t jump, I’m always on the floor in the

this avatar that was not cooperating with me, that was why

real world.

I couldn’t believe it is me. The exercise had me be this av-

Student 1:

atar but it wasn’t working with me, so I was like nah. But if

I just keep thinking I wish I had the ability to de-

we go about it another way where, there is a sense that this

sign my own world, because like when you’re trying to tell

action is me doing, even though it is separate.

someone about your dream and you swear there was a block down here and there was an alley that went on for-

JH:

It’s like contrasting to when we projected our own

ever and the person is like sure. So if you could design a

image to the wall and were watching a lagging version of

world, put a headset on them and let them experience it.

ourselves that we were so distracted by. So we know that’s

That would be so cool.

us because it looks like us. KK:

But it’s a different person.

JH:

It’s a different person because of the glitch and the

lag. Just like when we are in a zoom call with other people, or when you glitch yourself, which doesn’t happen often. KK:

But anyway, do you guys have any other thoughts

Student 1 (Black Dress): Guess I was always feeling like I was interrogating myself with every choice or movement I made, because of the lag, I was second guessing if this was really important, if this was what I wanted to put my energy into, which I thought was really cool. It made me think about what I want to do in this world, it made me more decisive KK:

You mean in the headset?

Student 1: Yes, and interaction became different. It felt like I was wearing glasses and like I wasn’t sure what I was doing. Like it wasn’t my body but I had control over it, like I was a puppeteer. That was really cool. PQ:

So there’s this distance right between what you

can see immediately and what I’m touching. Student 1: Yea because I swear I’m grabbing the ball but I’m not picking it up in real life, and then I threw it and it flew, and I’m like “I’m really strong” NC:

It’s very different physics right? But then I’m con-

Session # 7

139

SESSION #8 October 18, 2021 // Nine Years Theatre

Jam Sessions

141

ITINERARY 6:00 - 7:00

Set-up

7:00 - 7:45

Summary, questions from LASALLE Student

8:00 - 8:40

Michael Chekhov with Khairul Kamsani

8:50 - 9:10

Play 1 (VR)

9:10 - 9:35

Talk

9:45 - 10:00

Play 2 (VR)

10:05 - 10:15

Talk

10:20 - 10:30

Wrap up, pack up

Session #8

TRANSCRIPTIONS OF OUR SESSION TALKS PRE-EXPLORATION KK:

Embodying perception. Where philosophers and cognitive sci-

ence are at now, how do we understand perception and where is the mind? The Aristotelian, Enlightenment and empirical approaches in the past had made the mind a mysterious entity that was separated from the body. There is a big shift now in the world of phenomenology, actor training and cognitive science, towards the understanding that we must discard the notion that there is an absolute, empirical, actual truth that can be pinned down somewhere. We all understand now that every human being has a different perspective of the world, and we have to give agency to each of them to understand their own truth. And therefore this subjectivity of existences creates a contention with what is the actual reality of the world. And I think that is what we are looking at in our research: how do our own subjective perspectives reconcile with our previous thought of an objective reality. I think our research is in a new area because many of these

Cherilyn Woo

Joanne Ho

Khairul Kamsani

Nelson Chia

Thong Pei Qin

to frame the way we move forward in today’s exploration in feeling and

Timothi Lim

perceiving the experience and the body we have in relation to a virtual

theories look at the daily, while we are concerned with what is reality like for the actors in actor training, for actors playing a character. Obviously they haven’t even touched on what happened when you go into the virtual, the intermedial, so that’s where we are. So I just want

143

experience – how do we navigate that and how do we pin

world, that are gravity, electromagnetic, strong and weak

down this is how it happens.

forces of the atomic level. But in the virtual world, we can change the laws.

TL:

Related to the idea of the cognitive is the idea of

While my body is still grounded, I can see objects flying

the metaverse. The idea of metaverse is currently more

and when I throw something, it flies following a different

about transaction. However, to me it is more of a blurring

law. And even when my legs are on the ground, I can ex-

of the lines between digital reality and physical reality. I

perience my body flying. So, the evolution here is to lose

would like to explore more of the notion of metaverse be-

the body, so that the mind can no longer be bound by the

yond the transactional.

physical laws, and you can be in two places, in different dimensions, and you can go around time.

KK:

The metaverse for me is our post-postmodern

ability to move intra-temporary, to live in different time

PQ:

Conversely, you can also change your physical

and space – to live here but also understand the different

body experience. For example, if you are walking the plank

time zones through digital means. In a way, we already

in the VR world, you may feel vertigo even though you are

do live in a metaverse, but we need to step back and talk

still on the ground. Your fear of heights gets evoked.

about it. JH: PQ:

But I think that fear is conditioned by our bodies

Khai talked about the objective reality – how

being in this physical space. Because if you live in the vir-

do we define it, is there an objective reality? I was very

tual space your whole life, you won’t be scared. You know

interested, since our last exploration, in things being in

that if you fall off, you can literally re-start your life. There-

two different spaces at the same time. You talked about

fore, it’s interesting to think of training not just as actor

intra-temporal, but we are also in two different spaces

training, but mind training – like there’s a generation of

in all our sessions. I think Jo mentioned why don’t we

people who know intuitively how to use the interface of an

design the VR space as the actual NYT studio? This is

IOS, while there are others who will struggle because they

mind-blowing because then what is in the virtual reality

didn’t grow up doing it. We evolve with the technology that

is modelled against what is in the actual reality. Hence,

we create. Even the clock is a technology that conditioned

I was interested in the distance between the objective

our behaviour of keeping time, making schedules, etc. That

reality and the many different realities that we create. So,

is also what I’m interested in – what are the behaviours that

what is that space in between these realities. That’s my

we learn from new technologies.

question. TL: KK:

About time, when we talk about time in art, like a

That movement between “I am here now” and “I

piece of work such as Medea, it feels “timeless”. But when

am also there” – it seems like all of us can do it, but in fact

we think about the technology of creating time and sched-

not everyone can. In the area of Able-ism, it has to do with

ule, it is quite crazy the amount of work put into working

how the brain has to develop in a certain way so that it can

out calendar, time zones, etc.

perceive multiple time and space at once. The question

THOUGHTS AFTER MICHAEL CHEKHOV’S EXERCISE.

then is whether you have to force yourself in order to be able to do it?

PQ: NC:

Made me more aware of my body’s anatomy, each

What I’ve read recently that was interesting to

joint... And what my body even consist of, made me more

me was the idea that the next step for human evolution

awake. Just to go through the sensation... The first one is

is to lose our body. Because the mind is actually locked

“without skin”, I’m droopy, my organs can’t be contained,

up in this body. For example, Pei Qin’s question of what

when I move around I’m like falling or sloshing around. The

is the difference between the real NYT studio and the

second one, there’s “no muscle”, so I’m a skeleton. I feel

virtual one? The answer is the real NYT studio follows

rickety, very jaggered, a bit lighter.

the physical laws – the four general forces of the physical

Session #8

KK:

Also, about the sensation of lying flat on the

ground, being in contact with the floor, how gravity affects

world would make you sensationally go there. Because the use of words is always intellectual.

your sensation. That’s supposed to be the baseline, of being in contact with the floor with your skin. And then when

KK:

The point is to picture yourself outside of your

you get rid of it, how do you feel anymore. If you left your

eyes. We think and we live so much in our vision, that to

feeling organs over there, how do you navigate space and

put your eyes somewhere else and to look at yourself from

time without the organs that you rely on for feeling?

the inside can give you more of a sensational, sensorial understanding of the way of being, which is the point of the

CW:

I felt an existential crisis. Lying on the floor is easy

exercise.

for me to imagine the idea of my skin melting away, but when you say “ a puddle of you”, I was like, then who am

JH:

At first I started with an eye, an eyeball on top

I? I’m just “muscle”? What became the “Totem” for me was

of my head, but that is so difficult. I know how an eyeball

when I had to use my legs to stand up. Once I get up from

feels, the eyeball was stuck to the skin, it was an in-be-

the floor, I know I had to activate my muscles, if not I can’t

tween space between the outside and what’s outside. To

get up. Therefore, I had to temporarily suspend my imag-

move the eyeball inside was so difficult, so I changed it to

ination to get up. Then when I got up, then I was thinking

like a ball of light. When my eyeball became a ball of light,

how was me as a skeleton move?

then it became free to move. And then as soon as it came into my chest, I felt rally safe and comfy here, and then I fell

PQ:

So you are saying when you get up, there was a

asleep.

“kick”? TL: CW:

Yes, I had to temporarily close window, and then

I tried to give up being anatomically correct, and

try to be a metaphysical body. I try to imagine my body not

when I get up, then open window again.

filled with organs but is something that represents me.

NC:

KK:

Listening to this... Even when I was working with

Some people actually try to empty themselves, try

performers, this is what I encountered most, that they tend

to be hollow, and then voice their sensation of being hol-

to get to their brain first. They would analyse the fact that if

low. You see the journey from top to bottom of your body,

I’m a skeleton, how do... My muscles move my bone, but if

how does that guide your movement and discover a space.

I don’t have my muscles, what do I do? It’s not a bad thing

...So, when we do it in VR, it would help that if we

that we tend to go back to our pre-conception of what a

see things that aren’t logical, we don’t think about how do

bony person is, a skeleton... A puddle, what a puddle feels

we make it logical, as an actor, as a performer, but just get

like, and our body kind of consciously simulate it, or per-

carried away with it. I think that is where we can go off into

form it in a way.

when we try this thing.

This is usually the difficulty I have with actors, that whatever instructions you can give, if you’re playing music

PQ:

So, we should trust our bodies’ impulse more.

or whatever, it gets to their mind first, before they try to

There’s something I read about Chekhov, about the root

express through their bodies. What I’ve always been trying

impulse. Say, maybe it was the little finger that was twitch-

to get them to do is... You listen to your mind of course, but

ing, you trust in that and let that take to somewhere else.

you let your body experience it, and we are not asking peo-

Then that could become like, a push, and maybe that would

ple to act like a skeleton. If you let it go, what happens after

extend through the entire body, maybe the entire arm is

is a process of discovery... What does that mean? Could

pushing now. Afterwards, your whole torso is elongated

you find something? Something else maybe?

and the torso is pushing, so that it expands and expands.

So when technology comes in, this is what we are trying to find out, could technology help us to not get so much into the mind? In a VR world, no one’s going to give you the instructions and maybe something else in the VR

KK: This is what Michael Chekhov is all about, going back to the body. How is he able to act so realistically and organically? It’s because he trusted his body.

145

TL: When we do game development or the development of technology, I try to enter the cognitive flow, in terms of like the themes and the sense of being. But sometimes the toughest part is starting the project, again like aligning everyone’s imagination. But more than that is to get everyone beyond that logical thought. You can logically say that this might not work out, but again it’s all theoretical, all in the mind. For me it’s always like running with it, you have a good grasp on a key point set and then you run with it and develop it from there.

CW: Yes, I noticed previously we were further apart but this time we were closer. JH: Is it because you knew you were physically closer in real space? CW: I knew that later. But also previously we were normally opposite one another, but this time I realised at one point that Nelson was on my right in VR and he was actually also on my right in real space. PQ:

AFTER THE VR SESSION WITH THE MIRROR PQ: I was so rooted in front of the mirror and so fascinated with seeing myself, that I forgot I could move, that there is a 360 degree environment around me, and I saw everyone else’s relation to that mirror. NC: A few things here, not making sense of them but just articulating them now. Firstly, because the voice (giving instructions) is real, I couldn’t be totally immersed in the VR as before. The voice was a constant “kick” out of the VR. Secondly, the instructions were so focus on the bodily sensations. But when I opened my eyes, looked at myself in the mirror and realised I don’t really have a body, I can feel the sensations but it was a bit weird. That the made me think that I had a fuller experience of the Chekhov’s exercise then without the VR, rather than within the VR. However, I also had a fuller experience when I was in the VR without the voice instructions. Hence, when the two things, VR and the voice instructions, came together, they became lesser. Maybe it was the conflict in the understanding of the realities. Thirdly, because we had experience the landscape environment in VR the other time, this current environment was not as strong as the last time. But what was interesting this time round was that the other people / avatars were stronger. Maybe it was because of the mirror, that the mirror gave us a reference of where we were. JH: (To Timothi) Did you change anything with the proximity...

Session #8

So it’s like more mapped with the real world?

CW: Yes. Also, for a moment I thought I knew where each of you were but somehow that was not the case. PQ: Regarding the mirror... How does that relate back to our Chekhov’s exercise? We look into the mirror, we don’t see parts of our bodies but we are to sense parts of our bodies. And there was a difference between closing our eyes and seeing it, we did both and compared. I also thought about the actual weight of holding on to the joysticks, and the goggles on us. At one point, I wanted to lead with my neck but there was an extra weight, so the goggles actually impede the movement. NC: There’s also a sense that I might knock into someone? That in that way we were more reserved? JH: Is there a way to do the Chekhov’s exercises by directing the embodiment not to your own body but to the avatar’s? KK: I think that was what was explicitly trying to do. As soon as you opened your eyes, you feel the sensation here in your physical body but you see your body there in the VR world mirror. I was trying to see if you could live through the sensations in your physical body, and also the visual reference of where you feel you are. CW: I kind of got that, but also I thought the preface of this was that don’t think with your mind but think with your body. When I close my eyes I could

have one fiction, that was led by my body. But then when I opened my eyes and saw myself in the mirror, it didn’t quite reflect what I was feeling. Then I try to affect what I was feeling through the mirror, so there were two layers, one in me and one in the mirror... But also one not of my reflection. But you were saying the hope was to use the sensations to affect the mirror? But the I think it goes back to that – we don’t have the body. So I think I was trying to merge the body but all I could match were the hands. TL: Yes, having a full body in VR is very hard, because of the challenges in tracking.

tors to play with two different senses. Like if I listen to a piece of soft music while looking at a hard surface. That kind of contrast could happen? CW: Maybe what I’m saying is my two bodies – body in reality and body in VR space – are able to communicate to form one. PQ: Sometimes when I move my toe, I could see my head and arms will move and I feel I don’t really need that torso in between... So that mapped quite directly. It really looked like a mirror.

KK: I think if I were to do this sequence of exercise again, I would consider doing it with actors and a real-life mirror. We kind of jumped a step by using a mirror in VR.

KK: It is actually a notion of blending. That you take two things that don’t make sense and blend them to make sense. I think this is what we as humans do. There’s another notion also of how we combined realities to become one in order to keep sane.

CW: So the mirror in the VR is technically your outside eye of the VR avatar?

AFTER THE SECOND SESSION WITH MIRROR.

NC: If we look at the two things again, the Chekhov’s exercise brings us into our inner eye, our bodies, and when we start to move, we are observing our inner eye / bodies. The visual space that we see, may or may not affect us, but it’s there. However, in a VR world, we are doing the exercise and looking at the environment, and our bodies may react to the environment. But I think just now when we put the two things together, it cancelled out the strengths of each – the Chekhov’s exercise now has no body, but when I look at the VR world there are so many things that I just want to lose this body and go with the VR sensations. CW: I understand what you said, but there were moments I was experiencing a split personality. So I was wondering if there’s a possibility of the two eyes connecting? In which case it’s like real life, my perception is talking with your perception but in actual fact, nobody really knows who is talking to who. NC: That is also possible. Remember I talked about how we opened up our senses in one of my exercises, the body of wakefulness, when I tell the ac-

PQ: I’m wondering about the benefits of embodying the avatar, in theatre. Meaning, if it’s like a performance and all the actors are actually avatars, performing live. Then maybe that’s the point of it? KK: I sometimes think of an actor as, when they are performing, their body is an avatar. TL: Nowadays we enter into a world where the hollow life virtual avatars... In a way they are also acting as characters and not really as themselves. I wonder how it would be like in the future of acting, if it’s a hybrid model where you have digital avatars as well as physical avatars? ...I think it’s interesting to see that, like I could see you and I could see myself in the mirror, but both are equally interesting in the sense that you can mirror me and you can actually kind of touch but not actually touch. CW: You mean when you were “touching yourself”, you were looking at yourself in the mirror? TL: Yes, because the avatar is just a head and hands, you kind of have to imagine where the rest of your body is.

147

CW: So, in your fiction of the mirror, is it a reminder to you that you have a body as you are looking at the mirror? TL: I try to imagine the digital body in the mirror. Like I can’t put my hand through my physical chest but I can put a ball through my digital chest. CW: So you are using the physical sensation of the body in order to feed the imagination of the avatar? TL: Yes, and things like thinking about where the ball is or where I left it. If you pick up a sphere and put it there, it actually is there. Right now, it is very minimalistic because the avatar has no body but if you put a body, what would it be like? Or we can change body, take a body part and put it on, and that makes up your avatar, but in real life you can’t. ...Embodiment can actually be reinforced, if the idea was towards utilising the virtual to reinforce the idea of the physical being. The current one was more of your mind’s version of the physical, while the other one is using the virtual to reinforce your self.

Session #8

149

151

FINAL REFLECTIONS

Cherilyn Woo’s Reflections HOW IT STARTED When I first set out on this journey, the big theme for my own exploration was the idea of scenography. To broadly introduce the term in my reflection, “Scenography is a broad term that incorporates an intricate matric of overlapping practices any one of which is impossible to study in isolation.... Scenography encapsulates the whole event, including the performers and the audience; it is the discursive field in which these distinct and yet overlapping practices converge. “ (Nisbet 1) The aesthetic design of the space is one thing, but for me, the movement of bodies in the performing space was also one of great intrigue. Once there are different layers of reality, the term scenography encapsulates even more elements and rules of both the real and the virtual. In the case of my reflection, here are some broadly defined terms used: •

Real – the tangible

•

Virtual – the intangible, i.e that which is seen through the Head mounted display, what I will refer to as the VR head set.

•

Performance space – playing space demarcated for the playing sessions. Does not include space of technologist or archivist.

•

Performance – refers to the actions I made during participating in play sessions

HOW IT ’S GOING I was an active participant in most sessions. Taking part in the pre-play warm ups conducted by Khairul as well has entering into the virtual world and playing alongside in the real world while my teammates were in the virtual world. As I was in the play sessions, I constantly kept an outside eye on my spatial relationship with the real space, the virtual space, and my fellow team members. Our performance space had P ost-sessions reflections

two levels of reality. The Virtual Reality and the real,

certain restrictions in the movement of the limbs

the intangible and the tangible.

that hindered the performativity in the virtual. What I found was happening then to myself as a performer

For my reflection on scenography, I’m going

was that I could feel myself translating the knowledge

to divide the next part of my reflection into two areas:

of awareness I had of my full body into readapting

•

How the space affected my performance

that in the virtual I only had two hands and a face.

•

How my performance affected the space Whilst I know that in the virtual, I can only

SPACE AFFECTING MY PERFORMANCE

move hands, the real body I have outside of the virtual space still has to be engaged to move the hands

The biggest hurdle for me as a performer was

or the face. I found myself imagining the rest of the

negotiating the real and the virtual. My misconcep-

body whilst being in the virtual to form some coher-

tion was that the two are opposites of reach other.

ence of the self in the virtual. In this sense the virtual

One was real and the other was not. However, “virtual

space redesigned my perception of my virtual body.

reality is in a paradoxical relationship with the real.

My bodily representations begin to transform and are

On the one hand it is part of the real; but on the other,

remapped providing a physical sense that the virtual

it has to be constructed as different from the real in

body is mine (Jarvis 56)

order to be perceived as separate from it. “ (Giannachi 123)

Whilst performing I realized that the performer undergoes the sensation of being both immersed The virtual space comes from the real, but

and separate from the reality of either the real or vir-

at the same time it is also at a difference from the

tual. Hence, being in the virtual space allowed for

real. What begin to change inside of me was not that

me as the performer to be simultaneously inside and

I should relearn how to perform in the virtual, but how

outside the reality that the virtual space offered. The

I can bridge my processes in the real into the virtual.

performance arose out of dispersion and displace-

And whatever could not be translated into the virtu-

ment. (Giannachi 123) The performer will always be

al was that it should be excluded from consideration,

dispersed in and out of both the real and the virtual

but that it should be translated into it. For the parts

reality. This does not mean one is present only in one

that are not translated are the elements of what sets

reality at a time, but rather that you can be both pres-

the virtual at a distance from reality.

ent in the virtual and the real, or fully present in the real or fully present in the virtual reality. It is unpro-

Examples:

ductive to not be present in both realities.

A) The incomplete body It seems imperative then that should a perIn the virtual worlds we were given a face and two hands.

former want to perform in a performance space with multiple realities, this toggling between the different levels is necessary to the act of performing. Whilst it

I must quickly accept that my body in the vir-

might sound like a big leap from conventional perfor-

tual is without all my other limbs. However, in the real

mance, it in fact draws similar processes to it. This is

I had a full body with four limbs. Naturally there were

not unlike a conventional theatre performance where-

153

by there are people on stage performing. A perform-

are pulled together in a performance like this. It is

er is both cognizant that they are performing for an

pertinent that the audience experiences this, as it is

audience while simultaneously being immersed in a

only then that they realize the potential that virtual re-

narrative rehearsed where there are no audiences.

ality can bring to theatre. It can offer up a much more transportive and

It becomes apparent then as a process, by performing this duality of realities one performs less

Whilst the performer is feeding new transla-

and less in a marked reality, but rather performs the

tions of the experience into performance, it contrib-

duality of realities taking place. For this makes the

utes to the virtual reality reinventing itself through the

performer most truthful and engaged.

conduit of the performer. In doing so, the space (both real and virtual, both seen and unseen to either audi-

PERFORMANCE AFFECTING THE SPACE

ence or performer) is a performer. (Giannachi 124) It then became also revelatory that both levels of reali-

In incorporating both the real and virtual re-

ty combined in this expanded scenography can only

ality in performance, what became apparent to me

work in tandem with each other. You cannot have the

was the idea of an expanded scenography. One that

real within the virtual and the virtual without the real.

highlights the “experiential and embodied nature of

It is through the relationship with each other rather

scenographic experiences that facilitate encounters

than being a substitute for each other that the most

between participants, performers, spaces, sites and

dynamic use of the virtual space is found.

objects.” (Meyer 322) THE CONCLUSION It wasn’t just the space of the virtual reality, but in performing the duality of realities, we are also

Having been through the play sessions and

combining the real and the virtual space. Both reali-

reflecting with the team there are a couple of take-

ties become the scenography for where performance

aways I have as a theatre director:

can take place. This expanded scenography where it incorporates the scenography of the real and sce-

•

Immersion is always key. It is not enough for the

nography of the virtual space offers a way to investi-

performer to feel immersed in one setting, but it

gate the connection between two of these realities,

is about bringing awareness to the real and the

their elements as well as the presence and existence

virtual reality. It is not about forgetting but about

of the performer. (Meyer 323)

combining. Even in the negotiation of realising what is real in the virtual and what is virtual in

As a performer what I had hoped to commu-

the real is important. One level of reality does not

nicate to a viewer, should there be one, is that aware-

represent the other, but that it may be a location

ness that there are both presences of the realities.

from which the other is viewed from. I.e. The vir-

That beyond the space in which they see whether a

tual is not an accidental representation of the real,

projection of the virtual reality or the reality, they can

but a place where we can re-look or re-view the

see the one that is not immediately visible to them

real. (Giannachi 125)

through their eyes or through a projected screen.

•

Thus multiple locations and its individual elements P ost-sessions reflections

Hence it is pertinent to get away from binary thinking. I.e. Virtual reality is not reality, or

vice versa. It is more productive to instead consider the relationship and its effect on each other •

What I’ve found increasingly useful is the warm ups, whereby we engage in the training. It’s a familiar tool to ground the performer and an anchor in which it can be used in when entering the virtual. The warm up in itself is a primer for looking at how to adapt to the virtual reality.

•

I’ve found it incredibly useful to go through this workshop as a performer first. This would help in the facilitation as a director should I venture into works like these. This training could be useful for directors as a training ground and tester in order to navigate how to start their own processes in their work. What we are beginning to talk about is important in re-looking what is immersion in the work as well as for the audience. As society begins to live through technology, it will inevitable affect how they see reality, so looking into virtual reality in performance not only affects how you direct for a virtual reality performance but also in reality.

BIBLIOGRAPHY Nisbet, Jane Collins and Andrew. Theatre and Performance Design. New York: Routledge, 2010. Giannachi, Gabriella. Virtual Theatres: an introduction. New York: Routledge, 2004. Jarvis, Liam. Immersive Embodiment. Colchester: Palgrave Macmillian, 2019. Meyer, Rebekka Sofie Bohse. “The expansion of scenography in virtual reality theatre: Investigating the potential of double scenography in Makropol’s Anthropia.” Theatre and Performance Design (2020): 321-340.

155

Joanne Ho’s Reflections REFLECTING ON MY FIRST REFLECTION Comparing my first ever reflection to this one, it seems like the views that I had about this project started very broadly, because I had to have time to adapt to the ways in which theatre directors work with people in space. I also had many ideas that I thought could be tested by the end of this series of workshops, but over time it was proven that we needed to scale back the number of variables in our jam sessions in order to understand the different elements that affect us within the space of overlaid realities. We really started to formulate a clear direction for workshops after the first reflection, which couldn’t have happened without the fist four sessions, which is clearer to me now, than four sessions ago, why we’re calling this industry sharing workshop a prototype.

GENERAL SUMMARY Our workshops have now come to a close, and it’s been an insightful journey working with theater directors and dramaturgs on Architects of Realities. Since I’ve been attempting to expand my visual work into live performances, our sessions have helped me to better understand the interactions of mind-body and tech in space. My interest in moving towards performance actually lies in the presence of the mind-body in space that forms a connection between the performing mind-body and an receiving, interpreting mind-body. Where does your presence reside when you’re scrolling through social media? When you’re reading? Meditating? Daydreaming? Does the mind separate from the body when these happen? Is the transporting of presence something that is specific to us humans? We know animals dream… but do they dream, consciously? When a dog is conditioned to salivate at the sound of the bell, is it strictly a physiological reaction, or is imagination of food in the dog’s mind also involved? Perhaps some imagination of being present in the face of food has a part to play in the expectation of being fed.

P ost-sessions reflections

These are questions that are fairly new inquiries for me, prompted by the discussions we have af-

feel like I experience this very often even when I’m in the physical “in-real-life” world.

ter each jam session for Architects of Realities, driven by a need to understand more deeply my personal

The only time I was in the VR headset during

habit of dissociation as a result of repeating childhood

a jam session was with a set-up that overlaid many

events that I will not disclose. It is a common expe-

conditions that foster possible engagements for be-

rience for those who have had traumatic events to

ing present. First, there was the Chekhov technique

separate their mind from their body as ways to cope

that Khairul directed us through, which was to feel

with feeling trapped from whatever is being forced

your physical body in space. Second, there was the

upon their physical bodies. That’s why cold water

VR headset, where Tim had created a visually im-

therapy works for people like me - when submerged

mersive virtual world in which our minds perceive.

in ice-cold water, the immediate contact of cold to

Third, within the virtual world, there was a mirror that

your body cannot be imagined. Our mind-bodies typ-

we introduced for us to connect our physical body

ically do not recall discrete pain or pleasure sensa-

movements to the virtual avatar’s movements and

tions like it can with images. It is purely physical and

form. Fourth, although it was a glitch on the part of

inescapable. Although I’m not a fan of separating the

the technology, one of the peepholes of the headset

mind from the body and vice versa, it’s clear to me

showed a solid wall where the mirror texture should

that stimulating my body with intense and seemingly

be, and the other one accurately showed the mirror

inescapable physical sensations (that is welcomed or

that reflects our avatars.

self-imposed) is one that actively grounds me in the reality that my physical body is in.

When I was experiencing this jam session, I was constantly glitching in and out between being

Regarding the last few sessions, where we

present in my physical body and in my virtual body

introduced VR into our explorations, I was excited

that I was attempting to embody. By starting with the

to venture into VR, but knew that the unpacking of

Chekhov technique, I fully felt my body being ground-

our experiences within the VR headset would be a

ed on the floor. Even when I had my eyes closed, the

daunting task. We could have a whole other 8 ses-

image of being in Nine Years Theatre, as the space

sions where we focused on just VR. This technolo-

in which my body is inhabiting, still lingered in my

gy is a physical manifestation of our mental habits of

mind-body. Then, when we were told to open our

transporting presence. Now a place, representative of

eyes, I instantly lost all grounding. My spatial aware-

a three-dimensional space, is visualized for you.

ness launched into shock as I was in a virtual space framed by two mismatched images for each eye. I

The space of VR is one that most confuses

tried to close one or the other eye to be present in the

me. Even if I do create works that are specifically

virtual space, but the mismatch of images was active-

meant to be viewed and experienced in VR, there is a

ly kicking me back to my own body. I then became

sense of always needing to ground myself when I’m

acutely aware that I was using my eyes to perceive

within this space. As part of Architects of Realities

this space, and that my physical body was wearing

glossary of terms, we are using “glitch,” “totum” and

a headset. I couldn’t complete the jam session be-

“kick” to refer to the active process of becoming pres-

cause there was no way to escape the glitch. Even

ent in a space within our jam sessions, but I actually

when I closed my eyes again to bring my presence

157

out of the virtual world, I still felt my presence in there,

movement from an immersive environment, or it is

separating my body from my mind, which is still par-

an interaction based on the audience’s movements.

tially embedded within the physical heaviness of the

Coming out of these sessions, I can imagine and un-

headset on my face. But this was a different type of

derstand more of the variations of experiences that

dissociation I was used to. Instead of me following my

different people have when interacting with space

mind outside of my body, it felt like I was left partially

and technology. It was even more insightful to hear

in my body while the other portion of my mind was

and read reflections from professionals who work

inaccessible. I decided then that the glitch was too

within the field of narration creation with performers.

overwhelming so I chose to exit this jam session.

I can imagine using the knowledge I’ve learned here (in terms of thinking more deeply about “user experi-

There’s a strong reason why I chose to pur-

ence” in relation to space with overlaid realities) into

sue this project with those who work actively with

my work in the future. By better understanding how

techniques of communicating narratives of the imag-

actors think and react to elements in space, I can then

ination using the mind-body in space. The desire for

better craft elements in any type of space to serve as

me to share the experience of having multiple modes

moments of interactions for not only actors but audi-

of presence especially in a time where we have the

ence members that become actors to the work.

technology to do so (augmented reality, virtual reality, digital social media, immersive technologies) is one

FUTURE APPLICATIONS

that is driven by, perhaps, my compulsion to ground myself within a place, or a community. To connect

•

As events move towards virtual and hybrid for-

with others and explore journeys of presence in a

mats, this workshop can serve as a means to

space together.

understand the inner thoughts of actors in the space - through participation or through watch-

Just to be absolutely clear - I wouldn’t say that

ing/reading our documentation. There are many

I’m endorsing the idea of a hive-mind. The beauty of

events that are exploring virtual/hybrid formats,

connecting different people using these technologies

but I believe there aren’t enough documentation

is perhaps the increasing opportunities to experience

on audience/actor reflections on their individual

and understand everyone’s realities as best we can

experiences.

- a window into another person’s mind-body. The op-

•

Better crafted virtual or hybrid experiences using

portunity to live many lives, or live the life you want

our findings, especially with the glossary of terms

to pursue even for a few minutes. That’s also perhaps

that we have created to better describe these rel-

why they call VR the “empathy machine.”

atively new experiences.

This essay was not sponsored by VR technologies. HOW IT IS USEFUL FOR ME AS A MEDIA ARTIST WORKING IN PERFORMANCE Interaction is a huge component of my work, because the environments I create typically require P ost-sessions reflections

Khairul Kamsani’s Reflections This reflective document aims to encompass the overall process that our research team has undertaken in pursuit of experimenting with emerging digital interactive technology with the theatre making field. Early on, the National Arts Council board interviewing us had specifically questioned how this research was a separate entity from my PhD endeavour to avoid any funding overlap between this project and any developments for my funded PhD with De Montfort University. I had demarcated the difference between the research projects by the virtue that Architects of Realities (AoRs) was towards developing approaches and deeper understandings into performance and rehearsal making structures between theatre and creative technologist practitioners and that my PhD was between the relationship of an actor trainer and the acting subject interacting with Virtual Reality Technologies. In my earlier reflections, I had established that I personally would have to continue delineating AoRs from my PhD and vice versa and whether I would be able to for the entire process. While I do believe that the field of actor training with emerging technology remains a separate (newly establishing) in comparison to the field of theatrical performance making via conceptual structures and thematic blending, the overlap seems to be converging as we move ahead in our research and there will be a day that we cannot examine ‘theatre making structures and processes’ without an thorough understanding of ‘the actors perceptual and experiential process of virtual reality spaces for the practice of their craft’. In our design process for our first workshop and consequently from there, we aimed to start with ‘colliding physical bodies with the digital technology’ and ‘see what happens’ to examine the result for further extrapolation. However, we struggled to find a way to place the bodies in collision with the technology without any preparation for it. Instead of simply placing an actor opposite or within a projected space or virtual headset and examine their perceptual process from an external point of view, we agreed that the actor had to be prepared to physically engage with the digital/ virtual space and also be able to extrapolate their experience post exercise- they had to ‘do’ something with what they were presented with. We settled on a combination of my adapted Meisner Tech-

159

nique and a selected collection of Ann Bogart’s and

The final workshop which introduced the virtual mir-

Tina Landau’s Viewpoints (Bogart and Landau, 2014)

ror for participants in the virtual space illuminated

to guide the actor in physically and vocally engaging

several points of interest which potentially require us

with the perceived elements that were projected or

to return to the basis of our exploration before con-

virtually displayed. We did not predetermine the se-

tinuing further. The ‘theatre’, derived from:

quence of actions but rather provided a foundation of ‘what and how to perceive and react to’, procedural-

“The Greek word theatron, A theatre usually has a

ly generating a series of physical and verbal actions

stage area where the performance itself takes place.

that were observable and experienced by the actor

Since ancient times the evolving design of theatres

and the audience (observers). Drawing from our

has been determined largely by the spectators’ physi-

post-exercise discussions, it became apparent that

cal requirements for seeing and hearing the perform-

our original goal of devising and exercising new ways

ers and by the changing nature of the activity pre-

of theatre making with these emerging technologies

sented.” (Brittanica, 2021)

was a much more distant endeavour than where we are, at the moment since we are metaphorically only

Takes us away from the contemporary conventional

learning how to ‘see’ and ‘walk’ in this new world. To-

meaning attributed to the word, the ‘theatre’, which

wards the final workshops, the multitude of factors

tends to describe a physical building structure or

and variables at play when a new technology outside

place and back to the essence of how theatre is

the body is introduced became overwhelming for the

framed. Perhaps what is required of practitioners and

actor and audience to thoroughly explore and grasp

researchers in the theatrical field is a re-examination

within 10-15 minute explorations.

of this ‘place of seeing’ given the tremendous developments on how and where we can see. Erika Fisch-

A key discovery that our team has made is

er-Lichte describes how the frame of the theatre is

the omni-directionality of the ‘layers of reality’ when

created by the cultural contexts that itself is derived

applied to the playing space. Initially we had presup-

from and that it serves as a mirror to the culture it

posed that we would be able to ‘control’ the layers of

reflects in order for it to contemplate itself (Fisch-

‘realities’ by their terminological definitions- that Pro-

er-Lichte, 1992). With the ubiquitous nature of digital

jected Reality, Augmented Reality and Virtual Reality

technologies having changed the way our metropol-

could be separated and layered ‘on top’ of each other

itan cultures see and interacts with itself, we have to

and see how they interact among bodies in space.

return the theatre to be the place for us to see how we

What had occurred was that the body/actor being

see our selves; the physical, the digital and the virtual

caught between the layers had to spend much of

before we can consciously make an effort in a cre-

their time navigating on which layer to attend to and

ative practice that comments on this phenomenon.

how that layer affected them and provided agency

Just like how graduates of a BA Acting program take

and affordances to their prospective actions. At any

approximately three years to ‘craft’ their psychophys-

point the interactions between ‘realities’ converged

ical practices in the corporeal, yet have only just be-

on an x, y and z axis for the actor to engage with; the

gun in the application of that practice in the varying

depths of these interactions delved far deeper than

styles of acting while continuing the development of

anticipated and would require more time to explore

that craft. Architects of Realities must develop further

within with sufficient preparation for the engagement.

and deeper in scope and time to be able to function

P ost-sessions reflections

as a training ground for actors, directors and technologists to firstly have a handle on their practices being affected by emerging technologies before manipulating the affordances that these new factors bring to us. REFERENCES A, Bogart and T, Landau, 2014. The Viewpoints Book: A Practical Guide to Viewpoints and Composition. Nick Hern Books. Bay, Howard , Izenour, George C. And Barker, Clive. “Theatre”. Encyclopedia Britannica, 3 Dec. 2020, https://www.britannica.com/art/theater-building. Accessed 6 November 2021. E. Fischer-Lichte, 1992. The Semiotics of Theater (Advances in Semiotics). Indiana University Press; Abridged edition.

161

Nelson Chia’s Reflections REFLECTIONS ON SESSION #7 AND #8 : VR ELEMENTS During session #7 and #8 of our exploration, VR elements were introduced. In particular, during session #7, a “landscape” was presented as a VR environment. In session #8, a mirror was present in the environment. This is a personal reflection of some of the most significant take-aways from the experience.

SESSION #7 – LANDSCAPE: THE IMMERSIVE EXPERIENCE In this session, an underwater and a forest landscape were introduced separately. I had the opportunity to experience the former. It was an environment of seabed under the performers’ feet, underwater terrains around us, fishes swimming among us, and sunlight beaming through the water surface above us. The rendering of the environment had managed to capture some of the most significant characteristics of what constituted an underwater space. This is important because being able to capture the essential characteristics is key to the immersive efficacy of the VR environment. And when I say “immersive efficacy” I do not merely mean the familiarity or degree of realism of the renderings. What is referred to here are elements that may affect the performers’ bodies, changing them in the process of interaction. To be more precise, they are elements that would affect the performers’ sensations, or that which “speaks” to their senses. In my experience, when the underwater VR was turned on, the immediate effect I felt was a need to orientate or re-orientate myself – a very instinctual reaction of the mind and body in fact. I noticed the kind of ground I was “standing on”, the terrains that made up the spatial setting of my environment, their distances from me and their relationships with me and where I stood. I lifted my head and noticed the seeming distance between me and the surface of the water, highlighted by the light that shone through the water onto where I was ( I could also see the shadows I cast P ost-sessions reflections

when I looked carefully at the seabed ). The fishes

realised two points that were related to the experi-

were sparse in the water, and so they did not immedi-

ence.

ately provide a strong stimulus for me as a performer. In hindsight, this might be a good thing? For I would

A) Points of Reference - Lessons from the

think fishes as “living” elements would be a very

Real-World

strong image if they were present in larger numbers. The fact that my body knew where to While trying to orientate myself, I noticed that

search for the language to speak to the envi-

my breathing was already slowly changing and the

ronment made me realise this: that we were not

way my limbs were moving were also different. As

referring to a brand new language that was out

I started moving around the space, I noticed I was

of this world, that in actual fact, we would refer

walking in a way that I would if I were threading along

to our experiences and lessons gathered from

a sand bed – in a slower pace as if the ground was soft

real-world experiences – how would a body im-

and a certain resistance might be present. Soon, my

mersed in water feel and how would one move

torso and limbs began to experience that resistance

and breath in water. To put it another way, we

too, as I was affected by my senses to “make belief”

were searching for the familiar. There was noth-

or imagine that I was waving my arms and lifting my

ing unnatural about this for we knew this was

legs through a denser mass of water, a space that

how our mind functioned.

was denser than the air I was actually surrounded by in the real-world environment. As I moved around, my

What is interesting is this:

breathing became slower and shallower, suggesting that air might not be that present in there.

That we are bringing our real-world experiences into the virtual world, and adjusting them to

After exploring the underwater environment

relate to the virtual world accordingly. That ul-

in this manner for a while, something else hap-

timately the performers’ presence has little to

pened. I noticed myself ‘easing up” a little, as if I had

do with what is actually and objectively “real”

learnt quickly how to exist in this environment. What

around them, but how their subjective sensa-

I meant was that I had overcome the initial need to

tions come together to suggest to them and to

make sense of the space physically, and I was then

an observer who they are and where they are.

able to “perform” the space better. To be more exact, by “performing” the space, I do not mean to simulate

B) The Allowing Body - State of the Perform-

or fabricate the idea that I was moving underwater,

but that my senses had learnt the language to have a

In order to be truly immersed (as op-

meaningful dialogue with the environment, with the

posed to being a “tourist” in a VR set up), and

sensations that environment offered, that my body

in order to have a meaningful dialogue with the

found the way to react or resist these sensations ac-

elements presented in the VR environment, it is

cordingly. This was the experience when I was im-

crucial that the performers’ bodies and minds

mersed in the VR environment.

are in a state of openness, a state of allowing the sensations that surfaced to lead the bodies

Having articulated that, I had also afterwards

into reactions, manifestations, and expressions,

163

without reservations. This requires some prepa-

ately. The performers were drawn towards the mir-

ration before-hand, and may not be something

ror, a kind of interaction started to transpire among

a layperson could readily achieve.

them – we were waving to one another, mimicking one another, etc. This kind of interaction did happen

How is this useful?

during previous sessions without the mirror, however this experience was in a way more readily satisfying

Now, I do not think at this point in our

for the performers. It was perhaps due to the fact of

exploration that we are able to say how this

the mirror being a reflective medium that invoked a

is definitely useful to a performer in his or her

sense of self-recognition in a VR space where it could

practice, but there are two thoughts here:

be easily disorientating, given that VR is a space that traverses realities.

It does made us realise that an awareness to the sensation of the performers is key to the

Re-cognition of the Mind and Body:

imagination and manifestation of a fiction in performance.

But of course, the power of a mirror

2. That an immersive experience provided by

having this effect of re-cognition of the self is

VR technology may allow the performers to

also well-known and common in the real world.

react and relate to something that is uniform

However, I think what was significant was that

in intensity and scale across them, rather

when experienced in a VR environment, it be-

than leaving their work on fiction to the lim-

came something quite important, serving as an

itations of their individual ability to imagine.

anchor for a performer who was in the process of embodying an avatar (character). This recog-

One more note:

nition of the self was both visual as well as mental. In the first place, it affirmed the existence of

Just to remark on the point that during

the performer within these realms of uncertain

the session, because the landscape environ-

realities, by reflecting upon himself/herself its

ment was so overwhelmingly strong as an ex-

relationship with the other bodies or characters.

perience, the other performers’ existence as av-

Secondly, it affirmed the physicality of the per-

atars in the VR world paled in comparison and

formers’ character when the performer saw his

was quite readily ignored. A point worth noting.

or her creation of the characters come alive as a visual and physical rendering. Although I must add one point here that due to the limitations of available technology, the absence of a torso and legs were something worth taking note of as it

SESSION #8 –

definitely had an effect on the process of rec-

MIRROR: THE REFLECTIVE EXPERIENCE

ognition – namely the congruence of the bodily experience of the performers and their avatar ( I

In this session, on top of the usual abstract VR space, a large mirror was placed inside the space. Something quite interesting happened immediP ost-sessions reflections

feel my legs move but I don’t see it – so did I or did I not move my legs?).

Out-of-Body Experience: In the session with the mirror, I also had a sense of an out-of-body experience when I saw myself, or rather my avatar, in the mirror. What this could potentially mean may be that we are able to evoke a meta-sense of existence in the space, recognizing yourself, controlling yourself, and experiencing yourself from a higher realm. I wonder what does that mean in terms of Performance or how does that relate to the art of performance.

165

Pei Qin’s Reflections MY INITIAL INTEREST My initial research interest was to examine the efficacy of technology in supporting a theatre performer in manipulating the realities of time and space to create fiction, story-tell or make belief. This interest was formed with the understanding that technology can destabilise/ warp both the actor’s and audience’s sense of time and space, and enhance their imaginations. In our “Architects of Realities” ArtxTech Lab, I am interested in testing the concept of “phenomenology” and “liveness” in theatre, by playing with the duality between the real and fictional in performance, using the disturbances or provocations that technology can bring. “There is a tension between the actors’ phenomenal body, their bodily being-in-the-world, and the use of that body as a sign to portray a character. ... We will describe this tension as occurring between the ‘phenomenal body’ and the ‘semiotic body’” (Fischer-Litche, 26) For most of our Lab Sessions, we were constantly grounded by theatre acting training concepts and exercises such as Meisner’s Technique, Viewpoints and Michael Chekhov’s psycho-physical training, to locate the performer’s body in space, and to connect with our own ‘phenomenal’ bodies. These exercises remind our bodies and minds to exist in the immediate material and biological present, the “as is”, before embodying the “as if”. When I was immersed in the VR world as a player, my immediate sense of time and space and the surrounding environs shifted drastically, and I was transported into a different reality and fiction. It became harder to remain connected with my “as is”. This is not to say that the preceding actor-training work had come undone, but there were challenges to overcome once the VR goggles were put on.

P ost-sessions reflections

CHALLENGES FACED IN BRIDGING THE

VR world, even my sense of direction was disoriented

PHENOMENOLOGICAL WITH THE FICTION-

and I found it challenging to find my bearings. In VR,

my avatar could be right next to my team-mate’s avatar, when in actual fact, my team-mate could be at the I had to first recognise myself as a virtual

avatar existing within this alternate reality while si-

opposite end of the demarcated playing space from my phenomenological body.

multaneously grappling with the time and space my phenomenal body actually exists in. Embodying the

Despite these aforementioned challenges,

virtual avatar reminded me of mask work or puppetry,

I was determined to grasp or attain a more rooted

with these objects usually imagined as an extension

sense of phenomenology as a performer within the

of the performer’s physical and psychological self.

VR space.

The huge difference is, I could not physically touch the mask or puppet to form a connection, and the

THE SEARCH FOR PHENOMENOLOGICAL

only physical contact connecting me with the avatar

ROOTEDNESS IN VR

character is in the form of my VR goggles and touch hand controllers with their own set of control buttons

We tried to connect more with our “as is” in

(which are moulded functionally for the gamer-play-

Session #8, when there was a mirror in the VR world

er, but not for the same purposes of creating fictional

for our avatars to view our virtual selves (which we

characters like masks and puppets are intended for

previously could not see as we were designed to view

the performer). Therefore, there was already a gap in

the VR world through the camera lens/ eyes of our

conflating my phenomenological and fictional selves,

own avatars in the preceding Lab sessions), and to

and a sense that I could not have control over either,

view the other reflected avatars beside us and our

in the way I was used to in live rehearsals and perfor-

distance from one another. The presence of the mir-

mances. For example, the VR avatar could not morph

ror and being able to look at oneself in the VR world

its facial expressions according to my facial features,

did help us locate our bearings and make sense of

or perform the full range of movements as the torso

our relationships to one another as fellow players.

and lower half of the body’s anatomy was not configured and even if designed, would not be able to

For the first time in all of our Lab sessions, we

replicate joint or limb movements accurately. Hence

also decided to tweak another variable - the direc-

to embody a fictional self or character to suspend dis-

tor’s presence. There was Khairul’s guidance through

belief, it would require a bigger leap of imagination

his voice directing us to stay in tune with our physical

on the actor’s part, as compared to conventional re-

bodies, as we existed in the VR world. His voice was

hearsals/techniques for the theatre.

a “kick” out of the VR, but at the same time a “totem”

and reminder of the actual world, as he had previousAnother aspect for an actor to get used to (or possibly be trained in when employing VR to create fictions for rehearsals and performance in future), would be to bridge the gap in logic for both the body and mind, when existing in two (or more) different spaces at the same time. When I was immersed in the

The performer’s various techniques can be conscious and codified or unconscious but implicit in the use and repetition of a theatre practice. Transcultural analysis shows that it is possible to single out recurring principles from among these techniques. These principles, when applied to certain physiological factors - weight, balance, the use of the spinal column and the eyes - produce physical, opre-expressive tensions. These new tensions generate an extra-daily energy quality which renders the body theatrically “decided”, “alive”, “believable”, thereby enabling the performer’s ‘presence’ or scenic bios to attract the spectator’s attention before any message is transmitted. (Eugenio Barba, “The Paper Canoe: A Guide to Theatre Anthropology”, Chapter 2: Definition, Routeldge, 1995, pp.9) 1

167

ly guided us in finding connection with our phenom-

virtual world and the impulses it gave me. I stood still

enological bodies before we put on our VR goggles.

and either embraced or dodged a virtual fish coming

We wanted to see how that could bleed into the VR

my way, or let a wave “carry” my body as I “bobbed

world. By establishing a heightened awareness of

up and down” lightly by bouncing on my knees or the

my body and keenly observing my avatar and others

balls of my feet.

through the mirror, we started to mirror one another, or influence one another based on visual stimu-

I wanted to find out if Garner’s statement

lus or influence. For example, if one avatar tip-toed,

holds true for a performer and observers of the per-

another might do the same or the opposite such as

former in a VR space. On the most basic level of my

ducking down lower. It was as if we were doing View-

outward appearance, my own actual “material, phys-

points Lanes Work through the mirror, but relying on

iological facticity” could not fully emerge phenome-

the visual response much more than kinaesthetic re-

nologically in this virtual world, as I was represented

sponse. I was able to retain a sense of control over

as an avatar in the form of a robotic figure, just a face

my own fictional and phenomenological bodies this

and two hands. However, the biomechanics of my

way, and observe how my performative body could

physical movements or quality of movement could

influence others co-performing with me in VR.

be truthfully captured or translated into this virtual dramatic representation and “escape transformation”

“The actor in his or her “material, physiological factic-

to a certain muted extent. In fact, I posit that the full

ity,” inserts a “fundamental and intrusive actuality into

extent of my “rootedness in the biological present”

the field of dramatic representation... A rootedness in

would interestingly be more observed by the spec-

the biological present that always, to some extent, es-

tators outside of the virtual realm, who are viewing

capes transformation into the virtual realm”” (Garner).

my phenomenal body wearing the VR goggles experiencing an underwater fiction they may or may not

When we were immersed in Virtual Reality (VR) for Session #7, under the sea with underwater

see (depending on whether or not the VR world is being projected for viewing at the same time).

sights and sounds (e.g. Fish swimming around us), I found my body instinctively wanting to adapt to that

With all factors considered, at this point of our

fiction straightaway. I wanted to imagine my own ava-

Lab experiment (end of Session #8), we are still at a

tar blending into, or fitting into this new world, hence I

jamming and discovery phase. We have not yet done

started performing the artificial act of swimming such

the technical composition of the work of a performer

as extending my arms and waddling about. However

using performative signs/ semiotics yet, in creating

that did not feel truthful as my body was still bound by

layers of fiction. We are merely just getting used to

the laws of physics actually being in Nine Years The-

the conditions given by our VR design, and allowing

atre’s dry, air-conditioned rehearsal studio. Even as

our bodies to become neutral enough yet familiar to

my arms were “swimming”, my hip and legs remained

the VR conventions before they serve as ready ves-

upright and I was still standing vertically instead of

sels to take on layers of semiotic meaning.

fully swimming horizontally as imagined fictionally. Instead of imposing a fictional body or existence of

Once this aforementioned conditioning of the

self in this world, I then decided to allow my phenom-

body can be more established, will we then be better

enal body to just listen and respond naturally to this

equipped to address the next part of what I was in-

P ost-sessions reflections

terested in as well - testing for the efficacy/ effects of technology responding live to the actors’ movements and director’s prompts, such as whether “kinaesthetic response” is possible, and whether technology can enhance the jamming process of creating movement scores. Moving on for future experiments, I am keen to answer the following questions for myself as a theatre director who works with movement devising: •

Whether immediate “kinaesthetic response” is possible (or what we can do with the lags, if any?) - We tested this in Session #7 and #8, immediate “kinaesthetic response” is possible although with some time/ response lags.

•

How the body talks to technology and how technology talks to the body. What are the different levels of seeing and accessing spaces? How can these interactions inform the way Body and Technology can have a conversation with one another? How can the different levels have conversations or echoes with one another?

•

Whether technology can enhance the jamming process of creating movement scores (e.g. Help with recording and duplicating repeated patterns/ gestures, like a canon?)

REFERENCES Eugenio Barba, “The Paper Canoe: A Guide to Theatre Anthropology”, Chapter 2: Definition, Routeldge, 1995, pp.9 E. Fischer-Lichte, “The Semiotics of Theater (Advances in Semiotics)”, Indiana University Press; Abridged edition, 1992 Stanton B. Garner, Jr., “Kinesthetic Spectatorship in the Theatre”, Palgrave Macmillan, Cham, 2018

169

Timothi Lim’s Reflections The ArtXTech lab has been a pleasure to participate in, especially with the talented, infinitely inspirational, and wondrous individuals that make up the lab participants. The Architects of Reality group has ambitious goals that are feasible and ultimately game-changing, if one is open-minded to accept a vast vision. While I am careful to stereotype, Singapore does have a majority consensus of local culture that fast tracks mindsets towards practicality and safety, which in turn leads to aversion of risk and failure, two factors that Art and Technology inherently contain. To me, Art is an expression of the self through theory, thinking, and reflection in a given context and environment, and that expression will fail and it will carry the risk of rejection. Technology to me is also an expression of the self through theory, thinking, and reflection in a given context and environment. That repetition was intentional. Art and Technology are one and the same. Art has made use of technology to empower its expression, from paints to canvases, to paper making, and today to digital animation and computer graphics. Likewise, technology has also made use of art to empower the communication of its complex nature, thus we have user interfaces that allow us to humanely navigate a non-human digital world. As a whole, the ArtXTech lab has further cemented my philosophy that to segment and silo and industry would be the death of collaboration, creativity, and innovation. The art industry cannot exist alone, and likewise, the technology industry cannot exist alone. Nothing lives in a vacuum, and industries must collaborate, different mindsets and processes must clash and crash together to form new versions of the old. Mindsets are to me the crucial essence of human growth and innovation. As such, the ArtXTech lab is less a lab of new processes or skills, but rather a lab where mindsets must change, where mindsets must come to terms with new concepts and perspectives. Only then, when an individual is open to change, will collaboration flourish. There is a phrase that goes “only when you are at your lowest point, are you open to the greatest change” and for many, the context and environment of the COVID-19 crisis gave rise to many lowest points,and also great change. Prior to the crisis, many private companies met through my position as Creative Director of The Doodle

P ost-sessions reflections

People LLP., would rarely consider emerging technol-

ble to connect and bridge physical actors into digital

ogy such as Augmented Reality (AR) or Virtual Real-

reality and vice-versa. This technology, when devel-

ity (VR) as a need. The adage that was consistently

oped further, will allow international and global col-

thrown my way was “if it ain’t broke, why fix it?”, which

laboration through a wide variety of consumer tech-

showed a risk-averse, low innovation mentality in lo-

nologies that technologists and theatres will have

cal private companies. These local private companies

easy access to, especially now that e-commerce

which were very much reliant on government subsi-

allows us to purchase anything online. Imagine, two

dies and funding, would only move to act when the

theatres, one in Singapore and another in the UK,

government would offer them opportunities to miti-

both have Windows PC systems that are connected

gate the risk of innovation. Practicality and safety is

to projectors and depth sensors. The projectors cast

a must for a stable government, but when that men-

the virtual world onto the wall, a virtual world that is

tality trickles down to local private companies which

modeled to be like a physical stage (assuming it is for

are the lifeblood of business innovation, it becomes a

a physical performance). Depth sensors in the form of

cause of worry for the longevity of a country’s creative

inexpensive Azure Kinects, are pointed at the actors.

culture. In my business meetings, I have come to re-

On the PC system, a computer program coordinates

alize the co-dependency of the Singaporean private

the hardware and connects the data to the Internet.

sector with the public sector, and while it is helpful

Through technology, both theatres across timezones,

and stems from a place of good intentions, it creates

and oceans can practice and perform with each oth-

a comfort zone that few business leaders are willing

er with little to some latency depending on wireless

to step out of, especially in times of peace. If business

strength (which can be mitigated with the use of an

leaders only act when a crisis hits, our innovation will

ethernet cable). Director, writers, and perhaps even

be short term and will not bear fruit far into the future.

instructors, can also view the physical-digital perfor-

That is why I advocate for an increase in partnership

mance through Augmented Reality via a phone app,

and collaboration, of which requires both companies

or Virtual Reality via donning an Oculus Quest head-

to be independent and willing to step forward togeth-

set. The core component is the digital software that

er onto the road less travelled. The mission of The

allows it to connect AR, VR, and PC systems together

Doodle People is built around connection and uni-

through the Internet and thus empower collaboration.

ty, and that mission continues into this ArtXTech lab

In the digital world, actors can interact as avatars or

through myself as well. The technology developed

as point-cloud versions of themselves. While the dig-

through the Architects of Reality group has accom-

ital world is no substitute for a physical world, it can

plished a vision I have always had since 2019 and it is

open up new avenues for expression while also serv-

that of the Metaverse - an interconnection of technol-

ing the minimum function of facilitating international

ogy that bridges both the physical and digital reality.

collaboration.

The Architects of Reality group has created

To me, the ability to connect to a singular

a working minimum viable product of a portable de-

digital world regardless of technological platform,

ployable metaverse. Quite literally, with the help of

which is termed cross-platform, is the basis of the

computer systems, depth sensors, mobile phones,

metaverse. Through hardware such as the depth sen-

virtual reality headsets, and talent in the form of ac-

sors and projectors, we can then connect the digital

tors and technologists, our group has made it possi-

to the physical world and vice-versa allowing for the

171

metaverse to form. With the Architect of Reality group,

dle such ‘impossible’ tasks simply because the ap-

it has taken around 10 sessions to form the correct

proach is different and the aim is not to solve a prob-

mindset to utilize and perform with the metaverse

lem, but rather to identify and reveal key factors that

technology developed through the lab. 10 sessions of

could change mindsets and thus solve outstanding

around three hours each, totals to around 30 hours

issues. The Architect of Reality group to me isn’t here

of which it takes to align and formulate a ‘metaverse’

to solve a problem, it is here to explore, identify and

mindset or as I usually call it, a creative technologist

reveal a key flaw in our industry process, namely that

mindset. Artists, actors, stage managers, and direc-

the Singaporean Art sector is very isolated and is un-

tors would need to understand technology, not from

empowered to collaborate with other industries due

a utilitarian perspective, but from a philosophical per-

to fixed local mindsets viewing Art as ‘uneconomical’

spective. Technologists likewise must learn, respect,

or facing troubles in communication due to different

and empower the various artforms the artists seek to

mindsets and processes. However through our col-

express. A key component I have found to be a cru-

laboration and our creation together as the Architect

cial bonding factor is that of dramaturgy or the theory

of Reality group, I hope to reveal that the art sector

of performance. Theory might seem academic but it

and the technology sector can collaborate if only the

is a useful tool for alignment and for discussion. The-

various participants and the stakeholders are willing

ory allows both groups to coordinate around a vision

to set aside differences and to align with each oth-

or a pre-ordained mindset and evolve from there.

er as opposed to simply adopting a process or vying

This is what I believe the Architect of Reality group

for power, especially during development. Collabo-

did well, which was to utilize theory as a framework

ration must create its own processes and mindsets

and carve their own path from there. In their explora-

to be effective. Frameworks can be initially adopted

tions, I admire the ability of the group to upkeep the

but must quickly be customized to fit the group if it

spirit of the arts in creative exploration as opposed to

will certainly fail to connect. There is no ‘one fits all’

quickly adopting a critical thinking and solution ap-

solution when it comes to collaboration, which after

proach normally utilized in business and technology

all is about relationships between humans seeking to

development. While the critical thinking and problem

create together.

solving process is almost de-facto in today’s world, it is important to be open to the fact that it is not the

Communication and relationships between

only method and certainly not always the most effec-

humans are to me the crux of our creativity and in-

tive. The key issue I have with the critical thinking and

novation. Communication creates opportunities for

problem-solution approach is that it assumes there

creativity to grow as new perspectives can be learnt

is a problem to be fixed. The method proves ineffec-

and old perspectives modified. Relationships form

tive in the arts which rarely needs to fix a problem

because of value alignment and connection to one

but ultimately needs to express the impact of a prob-

another, thus leading to the opportunity to innovate

lem. Human problems such as biases which lead to

together and carve new paths forward. My belief is

racism, gender inequality due to fixed mindsets and

that creation is inherently human and that while we

rigid thought processes, and more are tough to “fix”,

would like to systemize creation and make it effi-

especially since every human is varied and differenti-

cient, humans are not a machine and our daily life

ated. However, critical theory and creative exploration

is certainly inefficient as a whole. As such, it is my

as an innovation process allows every group to han-

worldview that creation in of itself is a random natu-

P ost-sessions reflections

ral occurrence brought about by human inefficiency.

that we need to make the best choices always leads

Certainly you do not consider yourself a perfectly ef-

to a life of misery and suffering, because all efficiency

ficient human that is 110% more productive than your

and all choices are obsolete when faced with death.

fellow human, so why does it feel as if we are being

Death is a certainty for all living organisms and if we

pushed to be by our education, social-political, work

are to talk about the most efficient process, it certain-

systems? Humans will always be inefficient because

ly has to be death. As such, regardless of what we

that is nature. Nature is inefficient individually but as

do with our lives, we will always die, however what

a whole, efficient in the long-run, leading to evolution

doesn’t die? What doesn’t die is what we humans

and giving rise to life. Humans however, unlike na-

leave behind, our legacy for the next generations

ture, have agency, we have a brain that allows us to

to come. If humanity truly seeks to be efficient and

control our own development. However we are lim-

make the best choices, we would as a whole look to-

ited by what our human body can do, and as such,

wards the long-run and build a selfless legacy that

technology allows us to grow beyond our physical

inspires future generations to build upon our choic-

and logical limits. However, technology has yet to

es, of which collectively we have tried to make the

empower us to grow beyond our metaphysical and

best of what we have. However, seeing that we are

emotional capacities. We as humans are inefficient

individualistic selfish creatures that very much focus

and that is why we need to constantly innovate, and

on the short-run, we consistently give increasing-

to collaborate together. We need each other to grow

ly worse circumstances to the next generation and

and that is why we must always strive to communi-

then pass the blame to each other in a cyclical fash-

cate with each other regardless of our race, religion,

ion. So an individual who focuses on efficiency and

or worldviews, and through our struggles in commu-

making the best choices is, on a macroscale, ineffi-

nication, form relationships with those that align with

cient anyways. Perhaps efficient for themselves but

our values and create new mindsets together. That

ultimately inefficient for the species as a whole, and

to me is how creativity and innovation spring forth,

even the planet. Death should truly be a concept of

from being humble as a human. Humans are after all,

unification, a process that enables us to collaborate

a minute infinitely small organism floating on a rock

and create together for all of us. Who cares if what

in an ever-growing gargantuan universe. If we truly

we create doesn’t meet KPIs or if communities fail in

are ‘efficient’ creatures, we’d be out there in the uni-

their goals, failure is ever present in being a human,

verse right now being our best versions of ourselves,

that is how we must learn. We will always fail as hu-

not fighting amongst ourselves for limited resources

mans and while our history will show that time and

on a small rock of a planet that we are increasingly

time again we fail, our successes will show that we

destroying.

can rise above. Collaboration and innovation are key to building a human legacy that transcends time and

The average age of death is around 70ish

space, and lets us rise above our human failures.

years for a human in a developed city environment. We don’t live very long as an organism, especially

As a species we already face the certainty of

when compared to ‘less sentient’ creatures such as

hitting our lowest point, global warming and irrevers-

lobsters or turtles. As such, we may feel a need to

ible environmental change, which can lead to all of

be efficient with our limited life span. However, it is

our deaths. Perhaps that may lead you to focus on

my belief that a focus on efficiency or on the mindset

yourself, but really what we need now above and be-

173

yond, is collaboration. One spark can ignite a flame, and that could be all it takes to save us, and the best solution to problems usually comes from unlikely places and people. Share your ideas, express your thoughts and collaborate with your fellow humans who have learnt different skills and lead different lives. So what if we lose? We’ll do that together.

P ost-sessions reflections

175

177

LOOKING FORWARD

While our process has been rigorous, rich, and enlightening, there is a consensus among the research team that we have only begun to scratch the surface of this interdisciplinary field between emerging technologies and contemporary theatre practices. Being the (mostly) first encounter for the theatre practitioners at such a time period with these technologies, we have only begun to understand the ways in which we can arrange, format and engage with the digital tools at our disposal, knowing well that there are even more advanced technologies and tools that can deepen (and also further complicate) our inquiries. For our technologists involved, learning acting and theatre making techniques have been fun and eye opening but has made them acutely aware that it takes years of physical and practical actor and directing training to tacitly know and control their own body in a performance through space and time. A question arises: “how much does the theatremaker need to know about and how to use technology and similarly for the technologists in regards to being able to perform as an actor or theatre maker?” From our sessions it can be gleaned that it is not necessary for an actor or director to know how to use TouchDesigner, Unity, Unreal Engine or the Kinect Azure, but what definitely assisted the process was the knowledge of what these different technologies are capable of and how they can be arranged among each other to create several layers of realities. Potentially there are many more features in these components that we did not delve into due to time constraints but perhaps if we were to continue this research, we could go beyond the basic features of the technologies to see just how far they can alter the performance spaces that can be created in the actual studio space between practitioners and object. Similarly, the technologists may now see new ways that the technologies they employ can be engaged with. The end users of their

179

future work or experiments can be thought of beyond the usual types of interface that they may expect of a person, demanding perhaps a level of physical, focal and virtuous expertise in the surrounding use of their applied technologies. Since this lab was merely a prototype to imagine possibilities of what the outcomes can offer to relevant parties, the following lists possible developments that the Architects of Realities may undertake in the future, its requirements and how it may potentially benefit the stakeholders and the team: •

Students actors for participants: the team could work with theatre institutions to conduct workshops facilitated with/ through technology, targeted at existing theatre students and professional actors, with an aim to make meaning of the relationship between technology and theatre, and question how this discovery may inform the participants’ art work in future. A wider audience (live in physical attendance or streamed in) are welcomed to observe this workshop and listen in to the artists’ sharing of the process as well.

•

Other types of dedicated actor participants to experiment with so that the core team can focus on the process development, research inquiry and documentation.

•

Diving into the philosophy of “virtual” bodies, not simply in the sense of digital immersive environments, but affecting the memory embedded within he body, such as phenomena such as phantom limbs, transporting presence, touch as perceived by the mind, movement in various environments, etc.

•

Technical documents regarding training in performance, more specifically long term performer and theatre maker training with the technology, software/hardware skills and the incorporation of contemporary actor training processes with these emerging technologies.

•

Education and sharing for the public: School and public open workshops for participants to experience undergoing our workshop and training experience.

Looking For ward

•

Refining our techniques: Dedicated technologist setup time to test equipment configurations to explore a range of interconnected layers.

•

Expanding practice and research space: Acquiring a dedicated studio space for hardware and equipment storage for long term experimentation, continual research (with new funds) and development.

In the technological aspect of the future of Architects of Realities, the outcome of our lab should further the development of ArtXTech workshops in Arts schools such as SOTA, LASALLE, and NAFA, to allow burgeoning artists the opportunity to innovate and inspire the future of the Arts through immersive/interactive technology. Empower young and rising artists with freedom to explore Immersive Media as a medium for the expression, interpretation, and critique of our human condition. Our lab has connected actors and directors with a blend of technology that leverages Virtual Reality, Augmented Reality, and Mixed Reality. Through game engines such as Unity, we are able to create real-time interactions that allow us to input and output interactions to both a virtual reality and our physical reality, affording the group the literal ability to architect realities. Empowered with the ability to architect/create a new reality - artists, technologists, directors, producers, audience, and anyone who seeks to be a part of a new constructed reality; can participate, thus aligning all with a shared vision or idea made manifest through our senses.

181

ARCHITECTS OF REALITIES: Art x Tech Lab 2021 Compendium

Contributed by:

Cherilyn Woo Joanne Ho Khairul Kamsani Nelson Chia Thong Pei Qin Timothi Lim

This compendium was compiled in January of 2022.

183