SAC Journal 06: Breaking Glass

While teasing the disciplines with creative opportunities, the use of the media presents a staggering number of acute questions, not the least with respect to corporeal experience, the human-machine interface and what constitutes the “real.” Augmented and Virtual Reality invite to re-examine established ways of thinking and making within architecture and the arts and open onto an uncharted territory of what comprises architectural and artistic experience.

With Breaking Glass: Spatial Fabulations & Other Tales of Representation in Virtual Reality, select topics central to Augmented and Virtual Reality in architecture and the arts are addressed. It is published in conjunction with the conference Breaking Glass III: Virtual Space, the third and last in a series hosted by the Städelschule.

The publication includes texts by, among others, Martine Beugnet, Michael Young, Curtis Roth and Lara Lesmes and Fredrik Hellberg as well as conversations that Daniel Birnbaum respectively had with Sanford Kwinter and Sven-Olov Wallenstein. In addition, a series of visual portfolios by architects and artists presents works.

Finally, the publication features the award winning projects of Städelschule Architecture Class’ AIV Master Thesis Prize 2019.

The issue has been edited by Yara Feghali and the editorial team of the Städelschule Architecture Class. It has been made possible with the generous support of the Aventis Foundation and the Dr. Marschner Foundation.

© SAC JOURNAL is published by the Städelschule Architecture Class and AADR – Art Architecture Design Research (Spurbuch Verlag).

6 S AC J O U R N A L

ISBN 978 – 3– 88778–618 – 2 ISSN 2198 – 3216

B R E A K I N G G L A S S: SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY

Within a relatively short time, Augmented and Virtual Reality have emerged centre stage in architecture and the arts as novel means for exploring how their creative output is produced, mediated and experienced. Feeding the continuous spectrum between the “fully real” and the “fully virtual,” the underlying technology of these media present machine-generated sensorial input where to date the image-based dominate. With these inputs, corporeal existence see “virtual” experiences thrown on the scale with “real” ones as the concepts and models for how we understand perceptional dynamics are shifting.

BREAKING GLASS SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY

SAC JOURNAL STÄD EL SCH U LE A RCH I T EC T U RE CLASS

Cover SAC Journal 6.indd 1

26.03.21 14:07

BREAKING GLASS SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY

SAC JOURNAL 6 SAC Journal 6 1

31.03.21 10:04

4 EDITORIAL

BREAKING GLASS

8

INTRODUCTION JOHAN BETTUM

AN ECOLOGY OF SIGNALS & THE SPATIALISED IMAGE

14 A CONVERSATION SANFORD KWINTER & DANIEL BIRNBAUM

THE THIRD GLASS

28

ESSAY FREDRIK H ELLBERG & L AR A LESMES

TWEETING OUT LOUD IN THE SQUARE

34 ESSAY MARTINE BEUGNET

VIRTUAL REALITY AND THE GAZE ON FRAMELESSNESS, PANOPTIC VISION, AND PARRAGIRLS' MEMORY WORK

42 ESSAY CURTIS ROTH

GESTURING ELSEWHERE

48 ESSAY MICHAEL YOUNG

EXCESSIVE RELIEF

58 TRIALOGUE JONATAN HABIB ENGQVIST, CHRISTER LUNDAHL & MARTINA SEITL

AN ACCEPTABLE LEVEL OF REALITY

68

A CONVERSATION DANIEL BIRNBAUM & SVEN-OLOV WALLENSTEIN

A NEW CURATORIAL TOOLBOX

SAC Journal 6 2

31.03.21 10:04

CONTENTS

72

144

ESSAY FABRIZIA BANDI

PORTFOLIO LUNDAHL & SEITL

THE ARCHITECTURAL RELEVANCE OF VIRTUAL REALITY

82

PORTFOLIO KLING KLANG KLONG & ONFORMATIVE

COLLIDE

90 PORTFOLIO AINSLEE ALEM ROBSON

FERENJ A GRAPHIC MEMOIR IN VR

96 PORTFOLIO FOLLY FEAST LAB

MEDITERRANEAN SEA DIARIES ON THE AFTERLIFE OF OVERPRODUCTION

104 PORTFOLIO PAISLEY SMITH & LAWRENCE PAUL YUXWELUPTUN

UNCEDED TERRITORIES

SYMPHONY

152 PORTFOLIO MARCO BRAMBILLA

HEAVEN'S GATE (MEGAPLEX)

160 PORTFOLIO JACOLBY SATTERWHITE

WE ARE IN HELL WHEN WE HURT EACH OTHER

167 INTRODUCTION THE AIV MASTER THESIS PRIZE 2019

168

AIV MASTER THESIS PRIZE 2019 HAEWOOK JEONG

PATCHED CITY THE ARCHAEOLOGY OF THE DEBRIS

110

178

I CAME AND WENT AS A GHOST HAND

AIV MASTER THESIS PRIZE 2019 ARUNA ANANTA DAS

PORTFOLIO RACHEL ROSSIN

118

PORTFOLIO SPACE POPULAR

FREESTYLE

126

PORTFOLIO TIMUR SI-QIN

A NEW PROTOCOL VR

132

METABOLISM MUTATING NEW YORK

188

AIV MASTER THESIS PRIZE 2019 YEON JOO OH

SATURATED SPACE DESIGN PERFORMANCE IN VIRTUAL REALITY

198 PROJECT AND IMAGE CREDITS

PORTFOLIO MARSHMALLOW LASER FEAST

199

138

COLOPHON

WE LIVE IN AN OCEAN OF AIR

PORTFOLIO CURTIS ROTH

GESTURING ELSEWHERE

SAC Journal 6 3

31.03.21 10:04

EDITORIAL BREAKING GLASS: SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY 4

SAC Journal 6 4

31.03.21 10:04

For architecture and the arts, technology is a double-edged sword. The negative effects include those that typically deliver generic standardisation, degraded aesthetic qualities and uncritical consumption. In this respect, the rate at which technology becomes adopted in the disciplines does not help, and the influx of computerised processes and mediation during the past three decades is a case in point.

The grand upside of technology, whether in the form of new ways of working, novel material systems, construction or design procedures, comprises of opportunities for formal and/or aesthetic innovation. Thus, as the technology supplying the media of Extended Reality (XR), here principally in the form of Augmented and Virtual Reality (AR and VR), is increasingly adopted in architecture and the arts, there are good reasons to ask what this entails beyond reinforcing established ways of working.

With this sixth issue in the publication series of the Städelschule Architecture Class, the SAC Journal, we address some of the history and issues that are at stake at a crucial moment for architecture and the arts.

The use of AR and VR raises questions far beyond immediate disciplinary concerns as the media are inscribed in a larger field of speculations than those raised by the technology alone. For instance, they profoundly concern the production of images, our relationship to these regardless of technology, and thus how these feed our becoming subjects under the spell of the massive, contemporary influx of information. Thus, this radical momentum pertains to the image as a building block for AR and VR, but it goes beyond the assumed centrality of sight also to include the other senses. The media prompt us to reevaluate the strict division between that which we refer to as “real” and “not real.” This “real” is given through perception, and recent findings in neurophysiology enable a new understanding of the underlying processes for sensorial experience through which we relate to the world and construct our sense of reality.

With AR and VR the experiencing subject is situated in a novel and potentially interactive relation to a partly or fully simulated environment. For architecture this entails re-inscribing the human subject in discursive and practical contexts with a rejuvenated arsenal of insights, tools and processes. This re-inscription does not restore a conventional phenomenological approach to architecture but fills that subject’s absence in the annals of recent discourse and could eventually prompt the emergence of a new formal approach to architectural design.

5

SAC Journal 6 5

31.03.21 10:04

In the arts, the media open a new space for conveying art which may destabilise the art world’s dominant market-driven and institutional forms. This destabilisation is partly precipitated by the intimacy between the immersed subject and the art object as well as the interactive relation that AR and VR make possible.

All of the above is encompassed by the central and yet evasive phenomenon of virtual space that AR and VR present. Its pervasive immersive quality make these media the ultimate architectural tools. Hence, they are already generally used to present us with known spatial constructs and thus extend the existing presentation kits in architecture, the arts and the entertainment industries. However, they can also be employed to explore, research and experiment with the more intangible qualities of space. This presents us with a crucial opportunity given that we live in an extraordinary time when a global health crisis radically has constricted the spatial spheres that we previously assumed were ours and perhaps took for granted.

In sum, AR and, in particular, VR presents us with an unprecedented occasion for furthering our understanding of what space is and can be. Or, as phrased by Sanford Kwinter: ’It completely redirects our conception of space [and allows us to] re-understand how we experience the world and design for that re-understanding.’1 The subject, the image and space comprise the thematic concerns for the work with AR and VR undertaken in the Städelschule Architecture Class since 2015. This includes both the space that architecture and art present us with but equally important the space that architects and artists occupy when they design. For both, the technologically based media establish new inroads into fundamental aspects that underlie the formation of the disciplines themselves.

As an entry into the vast realm of topics at stake herein, this issue opens with a conversation between Sanford Kwinter and Daniel Birnbaum hosted at the Städelschule in 2017.2 The conversation took place during the academy’s annual open house exhibition, Rundgang, in which the Städelschule Architecture Class presented its Third Glass, a humorous interpretation of Marcel Duchamp’s Large Glass in AR and VR. Kwinter and Birnbaum’s exchange lays bare easily neglected historical strands of the media and presents a series of concerns which could have a radical impact on architecture and the arts.

In anticipation of the increasing importance of the virtual in our lives, Fredrik Hellberg and Lara Lesmes (Space Popular) in their contribution to this publication relate the virtual and the real worlds to one another and examine our use of spatial references, architectural metaphors and analogies for the virtual. Their interest revolve around social life and the affordances that architectural elements and spaces offer. Despite the virtual’s uncanny ability to simulate the real, they insist on their differences and call for precaution as we proceed.3

1) As stated in conversation with the editors.

2) Kwinter’s contribution has far exceeded that of a participant as he has advised on the unfolding

In her contribution, Martine Beugnet4 addresses the contemporary issue of surveillance in relation to the panopticon, the immersive condition of VR, and the frameless image that VR experience is based on. With her erudite background in film theory, Beugnet elaborates details pertaining to, amongst other things, ‘the freedom of the spectator turned experiencer’ and examines a specific project to make a case for how VR enables ‘polyphonic narration’ and evoke memory. Her contribution maps a political and cultural territory far beyond the immediate technological.

work for five years. The conversation in 2017 was the first of five Städelschule AR and VR related events that have featured Kwinter as a participant.

3) Space Popular also contributes a project in the portfolio section herein.

4) Beugnet’s contribution was presented in a lecture during Breaking Glass II - The Virtual Image in 2019.

The economy that is implied in our relationship to the image regimes so central to AR and VR, is of a mixed nature given the corporate control of the technology and social media. This control exploits our shifting attention within the saturated visual field where bodily movement is subtly connected to if not regulated by the visual input. Curtis Roth examines this mixed economy and makes a case for the connection between attention,

6

SAC Journal 6 6

31.03.21 10:04

EDITORIAL

wrist movement and the subject in relation to the image. Historically and technically anchored in how we engage with mediated imagery, his call is for resistance through creative engagement with the technologically driven media given the value of gestural movements - thus, sensory attention in immersive space.5 Returning to write for this series of publications, Michael Young tackles the quintessential architectural dyad of form and space. With great attention to detail, Young rereads Adolf Hildebrand’s seminal text The Problem of Form in the Fine Arts from 1893 and concludes that our challenge is to understand the problem of depth. He links this problem to contemporary techniques for designing, and his insightful and provocative text is an essential contribution to a renewed take on the relation between form, space and the contemporary technological image.

Addressing the role of the virtual in their own artwork, Martina Seitl and Christer Lundahl are joined by Jonatan Habib Engqvist to argue that ‘virtual reality is an ability rather than a form of technology.’ While VR is fully inscribed in the production of the artwork, it is never centrally thematised. Rather, they make a case for our organising perception into a living reality by activating our imagination and multi-facetted sensory capacity. Bodily experience and everything outside of it are no longer discrete binaries but open onto an “in-between” filled with friction and enriched experience.

Daniel Birnbaum, upon whose suggestion the engagement with AR and VR in the Städelschule Architecture Class commenced, enters into another conversation herein, this time with the philosopher Sven-Olov Wallenstein. Their conversation addresses the virtual as a phenomenon not invented with the latter-day technology. Their talk helps to situate current work in AR and VR in the context of a difficult philosophical phenomenon that at once has deep roots within the history of human thought and yet continues to be conceptually developed given contemporary technology.

Finally, with her background in philosophy and aesthetics, Fabrizia Bandi supplies the last text herein. Addressing ‘the architectural relevance of virtual reality, ’ Bandi explores three avenues through which the virtual and the real relate and are not antithetical. Both realms are anchored in bodily experience and its sensory apparatus, and Bandi concludes that ‘the technology of VR enables us to experience and inhabit that architecture which is conceived, imaginable, but not yet physically manifest.’

In the second of its two sections, the first comprising of the texts briefly accounted for above, this issue presents a collection of VR projects by various architects and artists. This entails the documentation of different approaches to working with VR, and the portfolios may be enjoyed for their beauty alone. However, notwithstanding their quality, obviously no documentation based on printed images alone does justice to the profound and often provocative experience with AR or VR. Hence, the visual part of this publication can only suggest the experience of what these projects offer.

Lastly, this issue includes student projects in the Städelschule Architecture Class that were awarded the AIV Master Thesis Prize in 2019.

Breaking Glass: Spatial Fabulations and Other Tales of Representation in Virtual Reality is published on the occasion of the third and last conference in Städelschule’s series, Breaking Glass, which addresses the emerging role of AR and VR in architecture and the arts. Breaking Glass I - Virtual Reality and Subjectification in Art and Architecture took place in 2018; Breaking Glass II - The Virtual Image in 2019; and now Breaking Glass III Virtual Space ends the series. This publication neither summarises the conferences to date nor prefaces Breaking Glass III. Rather, it is presented as a supplement to the latter and aims to give a glimpse into the vast intellectual and creative opportunities that AR and VR open up to.

5) Curtis Roth’s project, Gesturing Elsewhere, is also featured in the portfolio section of this publication.

7

SAC Journal 6 7

31.03.21 10:04

INTRODUCTION AN ECOLOGY OF SIGNALS & THE SPATIALISED IMAGE JOHAN BETTUM

8

SAC Journal 6 8

31.03.21 10:04

“Reality” is a funny word when used in architecture and the arts. It suggests a shared, essential experience, something that cannot be disproven and exists as a fact.

Conventionally, this “real” knows nothing of a notional or imaginative other.

However, whereas architectural design and the arts are devoted to the real, they are so only in the form of what does not yet exist or resists and queries “reality” as given. Each in their own way and architectural design in particular are entirely devoted to imagining what is not yet there. Whence, the business of architecture and the arts is to imagine alternative realities through invention, augmentation, critique and supplement to what is already here. Accordingly, technology that capacitates architects and artists to produce such “realities” presents the respective disciplines with a desirable and powerful tool to pursue their ends. And this is why the coming of Augmented and Virtual Reality (AR and VR) in architecture and the arts raises so many expectations and questions.

Both creative fields currently see a rapid adoption of the mediums of AR and VR for a wide variety of uses. With AR, artists can superimpose their work on any given setting through the handheld devices of their audience, likewise museums can provide their exhibited work with an associative or explanatory context, and VR offers an immersive and fully simulated theatre for narrational and compositional freedom. In architecture, AR is explored in construction to safeguard and optimise the outcome in relation to different qualities and time tables, and VR takes the clients and stakeholders in a project through the fully rendered building that is planned - all with the option to give the user realtime interaction with the computer-generated elements.

Yet, AR and VR raise questions and creative opportunities far beyond those tied to reifying established practice. These opportunities are concomitant to imagining the new, that “real” that does not yet exist.

1) Latter-day AR and especially VR is typically referenced to the 1960s when Morton Heilig patented a multi-sensorial immersive apparatus for

BEYOND VERISIMILITUDE The technology used for AR and VR is not new; it comprises of computer software and hardware which gradually emerged in the 1950s and -60s.1 During the last few years the technology and hardware have greatly improved, including that for monitors or screens as well as miniature accelerometers and gyroscopes for measuring velocity and spatial orientation. These improvements enable a tighter integration of the computer generated elements or environment with the user’s physical setting, the position of her/his body as well as visual attention within the simulated field.

experiencing films in three dimensions. From 1966 and on, Ivan Sutherland and colleagues at MIT conducted the first experiments with head-mounted displays to generate immersive experience. From then on until the 1990s, VR was mainly used in medical science, the car industry, and for flight simulations and military training. Only since the 1990s has the technology become commercially available with rapid industrial and technological development since 2010.

9

SAC Journal 6 9

31.03.21 10:04

AR and VR constitute different forms or mixtures of the same technological output and can be placed on the “reality-virtuality continuum” which extends from the completely real to the completely virtual environment.2 AR typically superimposes virtual objects over a live image given by a camera, such as on a mobile device, or smart glasses. The computer generated elements blend with the physical surrounding and can be interactive. VR, on the other hand, immerses the user in a fully simulated setting with the use of a head-mounted display (HMD) or multi-projected environment. Select objects or the environment can be interactive. Whereas AR always combines an image of the physical environment with virtual elements, which means that AR is near synonymous with so-called Mixed Reality (MR),3 VR can align the physical and the virtual environments through technology for positional tracking and highly detailed and accurate three-dimensional, computer generated models. The dominating sensory information conveyed by AR and VR is visual, yet auditory, including dynamic binaural input, as well as other types of sensory feedback with the use of haptic technology are possible.

The two mediums and especially VR are used in a wide range of fields, obviously by the entertainment industry, but also in education and - in social sciences, medicine and psychology, for select forms of therapy. In many if not most of these applications, the central issue is the systems’ capacity to generate verisimilitude with the known physical world. In other words, the success of the application depends on the degree to which the simulated environment appears true in reference to the appearance and/or behaviour of the physical environment.

For architectural design and the arts, however, novelty and invention - and therefore imaginative and critical projection beyond the horizon of the known - are essential and ingrained in the disciplines’ respective histories. This reflects how the phenomena of the “virtual” and the “augmented” are themselves not tied to technology.4 They present us with experiential dimensions that include the possible and supplemental to what we think we know and want to trust as they present us with a sometimes seductive, sometimes subtle and sometimes provocative disruption of the fine yet nervous economy of lived experience.

The idea of a “reality-virtuality continuum” suggests a fluid movement between the “real” and the “virtual” and therefore a possible flattening of different streams of sensory input. While this may be neurologically true, it also lessens the hypothetical productive tension between the physical and the digital, the “real” and the “virtual.”

This tension is lodged in the corporeal presence of the user or experiencer. While the technology can be seen as a prosthetic extension to that presence, the difference between them must be maintained as a disjunctive force in their productive exchange.5 2) The “reality-virtuality continuum” was formulated by the engineer Paul Milgram and his colleagues in 1994. Extended Reality (XR) denotes this entire spectrum between the real and the virtual.

3) Mixed Reality refers to a hybrid version of the real and the virtual, comprising of both physical and digital objects that can interact in real-time.

It is too easy to relegate technology to known regimes of thought and submit it to service existing systems of production. AR and VR in architecture and the arts are precisely at this crossroads: They may reify the existing or set free vast opportunities for the disciplines and what they produce. It is the human subject that occupies this juncture, not the technology; it is the human subject with its sensory capabilities that facilitates both that which we refer to as “real” and “virtual.”

4) The playwright Antonin Artaud is typically invoked to demonstrate this. Already in 1938 Artaud used the term ‘virtual reality’ in his book, The Theatre and Its Double.

5) The notion of a disjunctive force is loosely taken from Gilles Deleuze and Felix Guattari’s description of a ‘productive synthesis.’ See: Gilles Deleuze and Félix Guattari. Anti- Oedipus: Capitalism and Schizophrenia, trans. Robert Hurley, Mark Seem, and Helen R. Lane (Minneapolis: University of Minnesota Press, 1983), 75-76.

AN ECOLOGY OF SIGNALS Insofar as AR and VR concern a digitally produced realm, the user of these media constitute a physical one. This foregrounds the human subject rather than the technology, the experiencer’s perceptional capacity, imagination and creativity rather than the computational capacity to simulate that which we routinely know. It also ushers in an unprecedented intimacy between the technology, its hardware or devices, and the immersed subject - an intimacy that is predicated on an ecology of signals in the immer-

10

SAC Journal 6 10

31.03.21 10:04

JOHAN BETTUM INTRODUCTION

sive environment. These signals traverse the boundary between the machinic and the corporeal, the digital and the physical.

The centrality of the immersed subject with regard to architecture and spatial perception has been observed by the philosopher Elizabeth Grosz: ‘[T]he body … is already there, albeit shrouded in latency or virtuality. Bodies […] remain architecture’s unspoken condition […].’6 Elsewhere she elaborates to suggest that possible spaces arise from our corporeal presence: ‘I would contend that space and time are not, as Kant suggests, a priori mental or conceptual categories that precondition and make possible our concepts; rather, they are a priori corporeal categories, whose precise features and idiosyncrasies parallel the cultural and historical specificities of bodies […]. The limits of possible spaces are the limits of possible modes of corporeality: the body’s infinite pliability is a measure of the infinite plasticity of the spatiotemporal universe in which it is housed and through which bodies become real, are lived, and have effects.’7 Space, then, is not a pre-given and fixed entity but continuously constructed by the inhabiting subject. Being in space is to be in constant exchange with the environment, and critical advances in neurophysiology suggest that our sensing the world is far more malleable than previously thought. The plasticity of the brain and neural processes handle input from the surroundings in a far more complex, intricate and interrelated manner than what was previously thought.

This challenges the idea that we relate to the world as something entirely external to ourselves. Based on research on neural processes and modifiable synapses in the complex network of the brain, the neurophysiologist Wolf Singer describes a continuous negotiation between previous experiences and sensory input as incessant neural labour. His description of ‘cross modal integration’ accounts for multiple sensory sources and inputs being neurally and cerebrally negotiated to produce our sense of “reality.”8 This malleability and multiplicity in neurophysiological processes break down the clear distinction between the self and the space it inhabits. It expresses a dynamic relationship between the subject and its surroundings where physical movement is an integral component: Pulses, shifts and turns engender perpetual change, shifts in visual attention and an experience of depth.

There are different ways to structure the relationship between the ingredients at play in this drama. One is to think of the technology supplying the immersive simulation as a transactional realm between inner and outer “realities.” VR is then a third transitory realm between inner experience and outer setting, echoing how the British psychoanalyst Donald Winnicott spoke - not of Virtual Reality - but of an ‘area of individual development and experience’ given by ‘transitional objects [and] phenomena.’

His work was in part dedicated to conceptualise this transitional space, an ‘intermediate area of experience […] between primary activity and projection of what has already been introjected, …’9.

6) Elizabeth Grosz, “Embodying Space: An Interview”, in: Architecture From the Outside: Essays on Virtual and Real Space, (Cambridge: The MIT Press, 2001), 13-14.

Winnicott spoke of ‘precursors,’ which echoes the term ‘priors’ used in contemporary neurophysiology, and thought that both inner reality and external life contribute to this third realm. Addressing inter-personal relationships, in particular that between mother and child, he was concerned with ‘… the intermediate area between the subjective and that which is objectively perceived.’10 The French philosopher Bernard Stiegler has said of Winnicott’s third space: ‘In this beyond or beneath of both the exterior and the interior, there is something that holds between the mother and her child, and which nevertheless does not exist. What takes hold between the mother and child in not existing, but in passing through the transitional object, and which therefore finds itself con-

7) Ibid., 32-33.

8) Wolf Singer, “The Constructivist Nature of Perception: Why Virtual Reality Works”, in his presentation at Breaking Glass I - Virtual Reality and Subjectification in Art and Architecture (Städelschule, Frankfurt am Main, 2018).

9) Donald Woods Winnicott, Playing and Reality (London: Routledge, 2005), 2-3.

10) Ibid, 4. Winnicott goes on to suggests that ‘it is … only in playing [and] being creative that the individual discovers the self.’, 72-74.

11

SAC Journal 6 11

31.03.21 10:04

stituted by it, links and attaches them to one another through a wonderful relationship: a relation of love, of amour fou [crazy love].’11 The transitional object is a node of convergence and transactions, much like corporeal presence is a node for transactions between the virtual and the real. Yet, invoking the human body as a central node in these economies of exchange and experience is not concomitant with a reductive phenomenological approach that singles out individual experience and elevates individual perception to a philosophical phenomenon.

We are faced with a technical question that partly centres on the nature and serial structure of signals. These signals are given by digital technology as much as by neurological processes. They travel between the digital and the physical, the machine and the human, and infuse the space defined by technology and flesh with productive tension. An open-ended exchange between technological, corporeal and cultural systems ensues. The question then becomes: other than minuscule electrical pulses and chemical discharges across small differentials in molecular distribution, what is their shared currency?

THE SPATIALISED IMAGE There are signals everywhere, and when gathered, serialised and directed, they engender images.12 In AR and particularly VR, this signal-based status of the image is pervasive. While AR drapes its simulated elements over images of our surroundings, the space of VR is all image. It wraps itself 360 degrees around the immersed subject whose visual experience at any moment is only limited by bodily orientation and the visual cone. When immersed, we visually occupy this image; it has become spatialised and inhabitable.

Already more than a decade ago, the art historian and media theoretician Oliver Grau noted: ‘We are witnessing the transformation of the image into a computer-generated, virtual, and spatial entity that seemingly is capable of changing “autonomously” and representing a lifelike, visual sensory sphere. Interactive media are changing our perception and concept of the image in the direction of a space for multi-sensory, interactive experience with a temporal dimension.’13

11) Bernard Stiegler, What Makes Life Worth Living: On Pharmacology, trans. Daniel Ross, (Cambridge: Polity Press, 2013), 17.

12) This is one central part of an argument recently made by John May. See: John May, “Everything is Already an Image”, in: Log 40 (2017), 9-26.

13) Oliver Grau, “Introduction”, in: Media Art

The implications of this idea of the image as a spatial entity are formidable and open up for new conceptual paths and formal inventions. Borrowing from Edmond Couchot, we can imagine how this image no longer presents the subject with a fixed viewpoint, such as given in perspectival construction. Subjectivity can be conceptualised as spatially distributed,14 which in turn reflects the distribution of machinic and neurological signals and their production of images. The interactive capacity embedded in these images make them ‘autonomous visual artefacts capable of reacting to the gestures of [the immersed user and] reinstate the decisive importance of the body, in all its mysterious complexity, at the core of aesthetic relations. […] An unprecedented artistic situation springs from the choreographic interaction between real and virtual beings.’15

Histories, Oliver Grau (ed.), (Cambridge: The MIT Press, 2007), 7.

14) Edmond Couchot, “The Automatization of Figurative Techniques: Toward the Autonomous Image”, in: Media Art Histories, Oliver Grau (ed.), (Cambridge: The MIT Press, 2007), 183. Couchot refers to, amongst others, Roy Ascott for these

Gone is the opportunity for thinking in terms of “this and that,” in binary oppositional terms. A new construct must be invented for the relation between subject and object, between form and space. Signal, calculation, image, attention, incentive and action are one productive whole and contribute to the intimate, unfolding choreography in the image-space.

ideas.

15) Ibid., 188 - 189. Couchot discusses the digital image in general, not the image in VR, thus his observations have been adapted to the argument herein.

Yet, we should not be surprised by the centrality of the image in this perceptional economy. Consider, for instance, its role in Henri Bergson’s writing when he more than a hundred years ago addressed perception and memory:

12

SAC Journal 6 12

31.03.21 10:04

JOHAN BETTUM INTRODUCTION

‘[In] this aggregate of images which I call the universe, nothing really new could happen except through the medium of certain particular images, the type of which is furnished me by my body […]. The afferent nerves are images, the brain is an image, the disturbance travelling through the sensory nerves and propagated in the brain is an image too. If the image which I term cerebral disturbance really begot external images, it would contain them in one way or another, and the representation of the whole material universe would be implied in that of this molecular movement.’16 The image has always featured as a prominent part of Western thought, and its fundamental role for AR and VR may also reconnect with ideas, such as Bergson’s, and infuse them with new relevance for our understanding evasive contemporary problems.17 Yet, the spatialised image and our relation to it cannot be addressed merely as a technical problem. It must also be understood as culturally.

When accounting for the ‘image act,’ an ‘essential, facilitating counterpart to the philosophy of embodiment,’ the art historian Horst Bredekamp sees ‘the image not as a passive entity awaiting human scrutiny, but as an activating force in its own right.’18 He broadly defines the image as ‘every form of conscious shaping’19 and accounts for ‘the notion of living and active images’ that are ‘[…] no longer the instrument, but the actor - indeed, the “prime mover,” the protagonist.’20 He continues:

16) Henri Bergson, Matter and Memory (New York: Dover Publications (epub), 2004). The book was Bergson’s second publication and published in 1896.

17) A related example to Bergson’s use of the image can be found in the work of Gilles Deleuze and Gilbert Simondon where a theory of becoming and individuation is discussed in relation to images and

‘A new approach […] locates the image not in the place formerly occupied by the spoken word, but in that formerly occupied by the speaker. The image is, in short, no longer the instrument, but the actor – indeed, the “prime mover,” the protagonist. The image act […] adopts the dynamism inherent in the relationship between the speech act and its own social, political and cultural environment, but it finds its starting point in the latent capacity of the image to move the viewer.’21

the ideas of ‘modulation’ and ‘milieu’. The latter must be occupied, argued Simondon, which we can boldly associate to the image-space and the condition of immersion. See: Anne Sauvagnargues, Artmachines: Deleuze, Guattari, Simondon (Edinburgh: Edinburgh University Press (epub), 2016).

18) Horst Bredekamp, Image Acts: A Systematic Approach to Visual Agency (Berlin / Boston: De Gruyter, 2015), 11; Bredekamp argues that ‘it is now

With Bredekamp, a novel comprehension of the image emerges in the context of aesthetics, one that may satisfactorily begin to untangle the mystery of the spatialised image and our relation to it in immersive space. The problem, though, has been only just established, but given the creative opportunities with AR and VR in architecture and the arts, formal discoveries in both disciplines may be possible - including that which relates the human subject to the spatiotemporal complex in architecture.

indisputable that one is no longer in a position to address the contemporary world without first attending to the question of images.’ Ibid., 3.

19) Ibid., 16. Bredekamp refers to the French sociologist Henri Lefebvre for naming the image an ‘act.’

20) Ibid., 33.

21) Ibid., 33.

JOHAN BETTUM is professor of architecture at the Städelschule Architecture Class. He heads the studio, Architecture and Aesthetic Practice, where architectural design is explored through an explicit engagement with the arts and contemporary design technology. Recent work in the studio has used Virtual Reality as a laboratory for spatial inquiries in relation to subjective experience, the construction of reality and the role of images in regimes of representation. Bettum has taught, lectured and been a critic at various schools in Europe and the USA, and his writing has been published in edited books and journals.

13

SAC Journal 6 13

31.03.21 10:04

THE THIRD GLASS: A CONVERSATION SANFORD KWINTER & DANIEL BIRNBAUM

14

SAC Journal 6 14

31.03.21 10:04

On June 3, 2017, Daniel Birnbaum and Sanford Kwinter met for a public conversation hosted by the Städelschule Architecture Class as part of the Städelschule’s annual open house exhibition. The conversation aimed to shed light on the emergent role of Virtual Reality (VR) for architecture and the arts, account for various histories through which we can understand the creative opportunities of this medium, and, as Daniel Birnbaum formulated it in his introduction, ‘… what VR can add to our senses, to our understanding of certain practices, including art and architecture…’.

The immediate background for the conversation was the installation The Third Glass in the Städelschule exhibition, a free interpretation in VR of Marcel Duchamp’s The Large Glass. The installation was designed and produced by SAC’s studio, Architecture and Aesthetic Practice. It was based on a careful mapping of the version of Duchamp’s work that is part of the permanent collection at the Museum of Modern Art in Stockholm, and consisted of the different elements in The Large Glass interpreted three-dimensionally, digitally modelled, animated and staged in VR’s immersive space.

The exhibition also included a version of The Third Glass in “Augmented Reality (AR)” working on handheld devices.

Just prior to the conversation, Sanford Kwinter had the immersive experience of The Third Glass. Thus, the conversation became anchored partly to the history of Duchamp’s seminal work.

In what follows, Virtual Reality is written with capital first letters or denoted with its abbreviation, VR, when it is about the technological medium. When it refers to the broader phenomenon, also in a philosophical sense, the term “virtual reality” is written with small letters.

Birnbaum: There are artworks that somehow anticipate things that will come. It may be romantic to say that art can be prophetic, but one has the feeling that artworks sometimes try to do something that is not possible yet because the technological means are not in place. Years ago, I listened to a lecture by the philosopher 15

SAC Journal 6 15

31.03.21 10:04

and film scholar Slavoj Žižek. He said that Wuthering Heights by Emily Brontë is not a novel but rather a film. Whereas there was no cinematography in Brontë’s time, Žižek argued that Brontë was already thinking in cinematic terms based on the way that she structured the novel with flashbacks, double exposures, and other cinematic techniques. The hypothesis here is that Marcel Duchamp - key artist of the 20th century - similarly anticipated things to come. The Large Glass is a piece that we will never fully understand, yet it seems to anticipate some of what VR makes possible.

Around 1910, Duchamp and his brothers were members of a little group, La Section d’Or, which also included some lesser known cubists, and they were deeply involved in esoteric geometry through a dialogue with the amateur mathematician Maurice Princet. It was quite fashionable around 1910 to talk about the fourth dimension, and there were certain mathematical ideas that verged on mysticism. Meanwhile, Duchamp was also deeply involved in physical realities.

As he created or picked objects to make readymades, he imagined dimensions that cannot be seen with our normal eyes - maybe not in the most traditional, mystical sense but actually in a rather exact, geometrical sense.

The Large Glass, which Duchamp worked on from 1915 to 1923, is permanently installed in the Philadelphia Museum of Art. It is so fragile that it is never lent to other museums. Duchamp famously appreciated the fact that it came to be broken.

‘Finally unfinished,’ he said when its re-assemblage was completed. There are beautiful cracks in it, chance has entered the picture and contributed its own pattern. It is like when Francis Bacon finished his paintings, he always threw paint on them, as chance should have a role to play. The Large Glass is a picture, it is painted, but it is not a traditional painting. Duchamp called it ‘a delay in glass.’

There are a number of replicas of The Large Glass. The first one is the one that we have been working with at Moderna Museet in Stockholm. It is signed by Duchamp as a kind of official copy. So, it is treated as an original – whatever that means. It became The Large Glass that could travel and has been shown in Pasadena in 1963, at the Tate, the Pompidou and the Stedelijk Museum in Amsterdam. The pop artist Richard Hamilton also did a version of The Large Glass, and there is a Japanese copy. Duchamp, I think, did not know about the latter, but I hear it is very good. So one could say that there is the original one, then there is the generation two, which is those post-war, physical versions. Hence the name for the VR version here, The Third Glass.

And Sanford, on your way over here, you said that at one point the piece was probably the most interpreted cultural object of all time. At least in the modern era, it is a work of art that has generated the most absurd number of articles and attempts at explanation.

1) Daniel Birnbaum was the director of Moderna Museet Stockholm from 2010 through December 2018.

Kwinter: Permit me to apologise in advance for the combination of overstimulation and delirium I am feeling right now. I just got off a transatlantic flight and, on the floor above us in The Third Glass, I just moved through three or four manifestations or multi-sensorial responses to The Large Glass. The Large Glass was easily the single most precipitating event in my own education, so much of my thinking has pivoted around my early experiences of it. I saw the Swedish Large Glass before I saw the one in the USA. But I also encountered the more systematic elaboration of that work through Pontus Hultén’s approach to mid-century modernist culture as it was supported by the institution that Daniel is now leading in Stockholm1 most notably through the extraordinary catalogue produced around it with respect to the emergence of a “machine art.” Many will know the beautiful MoMA catalogue with the metal cover – collectible and pivotal even today if one is interested in a systematic approach to a revisionist understanding of culture.

16

SAC Journal 6 16

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

Sanford Kwinter in The Third Glass, Rundgang, 2017.

But there was also Harald Szeemann’s enormously important travelling exhibition, Les Machines Célibataires, in 1975, which I saw in Paris while I was a student. The title is from Duchamp, in English, The Bachelor Machines. I cite these episodes to help restore an historical framework for approaching current methodologies in art and architecture because these are often critically lacking, a lack that often blocks speculative capacities in design and art. In this vein, I would claim that the exhibition, Les Machines Célibataires, is actually part of The Large Glass and that The Large Glass was always intended to generate a continuous and episodic life within culture and in art.

There is a psychoanalytic term known as a “screen memory,”2 and there is also, of course, the term “screen” in late 20th century connected to the cinema. Cultural theory in this era related profoundly and inescapably to cinema. The screen was widely seen to have a relationship to processes that take place inside the head, inside of consciousness. The whole elaboration of the cinematic understanding in late 20th century through psychoanalysis and other phenomenlogical approaches has everything to do with that. There was an ever-evaporating distinction between things that were produced externally to us in the world and things that were generated and produced inside ourselves. The Large Glass took its place exactly in that new space. It was meant to operate as a screen onto which an infinite number of noetic entities could be projected and convene. The initial trope that animates it philosophically is that The Large Glass is not only full of opaque entities, and images; it is also a transparency. The standard dogma is that in any situation, as in the gallery or the museum, the spectators become part of the activation of The Glass. When you peer at it, not only do you encounter reflections of yourself looking, but your gaze also passes right through it; you behold the other spectators, indeed an entire aesthetic-experiential ecology.

The Large Glass thus operates as a manual of 20th century aesthetics some would challenge this, but neither the two of us would.

2) The term “screen memory” was coined by Sigmund Freud in 1899. It describes a distorted memory from childhood. This memory is rather visual than verbal. See: Sigmund Freud, “Screen Memories” in: The Standard Edition of the Complete Psychological Works of Freud, Volume III (18931899): Early Psycho-Analytic Publications (Vintage Publishing, 2001), 299-322.

One could say that a cryptical part of many writers’ projects – my own certainly – is to displace the platitude of a cubist origin of modernist consciousness. In relation to this, The Large Glass is projected as an infinity, precisely because it is built for perpetual remapping, and its project is to constantly receive and transmit novel mappings.

I would further argue that we live in a universe where The Large Glass is intimately sutured into our nervous systems. Let us not imagine that our computer screen is anything other than “a large glass;” it is not. If you consider the environment that we en17

SAC Journal 6 17

31.03.21 10:04

From left: Sanford Kwinter, Daniel Birnbaum and Johan Bettum at Städelschule.

gage with, if we account for our screen time and the promiscuity of connections that it permits, we realise that we are now living in Large Glass. And yes, it is a nightmare because we cannot exit it.

Now the question, of course, is whether The Large Glass can be made four-dimensional or five-dimensional? What was important about the Les Machines Célibataires exhibition in France in the early to mid-1970s (and the exhibition travelled to four or five further locations in other countries), was the project to remap The Large Glass into the cultural environment of the era. One of the primary tropes deployed to receive it came from the work of Gilles Deleuze and Félix Guattari which also notably included a great deal of radical psychoanalytic ideas and theories. Yet for our generation there was a term beyond all others from Deleuze and Guattari that seemed to perfectly explain, or explode, The Large Glass: the French phrase “plan d’immanence” - or the ‘plane of immanence.’ We saw The Large Glass as an actual plane of immanence.

The plane of immanence is a geometric idea in which there exists an infinite number of dimensions that cross through a section, a section of existence, a section of reality.

With everything crossing through it, it becomes a place where everything can communicate and be recombined. The plane of immanence is where we live and experience physical reality. In fact, every determination of our physical reality exists virtually on the plane of immanence. So it is the place of infinite combinability, a place where all creation takes place through a combinatorial process. You can take things apart and put them back together in ever new ways; it is where life happens.

Birnbaum: [Turning to the audience] How many people here have actually experienced VR? …

Kwinter: … in the 1960s they would say: “Are you experienced?”

Birnbaum: ... maybe five or ten in this big room? Nine months ago, I had heard the name, and I thought that was exactly the kind of stuff that I am not interested in - something very nerdy, ridiculous, silly looking. It is still silly looking.

When you have that headset on, it is so unattractive, even preposterous. Yet today I would say, ‘try it,’ because inside it is really fascinating. I have been looking at a few things the last few months, and for the present Venice Biennale, a small 18

SAC Journal 6 18

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

number of artists have been working with this new medium. There is for example a new piece by Paul McCarthy and a highly reduced, almost minimalistic piece

by Olafur Eliasson. Soon those glasses may take on the aura of Ray Bans, the machines will become smaller, and in the end they will find a home in our smartphones.

Kwinter: Google glasses were a multi-billion-dollar research project that basically …

Birnbaum: … flopped somehow!

Kwinter: Not somehow! Nobody was going to be caught dead wearing one precisely because it was so egregiously stupid and ugly. [Laughter] Birnbaum: When did you first hear about VR? Actually, the term “réalité virtuelle” is in Antonin Artaud’s The Theatre and Its Double which was published in the late 1930s.3 Kwinter: It was in 1989. There was an emerging culture in San Francisco in the late 1980s after the demise of the Whole Earth Catalog.4 The magazine finally collapsed as did aspects of the project. But in its wake, there emerged the virtual reality movement around a new publication called Mondo 2000. Anyone who is serious about understanding VR and its DNA, ought to find those early issues of Mondo 2000. The movement revolved around two things: one was the psychedelic use and expansion of the nervous system and, two, the political component that construed our nervous system as a flashpoint of “cognitive liberty,” human rights and emancipation. In other words, the genius and project of Mondo 2000 was the union of the psychedelic and political. The activists behind it believed that VR had the capacity to transform society in a very particular way, much like the inheritance, if you like, from the 1950s and 1960s with the experiments with LSD and other hallucinogenics. Around 1991 some researchers at MIT, obviously noticing the VR phenomenon that was taking place mostly with military-industrial and entertainment interests in California, started to get interested. The result was that Nicholas Negroponte became part of a group that decided to start a new publication – Wired Magazine – that essentially hijacked the entire Mondo 2000 mandate and transformed it into a corporate enterprise. But VR was a huge cultural phenomenon for about four years, from 1988 through at least 1991. That represents its early history; it is when the term VR was coined and when all the significant figures played, among whom was Jaron Lanier, widely credited as the guy who “invented VR” in the mid-1980s.

3) See: Antonin Artaud, Le Theatre et son Double, in: Collection Metamorphoses No. IV (Éditions Gallimard, Paris, 1938).

4) The Whole Earth Catalog (WEC) was an American counterculture magazine published between 1968 and 1972 with a focus on alternative education, do-it-yourself culture, holism, ecology, and selfsufficiency. See: Stewart Brand (ed.), Whole Earth Catalog, published from 1968 to 1971 (United States).

Birnbaum: He is a musician?

Kwinter: Yes, and he is interested in combinatorials. He does a lot of work with music and has remained a guru of a certain, although now rogue, type. But important to note as well is the concomitant rise of what came to be known as the psychedelic revival that took place in the early 1990s. The main figure there was Terence McKenna, although it was part of an explosive matrix emerging in the early 1990s of California culture reasserting itself. One ignorantly wonders too often out loud “how did California become such a serious player?” when they were always so completely dismissed by us on the East Coast. But this is exactly where it began and long before the 1990s. This is the true DNA of VR and where the real interest needs to be directed. But for our purposes in this conversation, let us note that VR went underground for fifteen years before it again exploded very recently.

Birnbaum: Are the reasons for its re-emergence simply technological, or technological-entrepreneurial, that to begin with one could not sell it?

19

SAC Journal 6 19

31.03.21 10:04

Kwinter: There was an economic and technological history that had to take place in between. One thing had to do with the fact that we all believed that fibre optics, and massive increases in throughput, were going to make it all possible. VR had a lot to do with remote sensing in the old sense of the term, meaning it had to do with being sensorially in many places at once. That could only be achieved if you could collapse the information flow to a high enough degree so that the brain could experience being in many places at once as a unified sensation. The fibre optics initiative fell apart, and then a lot of years passed. The technological aspects meant that it was not until the great West Coast, social media conglomerates – YouTube and all of the ancillary enterprises – came together to create an entirely new environment for VR, that it emerged with military and industrial resources behind it as opposed to being a counter-cultural phenomenon as it was in the early 1990s. And with that kind of capital behind it and those kinds of interests: BINGO! Birnbaum: It seems now that all the big companies, such as Sony and Facebook, are competing to dominate this new field. We must also remember that VR is already in use, not so much in the arts yet, but for instance in medicine. If you are a leading surgeon in Frankfurt but a little bit uncertain about something in a heart operation, then you bring in another surgeon in San Francisco. They can both operate live on the same real heart. It does not matter where you are; you are present, and there is no other presence in surgery today because the process is mediated through instruments. It is also in use in aviation to train pilots, or if drilling for gold, one does not have to go one kilometre down into the mountain anymore because it can be done via remote VR. But what are the driving forces?

With photography we know it was pornography, and with VR I do not doubt that pornography and the gaming industries are primary driving forces.

With art, is it not always the case that with a new technology or medium, artists will wonder what can be done with it, and – when it happens at all – rarely in the way that the engineers or those inventing the medium were expecting or hoping for? Artists may sabotage or short-circuit the technological system, as in Joan Jonas, Bruce Nauman and others when video was introduced in the 1960s. They did not produce wonderful, commercially compatible work. They would turn the camera on itself and basically deconstruct the medium. We do not know what is going to happen with VR; right now there are a lot of people approaching it. As I mentioned, there were at least two examples in Venice. At the current Whitney Biennial there is a brutal and much written about piece by the young American artist Jordan Wolfson. It is horrible, a guy being beaten with a bat, so brutal that it would be hard to watch the scene in a normal film.

There are a few artists who have been trying out VR for a while, and there are writers who really believe in its capacity to change not just the way we live on this planet and communicate, but also the very meaning of what art can be – or, at least, the possibilities of art. One of them is Douglas Coupland, the famous Canadian writer with twelve novels - including Generation X: Tales for an Accelerated Culture. Coupland is also a journalist interested in new technologies and was an editor at Wired. I had a conversation with him recently and asked him simple questions like: ‘What will this lead to; what does it mean? Is it like the introduction of television or is it more like the introduction of electricity?’ And he said: ‘Electricity, clearly! More important than electricity!’ I do not know that it is really going to change the perception of what it is to be here and now or to be present in a body in a room. However, if such a thing actually enters our life-world, why would it not also totally transform art, as photography did?

Kwinter: Well, we need at this point in the conversation to introduce a new term. But before we do, allow me to preface and contextualise it with a quick historical sum20

SAC Journal 6 20

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

Visitors in The Third Glass, 2017.

mary that many of us who were theorising media in the 1980s and -90s often made.

The arrival of television into our psychic, urban and late 20th century universes can retroactively be understood as a preparation for what came to be the unforeseeable domination of our historical experience by the screen. But even more significant is the fact that his new dominion of the computer and portable gadget screen is the foundational element that we now consensually refer to social – in other words, our social and psychic environments. We all recognise the profound, even total, economic and global transformation that took place with the arrival of digitisation. Midcentury television would have been the social, economic, cultural but also psychic preparation for that.

Now, the question, of course, is: what is significant here; what is the moment; what is the hinge principle? I would suggest that it is immersion. What happens, what changes, when these experiences and the relationships become immersive?

At a time when nobody understood what “virtual reality” was - I refer to the late 1980s – everybody was talking about cyberspace and “virtual reality,” and a lot of people could not get their head around it. The phrase that was continually invoked was: “virtual reality” or cyberspace is “where you are when you are talking on the telephone.” Of course, as Marshall McLuhan pointed out, the older technology is always initially used to frame the novel one, before the new modalities fully arrive. But these occurrences signal to us that something new has arrived on the political horizon and that gives rise to a certain tension. Why are we “somewhere” when we are on the telephone? It is because our attention is “somewhere,” and we began to realise that this has radical implications for our bodies. There is a profound hatred of the body embedded in the DNA of VR. Yet we are also discovering today that the nervous system is an unfathomably rich and radically under-explored environment. VR actually allows us, via technological intermediation, to access aspects of the nervous system and human nervous response, to access mo21

SAC Journal 6 21

31.03.21 10:04

dalities of experience that lie embedded or unused in the vast human endowment. My belief is that VR has an incredibly important potential role to play here.

Immersive experience and the transition to an immersive environment tell us something because all of our attention is there. And something does change; it is a new problem of the senses, the senses operating in a synthetic environment. It becomes biological and bio-historical, even onto-historical problem. That is perhaps where artists are going to lead us into something really entirely different.

In order to thicken the problem of VR so that we do not merely see it as a “tech problem,” we need to realise that there is something important taking place today in terms of the application of new developments in neurobiology. These things are being understood in cultural theory and perhaps especially in political theory. There are productive groups working on this, and – at least in my assessment – the most interesting one is under the rubric of “cognitive capitalism.” It is an attempt to account for this profusion and transformation of knowledge in the last ten years with respects to how our nervous systems work. It uses neurobiology and focuses in part on how our environments and the shaping of our brain and experience are inseparable phenomena. Most people have probably heard of neural plasticity and other terms such as default mode networks, neurodiversity and so on, that are increasingly being bandied about today.

5) For a thoughtful if rudimentary early engagement with neuro-political dimension see: Catherine Malabou, What Should We Do with Our Brain? (New York: Fordham University Press, 2008).

VR merits theoretical and artistic attention because it provides a new set of relations to explore that lead to new things we can do with our brain.5 By brain we now of course understand both body and senses, and we also understand by it the set of forces that determine the formation of subjectivity. That question was endlessly brought up by Guattari, and his project is simply to acknowledge that what you do with your brain, with your body, and what you do with the senses is who you are. VR thinking could give us expanded insight access into these formation processes. That said, most the VR work that I see is rubbish.

[Laughter] However, I also know today I had an experience that was deeply moving. I entered into The Large Glass - that is, The Third Glass in VR upstairs. It was a profoundly disturbing and ecstatic experiment of being in a different place in my head, indeed it was an incredible feeling. And like Daniel, although I spent several decades looking at, talking and thinking about The Large Glass, today I received a whole new way of knowing it. [Laughter] Birnbaum: Perhaps we may now start a new chapter; I mean, there are only 12,000 books on Duchamp. Maybe there will be another 3,000 just dealing with Duchamp and VR? However, what you say about the senses is fascinating, but so far VR is only about vision and sound, right?

Kwinter: That is an interesting question! One of the things I am working on right now, and I did not know until very recently that it was in fact an exploration in the noetic effects of virtual reality, is sound.

Birnbaum: What? It was VR and sound?

Kwinter: S-o-u-n-d! [Laughter] It has to do with what is often called three-dimensional sound, also known as binaural sound – that I found game-changing for myself. One can easily experience binaural sound online with a simple set of headphones. With binaural sound, you are hearing the world, one might say, “as it is,” and “as if” you were part of that world – in other words, immersively. Binaural simply means two ears.

The reason we are able to spatially locate sound has to do with the fact that the signals being received by each of our ears are infinitesimally different. But that infinitesimal difference is extremely efficiently and accurately processed by the brain. Even a few milliseconds of difference in the time that it takes for a signal to arrive in the left ear and the right ear, if properly processed as distinct signals, will give you a pinpoint spa-

22

SAC Journal 6 22

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

tial localisation. But it is not only arrival time difference; it is also because the signals are timbrally different. Pending origin, the signal will be shaped by its trajectory across your face so that the signals that which arrive at your right ear will differ from those at your left, and vice versa. Moreover, all signals that arrive in your ears are shaped by your pinna, which are the uniquely-shaped earlobes that each listener has. Binaural sound is a very simple thing; it simply and accurately brings to the foreground precisely the information of what we hear when we hear something. In daily life this is difficult to experience because our own ears, just like our eyes, are so inured to default modes of hearing, the equivalent of perspectival “seeing” learned through experience of cinema, screens, and so on. In a way, one has to trick one’s ears – really brain – to actually enter into the binaural universe even though one’s ears are natively largely binaural; this is why psychoacoustics will need to play a major role in VR research. You simply put on a set of headphones, or record sound by putting the microphones in your ears (special binaural mics) so that you record what your ears are hearing rather than what they think they are hearing. Some of the binaural recordings online are not at all that dramatic, yet some of them will knock you out of your seat. You will feel like you have become inseparable, three-dimensionally or materially, from the entity that is being heard.

There is one that I find especially lovely and that I have my students hear. It is a binaural recorded of the flushing of a toilet. When you listen to it, your head is located right in the middle of the vortex, and you actually can experience the vertical flow acoustically, aurally, as an intimate version of your own material, receptive body. It is a staggering experience. I have often seen people break into tears upon first experiencing a binaural sound, particularly certain types of music, and sometimes acoustic landscapes in nature. That is why I found the reflexive inclusion of “sound” in your question just now to be so interesting. Most people think of VR as uniquely visual.

But the problem is an immense one and we do not even know how yet to frame it. I, for one, believe it is the problem of the frame itself that is at stake in VR. One way of thinking about it is like this: When we are born, the very first sensation we receive of an external, non-self, world is sound. Even before we have been fully formed, we are already anchoring ourselves in “place” with acoustic orientation whose origins are from reception in the womb. We know, for instance, that newborns, even 18 hours old, can already recognise and respond to their parents’ native language specifically, simply because it is already picked up and patterned by their brains; they know its characteristic prosody, rhythmic patterns, its peculiar sound contours because they hear it in the womb. But long before that, the embryonic material is already picking up these sounds. As Buddhists point out, hearing is the last sense to leave us, hence why they always read to the dead and dying; they read and in so doing acoustically prepare them for their transition. Indeed Buddhists conceive not only of life, but of death as acoustic. The point here is that the flesh, our nervous system, is laden with these extraordinary pre-subjective memories and traces of early pre-ego connection with the universe through an acoustic continuum. Through certain experiences of sound and obviously of music, one can continuously change one’s mood, one’s body state, one’s chemistry. Thus, in effect, the experience of three-dimensional reality, and the immersive idea, are not uniquely visual phenomena. In point of fact, the visual frequently tracks the aural. The aural is the primordial armature of how our senses enmesh with the external world, with the body’s own materiality. In a certain sense, VR’s power may be conceived as an increasingly aural use of the visual system. Marshall McLuhan was already aware of the fluid relationship between the ear and the eye. The acoustic problem also bring us to the cochlea, which is the spatial orientation organ. Why is it in the ear? Why are space and sound reception anatomically close? I have no idea the answer to this, but it is very fascinating to me. The binaural experience is so profoundly spatial, you may feel like you never knew what space was until you experience this. My feeling is that these are the clues as to what changes when we enter an immersive environment; it has something to do with things that have profound origins in our senses, in our flesh, from the time before we became divided from the world.

23

SAC Journal 6 23

31.03.21 10:04

Birnbaum: Sounds to me that a lot of what you speak about explains how a materialist can land with such a fascination for VR. Kwinter: I wish I had said that!: A materialist approach to “virtual reality.” [Laughter] I know, but the thing is, I cannot let a wonderful occasion go by. [More laughter] Birnbaum: We are still talking about vision and sound, but, of course, there are other senses. We can taste things; we can smell and touch things. And maybe it is a bit of science fiction to say that all of these will be simulated one day in a oneto-one, perfect way, although I think a lot of people believe this will be the case.

It may take a few decades or perhaps a century, but then from there we come to even more science fiction-like speculation that brings us right into science fiction movies, like The Matrix, where we are basically already immersed in a simulation but hide this fact from ourselves. The collective hallucination is what we normally refer to as reality, and that is a kind of simulation. There is something called “the simulation argument;” there are one trillion philosophy seminars about it.

6) Jakob Johann Baron von Uexküll (1864-1944) was a Baltic German biologist, philosopher and zoologist.

7) See his publication: Jakob von Uexküll; Georg Kriszat, Streifzüge durch die Umwelten von Tieren und Menschen. Ein Bilderbuch unsichtbarer Welten, (Berlin, Springer-Verlag, 1934).

The philosopher Nick Bostrom at Oxford argues that future generations will have access to super-intelligence. If they would be interested in their own history, they would enter so-called ancestral simulations – that is us. We are ancestral simulations of future super intelligences; it is just that we do not know because we are so limited. However, I am more interested in non-human – not post-human – perception, and Jakob von Uexküll offers a phenomenology of animals.6 I think Heidegger has a footnote on him here and there. The question is what does the world look like if you do not have the twelve categories and space and time, but you have others, to put it in a Kantian way? What is it like if you are a bird, if you are a fly, or if you are an even more primitive insect? Von Uexküll laid it out in this tiny book which is quite amusing and called A Stroll through the Worlds of Animals and Men: A Picture Book of Invisible Worlds.7 It is about how animals and humans navigate the world. He had an artist visualising how we could imagine that the world would look like if seen by a lark, an eagle, or a fly. I remember one of the illustrations which look just like a Sigmar Polke painting.

Kwinter: [Laughing] Von Uexküll was a theoretical biologist and one of the founders of ecological thinking. He always placed the mind, the nervous system, in an ecological context. What he showed is that for every nervous system there is a corresponding world, and he made a pointed distinction between the world and the environment.

Each one of us has our own universe that corresponds to what our capacities for sensing happen to be. Every perceiver is different in the sense that every one of us has a different brain and hence a different world that corresponds to it.

Birnbaum: Now you limit this to us here in the room, but non-humans are also perceiving entities …

Kwinter: Absolutely, absolutely! That was the beauty of his work, it told us that there is an infinite number of worlds or universes. Von Uexküll used the term “function circles,” to show that the sensing apparatus and the world sensed were circuits, and the circuit did not operate if one part of it was not there. That is also an important idea for our current problem, our understanding, and the 21st century. There is not a single significant theory in neurobiology since von Uexküll that goes against that idea. That is why von Uexküll is invoked by not only those you mentioned but also by everyone from Oliver Sacks to the mathematician René Thom. It is because of just how important that fundamental insight was. Yes, you talk about von Uexküll as somebody obscure, but in this new VR world that plays so freely with the circuits, he is, as you rightly point out, sure to become very prominent.

24

SAC Journal 6 24

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

Birnbaum: All of those experiences are premised on movement or mobility, are they not? In the 19th century in Germany, when space actually emerged as something that one actively thought about in the arts and specifically in architecture, it was partly premised on a certain interest in movement, the movement of the eyes but also the movement of bodies. Art and architecture theorists began to pick it up from physiologists and the emerging field of psychology.

Kwinter: Yes, another significant thing about VR emerges, not only from von Helmholz as you point out, but out of the later work of James Jerome Gibson, who was a perceptual psychologist. Gibson has been another fringe character in architecture and the arts for a long time, but whose works are going to become widely used now. Gibson’s late career work was based on the primacy that he gave to movement. For example, one of our favourite artist friends, Olafur Eliasson, whose published books and infinity of experiments and works have been arguably based on a single idea: If it does not move, you cannot perceive it and hence not find it interesting. You cannot see anything that does not move – or change – and Gibson places the motion of a body through space as the foundation of perception and refers to this as a flow, an optical flow. He talks about an “ambient optic array” and how it continuously morphs and transforms. How it is only through the transformation that the information required to interpret the environment is picked up. It is a system and a beautiful thing. It is clear that this is the shift that one needs to make in terms of beginning to understand things in a new way. Let us say that VR is delivering us from the illusion that we used to have that space was fixed or that perception could actually see something that did not move like us.

Now some of you may say: “Well, that is not true! I can see things that do not move.

I can even fix my head in place and see something that does not move.” Yet the truth of the matter is, that in such cases we rely on saccadic eye movement. The saccades of the eyes vibrate at a speed that is so great that most people cannot see another person’s saccades. But without the saccades you cannot see, you have to produce a differential, a spatial-temporal differential in order to discern. These are important foundational questions because they change the way we understand space and time. So it is the conjunction of space and time and the understanding that they are always given together.

We seem them as distinct because of our training.

Birnbaum: The space-time problem is obviously also the central concern of The Large Glass.

Kwinter: Absolutely! And that leads me to another thing. When I put on the headset today, I saw something absolutely unbelievable. I hope you can follow me with this; I will try to be brief. There is a legendary hallucinogen that is known as DMT, dimethyltryptamine. It is the principle ingredient in ayahuasca or yage.8 In the USA it is written about quite a bit including in medical circles. DMT is said to induce an extraordinary transformation in the brain, a profound hallucination, of which one characteristic is that there appear all kind of little helpers or elves and this experience is almost universal.

You take a DMT trip, and you are going to see elves, and they are going to communicate with you. There is a remarkable scientific study done in New Mexico in the 1990s by Rick Strassman. He published a book called DMT: The Spirit Molecule.9 Most of the protocols that he developed are those used today for the study of all these types of medicine experiences. What DMT does is de-differentiate the human composite structure, and you get a non-ordinary experience. It only lasts 10-18 minutes, and usually it is an unforgettable and overwhelming experience. With ayahuasca the experience of DMT is extended over 4-5 hours because it is ingested rather than smoked; this permits a more sustained encounter with its universe.

These studies were conducted for specific medical and psychiatric purposes. The dedifferentiation process they produce is seen to be highly reparative. It is also accessible

8) A tropical vine of the Amazon region, noted for its hallucinogenic properties.

9) Rick Strassman, DMT: The Spirit Molecule: A Doctor’s Revolutionary Research into the Biology of Near-Death and Mystical Experiences (Rochester, Vermont: Park Street Press, 2000).

25

SAC Journal 6 25

31.03.21 10:04

in other ways; through meditation for example. The de-differentiation process has one common and profound effect: multiple spaces and times are able to present themselves to the nervous system simultaneously and yet in a non-paradoxical fashion. Furthermore, this experience can be remembered as knowledge, and transposed into the sober state. People who experience these things report that the insight remains and can be accessed and used in their sober states.

In Strassman’s work, there is a crisis that develops during the four-year study. The patients, who are undergoing multiple DMT experiences begin to report that these experiences, including the encounters with the elves are not hallucinations. Well, this is exactly what I feel I saw this morning with the chocolate grinders bobbing up and down in front of me while immersed in The Third Glass. [Laughter] You can find similar reports in the ancient Hindu texts as well. Reports of these experiences are documented throughout the works of Stanislav Grof, the scientist who did the major work on LSD therapy in Czechoslovakia, Germany and then in the USA in the 1960s.

In the end, it is a space-time problem, and one learns that one can access layered memories and thick and multiple spatio-temporal experiences as completely unified and non-paradoxical. VR is actually moving our experience into a place where that next level integration can take place. The way each of our senses are organised is in fact specifically only true for oneself. Synesthesia for example is a commonly reported but wildly various thing and probably applies in some manner, but differently, to everyone. We can learn a great deal about how others’ senses are organised through study of neuro-atypical people. The works of Temple Grandin for example are incandescent in their capacity to illuminate.

What one begins to realise is that what architects and artists have commonly presupposed to be the fixed and essential armature of space-time for experience is simply not the case, and never has been. We have access today to entirely different ways of producing experiences and producing images, etcetera. I encountered a lot of this today, even in a smaller work in which somebody began to speculate from the point of view of animals, in the sense of von Uexküll. What does a machine see or what does its world look like? What is the unity of the world that corresponds to machine vision? Because as von Uexküll points out, every world is complete or full, even if it is that of a tick which has only two senses, or if it is that of a human, who has a multiplicity of ways to access the universe, or – for example – a shark or a bird who can pick up electrical or magnetic fields and signals. It is a huge, damn universe out there whose depths we have not known how to think about. This is what Duchamp knew!

Birnbaum: And as far as this new medium and art go, there are a number of big projects that are being developed. Meanwhile, the critics and sceptics are there, and I can understand that, because there is something a bit autistic, solipsistic, masturbatory about VR, that you are alone with some machine. And what we like about art exhibitions – most of us – is the fact that we share them, they are an intersubjective medium, you walk through a show, you talk about it. It is a critical kind of environment.

Kwinter: I have, of late, become allergic to art. I do not love to go to museums and specifically not to galleries. But on occasion when I must, I have noticed that in New York everyone – young and old – is there looking at the work through their phones and interacting with others simultaneously and laterally.

Birnbaum: Yes! And it is not only to document it and look at it afterwards, it is because it triggers certain things. Now there are artworks that you have to see through the phone. The French artist Pierre Huyghe is quite close to these animal speculations that we have spoken about, and he is doing a piece for Kasper König’s curated show, Skulptur Projekte, which opens next week in Münster.

26

SAC Journal 6 26

31.03.21 10:04

SANFORD KWINTER & DANIEL BIRNBAUM THE THIRD GLASS

Huyghe said he is doing an Augmented Reality piece there. I am very much looking forward to see it. He also said: ‘I do not like VR; I only like Augmented Reality.’

But then I mentioned this to one of the technicians who works full time with this, and he said: ‘It is like saying I do not like fruit but I love bananas!’ Because it is largely the same thing. It just depends on how you project it. The same technologies generate these things, and you can place them in a room or you can have them in a closed, immersive space.

I have consulted and assisted some people who produce art works in VR.10 They have done a piece by Olafur Eliasson after I introduced them to him. It is very interesting, but it is almost like a poorer virtual version of something he has already done. There is an early, famous piece by Olafur called Beauty, which is a rainbow but an artificial rainbow with a primitive setup featuring a lamp and water. He has also done a waterfall in virtual space. There are other works emerging – a Jeff Koons piece as well as a piece by Marina Abramović – both will be launched soon. I really do not know what they will look like, but it is interesting to speculate on what could happen to the art world, which we both kind of dislike actually, though I am totally in it.

10) In 2019 Birnbaum left his job as director of the Moderna Museet to become artistic director of Acute Art, a London based laboratory exploring new immersive media in close dialogue with artists.

The art market is so powerful now that almost anything can be commercialised.

The market has such a strong grip on the big museum world, and maybe I am totally naive but let us hope that things can be different if those artists, who I just mentioned, do things that are not an edition of five and just sold to Abu Dhabi but would actually be for everyone, a bit like Netflix or Spotify. Someone will make money, for sure, but it is not the same thing. The distribution of that kind of art could mean, at least for the viewer, a certain kind of de-monetisation of art. That may be a naive optimism of mine, but you do not have to be a Marxist to be a bit negative about certain aspects of the art world today.

Kwinter: Changing the subject is what is at stake; it is what ought to be at stake. Changing us, changing our own destinies is really what ought to be clearly brought in. That is why The Large Glass is so wonderfully instructive – it is a work which continually interacts with the existing culture and produces unexpected, and at least initially, radical results. Works like this are letters to the future. We must read them.

SANFORD KWINTER is a writer, architectural theorist and co-founder of Zone Books publishers. His work includes influential essays and books: Far from Equilibrium: Essays on Technology and Design Culture (Actar Press, 2008) and Requiem for the City and the End of the Millennium (Actar Press, 2010). Kwinter is professor of Architecture and Urban Design at the Pratt Institute, New York

.

DANIEL BIRNBAUM is a philosopher, critic, curator and professor at Städelschule. He is artistic director of Acute Art in London. From 2010 to 2018 he was the director of the Moderne Museet Stockholm. He was the dean of the Städelschule and director of Portikus from 2001 to 2010 and the director of the 53 rd Venice Biennale in 2009.

Birnbaum is the author of numerous books, amongst others: Spacing Philosophy: Lyotard and the Idea of the Exhibition (with Sven-Olov Wallenstein, Sternberg Press, 2019).

27

SAC Journal 6 27

31.03.21 10:04

PORTFOLIOS

81

SAC Journal 6 81

31.03.21 10:05

SAC Journal 6 82

31.03.21 10:05

COLLIDE

KLING KLANG KLONG & ONFORMATIVE

The digital art installation Collide combines original chamber music and painterly visuals to interpret recorded motion data and act as a conductor for the performance of the musical score composed for the installation. The animations provide visual inspiration for the musicians who observe them through VR goggles and react to the “visual timbre” while playing. Collide explores how Virtual Reality can affect the physical world. As a multi-sensorial experience it linkes visuals to an engaging soundscape and absorbs the viewer in an immersive space.

Collide was a collaboration of studio kling klang klong with onformative and displayed in the Dolby Gallery in San Francisco, CA. kling klang klong’s team consists of composers, sound designers, producers, programmers, and scientists who spatially explore contemporary culture and the world around us. Their acoustic works pioneer communicating with audiences through exhibitions, motion pictures, interactive installations and performances in public spaces. onformative is a studio for digital art and design. Their works, which range from interactive media installations, generative design and dynamic visuals for data-driven narratives, have been exhibited worldwide at various galleries and festivals.

SAC Journal 6 83

31.03.21 10:05

84

SAC Journal 6 84

31.03.21 10:05

KLING KLANG KLONG & ONFORMATIVE COLLIDE

Top left: Performace view of three cellist wearing VR goggles Top right and bottom left: Stills from Collide

SAC Journal 6 85

31.03.21 10:05

86

SAC Journal 6 86

31.03.21 10:05

KLING KLANG KLONG & ONFORMATIVE COLLIDE

87

SAC Journal 6 87

31.03.21 10:05

SAC Journal 6 88

31.03.21 10:05

KLING KLANG KLONG & ONFORMATIVE COLLIDE

SAC Journal 6 89

31.03.21 10:05

FERENJ: A GRAPHIC MEMOIR IN VR AINSLEE ALEM ROBSON

90

SAC Journal 6 90

31.03.21 10:05

Ferenj is a visual dialogue engaging with memory, reality, racism and the digital in an afrosurreal dreamscape. The viewer is guided through fragments of the author’s memory of her parents’ Ethiopian restaurant in Cleveland, Ohio, and the house where she grew up, to the streets of her mother’s home country, Ethiopia. Ferenj is an experimental form of emancipatory expression that reclaims Ethiopian-American, mixed-race identity, redefines boundaries between fragmented memories and the digital imaginary, and contributes to the movement to change reductive and stereotypical narratives of Africa.

Ainslee Alem Robson is a director, writer and media artist who crafts stories about tensions embedded within the intersectional layers of race, gender, perception, nostalgia and the digital. Her background in philosophy renders a critical approach to her emancipatory narrative style, deconstructing colonial legacies that centre on underrepresented perspectives with emerging technologies in digital art and film.

91

SAC Journal 6 91

31.03.21 10:05

SAC Journal 6 92

31.03.21 10:05

AINSLEE ALEM ROBSON FERENJ

Left: Ferenj, VR screenshot Bunna Right: Installation view of Worlding Worlds in MU, The Netherlands , 2020

SAC Journal 6 93

31.03.21 10:05

94

SAC Journal 6 94

31.03.21 10:05

AINSLEE ALEM ROBSON FERENJ

Top right and bottom left: Ferenj, VR screenshot School Bus and Piano Bottom right: Installation view of Worlding Worlds in MU, The Netherlands, 2020

SAC Journal 6 95

31.03.21 10:05

IMAGE AND PROJECT CREDITS

COVER IMAGE

The Thrid Glass is a project by Städelschule Architecture Glass.

Students: Yara Feghali, Viviane El Kmati, Miriam Kuhlmann, Nistha Mehra, Keyur Mistry, Provides Tsing Yin Ng, Jayakar Priyadharshan, Anokhi Shah, Madhumathi Shankar, Vamsi Vemuri, Nopnida Vera-Archakul.

Concept: Daniel Birnbaum, Damjan Jovanovic and Johan Bettum.

VR/AR Software Development and Project Lead: Damjan Jovanovic. Cover Design: Anna Arlyapova. Cover Illustration: Damjan Jovanovic and Yara Feghali.

SANFORD KWINTER & DANIEL BIRNBAUM A CONVERSATION: THE THIRD GLASS Page 17 and 21: Photo courtesy of Städelschule Architecture Class.

Page 18: Photo by Kishan Kumar.

MARTINE BEUGNET VIRTUAL REALITY AND THE GAZE Page 35: Parragirls Past, Present. Unlocking Memories of Institutional ‘Care’, 2017.

Immersive 360 – degree 3D steroscopic film, ambisonic sound.

Image courtesy the artists.

MICHAEL YOUNG EXESSIVE RELIEF Page 49: Ludovisi Sarcophagus – CE 250-260, National Roman Museum, Palazzo Altemps, Rome. Detail View.

Page 50: Ludovisi Sarcophagus – CE 250-260, National Roman Museum, Palazzo Altemps, Rome. Page 51: Photogrammetry scan of the Ludovisi Sarcophagus, Palazzo Altemps, Rome (2019). Distant View.

Page 52, 53: Photogrammetry scan of the Ludovisi Sarcophagus, Palazzo Altemps, Rome (2019). Near View. All photos and images by author.

JONATAN HABIB ENGQVIST, CHRISTER LUNDAHL & MARTINA SEITL AN ACCEPTABLE LEVEL OF REALITY Page 61: Detail of hands. Royal Academy of Arts. Photo Julian Abrams.

Page 62: Both images: Monograph ll, 2021. Installation view. Photo by Joakim Olsson.

Page 63: Captured by Lidar Terrestrial Laser Scanner. Courtesy ScanLAB Projects.

Page 64: Monograph ll, The Memor, 2019. Photo by Joakim Olsson. All other images: courtesy the artists.

FABRIZIA BANDI THE ARCHITECTURAL RELEVANCE OF VIRTUAL REALITY This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. [834033 AN-ICON]), hosted by the Department of Philosophy “Piero Martinetti” – Department of Excellence at the State University of Milan.

KLING KLANG KLONG & ONFORMATIVE COLLIDE (2016) onformative: Creative Direction: Cedric Kiefer. Design: Bernd Marbach.

Production: Imprint Projects, onformative. Research: Christian Loclair. kling klang klong: Sound Concept, Composition, Music Production.

AINSLEE ALEM ROBSON FERENJ. A GRAPHIC MEMOIR IN VR (2020)

PAISLEY SMITH & LAWRENCE PAUL YUXWELUPTUN UNCEDED TERRITORIES (2019) Created by Paisley Smith, Lawrence Paul Yuxweluptun with music by A Tribe Called Red.

RACHEL ROSSIN I CAME AND WENT AS A GHOST HAND (2015) Page 110, 112, 113 and 116: Installation view, Rachel Rossin, ‘Lossy’ at Zieher Smith & Horton. Photo: Matt Grubb. Courtesy the artist and Zieher Smith & Horton.

Page 114 and 115: VR screenshot from Rachel Rossin, “I Came And Went As A Ghost Hand” (2015). Courtesy the artist.

SPACE POPULAR FREESTYLE (2020)

Page 120 and 123: FREESTYLE: Architectural Adventures in Mass Media, curated by Shumi Bose, Royal Institute of British Architects, London, 2020. Photos: Francis Ware.

TIMUR SI-QIN A NEW PROTOCOL VR (2018) 2D video version of “A New Protocol VR” by Timur Si-Qin Sound.

Music: Darri Lorenzen. Narration: Moira Barrett.

MARSHMALLOW LASER FEAST WE LIVE IN AN OCEAN OF AIR (2018) Marshmallow Laser Feast: Barnaby Steel, Ersin Han Ersin, Robin McNicholas, Nell Whitley & Mike Jones. In collaboration with Natan Sinigaglia and Mileece I’Anson.

Executive Producer: New Balloon, supported by RYOT, Haglofs, HP, QUIXEL, Lexhag, The Feelies, Pollitt and Partners, The Eden Project, Artists&Engineers.

CURTIS ROTH GESTURING ELSEWHERE (2020) All images courtesy Curtis Roth.

LUNDAHL & SEITL SYMPHONY (2009-2014)

Page 144: Symphony of a Missing Room, Royal Academy of Arts, London, 2014.

Photo: Julian Abrams.

Page 146: Symphony - Mnemosyne Revolution, 2nd Kochi Muziris Biennale, 2016.

Courtesy Kochi Muziris Foundation.

Page 150, top: Paul Almasy, Louvre, Paris, 1942, © the artist and akg-images / Kleinschmidt Fine Photographs.

Page 150, bottom: Symphony of a Missing Room, Groupius Bau, 2016.

Courtesy Berliner Festspiele. Immersion.

MARCO BRAMBILLA HEAVEN’S GATE (MEGAPLEX) (2021) 5:45 minutes continuous loop, 8K video with sound. All images courtesy Marco Brambilla Studio.

JACOLBY SATTERWHITE WE ARE IN HELL WHEN WE HURT EACH OTHER (2020) Artwork and installation images © Jacolby Satterwhite, courtesy of the artist and Mitchell-Innes & Nash.

Written, directed and created by: Ainslee Alem Robson.

Producers: Liam Young and Ainslee Alem Robson. Art Assistance: Kidus Hailesilassie.

FOLLY FEAST LAB MEDITERRANEAN SEA DIARIES (2020) Developed and designed by Yara Feghali and Viviane El Kmati.

YEON JOO OH SATURATED SPACE (2019) Page 190: Frederick Kiesler working inside his sculpture “Bucephalus”, Amagansett/NY, c. 1964-65, black and white photograph, photograph by Adelaide de Menil. © 2020 Austrian Frederick and Lillian Kiesler Private Foundation, Vienna.

198

SAC Journal 6 198

31.03.21 10:05

SAC JOURNAL 6 BREAKING GLASS SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY Series Editor: Johan Bettum Issue Editor: Yara Feghali and SAC Editorial Team Executive Editor: Annika Etter and Sylvia Fadenhecht Editorial Office: SAC Städelschule Dürerstrasse 10, 60596 Frankfurt am Main, Germany Tel +49 (0) 69 60500869, architecture@staedelschule.de www.sac.staedelschule.de www.breakingglass.staedelschule.de Design and Layout: Jacqueline Jurt Image Editor: Jacqueline Jurt Image Compilation and Design Assistance: Anna Arlyapova and Yeon Joo Oh The editor has conscientiously endeavoured to identify and acknowledge all sources and copyright holders. All those holding illustration copyrights who have not been identified or credited can contact the editor.

SAC Journal is published once a year.

Publication © Copyright 2021 by Spurbuchverlag Baunach, Germany; Städelschule Architecture Class, Frankfurt am Main; and authors.

All rights reserved.

No part of the work may in any mode (print, photocopy, microfilm, CD or any other process) be reproduced or – by application of electronic systems – processed, manifolded or broadcast without approval of the copyright holder.

The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic information is available on the internet at: http://dnb.de AADR – Art, Architecture and Design Research publishes projects and research with an emphasis on the relationship between critical theory and creative practice.

AADR Curatorial Editor: Rochus Urban Hinkel, Melbourne Production: pth-mediaberatung GmbH, Würzburg Publisher: Spurbuchverlag Am Eichenhügel 4, 96148 Baunach, Germany Tel +49 (0) 9544 - 1561, Fax +49 (0) 9544 - 809, info@spurbuch.de Spurbuchverlag: www.spurbuch.de and AADR: www.aadr.info

ISBN 978 –3– 88778–618–2 ISSN 2198 –3216

SAC Journal 6 is published in conjunction with the symposium Breaking Glass III – Virtual Space and made possible with the generous support of “experimente#digital – eine Kulturinitiative der Aventis Foundation” and Dr. Marschner Foundation.

199

SAC Journal 6 199

31.03.21 10:05

200

SAC Journal 6 200

31.03.21 10:05

While teasing the disciplines with creative opportunities, the use of the media presents a staggering number of acute questions, not the least with respect to corporeal experience, the human-machine interface and what constitutes the “real.” Augmented and Virtual Reality invite to re-examine established ways of thinking and making within architecture and the arts and open onto an uncharted territory of what comprises architectural and artistic experience.

With Breaking Glass: Spatial Fabulations & Other Tales of Representation in Virtual Reality, select topics central to Augmented and Virtual Reality in architecture and the arts are addressed. It is published in conjunction with the conference Breaking Glass III: Virtual Space, the third and last in a series hosted by the Städelschule.

The publication includes texts by, among others, Martine Beugnet, Michael Young, Curtis Roth and Lara Lesmes and Fredrik Hellberg as well as conversations that Daniel Birnbaum respectively had with Sanford Kwinter and Sven-Olov Wallenstein. In addition, a series of visual portfolios by architects and artists presents works.

Finally, the publication features the award winning projects of Städelschule Architecture Class’ AIV Master Thesis Prize 2019.

The issue has been edited by Yara Feghali and the editorial team of the Städelschule Architecture Class. It has been made possible with the generous support of the Aventis Foundation and the Dr. Marschner Foundation.

© SAC JOURNAL is published by the Städelschule Architecture Class and AADR – Art Architecture Design Research (Spurbuch Verlag).

6 S AC J O U R N A L

ISBN 978 – 3– 88778–618 – 2 ISSN 2198 – 3216

B R E A K I N G G L A S S: SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY

Within a relatively short time, Augmented and Virtual Reality have emerged centre stage in architecture and the arts as novel means for exploring how their creative output is produced, mediated and experienced. Feeding the continuous spectrum between the “fully real” and the “fully virtual,” the underlying technology of these media present machine-generated sensorial input where to date the image-based dominate. With these inputs, corporeal existence see “virtual” experiences thrown on the scale with “real” ones as the concepts and models for how we understand perceptional dynamics are shifting.

BREAKING GLASS SPATIAL FABULATIONS & OTHER TALES OF REPRESENTATION IN VIRTUAL REALITY

SAC JOURNAL STÄD EL SCH U LE A RCH I T EC T U RE CLASS

Cover SAC Journal 6.indd 1

26.03.21 14:07