Medium of the Future, Medium of the Now A Roundtable Discussion on Narration and Immersion in Advanced Cinema Technology and VR
Wolfgang Brßckle and Fred Truniger in conversation with Neal Hartman, Johan Knattrup Jensen, Florian Krautkrämer, Marco de Mutiis, and Alia Sheikh
The advent of immersive filmmaking practice fundamentally changes the media system and how we tell stories. So, what is the impact of VR and 360-degree filmmaking on the present and future of storytelling? Wolfgang Brückle and Fred Truniger asked Neal Hartman (Director of the CineGlobe Film Festival at CERN, Geneva), filmmaker Johan Knattrup Jensen (Director of Macropol, Copenhagen), Florian Krautkrämer (Visiting Professor, Mainz University), Marco de Mutiis (Digital Curator, Fotomuseum Winterthur), and Alia Sheikh (Senior Development Producer, BBC Research & Development, Manchester) to share their thoughts in a panel discussion that took place on the occasion of the conference «Display, Disruption, Disorder: New Formats, Actors and Places in Audiovisual Media» at Lucerne University of Applied Sciences and Arts on 18 November 2017.
← Johan Knattrup Jensen, Skammerkrogen (The Doghouse), 2014 Jensen’s installation invites viewers to step into a virtual world for an immersive first-person film experience which, while absorbing the individual participants, at the same time connects them with an interpersonal reality. They are invited to sit around a table wearing an Oculus Rift headset. Each participant assumes the role of one of five family members at a dinner party so that each of them experiences the unfolding story about the first visit of the older son’s new girlfriend from a unique point of view. The camera’s movements are completely defined by the characters.
Wolfgang Brückle: Traditional TV appears to be doomed in the eyes of many experts, whose opinion is based on how little interest the medium generates among young audiences. At the same time, not only individual directors but also media corporations have in recent years started to embrace the possibilities of technically advanced filmmaking, and 360-degree cinema and VR seem to be a promising field for experiments today. The BBC’s production lab is active in the field; the CineGlobe Film Festival at CERN is encouraging experiments in the fields of both fiction and documentary; a team working at the Hochschule Luzern is currently cooperating with the Swiss Broadcasting Corporation SRG to enhance the quality of narrative 360-degree filmmaking. But it is not always clear whether these activities are driven by similar ambitions and dreams. Are we merely wit nessing a modern variation of the spectacles that panorama paintings once pre sented to an audience attracted by the lure of extreme realism? Or is this the rise of new ways of storytelling that bring unforeseen disruptions to what our senses used to consider the real, and that allow new forms of combining the fictional and the factual? You probably don’t want to see storytelling limited to traditional con cepts of literary narrative in our discursive context… Fred Truniger: Actually, there is an undercurrent of dissatisfaction with the way storytelling functions in VR filmmaking, and we probably want to think about what we can hope to get in exchange for the disadvantages that VR filmmaking has in relation to traditional cinema. A general notion is that we have yet to understand what VR can actually achieve. Do you consider what you’re working on right now simply the beginning of a new practice and an apparatus that’s bound to change, and are you constantly envisioning what this change could be? Florian Krautkrämer: Thinking about the present and the possible futures of this format has a lot to do with how we see our present and past media. We’re always comparing it to cinema. But what exactly is this cinema that we hold up in comparison to VR and 360-degree filmmaking? My impression is that when we contextualize VR and 360-degree filmmaking, we unnecessarily limit the discus sion to the paradigm of mainstream movies, where you offer the audience all the answers on a silver platter. Blockbuster cinema may be the most visible type of cinema, but it’s only a small part of the whole thing. We have the films of Lav Diaz, Lisandro Alonso, Chantal Akerman; we have cinema that raises more questions than it answers. Discussions about 360-degree filmmaking often revolve around how we can draw the attention of the audience to this or that detail just because we don’t want them to get frustrated in the act of watching. But I would like to suggest that we focus less on finding ways of how to satisfy the viewer’s desire to be guided through a linear narrative, and more on how we can grow a culture of higher frustration tolerance. We need to find ways to accept frustration, and from this starting point develop new questions and interests on the side of the spectators.
68 ⁄ 69
Alia Sheikh: It is for two reasons that people associate the commercial block buster aspect with cinema at large: because it’s ubiquitous and because it makes money. I think that in the long run, VR and 360-degree cinema will let us have stories that were previously impossible, but while we have our heads stuck in reinventing the narratives we already have, we’re never going to get there. And yet a large part of my work is about teaching people how to use VR to tell the sto ries they usually tell, and what the new rules for directing attention are, even though there’s a possibility in VR to tell stories that may not need as tight a con trol of attention as cinema does. When I’m talking to people who come from the world of traditional film and TV formats, I can’t afford for them to get frustrated with the medium before they’ve been able to create something good. So, for me, it’s about giving them enough pointers and an understanding of how to approach it in terms of their usual subject matter, and then hoping someone will realize that «some of these things are new, and I couldn’t do them before.» Once people start thinking this way, things ping off in other directions, and then you find people are making things like Alice: the Virtual Reality Play – theatre experiences where you embody a persona and live actors embody the other characters. And that’s not cinema at all. So, while it’s maybe a bit of a shame that so many people rumi nate over how to create cinematic experiences in VR, they’re making a step on a journey to a better, more interesting place. We have to start somewhere, and it isn’t surprising that people often approach VR through the lens of familiar formats. Neal Hartman: If you look back at the history of film, we basically had cinema for the first fifty years or so. Then we had cinema and TV, but no one would ever have proposed watching cinema and TV at the same time. That would have been completely absurd in the fifties or sixties. Now, however, we have the concept of second screen, which refers to the auxiliary content that can be created to accom pany a film. This phenomenon began with DVD extras, but has quickly come to mean content that will be consumed on a digital device such as a tablet or smart phone. We’re not talking about the film trailer here, but about content that is meant to be consumed as part of the viewing experience, perhaps not necessarily simul taneously, but sometimes even in unison with the original viewing. Especially with web video, in that you have multiple sources of content for the same core story, or the same project. Outside of gaming, we see the biggest commercial use of VR in the marketing of big films and online productions, like Stranger Things or Harry Potter, sometimes amounting to essentially amusement park applications. The question is whether VR will remain a marketing tool, which is akin to a second screen, or whether it will establish its own space as primary creative content. This last possibility is exciting because of its immersivity and the way VR excludes external stimulation. Perhaps it might even permit us to actually return to a more focused way of viewing. Marco de Mutiis: There has been a desire to dissolve into the medium as early as in the ancient Pompeii frescos. At the same time, however, there’s a resistance to it. We keep inventing mediums that offer illusionistic experiences, but we also resist them, maybe because we fear being overwhelmed, and we fear losing con trol. This is one of the interesting dynamics of immersion, and is perhaps one of the reasons why we like to push it forward into the future: we’re not really com fortable about exacting it in the now. Alia Sheikh: True as this may be, now is the time for us to do the thinking about what these new possibilities could be, because we have an opportunity to have an input into the technology that is invented. We’ve seen things come and go – we saw 3D rise and fall, and we’d had VR before, the Victorians had dioramas and stereoscopy and stereopticons… We already know that at the content creation side, we can produce complex, technologically advanced experiences, but at the consumer side, the most easily accessible 360-degree experiences are on You Tube or Facebook or cardboard phone headsets. At the moment, I can’t deliver to everyone in the home the kinds of personal headset-based experiences that we’re exploring, simply because unlike a TV set, the technology isn’t in everybody’s homes…
Wolfgang Brückle: Could you detail what this experience might possibly be? And what, apart from the BBC’s incentive to keep pace with the development of visual devices TV can’t easily embrace, makes anybody’s private home the ideal place to share this kind of experience? Alia Sheikh: With VR and 360-degree filmmaking, which I always regard as almost the smallest unit of virtual reality, we can amaze people, we can get you closer to the world of the story, and we can put you in someone else’s shoes. We can create more of a feeling of empathy. This may be just a better version of things we could do already. But then there are some things that are just completely magic, and those are right on the fringe of the technology. For example, when you’re using headsets and 360-degree cameras combined with human actors as well, and you play with the space, and the technology is really itchy and fuzzy and doesn’t really exist. My favorite example is BeAnotherLab’s Machine to Be Another, which is a performance installation that started out as low budget experiments into Embod iment and Virtual Body Extension. They’ve created a system where it feels entirely plausible that you have inhabited another person’s body, complete with a sense of ownership. I really want the momentum to keep going and for us, as an industry, to keep doing these experiments to push the medium. We know it hasn’t reached its full potential yet, and experimenting is the only way to realize it. Fred Truniger: Johan, I remember you saying in a different context that in your project The Doghouse you tried to get something out of the 360-degree, or in this case 180-degree, film experience that you could incorporate into more traditional filmmaking. What is this «something» that traditional forms of cinematic story telling can gain from? Johan Knattrup Jensen: I think the biggest promise of VR is that it can turn cinema from a mass medium into a solo medium, or into an exclusive, even ego istic medium. From the perspective of an artist, I see a huge potential in creating experiences that people are actually willing to pay for. They’re willing to spend the same amount of money as they would for, say, opera tickets, to have these experiences. The reason why we have big movie theatres and create rules for a medium is so that we can distribute it. It becomes easier to distribute and the media becomes sellable in a mass-market way. That’s why we’re trying to put rules on VR, so that we can distribute it. But that’s initially not my intention… VR has changed the role of the viewer not so much in terms of interactivity as in terms of the viewer’s responsibility. You’re no longer just a passive witness with no responsibility for anything. You can become an activist in the story. Producers and filmmakers should respect that the audience is ready to commit to such an experience, both emotionally and financially. Fred Truniger: In your concept of VR as a solo medium, do you refer to the purely economic promise that may accompany the new form of entertainment commod ity you just evoked? It strikes me that the most promising result of your cine matographic experiments is a form of storytelling that is able to simultaneously include a core story that is delivered to all viewers and a highly individualized, maybe even singular story for each of the viewers – and thus extend the story telling experience to the moment, when different participants of a VR-piece engage in discussions after the actual viewing. Johan Knattrup Jensen: People are talking about the single or mono audience experience as the downside of VR, as it is not a collective experience. I don’t think cinema necessarily needs to be a collective experience. More often than not, it’s an individual experience. However, that doesn’t mean we can’t create a space for collective reflections. In fact, I believe filmmakers should reclaim both the ante room and after room of the experience in an attempt to design not just the prayer, but the full ceremony of the audience. I strongly believe in the power of the expe rience triggered by the rituals of watching a movie. The more specific and per sonalized the rituals are, the more important and significant the experience becomes. Still, that doesn’t change the communicative nature of cinema and storytelling. It is meant for many people, but only one at a time.
70 ⁄ 71
← Édouard Castres, Entrée de l’armée de Bourbaki aux Verrières (The arrival of Bourbaki’s army at Les Verrières), 1881, oil on canvas, 9.80 × 115 m, Lucerne In 1881, Swiss painter Castres created a panoramic 360-degree painting of a military event that had recently made his home country an active player in the Franco-Prussian war. Almost 87,000 members of the French Bourbaki Army, defeated and demoralized, were allowed to cross the border to Switzerland, where they were decommissioned and detained in what became famous as one of the first operations of the Inter national Red Cross. In order to effectively captivate and immerse the audience, Castres put sculptural elements on a so-called faux terrain in the foreground of his painting: life-size figures in uniforms huddle up to a fake fire; local civilians approach to attend them; guns and knapsacks lie about in the artificial snow. Like other panorama painters, Castres aimed at blurring the borders between reality and image. In effect, he created an early example of what we now like to call a «virtual reality».
Wolfgang Brückle: Your comments are interesting in that they point us to a very ambivalent situation, where what we know from post-conceptual currents in art is that they put aside storytelling for the sake of the distantiating forces that encourage knowledge or its crisis, reflection, and critical thinking. But in so many innovative forms of VR and 360-degree movies, immersion and a rather old-fashioned fascination with illusion are key. How do you think we can negotiate these apparently opposing distantiating concepts, on the one hand, and immersion, on the other? I can accept that users, viewers, visitors, and beholders are per haps granted more power than before. However, we usually associate immersive powers with the society of the spectacle, with entertainment and amusement. Let’s face it: the invention of cinema is not the only reason why panoramic pic ture installations like the Bourbaki Panorama Luzern have fallen into oblivion for many decades. In their focus on sensational visual experiences, they also came to be considered too popular to be regarded as serious art – they came to be regarded as mere products of the entertainment industry. We probably need to evaluate the criteria we want to be essential in any critique of today’s emerging VR worlds. Fred Truniger: I agree that there is a suspicious excitement about the promises of VR in the comments of so many practitioners and critics, which reminds me a lot of the «frenzy of the visible» that accompanied the advent of film at the end of the nineteenth century. I wish there was more critical reflection on what VR really is beyond what Tom Gunning labelled the «cinema of attractions.» Take, for instance, the VR project Tree by the New Reality Company, which was presented at the Sundance Festival. You put on the head-mounted display, and you are put into a seed’s POV and develop into a rainforest tree that is threatened by a slashand-burn campaign. As a viewer, I was a bit annoyed by an imagery the main pur pose of which was to make me feel the vertigo of the extraordinary viewpoint from the top of a giant tree: it posits a strongly immersive, but narratively naïve sensa tion of being a growing seed or a grown-up tree at the center of my attention, and I would argue that this preference of immersive effects over the argument dis tracts the viewers from focusing on the economic and ecological implications of the story. I am far more interested in the kind of dissociative POVs that the Felix & Paul Studios developed for the Cirque du Soleil’s Dreams of «O». While some of the effects used in this piece are reminiscent of the more or less silly techniques used to increase visual appeal in the 3D era, there is an intriguing short sequence where you are watching a group of synchronized swimmers performing beautiful blossom-like formations. All of a sudden you occupy two POVs at the same time: when you look up, you see a formation of synchronized swimmers as if from below the water surface. And when you look down, you see the same swimmers from above. To me, this is a genuine and exclusive potential of image-making in VR: to make my perception disintegrate into two possible embodied POVs – and still be able to «read» the situation properly. Even though they don’t seem to be located at the core of the hallowed promise of immersion, such VR possibilities of the VR medium are intellectually challenging experiences, which seem more interesting to me than even the most skillful repetition of those pretty intrusive immersive effects VR often seems to be satisfied with. Johan Knattrup Jensen: We mentioned The Doghouse earlier – actually, immer sion is an ambiguously complex concept in this installation, too. I didn’t necessar ily intend a complete immersion of the viewers. They are not anonymous bystand ers: they live inside someone else’s mind, they experience a bona fide emotional transformation, and they undergo a short identity split which is very revealing beyond mere visual excitement. They will realize that there are other truths than what they thought was real. It’s both an intellectually and emotionally revealing experience. Marco de Mutiis: As Wolfgang said, the practices have been there for quite some time. However, 360-degree and VR are not really the same thing when we’re talk ing about the scene of your experience, whether it’s Jeffrey Shaw’s experiments
72 ⁄ 73
with iCinema or dome projections. We could also trace all the different forms of immersivity right up to the ridiculous attempt, in my view, to make screens slightly bent, or 3D television. But also, so many artists in the art world today are working with VR as a tool. Take, for example, Jon Rafman’s contribution to the Berlin Bien nale in 2016. At the Fotomuseum Winterthur, we’re now showing The Sensible Spectrum by Alan Bogana in which you wear a headset and a projection on top of your head; you’re seeing something within the headset while at the same time, it’s projecting something else outside. So effectively you’re always conflicted in that tension of not knowing what is actually going on outside of the view you’re immersed in. In terms of FOMO, in this case it’s the fear of missing out of what’s outside of your head. I think these are all interesting practices. Wolfgang Brückle: These aspects obviously offer valuable points of departure for defining the promises VR holds for discussions on media aesthetics. The view ers’ individual choice of focus may also be essential. We may say that when, in 2007, the Wooster Group designed their panoramic installation There Is Still Time… Brother, they married the old panorama paradigm, complete with the latter’s pref erence for battle scenes, to the theatrical narrative whose home base is the movie theatre. Actually, they were inspired by a feature film from 1959. What is essential in their approach is the chaotic complexity of a story told in visual fragments and in a palimpsest of competing voices, with the viewers experiencing a variety of meanings depending on what they decide to focus on. Fred Truniger: This brings us back to the question as to whether we need to extend the scope and range of our notion of storytelling. Film has always offered one and the same story to a large number of people. While each individual has their own experiences, film has always tried and is still trying to control what’s going on in your mind. In installations like The Sensible Spectrum, however, there’s a narrative trap for the person in the helmet that is very interesting as a basic set ting for storytelling. The installation leaves a lot to the imagination of the viewer, his or her mind is expected to deal with insufficient information in a situation of visual uncertainty. And the author has to give up his or her full control over the narrative in the process of exploring the field of storytelling. Perhaps, as Johan said, storytelling is starting to become individualized again. Marco de Mutiis: I think storytelling is a tricky concept in this context, as it tends to restrict VR to traditional film narrative ideas. VR exists within an ecosystem of digital media that might have nothing to do with stories. I’m thinking of the notion of databases and networks of images, for example, and in this sense VR allows many different visual cultures and image traditions within itself. Immersion is not necessarily linked to storytelling in the traditional sense. We probably should not underestimate the implications of Lev Manovich’s insistence on databases being, in the computer age, a key form of cultural expression and an alternative to what novels and cinematic narratives meant for the modern age. Neal Hartman: I think it’s important to make a distinction here when considering the question of «individualization» in storytelling, especially in relation to VR. Immersiveness is a quality of VR experience – what length it goes to in order to create the feeling of «being there.» Presence, in turn, is what the spectator feels while having that experience. The level of presence generated in the participant can be affected by multiple factors, including the physical procedures that they use to get into the experience. VR projects can be very immersive but still fail to generate real presence in the participant, and vice versa. In particular, the role of interactivity and the customization of individualized narratives sometimes risks being overestimated. This seeming paradox is well exemplified in Johan’s DocLab performance of The Shared Individual, where everybody in the audience is basi cally experiencing the same thing: there’s no tailored individuality for all those people, and yet they all have an individual experience. This shows that it doesn’t have to be adapted for each person for it to be individual, and so an entire audi ence could watch exactly the same film. And in The Shared Individual, for example, they use touch, mimicking motion and all these things that, through research, are now widely known to enhance presence in the VR experience… I think you can have immersive experiences that don’t generate a lot of presence because they
→ New Reality Company, Tree, 2017, installation at the Camden International Film Festival, Maine, 14 to 17 September, 2017 Milica Zec and Winslow Porter’s studio uses virtual, augmented, mixed, and actual reality to create VR experiences in collaboration with corporations, foundations, and artists. While their 2016 project Giant was one of the first VR projects to combine semi-volumetric liveaction video with game-engine software, Tree offers a combination of a VR film with real-world props which are supposed to gently prepare participants for their VR experience and additional touch and smell experiences. The installation puts the viewer in the position of a Kapok tree which, after growing to considerable height, is threatened by the ecological disaster of deforestation across the Amazon basin. Porter explains: «We wanted you to feel what a tree is like, what it means to be immersed in nature.»
↙ Alan Bogana, The Sensible Spectrum, 2017 In Bogana’s VR installation, a membrane marks the interface between intimate and public space: users are cocooned by the artist and obliged to watch while immersed in various VR sequences taken from online platforms such as YouTube or Vimeo. This footage stems from different contexts, includes different gazes, and aims at distinctive viewer reactions. Via a customized VR headset, the users’ viewing behavior can be tracked by outside observers who experience the sequences in a unique, spatialized way. Confronting various forms of spectatorship, The Sensible Spectrum highlights the relation between the immersive media we consume and the potential privacy issues associated with recent developments in VR technology and the capacity to record our behavior.
haven’t been designed properly, and you can also have cinematic kinds of expe riences in a collective group that generate a lot of presence because of the way the spectator enters, because of the content, or because of the empathy that’s being generated. So, when we’re talking about VR I think it’s important to differ entiate between the actual medium and the content that is practiced within the medium. Alia Sheikh: I agree that we shouldn’t talk about a tool as if it were a format. Mov ing visuals – video for example – can be anything: a video of your grandchild, CCTV footage, or something you watch on TV or in a cinema. Video isn’t the format, it’s the thing you put into the format. I often hear people talk about VR as if it was a specific format rather than a set of techniques that convey whatever you want them to convey. I don’t think that there’s one definitive, generic VR experience. Even for movies, you always bring your experience to them, and I feel that VR tools are like that as well. Johan will look at VR tools and say, «I want to use them to put myself in a movie.» And I’ll look at the same tools and say, «I don’t want to be in the story, but I want to see it from much more closely.» And it’s the same tools that enable all of those approaches. Florian Krautkrämer: I’d like to add another aspect. Perhaps for the first time in the history of motion pictures, we are offered an inexpensive tool that is not reserved for a few people, but is open to a broader community of contributors to our visual culture. You can get 360-degree cameras for about €200. The new GoPro Fusion model has a built-in 360-degree function. Maybe what we will do with the new medium is not entirely in the hands of the professionals; maybe we should also consider what amateurs are offering on their YouTube channels. Alia Sheikh: When cameras became cheap and ubiquitous, we stopped getting static posed family photos; we got pictures of the cat, the dog, the baby, oppor tunistic unscripted moments. When technology becomes more widely available, that’s where we get the momentum and the enthusiasm. Because if it’s only in the hands of a few gatekeepers who decide, like, «well, we can’t use it to make money yet, so it has no value,» that puts the technology a lot more at risk than if a crowdfunded VR headset gets enough momentum that people buy it, get excited about it, and start doing really unusual things with it. These things might not nec essarily be financially successful, but at least there’s that democratization. Wolfgang Brückle: You’re raising an interesting question: can democratization guarantee innovative strands of production in our field? In the field of photogra phy, Kodak may be a case in point although I’m not sure the smartphone is. Neal, you are organizing a festival with 360-degree films made by everyone who wants to take part. Do you think that one of the possible ways of developing this medium is to encourage production on all levels that hobbyists and professionals have to offer, and then see what will last and what will go? Neal Hartman: I think this all falls back into the traditional film industry paradigm: the development of the tools is extremely expensive, so that has to be done by large companies. It’s what we’ve seen so far with VR: we have Oculus and Sam sung and now HTC, but they require take-up on the scale of millions or else the development doesn’t pay off. So, although the Samsung gear has been distributed in the few millions, and the Vive in the few 100,000s, so far it hasn’t been radically commercially successful for the hardware. I think that part of the reason for this is that, although there is now a range of «cheap» cameras and head mounted displays (HMDs) available, we don’t yet have a broad ecosystem of creators and content consumers with a diverse, healthy foundation layer of young people mak ing projects. VR needs a large group of creators that aren’t necessarily very good at it, but that are ready to flesh out the creative landscape, providing the base of the pyramid of production where at the top you have your expensive projects that are extremely well done. But without enough critical mass, those expensive pro jects will only be few and far between. That’s why it’s particularly interesting to see these tools get into the hands of lots of different people that don’t have a lot of means: they’re the ones that can grow this base layer of the ecosystem that you now see in traditional cinema with short films. We have an extremely healthy, though not in a financial sense, short film industry in the world, with tens of thou
→ The Wooster Group, There Is Still Time… Brother, 2007, installation at the Pano rama Festival, ZKM (Center for Art and Media), Karlsruhe, Germany, 15 December 2007 to 6 January 2008 Directed by Elizabeth LeCompte in cooperation with Jeffrey Shaw for the iCinema Research Centre, this installation presents an experimental Spherecam film about individual reactions to the experience of war. Its title alludes to a banner in a scene from Stanley Kramer’s On the Beach of 1959, a film that evokes a post-apocalyptic planet Earth after nuclear fallout. Visitors sit in revolving chairs at the center of a 360-degree AVIE system panorama, with the option of focusing on discrete aspects of a story about an eighteenthcentury clash between British and French troops in the French and Indian War. Children’s toys vie for attention with politically minded bloggers, unsavory YouTube videos, and a mercurial host who, in Shaw’s words, attempts to «articulate the implications of this unique narrative space.» Audience members continu ously generate new cinematic experiences in this collage of multinarrative threads.
sands of short films being made very year. That drives a lot of quality upper level production. Florian Krautkrämer: And that’s the important point here! It’s not only the authors of the scenarios that invent the stories, it’s also the audience. As long as the pub lic remains only ever interested in the same stories and contents, nothing will change. We need to find a way of triggering interest in new forms and experiments. And in the end, that comes down to curiosity and education because we can’t guarantee that a good film will always find its audience only because of its inher ent quality. Fred Truniger: Still, the education of filmmakers is part of this question. It’s not just an issue of money versus no money; it’s also about training versus no training. If we accept that there is a lively production culture of prosumers, what is our role in the training of young people who want to become skilled filmmakers? Alia Sheikh: You want to provide them with the opportunity to make more inter esting mistakes quicker. I don’t want everyone to make the same twenty boring mistakes. You never want to have a situation in a film class where people don’t know how to operate the camera and which end of the light is hot, so it’s impor tant to know the basics. But I’m really excited about the interesting mistakes and how quickly you can get to them. Fred Truniger: Speaking from an art school’s point of view, it still makes sense to discuss perspectives beyond trial and error: we don’t want the medium to be limited, and we don’t want a normal narrative; we want to encourage artists and directors to break the rules. It’s the same discussion that’s been going on for nearly a century in the quarrels between avant-garde or arthouse versus mainstream cinema. But if we really believe that 360-degree VR is a cheap medium and every body can use it, who will develop the new forms? Who is the driving force behind it? Who should be the ones to play with it and find new ways to use it? Is it who ever tries, or is it the trained and skilled animators and artists trained at our art schools: who is doing the work? Is it really democratized, or is the development still in the hands of a certain type of skilled practitioner? Is it at BBC R&D, with the money they’re able to invest in experiments? Is experimental work so expen sive as to require an institutional background?
76 ⁄ 77
Alia Sheikh: It’s not that it’s cheap, it’s that it’s cheap to play in this space in the first instance. To make the VR experiences that are going to win Oscars, you need a lot of money. Take our BBC project Home, which offered the experience of going on a spacewalk using the HTC Vive and the Oculus Rift and, ideally, a haptic feedback chair. It’s one of the most successful VR experiences the BBC has produced. It offers a high sense of drama, excitement, adventure. A lot of the potential prob lems were solved by choosing a fitting story which let us ensure that people retained a high sense of presence and immersion. For example, you look down at your arms and you’re in a space suit, and you don’t have any dissonance in seeing that your hands are the wrong color, or you’re a man and not a woman, because most astronauts look about the same when they’re in a space suit. Now, Home was not a cheap thing to make. But it’s not necessarily expensive to play in the VR space and do something interesting. And in terms of who invents the interest ing experiences, I think maybe it’s the people who have a little bit of time and breathing room. As a research department, we’re not swimming in tons of money, but we can spend a relatively long time allowing ourselves to try out different things. Likewise, in a design and art school, you may have students who do cross-disciplinary work in film and animation, all interacting and talking with each other. And that’s why machinima and gaming is so interesting: they show what can happen if there are sufficiently motivated individuals who have the luxury of some time, and who have been offered an access point to make the thing they want to make. Fred Truniger: When we looked at ultrashort filmic forms in a research project, we saw that, for instance in Vine, lots of people were trying it out. Yet the most successful ideas came from individuals who had a certain basic training, or even from big companies that used the channel to promote their own product. I think we have the same situation with GoPro. Who is actually playing with it? Who is making the really interesting stuff? I think we should find a way to inject this cre ative output into the development and not only rely on the power of the company. Florian Krautkrämer: Exactly. When I was talking about education, I didn’t mean education in film schools, I meant education in general. A large part of the gener al public is interested in diverse forms, and in a broader possibility of different formats. Usually, however, the most successful films or contents are driving minor forms out of the market; they prevent their visibility to a larger audience. Com
← BBC and REWIND, Home: Immersive Spacewalk Experience, 2016 In cooperation with BBC Science, BBC Learning, BBC Digital Storytelling, and the REWIND team of technologists and digital artists who specialize in immersive technologies for VR, AR, animation, DOOH, VFX, and 360-degree video projects, director Tom Burton created a 15 minute visual spacewalk narrative delivered to Steam VR, i.e. HTC Vive and OSVR. Viewers go through a simulated spacewalk developed in conjunction with NASA and ESA. Besides the VR head-set itself, they experience the walk using haptic feedback chairs such as these movable flying or racing chairs. Immersive Spacewalk Experience also supports a full body biometric system that allows viewers to hear and monitor their own heartbeat during the mission.
pared to twenty years ago, we have fewer opportunities to show different formats, contents, and other forms of film simply because the market is now dominated by a limited number of big content providers. The only way to stop this vanishing of non-mainstream content channels is by raising awareness through education, not only in film schools but everywhere. We need to spark curiosity for different forms, formats, narrations, narratives, and so on, in order to generate other forms of storytelling. Marco de Mutiis: Going back to VR, I’d like to mention the research group BeAn otherLab as one of the critical examples in this discussion. While they’re doing some of the most innovative and interesting experiments with the medium, they don’t come from the big money of the establishment; they rely on funding and research grants. So, from a financial perspective, they’re not successful, that is, they aren’t in terms of the views and likes that regulate the distribution of media on online platforms. But if you want to talk about successful VR, then we have to talk about the pornography industry… Wolfgang Brückle: Even if we accept the idea that the prosumer culture inspires, or indeed challenges, the big machinery of blockbuster production, I imagine that you all have some personal utopian visions, or at least aspirations, when working with 360-degree and VR tools in the run-up to creating a format. Obviously in your research and experiments, you are geared towards something that you may or may not call your own vision of a sufficiently complex narrative and visual expe rience. Is there anything that you particularly hope for, or something you’d like to see avoided in the way that the emergent tools are used? Johan Knattrup Jensen: The one thing that people in the industry talk about when discussing VR is whether this is the medium of the future or not, which then raises the question as to when it is ready to be the medium of the now. For me, it is very much a medium of the now. I don’t care about what it can do in the future. I care about what it can do now. How it can affect my audience now. How it can help me tell a certain kind of story that I couldn’t tell otherwise. The Doghouse would never have made it as a «flattie». Audiences would never have understood Ewa and her out-of-body experience if they hadn’t been able to actually become her. The Shared Individual would only have been an idea if we hadn’t gathered the whole audience in one person. The most important thing for me is not to under stand the medium, but to make the medium understand me and fit the medium to whatever story or whatever vision that I have inside me. Make the medium just that: a medium for expressing my feelings. It’s the medium that serves the mes sage, and not the other way around. Alia Sheikh: I want to leave things open for the future. I really don’t want people to tie down what’s possible, to decide that there is one specific format, and one specific kind of headset. I’ve worked with projections as well as headset experi ences and I’d love to see a future with no headsets. Things like smart contact lenses or little projectors sending images onto our eyeballs sound like science fiction, but the reality is that headsets do need to get lighter and more invisible to be usable. If you think about television, the basic design of the TV set hasn’t changed that much in terms of the way you experience it. It’s a screen, it’s removed from you, there are buttons, and you can change the channels. Inside the device, it’s very different – we’re not using cathode tubes anymore – but someone from the very early age of television would be able to identify a TV in a modern living room. That format hasn’t changed very much. With VR and immersive formats in general, however, there are numerous different ways to create an immersive experience. Neal Hartman: What I find particularly interesting about VR is that it has equally useful applications completely unrelated to art. I guess I’m going into the direc tion Marco already implied: as a tool for research, in therapy, treatment, training, tourism, and so on, there are a multitude of uses for VR that might involve a cre ative element, but aren’t cultural projects per se. VR is a ground-breaking tech nology in terms of examining a lot of areas of life and the world that have little to
78 ⁄ 79
do with what we consider artistic culture. Yet bringing culture and media into the same tool that you can use for research and education, and bringing everything to the same plane, offers an opportunity to foster a very multidisciplinary appre ciation of how we both entertain ourselves and learn about our world. Showing how all this is interconnected could be really beneficial to how people relate to the place that we give to these diverse activities in our lives. Alia Sheikh: Your comments bring us to augmented reality. We’ve been talking about going into a medium but still being a spectator and being removed from it. What if virtual reality was considered as real as, say, my dad texting me. I don’t see that as a text message on my phone; it’s my dad who wants to say something to me. It’s real communication, with a real person behind it. So, what happens when virtual spaces become so configurable and so ubiquitous that they’re actu ally amongst our real spaces? For example, one thing I really want is a virtual editing room where I can just throw clips around, stick them on the wall, and see lots of stuff all at once, because I can never have enough screens. I can’t make that in real life and I can’t make that in VR either, the resolution isn’t there. But you can imagine virtual workspaces, and once you start using VR as a tool to make stuff, even to make more VR, I think that the space stops feeling virtual. It’s just over there, in the same way that I don’t think of my dad texting me as my receiving a virtual electronic message, I think of it as my dad telling me something. I think the «virtualness» of VR will somehow reduce. It will just become more reality… My department is split between London and Manchester. In the London kitchen, they have a big screen that shows our kitchen, and in our kitchen we have a big screen that shows theirs. When you go past, sometimes you see someone you know and you wave. That has a validity to it that me putting on a headset and watching a cartoon cat jump around doesn’t. Why? Because I’m used to it and because they’re real people. Also, I’ve incorporated it as part of my real life, and it’s not just escapism. Maybe VR experiences feel like some virtual other removed thing because of what we’re using it for: for escapism instead of something that is incorporated into our lives. Marco de Mutiis: If we will look at the current practices of artists working with VR, I see a big trend that is really addressing this ambiguity between the physical world and the virtual world, in a way similar to how we perceived our online and physical realities and identities in the nineties, before social media and web 2.0 collapsed the boundaries between them. If we want to pursue the exercise of foreseeing, we could try to draw a parallel and see how we’re really concerned about the body and the virtual place. But perhaps, once we start to get comfort able with it, it would become possible to imagine virtual spaces that take over, and that become an integral part, an intertwined and indistinguishable part of reality, just like Facebook or texting is now.
Contributors
Wolfgang Brückle has been working as a lecturer in the fields of art history, photography, and cultural critique at the Lucerne University of Applied Arts and Sciences since 2013. He studied art history and letters at the universities of Marburg, Dijon, and Hamburg, where he was awarded a PhD in 2001. He worked as an assistant curator at the Staatsgalerie Stuttgart and as an assistant professor at the universities of Stuttgart, Bern, Essex, and Zurich. His fields of research include medieval art, art theory, museology, contemporary art, and the history of photography. He is currently leading a research project on postphotography at the Lucerne School of Art and Design. Marco de Mutiis holds a Master’s of Fine Arts from the School of Creative Media, City University of Hong Kong, where he continued to teach and research before starting work as a curator of digital collections at the Fotomuseum Winterthur, and, at the Lucerne School of Art and Design, as a teacher in the fields of physical computing, creative coding, and computational photography and video. He continues to practice as a media artist whose works are shown at international festivals and in galleries. In 2018 he began a PhD on in-game photography in the context of the post-photography research project at the Lucerne University of Applied Arts and Sciences. Gabriel Flückiger is an art historian and an artist. He studied art history and social anthropology at Bern, fine arts and photography in Zurich. He is a research assistant at Lucerne School of Art and Design and a guest lecturer at Zurich School of Arts and the Design School Bern/Biel. He has co-edited Bern 70, Bern 2017; Norbert Klassen – Warum applaudiert ihr nicht?, Bern 2015; and New Institutionalism, OnCurating (2014). His artistic practice deals with the conditions of stage and appearance as well as the visual representation of the body and labor.
Dieter Geissbühler (Prof., Dipl. Architekt ETH SIA BSA SWB) works in the fields of architecture, town planning, criticism and writing, teaching and research. He runs an architects’ practice in Lucerne with Gerlinde Venschott and teaches at the Lucerne School of Engineering and Architecture, where he has held a professorship since 2000. He is responsible for the MA course teaching on material technologies in architecture and is currently conducting research on materiality and architectural cultures as part of the Technology and Planning research group. Andrea Glauser teaches and researches at the sociological seminar of the University of Lucerne. She studied sociology, art history, philosophy, and political economy in Bern and New York. In 2008 she was awarded a PhD in Bern for her thesis on the interaction between cultural politics and artistic practice in the case of artist residencies. She gained her postdoctoral qualification at the University of Lucerne. She has been a visiting scholar at Columbia University in New York (2006, 2011–12) and at the Institut Français d’Urbanisme, Université Paris-Est (2010–11). Neal Hartman is director of the CineGlobe Film Festival at CERN (the European Particle Physics lab and home to the Large Hadron Collider) in Geneva, Switzerland. He also serves as director of production for TEDxCERN. He was the invited chairman of the World VR Forum in 2017 and is on the committee of the Geneva International Film Festival. Neal has been producing the 48-Hour Film Project in Switzerland since 2008 and is the director of Human Power, a documentary film on high-speed bicycles. He is currently producing a documentary on CERN shot entirely by the engineering staff in the underground caverns.
Hans Kaspar Hugentobler studied human-centered innovation (IIT Chicago), communication sciences (HdK Berlin), and information sciences (FU Berlin). He began work as an academic in 2003. Interested in innovation, organization, and design, and having gained teaching experience in Canada, China, Taiwan, and Germany, he started teaching the Strategic Design Planning course for the BA Design Management, International in 2007. He is currently also part of the program management team. He has been involved in a range of projects related to health and business development at the Design & Management research group. Catalina Jossen Cardozo is an industrial designer with Master’s degrees in product design and service design. Her work as a designer, entrepreneur, and businesswoman draws on her professional expertise in product design and marketing strategies. Her research on the furniture and footwear industries has led her to inquire into the role of service design as a tool for making the traditional supply chains of both industries more sustainable. She is the recipient of a 2017 SNF Bridge Proof-of-Concept Grant for her project «By María! – Building a Sustainable Designer–Shoemaker–Customer Network.» Besides this project, she brings unique insights from Colombia and Latin America to the Design & Management research group. Her project is also supported by the HSLU Smart-Up program.
Sabine Junginger holds a PhD in design and a Master’s in communication planning and information design (both Carnegie Mellon University). She is an internationally recognized expert on human-centered design in government and organizational change. In addition to heading the Design & Management research group, she is a Research Fellow of the Hertie School of Governance, Academic Advisor to the European Forum Alpbach, and member of the Research Committee of the Free University Bozen. Previous academic positions include lecturer and founding member of ImaginationLancaster at Lancaster University (UK) and Associate Professor at the Kolding School of Design, Denmark. Faith Kane is a design researcher and educator working in the area of textiles and materials. Her research interests include design for sustainability, collaborative working in the design/science space, and the role and value of craft knowledge within these contexts. She is a Senior Lecturer and the Programme Coordinator for Textiles at the School of Design, College of Creative Arts at Massey University in Wellington, New Zealand. She is also an editor of the Journal of Textile Design Research and Practice. Johan Knattrup Jensen graduated as a film director from the Copenhagen film school Super16 in 2012. His works have since been shown at most major film festivals, including Cannes, Locarno, IDFA, and New York Film Festival, as well as at biennials, in art galleries, and museums worldwide. He is the artistic director of Makropol, a progressive production studio in Copenhagen. His work spans cinema, installation, and performance and he is considered to be among the pioneers of cinematic virtual reality. His works include Skammekrogen (2014), EWA, Out of Body (2016), and, most recently, Anthropia (2017).
Florian Krautkrämer studied at Braunschweig University of Arts, where he wrote his PhD thesis on concepts of writing in film. He remained at Braunschweig as a postdoctoral researcher before accepting a guest professorship in film studies at the Johannes Gutenberg University in Mainz. In 2018 he became Head of Interdisciplinary Programs in Design and Arts at the Lucerne School of Art and Design, where he also teaches film studies in the Master’s program. His research interests include mobile media, digital film, post-cinematography, screen studies, documentary film, and experimental film. Vera Leisibach was one of the initiators of the self-organized cultural center Zollhaus Luzern and is active at the Tatort Garten. She is a member of the Kollektiv am Strand with peers Laura Bider and Corina Schaltegger. Her doctoral thesis, based on artistic activities, investigated long-term collaborative strategies in artist collectives and artist groups. She has taught on the Master’s in Art program at the Lucerne School of Art and Design since 2015, where she is also an assistant on the Art, Design & Public Spheres research group. www.veraleisibach.com
Christiane Luible is professor and head of Fashion & Technology at the Kunstuniversität Linz. Her main fields of interest are the 3D virtual simulation of fashion and the influence of digital media on fashion design. Luible has always been fascinated by clothing and technology. After training as a women’s tailor, she studied Fashion Design at the University of Pforzheim and the F.I.T. in New York. In 2000 she developed a virtual fashion show that received widespread attention and was awarded the Lucky Strike Junior Design Award. Having worked with various apparel companies, Luible continued on the path of technology in fashion, writing a PhD on the simulation of fabrics and clothes at the University of Geneva. Her main focus now is practice-led research in fashion design.
Bettina Minder is completing her doctoral dissertation at the University of Aalborg in Denmark. After studying graphic design, she completed a degree (Lic. phil. I) in Slavic Studies, Film Science, and Modern German Literature at the University of Zurich. Since 2007 she has been involved in design research at the Lucerne School of Art and Design. In 2017 she joined the Design & Management research group. She has extensive experience in working with interdisciplinary teams and has co-developed and co-led the interdisciplinary course program SocialLab. She is currently a member of the core team at CreaLabs, the future laboratory of HSLU.
Astrid Mody, architect MAA (2004) and PhD (2016). Astrid Mody’s core knowledge and experience is in architecture, textiles, and LED technology, research, and practice. Rachel Mader is an art historian. Since September 2012 she In her research she investigates textile has been director of the Art, Design & technology, techniques, properties, and aesthetics as inspiration and Public Spheres research group at the Lucerne School of Art and Design. methods for expanding the design Between 2009 and 2014 she directed space of LED technology within a project entitled «The Organization architecture. For more information on her PhD see https://kadk.dk/ of Contemporary Art: Structures, case/textilisation-light. Production and Narrative» at the Zurich School of Art. Her publications Isabel Rosa Müggler include radikal ambivalent, Zurich/ Zumstein (FH Textildesign, Berlin 2014; «How to move in/an MAS Digital Design & Management) institution», in: New Institutionalism, OnCurating (2014), www.on-curating. is a designer with a background in textiles. She is a researcher at org; and Kollektive Autorschaft in der the Lucerne University of Applied Kunst, Bern 2012. Sciences and Arts, where she also teaches on the MA in design. Her Sarah Merten read art history research interests include materiality at the University of Zurich. She has been a research assistant at the and the interfaces between design Lucerne School of Art and Design and technology, high-tech and since 2015 and is currently conductlow-tech. Her company, Tiger Liz ing PhD research under Professor Textiles, develops and produces soft Beatrice von Bismarck (Leipzig) in and functional materials for a variety the context of the SNF funded project of architectural applications. «Off OffOff Of? Swiss Cultural Policy and Self-Organization in the Arts since 1980.» She also works as a researcher at the Kunstmuseum Bern and regularly writes reviews and texts on contemporary art for various journals, monographs, and catalogues.
82 ⁄ 83
Pablo Müller is an art critic and an art historian. He works within the Art, Design & Public Spheres research group at the Lucerne School of Art and Design. He is currently at the University of Zurich conducting doctoral research on socially engaged art criticism in the examples of October, Texte zur Kunst, and Mute. He writes for Kunstbulletin, Neue Zürcher Zeitung, and WOZ Die Wochenzeitung. He co-edits Brand New Life, an online magazine for art criticism. Claudia Ramseier studied social anthropology, media, and communication sciences as well as religious studies at the University of Berne (Lic.phil.hist.). She has extensive experience working in cultural and tertiary education and has taught courses on scientific writing and film analysis at the Zurich University of Applied Arts. She joined Lucerne School of Art and Design as a scientific researcher in 2014 and has helped to develop new curricula for the BA program Design Management, International (DMI). Her current work includes research and teaching in addition to coordinating the project SwissGradNet and supporting the school’s development department in its projects. She has been contributing to Design & Management research group projects since 2017. Alia Sheikh is a filmmaker and senior development producer working in the BBC’s Research and Development Department, Man chester. Having formerly directed the Production Labs project, she now runs the department’s experimental filming projects. She has been investigating immersive video formats since her work on BBC R&D’s Surround Video system in 2010, and previously designed experiments to test a variety of immersive filming techniques, including for ultra-high frame rate and high dynamic range video capture. At present her work is focused on the research questions of how to effectively convey a narrative and unobtrusively direct attention in 360-degree video.
Peter Spillmann is an artist, curator, and lecturer. Co-founder of various self-organized platforms such as the Center for Post-Colonial Knowledge and Culture (2008), he develops thematic projects and exhibitions in alternating interdisciplinary contexts. He teaches and researches at the Lucerne School of Art and Design, where he is director of the Master’s in Art program with a focus on art in public spheres. His recent projects and publications include Destination Kultur (2012), www.transculturalmodernism.org (2012), mapping.postkolonial.net (2013), tricontinentale.net (2015), and Viet Nam Diskurs (2018). Dagmar Steffen holds a degree in product design (HfG Offenbach) and a PhD in design (University of Wuppertal). She has worked as an author, an exhibition curator, and as a freelance journalist for various professional journals. After several teaching assignments and research projects at German and Finnish universities, she joined Lucerne School of Art and Design in 2008. She teaches design semantics and theories of design. As a core member of the Design & Management research group, she has led and contributed to a range of research projects involving health applications and product development for businesses. She is the representative for Smart-Up, a support program for HSLU entrepreneurs. Fred Truniger works as a film historian, curator, and researcher at the Lucerne University of Applied Arts and Sciences. He holds an MA in film theory from the University of Zurich and a PhD from the Swiss Federal Institute of Technology, ETH Zurich. In the past he has directed the graduate studies program, led the Visual Narrative research group, and run the Ultrashort research project at the Lucerne University of Applied Arts and Sciences, where he is now Head of the MA Film program.
Axel Vogelsang leads the Visual Narrative research group and is a senior lecturer at the Lucerne School of Art and Design. He holds a PhD in design from Central Saint Martins College – University of the Arts London. Since 2010 the focus of his research has been the use of digital and visual media in cultural contexts, particularly museums. Axel Vogelsang is also a board member of the German Society for Design Theory and Research (DGTF). Andrea Weber Marin (Prof. Dr.) is head of the Product & Textile research group at the Lucerne School of Art and Design. She studied environmental sciences and did her PhD at the Federal Institute of Technology in Zurich. She worked in the textile industry for several years before becoming a lecturer and researcher at Lucerne University. Martin Wiedmer (Dipl. Arch. HTL) has been Vice Dean of the Lucerne School of Art and Design since 2012. He is in charge of its four research groups and six bachelor's and master's programs. From 2001 he expanded research at the Academy of Art and Design, University of Applied Sciences and Arts Northwestern Switzerland. Between 2005 and 2012 was director of its Institute of Art and Design Research. During this time he carried out multiple research and service projects in art and design, especially in the fields of interaction and design thinking. Martin Wiedmer is active on various research boards, juries, and conference committees.