Development tools and platforms
AU
AARHUS UNIVERSITY
2
Contents
3
CAVI – development tools and platforms
4
Interactive tables
6
Sound-O-Matic
8
Advanced 3D projection
10
Virtual studio
12
3D cinema and headmounted displays
14
Media architecture
16
BullsEye
16
DUL radio – sensors
17
Guard-I-aM
17
Camera tracking
18
Process reflection tool
18
Unity3D
19
ProxiMagic
19
Webstrates
3
CAVI – development tools and platforms The Centre for Advanced Visualisation and Interaction (CAVI) is a research centre at the School of Communication and Culture at Aarhus University, Denmark. CAVI was established in 2001, and focuses on interaction design, design processes, digital aesthetics and digital technologies – forming the basis of a range of cross-disciplinary research projects and collaborations with the business community and cultural life. Projects and collaborations constitute the framework for the design, development and production of experimental and permanent installations in which the use of new technologies and new ways of interacting are tested in the real world. In this catalogue we present a range of CAVI’s digital development tools and platforms, which are used to carry out tests, experiments and studies in the following fields: interaction design, 3D, digital experiences and digital aesthetics with regard to cultural heritage, media architecture, product presentations, digital creativity, design, digital art, play, learning and a wide range of other areas. The intention of the catalogue is to provide insight into and inspiration for collaboration with CAVI when it comes to specific research issues to which CAVI’s competences and technologies can contribute. In many cases, CAVI’s development tools and platforms
comprise an effective and innovative resource for research with an application-oriented element. Our development philosophy is that we base the development and production of new products systematically on the knowledge and technological platforms that we possess. We also seek to ensure that the systems we develop can be used as the basis of future projects. And thanks to this development philosophy, we now have an extensive portfolio of development tools and platforms with a high degree of utility. This catalogue contains a summary of CAVI’s development tools and platforms. Some of these tools are described independently, but can also be used in other tools. If you are interested in further information about CAVI’s resources and competences, or if you have any projects or applications to which CAVI can contribute, please feel free to contact us. Kim Halskov, centre director, halskov@cavi.au.dk, +45 2899 2251 Morten Lervig, production manager, lervig@cavi.au.dk, +45 2899 2100 www.cavi.au.dk
Foto: Jakob Halskov
5
CASE: RadarTable The RadarTable is an interactive social music installation that can be set up in public spaces on an interactive table with sound. The participants have two types of physical blocks at their disposal: cubes and flat cylinders. When a cube is placed on the table, a music loop is played that is linked to the tag (graphical pattern) which the user places face down on the table. The musical content is created so that all the music loops that can be played simultaneously match each other and therefore generate a consistent musical effect. The flat cylinders represent sound effects which affect the music loops in the vicinity.
Interactive tables
The design of the installation encourages several people to take part at the same time. The RadarTable brings people together and motivates them to learn simple functions by watching each other’s activities. The result is a surprisingly varied and cohesive musical effect. The users feel that they are playing instruments, but they don’t need any particular qualifications or skills to take part.
Interactive tables are horizontal interactive surfaces, often shaped like a table, on which one or more people can interact simultaneously with the digital content displayed on the screen. CAVI has developed its own interactive tables, as well as using interactive tables produced by other manufacturers, in interactive installations. CAVI’s interactive tables make fingertouch interaction possible as well as interaction based on tags placed on the bottom of physical objects. For this purpose, CAVI has developed its own standardsetting tag-tracking system, called BullsEye. In addition to these forms of interaction, CAVI has used advanced 3D projection onto physical interaction objects on the table surface, thereby making it possible to use these physical objects as displays.
www.cavi.au.dk/Radartable
www.cavi.au.dk/Interactivetables
7
Sound-O-Matic
CASE: The Greenhouses The Sound-O-Matic system was originally developed for use in the Greenhouses at the Botanical Garden in Aarhus, where more than 100 loudspeakers are placed in the beds and plants – adding to the authenticity of the experience. The sounds are played in real time, so the experience varies. A flock of birds are put to flight with wings beating through the tropical rainforest. The direction in which they fly varies based on random algorithms in the Sound-O-Matic system, so the flock never sounds exactly the same as last time and never takes the same route through the rainforest. www.cavi.au.dk/Vaeksthuslyd
The Sound-O-Matic is an advanced sound system for use in spatial, interactive installations. It can play sounds from several loudspeakers at the same time. You can position the speakers anywhere you like, depending on where you want the listener to hear the sound. The Sound-O-Matic is a sound engine in the sense that sounds which the user has stored in the system’s database can be played in a random set-up of speakers. The sound is controlled by the Unity3D program, which has a graphical 3D interface. You could say that Unity3D is the Sound-O-Matic’s graphical user interface. The ability to use a graphical 3D program (in this case Unity 3D) as the editor of a sound system is one of the special features of the Sound-O-Matic. The result is a uniquely graphical and spatial approach to the editing and production of sound. The Sound-O-Matic can also handle live sound input. The experience of a Sound-O-Matic system with a great number of speakers can be compared with a surround-sound setup in a cinema, but there are a number of differences. The spatial location of the speakers in the Sound-O-Matic system is variable. And the sounds that are produced (where and when) are decided by the users themselves in Unity3D. CAVI has developed a wireless version based on wi-fi. This means that it takes very little time to set up a number of wireless and battery-driven speakers in any spatial setup, and then use them to play spatial sound installations. www.cavi.au.dk/Sound-O-Matic
9
Advanced 3D projection
CASE: The terracotta warrior at Moesgaard A 1:1 copy of a warrior from the terracotta army was featured in a special exhibition at Moesgaard Museum called ‘The first emperor: China’s terracotta army’ in 2015. The warrior was made of unglazed terracotta, and his clothing was projected onto him in colours chosen by the public. Visitors to the museum were invited to choose the colours of individual items of clothing using a touch screen – enabling them to design the warrior’s clothing themselves. And when they had finished, the warrior joined the army projected onto the wall behind him. www.cavi.au.dk/TerracottaWarrior
Advanced 3D projection adds digital content to physical objects by projecting digital material onto them. At CAVI the technology involved has been used in various ways in design processes in which the visual appearance of physical objects (or scale models of them) can be changed by projection alone. This makes it possible to create illusions and simulate dynamic processes. You can work with both colours and patterns on the surface of objects, as well as working with light and reflections from simulated light sources and dynamic light. This makes it possible to examine a wide range of opportunities in the interaction between physical objects and digital material. CAVI has also used advanced 3D projection for various forms of communication, with projection onto physical objects making it possible to communicate the object’s narratives, properties or history. Advanced 3D projection is also used increasingly on buildings and other large objects in public spaces – either as an event in itself, or to support other events. CAVI’s tools for advanced 3D projection have extremely high precision, making it possible to achieve aesthetic and magical effects. www.cavi.au.dk/ProjectionMapping
11
CASE: 3XN Odenplan The virtual studio was used as a tool to design a media architecture for Odenplan, an underground station in Stockholm, in collaboration with architectural company 3XN. A virtual model of the prototype for the media architecture was created in Unity3D and then used to experiment and explore the potential for interaction in a workshop. The idea involved a large amphi staircase with built-in interactive light sources supporting a playful, experimental experience for the users. In the virtual studio it was possible to interact physically with the prototype and move the camera round the studio to explore the spatiality of the design. Testing the prototypes in the virtual studio made it possible to test various scenarios and discover new opportunities for interaction. Finally, a virtual video prototype was produced in the studio to present the concept. www.cavi.au.dk/Odenplan
Virtual studio The virtual studio is a form of video technology that makes it possible to combine physical objects with digital 3D objects in a virtual scene. Instead of constructing prototypes in wood and other materials, the scene can be created by using digital 3D objects. The virtual studio at CAVI is a blue studio with three walls, a floor and several objects in blue, with the movements and properties of the camera being registered by motion tracking. This makes it possible to move the camera and make recordings in real time, with physical objects (people, for instance) being combined with this virtual world. The virtual studio can be used as a tool for creating virtual video prototypes and conducting design workshops in which digital 3D worlds can be incorporated. www.cavi.au.dk/VirtualStudio
13
3D cinema and headmounted displays
CASE: The Virtual National Exhibition (1909) In 2009 there was a virtual reconstruction of the National Exhibition in 1909 at Tangkrogen in Aarhus, celebrating the centenary of this exhibition. The 3D model generated a historical universe which could be navigated by using a Wii remote in the 3D cinema, enabling users to move around a digital source representation of the 1909 exhibition. A virtual universe could be created using source material including the original hand-drawn architectural drafts. In other words, the model was not only a representation of the exhibition itself, but also a way of presenting these wonderful old drawings. www.cavi.au.dk/Aarhus1909
In the 3D cinema you can use 3D glasses to create an illusion of depth in the images you can see. The image shown may be created in a 3D engine such as Unity3D, in which it is possible to insert two cameras to simulate two human eyes. This kind of 3D visualisation can be interactive. In the 3D cinema you can interact in real time using a Wii remote, for instance. The 3D cinema gives you a joint experience with the rest of the audience and participants. CAVI also works with headmounted displays, which can only cope with one user at a time. They differ from the 3D cinema because their virtual universe surrounds the user entirely, generating a greater sense of presence in the digital world. Sensors make it possible to register the position and movements of your head, enabling the user to look around and move through a virtual universe. A controller is also connected to each hand, enabling the user to interact with their hands. www.cavi.au.dk/3Dcinema
15
CASE: EXPO 2010 Denmark Pavilion at the world exhibition in Shanghai Roughly 3,600 light sources were embedded into the facade of the Denmark Pavilion at EXPO 2010 in Shanghai, designed by BIG, an architectural company. The result was a media facade which ran around the entire circular structure. The unique form of the pavilion and the low resolution of the media facade (max. 9 pixels in height) made it necessary to find out what could be displayed and what the overall effect would be. A range of specially developed design tools made it possible to carry out the necessary tests and experiments before the building was constructed. The design tools were developed so the same software could be used to control the lights in the building in Shanghai.
Media architecture
www.cavi.au.dk/Expo2010
www.cavi.au.dk/Mediaarchitecture
The concept of media architecture covers the use of displays and light sources as an integrated part of buildings and urban spaces. At CAVI we have been working on various technological and aesthetic systems in this field, often focusing on user interaction. We can now manage the design and development process based on experiments, studies and tests, as well as managing the actual production of media architecture.
BullsEye BullsEye is a software system that makes it possible to track graphical markers and fingers on interactive tables. The position and movements of objects equipped with BullsEye markers are converted into a stream of real-time data, making it possible to use the objects as interaction tools. BullsEye is a system developed by CAVI that is similar to a commonly used system called ReacTIVision. But BullsEye offers more precise tracking. BullsEye can supply output for a common protocol for interactive tables (TUIO), and for WebSocket. www.cavi.au.dk/BullsEye
Incorporating DUL radios into old books converts them into interactive tools, with the movements and pressure of the public on the books helping to compose a poem on the poetry machine. The project was developed by CAVI in collaboration with Roskilde Library.
DUL radio – sensors The DUL radio is a compact, wireless sensor board which can achieve a range comparable to Bluetooth using radio communication. The DUL radio has a built-in accelerometer and gyroscope, equipping it to immediately register movements and transmit information about them. Other sensors and external units can also be connected. The DUL radio has been optimised to achieve an extremely low battery consumption. BullsEye tracking tags mounted on seven dice which are used as an interaction device to produce and control music on a RadarTable – an application for interactive tables.
www.cavi.au.dk/DULradio
Guard-I-aM The Guard-I-aM (GIM) is a control and monitoring unit used to monitor and control the start and shutdown of interactive installations via the internet. The GIM consists of a micro-computer and associated software. Using the GIM makes it possible to tailor make the required monitoring system. The operation of individual units in an installation can also be monitored. A camera and microphone can be connected to ensure that you do not only receive technical information, but can also monitor whether the installation works as it should for the users concerned.
The climate wall was an installation that projected climate-related statements onto the walls of Ridehuset in Aarhus. The movements of passers-by could place the words in sentences to make statements about the climate.
www.cavi.au.dk/Guard-I-aM
Camera tracking
The Guard-I-aM is a micro-computer which is mounted on the installation you want to monitor and control.
Camera tracking is used to extract information from a video stream from a camera. CAVI has developed and used camera tracking for a range of special purposes over the years. For instance to track a person’s location and movements with a view to provide input for interactive installations. Data from camera tracking can also be logged to find patterns in people’s movements in space and time. www.cavi.au.dk/CameraTracking
Process reflection tool The process reflection tool (PRT) is a blog-based tool that was developed originally to document and reflect on design processes, it can also be used as a general process documentation tool. The central element of the PRT is events, which are documented in a descriptive part and a reflection part. Events can be divided into sub-events with the same structure. In connection with both elements it is possible to upload and connect still images, videos and documents.
Illustration(s) from a smartphone and tablet opera by Signe Klejs and Niels Rønsholt: Breathless Moment. An interactive music/ drama experience developed and produced at CAVI using the Unity3D system. Available at the App Store and Google Play.
www.cavi.au.dk/PRT
Unity3D
The process reflection tool was used in connection with the design process preceding the production of the Zaha Hadid exhibition in Danish Architecture Center.
The Unity3D system is a commercial game engine which is developed and sold by a company called Unity Technologies. It is mentioned here because at CAVI we use the Unity3D system for a range of productions and platforms. It is a software system designed for the development and production of digital games. One of the core functionalities of the Unity3D system involves the use of 3D models as an environment. The combination of 3D worlds and real-time interaction is a fundamental feature in many of CAVI’s projects. The Unity3D system is a relatively open piece of software which makes it possible to develop its functionality to a considerable extent. At CAVI we have developed and constructed a great number of highly specialised Unity3D add-ons which we use in our development tools and projects. www.cavi.au.dk/Unity3D
ProxiMagic
A figure is edited with various tools, while updating the document in which it is inserted.
15
The ProxiMagic system is a web-based infrastructure which makes it possible to locate people who are logged onto a specific wi-fi network, thereby creating location-determined digital experiences and digital communication. The special feature is that a situation can be created in which you have to be in a specific physical location with your smartphone, tablet or computer to gain access to digital content. www.cavi.au.dk/ProxiMagic
Webstrates Visitors to the Local Area Artworks exhibition at Kunsthal Aarhus were able to comment on and communicate about the works on display. ProxiMagic ensured that their comments related only to the work in front of which they were standing.
The Webstrates system is a web-based software platform which makes it easy to develop software that also supports collaboration. The system can be used for all laptops, smartphones, tablets etc. and can be reprogrammed easily. The Webstrates platform forms the foundation for a range of experimental ways to use software – including software to support collaborative, creative work and software in which the gap between using and developing software is reduced. www.webstrates.net
AU
SCHOOL OF COMMUNICATION AND CULTURE AARHUS UNIVERSITY