Augmented Reality Magazine

AUG MEN TED REALITY Hochschule Neu-Ulm | University of Applied Sciences | Summer Term 2014 Multimedia Management and Production

CONTENT

1 2 3 4 5 6 7

2

Definition and History

p. 4

Usage and current Applications

p. 10

Devices and Google Glass

p. 18

Projects

p. 30

Augmented Reality in Future

p. 38

Sources

p. 44

About us

p. 50

EDI TO RIAL Mareike Pfennig

The purpose of this magazine is to give information about the topic ‘Augmented Reality‘ (AR). Therefore the term ‘Augmented Reality‘ is going to be defined and its background will be assumed. A quick excursion to the different operational areas is going to follow next.

Providing an insight into some current projects will give the reader a better understanding of the subject. Last but not least a forecast for upcoming AR technologies will complete our report and will bring down our magazine to a round figure.

3

DEFINITION & HISTORY Mareike Pfennig

4

DE FINI TION

“People now spend a better part of their life “jacked into” a digital world while (still) walking the real world.”1 Clyde DeSouza

Augmented Reality is a topic, which becomes more and more important, not just for industries but also for us. As the quotation above says, Augmented Reality is already an appreciable part of our life. But what is Augmented Reality and why is it becoming such an important part in people‘s life?

world either based on a real model or completely fabricated“3.

Three important characteristics that have to be presented to talk about proper Augmented Reality: 1. AR is a combination of real and virtual information 2. AR is an interaction in realtime 3. AR is operating and used in 3D environment

“Augmented Reality is taking digital or computer generated information, whether it be images, audio, video, and touch or haptic sensations and Therefore you can say, that Augoverlaying them over in a real-time mented Reality is a blend between environment.“2 the completely synthetic and the Augmented Reality allows the user completely real.4 Moreover AR is to see and experience the world not just one particular technology as it is, it only supplements reality that is used to give the user for exby superimposing or compositing ample information about the locainformation about the environ- tion he is currently at. Augmented ment the user is currently located Reality is a “combination of several at. It has to be clearly distinguished technologies that work together from Virtual Reality, which „is the to bring digital information into vicomplete immersion into a digital sual perception“5.

5

HIS TORY The beginning of Augmented Reality was in the late 50‘s.

From 1957 to 1962 Morton Heilig invented and developed a motorcycle simulator, which was called ‘Sensorama‘. The simulator was working with visual, acoustical, vibrating and odorous effects and gave the user the impression and feeling of riding a motorbike6.

6

In 1966 Ivan Sutherland created the first Augmented Reality head mounted display system.

Sutherland‘s head mounted display was very heavy, therefore it had to be suspended from the ceiling and the impressive appearance of the device inspired its name ‘The Sword of Damocles‘. The device displayed output from a computer

program in the binocular display as you can see on figures two and three. What point of view the user was shown by the system depended on the user‘s sight. To achieve this result, head tracking was necessary7.

1992 the first organization made use of Augmented Reality. The aircraft manufacturer ‘Boeing‘ used

Figure 1: Sensorama – The motorbike simulator

7

Figure 2: The Sword of Damocles

AR for cable installations within a plane. Its purpose was to localize the position of a cable and tell the user where the cable is supposed to be8.

In the same decade and at the beginning of the 21st century, AR is becoming an increasingly interesting topic for a lot of different scientists. Ronald Azuma defines in 1997 Augmented Reality as “combining real and virtual environment while being both registered in 3D and interactive in real time“9. Three years later, the first AR game, ARQuake, is developed.

8

And in 2008 Wikitude launched its first mobile AR software in the form of a Travel Guide, where people get through location-based systems (e.g. GPS or Wifi) additional information about the object, they are currently looking at10.

Today Augmented Reality is part of our daily life and is used in multiple sectors and branches. Advertisement, navigation and art are just some of them. But it is worth it to take a deeper look at some technological sectors, where AR is being used primarily.

Figure 3: Sword of Damocles – complete sight

9

USAGE & CURRENT APPLICATIONS Natalie Litzl

10

WIKI TUDE Figure 4: Scan from Wikitude World Browser App

After the definition of Augmented Reality, we want to present different ways of using AR on mobile devices.

The easiest way users can access AR implementations is with their smartphone. In order to use a service they need to download an Application which is able to process their geographical standpoint and the information their camera is scanning.

One App that can be used to gain more information about our surroundings is Wikitude.

Wikitude offers two main implementations: Wikitude World Browser und Wikitude Drive. Wikitude World Browser offers users information to their current surroundings to different main topics like accommodations, restaurants, sights and events. It is “most commonly used as a travel guide�11.

For example if they are looking for a place to sleep in a new city, they just scan around them and they will see possible options.

With Wikitude Drive users can turn their smartphone into an AR based navigation system12.

11

APPS FOR PRINT

12

There are also Apps that focus on print. Since the Internet is becoming the first news ressource for people the future of newspapers and magazines is unsure. Compared to the internet print is not as connecting and enriching.13 On-going research by Professor Soussan Djamasbi of the Worcester Polytechnic Institute shows that using AR in print has a “positive impact on engagement”14 while not disrupting attention on textual information. And engagement is what is important to keep readers and customers interested. With AR implementations embedded in

print video, audio, and imagery can be inserted into pages – more elegantly than without.

For companies there are mainly two ways of doing this: either creating a whole new AR App or using an existing platform.

One example for making an own App would be the German magazine ‘STERN‘. They did a special interest magazine on 9/11 in which they excessively rely on AR to illustrate the printed information further via videos, interviews and 3D-graphs. To view the AR content, a special App is needed (Figure 5 and 6).

Figure 5: Accessing the AR data in the STERN magazine

Figure 6: Scan QR-Code to see whole video about 9/11 AR edition of the STERN magazine

13

IKEA Another example: IKEA.They offer an App with which users can access additional information in their catalogues.

Figure 7:View of the IKEA AR App

14

LA YAR The second way would be to use an already existing and well-known software platform for example Layar.

Figure 8: Example on the Layar website to try the Layar App

15

If magazines want to use AR not only to offer additional content but to offer additional information for marketing purposes they often rely on Layar. Layar is a software platform that helps to embed AR content into print. To use it for a print piece it needs to be scanned and the additional information must be added with the Layar software. If the print piece is then scanned with the App again, the additional information appears.

There are several magazines that use Layar for example GRAZIA and net-a-porter. Both fashion magazines offer AR advertisements.

Magazines create revenue out of advertising. AR in an App that links to their website or TV spot creates additional engagement, awareness and traffic for them. It could be a new selling proposition for print advertisements.

16

Figure 9: Outcome after example is scanned with LAYAR App

17

DEVICES & GOOGLE GLASS Lilie Franz

18

DE VICES

Figure 10: AR-Devices

Augmented Reality can be used on all screens and connected devices: Smartphones, tablets, PC, TV and even connected glasses and lenses.

On smartphones and tablets Augmented Reality feels like a magic window. Therefore you need to download special Apps which are available on iPhone, iPad and Android devices.

If you want to explore Augmented Reality on a PC or a TV, you need a webcam.

The latest developed devices are glasses15. The most known glasses are the so called ‘Google Glass‘.

This article focuses on the Google Glass technology and its usage.

19

GO OGLE GLASS In April 2012 Google Glass was presented to the public the first time. The idea was born in the research project ‘Google Project Glass‘ which is lead by the Google founder Sergey Brin. Interesting is, that Google doesn´t call it ‘Google Glasses‘ but ‘Google Glass‘ – the singular form. It is due to the single glass prism which is used.

20

Picture11: 2:Sketches of Google Glass Figure

21

CON STR UCT ION

Google Glass is like a little smartphone which is installed within the right leg of the glasses.

It consists of a glass prism, camera, microphone, loudspeaker, touchpad, battery and a control unit with GPS function16.

The little glass prism is positioned above the human field of view, so it is not irritating, if Google Glass is not used at the moment.

touchpad

battery

(incl. GPS)

loudspeaker

Figure 12: Google Glass Construction

22

microphone

camera

glass prism

glass prism

projector

Figure 13: Display an overlay

The camera takes your existing environment and a projector displays an overlay of additional information about the things you are seeing17. In this way the real world merges with the virtual world.

Figure 14: Merge of real and virtual world

overlay reality

23

FUN CTIO NALI TY

Figure 15: Functionality

24

In order to use Google Glass there should be some prerequisites fulfilled.

You need a smartphone that uses the operating system Android or IOS18 and you have to download the App ‘MyGlass‘ which you need for connecting the smartphone with Google Glass. According to the connection, the device has all functions of the smartphone and the ability to use the Internet.

You operate Google Glass with head movements, voice or with the touchpad on the right leg.

To activate the display you have to nod your head in a 45 degree angle, with a tap on the touchpad or with the command “Okay, Glass…” . With Google Glass you can easily take pictures, listen to music, check your Facebook account or the weather, use Google

Maps, record a video or use the Google search engine. With direct commands like “Okay, Glass take a picture” or with the touchpad, you can navigate through the menu19.

An interesting App is ‘Word lens‘.

With this function, Google Glass translates printed words for example road signs from one language to another one, in real time20.

25

US AGE In the near future, Google wants to integrate its Glass into the workplace. Using Google Glass means having benefits. For instance, Google Glass could be used as a virtual assistant. Step by step instructions and illustrations are displayed in the prism and the user has the possibility to work with both hands. In this way, Google Glass enables new employees an easier start. They don´t have to learn complex processes by heart or need any support by

26

other employees. Furthermore, experts who are not on site could give a worker recommendations to a task via video sharing. That could be very useful for doctors in hospitals21.

In privacy, Google Glass can be used in all every day situations, just to add value in form of comfort to life. Together with the Italian eyewear manufacturer Luxottica (RayBan, Oakley) Google develops new modern designs for its product22.

Where benefits exist, there are always some challenges. The biggest one is probably the penetration of people´s privacy. With the camera it is possible to record people anytime and they are not able to notice that23. Thus, strict regulations exist in the USA. For example, people are not allowed to enter casinos, cinemas or night clubs with wearing Google Glass.

Figure 16: Modern Designs

27

FUT URE

28

At the moment Google Glass is in the beta stage and is sold in the USA for 1,100 EUR within the ‘Explorer Program‘24. In this program, people share their experiences with Google Glass and help to develop the device further. The official date of the market launch is not known. In the future it is expected that Google introduces contact lenses with micro camera and sensors embedded to the market, because they have already a patent on it25.

Figure 17: Google founder Sergey Brin with Google Glass

29

PROJECTS Julia Maiwald

30

VISU ALI ZA TION

Foto: The Advertiser

Figure 18: Office Technique in Real Estate Business

In the following, some projects are shown which get visualized with Augmented Reality technology.

There exist many companies and agencies for modern (marketing-) technology.They are offering visual solutions for companies in order to improve sales performance and

presentation in front of customers.

The real estate business sector is more and more using Augmented Reality technology to make projects and building plans visible – long before liquidation. It helps to understand and to imagine concepts in a very descriptive way26.

31

THE AD VER TISER

32

The first project that will be presented here is ‘The Advertiser‘.

It is a free-software project for all computer system providers in order to replace billboard advertisement with art in real-time.

Computers will be trained for recognizing individual advertisement and to replace it immediately by alternative content like images or video when viewed through the hand-held device27. It doesn’t matter whether the advertisement is on a building, in a magazine or on the side of a vehicle.

Figure 19: Advertisement substitution

33

This product-replacement idea was initiated by Julian Oliver in February 2008 and has been developed continually. It has appeared already in full working form in street exhibitions in Berlin, Brussels, Rotterdam and Helsinki. This new thinking of improved reality can be seen as a new platform and as an exhibition space for public art whereby pedestrians are invited to use the looking device to view an exhibition on the buildings of the city around them.

34

Currently this technology runs only on digital binoculars, which are owned by the team. But also Android devices and laptops with webcam function. It is possible for developers to download the code in order to help improving the software.

For the future The Advertiser targets following devices:

•

• •

Billboard Intercept Unit: This hand-held device offers highquality advertisement substitution and is the currently best performing AR App available. It is equipped with a wide-angle lens, fast CPU and GPU (Central and Graphics Processing Unit), powerful wireless adaptor, long-lasting battery and a solid state storage space.

A currently working version for the Android OS For standard phones with camera function: For the Symbian OS (developed by Nokia) The Advertiser is drafting up a version to provide ‘single shot‘ photo substitution28.

Figure 20: Billboard Intercept Unit

35

DSA GAME Online-Role-Playing Game ‘Das Schwarze Auge‘

36

Augmented Reality technology is also used for gaming – ‘sprylab‘ developed with their product ‘tripventure‘ an augmented reality game for smartphones in cooperation with ‘Chromatrix‘, a service provider for entertainment projects.

The location-based App is made for the biggest European roleplaying game DSA (‘Das Schwarze Auge‘) that exists since almost 30 years. It happens in a detailed designed fantasy world game area with middle age atmosphere.

For this particular App the game takes place in the Berliner Tier-

garten (zoo of Berlin). Via GPS the App turns the environment into a magical world and the user will be lead from the Brandenburger Tor deeper into the forest. Via Augmented Reality other magical characters are pictured in the smartphone screen at certain positions.

During the game the player has to fulfil tasks in interaction with the real environment and the surface of the game.This experience enables a new and innovative feeling of gaming. GPS, speed- and direction sensors are combined with handson content and activities29.

Figure 21: AR-Game DSA in Berlin

37

AUGMENTED REALITY IN FUTURE Tatjana Wiest

38

CUR RENT PROB LEMS

We are already pretty far in this AR technology. But especially the technology refrains us going ahead with Augmented Reality. At the moment the end devices are the main brakes. They have not the needed equipment.

The problem of the Augmented Reality technology: It consists of the three Dimensions (3D)30. 3D files are huge! Therefore the end devices require a big amount of

computing power. But for now, they don´t have it.

The main reason is that there is not enough public demand for this advanced Augmented Reality technology. For the simple and funny games, which exist already, the smartphones and tablets have enough computing power to handle it. But the prospective technology requires much higher computing capacity.

39

IM PROVE MENT Improving this, the producers have to integrate better processors, sensors and the Internet has to be much faster31.

1

The processors do the counting-work. The producers have to integrate processors with much better and higher efficiency. This will be the first step into the daily use of Augmented Reality.

40

2

Second important improvement in the end devices must be the sensors. They get the information from the environment of the user and transmit it into electric signals and send it to the software and the processor. For the Augmented Reality there must be sensors like for example the Structure Sensor32 which is produced for the iPad. It enables the end device to scan objects in 3D and build a digital manipulable model of a room.

Furthermore, it includes Infrared lasers, which allow seeing in the dark.

3

Third thing, what is independent of the end devices is the Internet speed. The better processors and the advanced sensors are useless without a fast Internet connection. The sensors get the information from the environment and the processors are responsible for the computing power. But from App´s – which are mainly dependent on Internet – they can compare the information and bring it together to the augmented picture on the smartphone.

At the moment, many Internet providers like Telekom, Vodafone and O2 are increasing and improving their electrical network to offer higher data volumes for ‘faster‘ Internet33.

Another interesting matter will be the user interfaces. They have to be very easily understandable. In the best case, the end devices will scan the environment much faster, and the user doesn´t have to do anything than holding the end device. Very often, the user has to select something or has to confirm facts, where the Augmented Reality software is not sure. But the plan of the innovators is that the App will do it all on its own.

Figure 22: Easy User Faces of the AR-Apps

41

IN DAILY LIFE

42

To get the project Augmented Reality started in the daily life, the known producers like Apple, Samsung, HTC and so on, need to understand the huge potential of the usage and demand of Augmented Reality.

They have to provide it and make Augmented Reality with all its advantages more known in the public. The people will learn to use it, what possibilities there are and finally Augmented Reality will be integrated in the daily life.

Figure 23: Possible integration in daily life

43

SOURCES

44

BIB LIOGR APY DEFINITION & HISTORY 1 Quotes Daddy (2014) Augmented Reality [Online]. Available from: <http://www.quotesdaddy.com/tag/ Augmented+Reality> [Accessed 23rd May 2014].

2,3,4,5,6 Kipper, G., Rampolla, J., (2012) Augmented Reality: An Emerging Technologies Guide to AR. Else vier. pp. 1,22, 3, 4,8.

7 Mehler-Bicher, A., ReiĂ&#x;, M., Steiger, L., (2011) Augmented Reality: Theorie und Praxis. Oldenbourg Verlag. pp. 13,8.

8 Inglobe Technologies Sr (n.Y.) Augmented Reality and the Future of Printing and Publishing [Online].

Available from: http://www.inglobetechnologies.com/docs/whitepapers/AR_printing_whitepaper_en.pdf [Accessed 17th June 2014].

9,10 Furth, B. (2011) Handbook of Augmented Reality. Springer. p. 5

USAGE & CURRENT APPS 11, 12 Kent, J. (2011) Augmented reality. Kent James. pp. 41,41 13, 14 Djamasbi, S. (2014) Augmented Reality and Print: Research-in-Progress [Online]. Available from: <http://aisel.aisnet.org/amcis2014/Posters/HumanComputerInteraction/6/> [Accessed 17th June 2014]. pp.1,8 45

DEVICES & GOOGLE GLASS 15 Callari, R. (n.Y.) Top Ten Augmented Reality Devices [Online]. Available from: <http://inventorspot.com/ articles/top_ten_augmented_reality_devices_32462#sthash.FrRBf1wX.dpuf > [Accessed 18th May 2014].

16, 19 Mangelmann, A. (n.Y.) Google Glass [Online]. Available from: <http://www.mygoogleglass.de/googleglass/> [Accessed 18th May 2014].

17 Dรถrner, R., Broll, W., Grimm, P., Jung, B. (2013) Virtual und Augmented Reality (VR/AR) [Online]. Berlin Heidelberg: Springer. p. 247 18 Donath, A. (2013) Google verbindet Glass und iPhone [Online]. Available from: <http://www.golem.de/ news/myglass-google-verbindet-glass-und-iphone-1312-103509.html > [Accessed 19th May 2014].

20 Google Inc. (n.Y.) What it does. [Online]. Available from: <http://www.google.com/glass/start/what-itdoes/> [Accessed 18th May 2014].

21 Schmitz, A (2013) Die Vorteile von Google Glass & Co. [Online]. Available from: <http://de.news-sap. com/2013/11/18/die-vorteile-von-google-glass-co/ > [Accessed 25th May 2014].

22 Frankfurter Allgemeine Zeitung GmbH (2014) Googles Datenbrillen sollen stilvoller werden. Frankfurter Allgemeine Zeitung [Online], 25 March 2014. Available from: <http://www.faz.net/aktuell/wirtschaft/ kooperation-mit-luxottica-googles-datenbrillen-sollen-stilvoller-werden-12862537.html> [Accessed 18th May 2014].

23 Grabmair, M. (2014) Google Glass: Zum Scheitern verurteilt. [Online]. Available from: <http://www.tech. de/news/google-glass-zum-scheitern-bestimmt.html> [Accessed 18th May 2014].

24 Google Inc. (n.Y.) The Glass Explorer Program. [Online]. Available from: <https://www.google.com/glass/ start/explorer-stories/> [Accessed 18th May 2014].

25 Trenholm, R (2014) After Google Glass, Google developing contact lens camera [Online]. Available from: <http://www.cnet.com/news/after-google-glass-google-developing-contact-lens-camera/ > [Accessed 25th May 2014].

PROJECTS 46

26 Schart, D. (2014) Wie Augmented-Reality-Anwendungen die Immobilien-Wirtschaft revolutionieren [Online]. Available from: <http://www.wearear.de/augmented-reality-anwendungen-immobilienwirtschaft/> [Accessed 27th April 2014].

27, 28 N.a. (n.Y.) the advertiser: improved reality [Online] Available from: <http://theartvertiser.com/> [Accessed 27th April 2014].

29 Kundler, W. (2014) Ingress Spielanleitung – Augmented Reality von Google [eBook]. Available from: <http://www.amazon.com/Ingress-Spielanleitung-Augmented-Reality-Google-ebook/dp/B00EMI4HG4/ref=s r_1_9?s=books&ie=UTF8&qid=1400140218&sr=1-9&keywords=augmented+reality+projects> [Accessed 23rd June 2014]. p.2

AUGMENTED REALITY IN FUTURE 30 Friedrich, W. (2004) ARVIKA Augmented Reality für Entwicklung, Produktion und Service [PDF]. Avaiable from <http://www.arvika.de/www/pdf/IVIP_Forum_d.pdf> [Accessed on 15th of June 2014].

31 Ambrus, M. and Lonau, Rainer (2014) Was ist Augmented Reality? [Online]. Available from < http://www. augmented-minds.com/de/erweiterte-realitaet-anwendung/was-ist-augmented-reality> [Accessed on 16th of June 2014].

32 Hoopes, H. (2013) Structure Sensor: 3D scanning and augmented reality for iPad [Online]. Available from:< http://www.gizmag.com/structure-sensor-3d-scanning-augmented-reality-ipad/29094/> [Accessed on 15th of June 2014].

33 Schöne, S. and Wildemann, M. E. (2014) LTE Netzausbau & Netzabdeckung inkl. Ausbaukarte [Online].

Available from < http://www.lte-anbieter.info/verfuegbarkeit/netzausbau-abdeckung-lte.php> [Accessed on 16th of June 2014].

47

LIST OF FIGU RES DEFINITION & HISTORY Figure 1: Heilig, M. (1960) Sensorama [Online]. Available from: <http://www.telepresence.org/sensorama/ images/sensorama-3.jpg> [Accessed 23rd May 2014].

Figures 2,3: Wordpress (2007) The Sword of Damocles [online]. Available from: <http://sr789.files.wordpress.com/2007/11/swordd.jpg> [Accessed on 23rd May 2014].

USAGE & CURRENT APPS Figures 4, 5, 6, 7, 8,9: Own photographies.

DEVICES & GOOGLE GLASS Figure 10: Le Saloon (n.Y.) Augmented Reality Devices [Online]. Available from: <http://augmentedev.com/ augmented-reality/> [Accessed 20th May 2014].

Figure 11: QWRT (n.Y) no title [Online]. Available from: <http://images.qwrt.ru/images/2014/01/28/Google%20Glass-862116.jpg> [Accessed 24th June 2014].

Figures 12, 13, 14 (modified): cf. Duplicon (2013) Die Google-Brille (Infografik) [Online]. Available from: <http://www.brillen-sehhilfen.de/google-brille/> [Accessed 21st June 2014].

48

Figure 15: Timedoll (2014) Google Glass View Wallpaper [Online]. Available from: <http://timedoll.com/ google-glass-view-wallpaper/> [Accessed 21st June 2014].

Figure 16: SWIDE S.r.l. (2014) Luxottica to put the chic into geek for Google Glass [Online]. Available from: <http://www.swide.com/art-culture/google-glass-partners-with-luxottica-to-design-new-versions/2014/03/25> [Accessed 21st June 2014].

Figure 17: TWT Interactive GmbH (2014) Neue Partner f端r Google Glass [Online]. Available from: <http:// www.digitalnext.de/neue-partner-fuer-google-glass/> [Accessed 21st June 2014].

PROJECTS Figure 18: Schart, D. (2014) Wie Augmented-Reality-Anwendungen die Immobilien-Wirtschaft revolutionieren [Online]. Available from: <http://www.wearear.de/augmented-reality-anwendungen-immobilienwirtschaft/> [Accessed 27th April 2014].

Figure 19: N.a. (n.Y.) the advertiser: improved reality [Online] Available from: <http://theartvertiser.com/> [Accessed 27th April 2014].

Figure 20:Youtube DE, (2010) MFF2010: Artvertising Brussels - Brussels 2010 [online], Available from: <http://www.youtube.com/watch?v=epKqR9cQfl8#t=103> [Accessed 23rd June 2014].

Figure 21: sprylab technologies GmbH (2014) Das Schwarze Auge als Augmented Reality App [online], Available from: <http://sprylab.com/de/projekte/das-schwarze-auge-als-augmented-reality-app> [Accessed 23rd June 2014].

AUGMENTED REALITY IN FUTURE Figure 22: Google+ (2014) Augmented Reality [Online]. Available from: <https://plus.google.com/+solopress/ posts>[Accessed 16th of June 2014].

Figure 23: Jaunted (2014) Augmented Reality Apps for Travel Attempt a Comeback [Online]. Available from: <http://www.jaunted.com/tag/Augmented%20Reality> [Accessed on 16th of June 2014].

49

ABOUT US

50

Authors: Mareike Pfennig Natalie Litzl Lilie Franz Julia Maiwald Tatjana Wiest

Design: Julia Maiwald Lilie Franz

51

Hochschule Neu-Ulm University of Applied Sciences Summer Term 2014 Multimedia Management and Production Augmented Reality