Mobile Based Augmented Reality

MOBILE BASED AUGMENTED REALITY WHITE PAPER

Author: Arvind Chikkalavalasa

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

Abstract Augmented reality adds information and meaning to a real object or place.

Unlike virtual reality, augmented reality does not create a simulated reality. This paper surveys the field of Mobile based Augmented Reality. It describes the manufacturing applications that have been explored. Different types of augmented reality applications are also covered. The paper also describes the characteristics of Mobile based Augmented Reality systems. Development platform for developing such applications is discussed. The challenges in developing such applications are also explained.

ďƒ“ Wipro Technologies

Page 2 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

Table of Contents

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

INTRODUCTION ........................................................................................................................................... 4 HISTORY .................................................................................................................................................... 4 DEFINITION ................................................................................................................................................ 4 TYPES OF AUGMENTED REALITY...................................................................................................................... 4 APPLICATIONS OF AR IN MANUFACTURING ...................................................................................................... 5 MOBILE BASED AUGMENTED REALITY .............................................................................................................. 6 DEVELOPMENT PLATFORM ............................................................................................................................ 6 JUNAIO : APPLICATION FRAMEWORK ............................................................................................................... 7 CHALLENGES OF MOBILE BASED AR ................................................................................................................ 9 APPENDIX............................................................................................................................................ 9 REFERENCES ..........................................................................................................................................10

 Wipro Technologies

Page 3 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

1. INTRODUCTION As computers increase in power and decrease in size, new mobile, wearable, and pervasive computing applications are rapidly becoming feasible, providing people access to online resources always and everywhere. This new flexibility makes possible new kind of applications that exploit the person's surrounding context. Augmented reality (AR) presents a particularly powerful user interface (UI) to context-aware computing environments. AR systems integrate virtual information into a person's physical environment so that he or she will perceive that information as existing in their surroundings. Mobile augmented reality systems (MARS) provide this service without constraining the individual’s whereabouts to a specially equipped area. Ideally, they work virtually anywhere, adding a palpable layer of information to any environment whenever desired.

2. HISTORY Augmented Reality has evolved from Virtual Reality. Virtual Reality is a completely computer generated environment. Augmented Reality in contrast adds computer generated information to the real world.

3. DEFINITION Augmented reality is related to the concept of virtual reality (VR). VR attempts to create an artificial world that a person can experience and explore interactively, predominantly through his or her sense of vision, but also via audio, tactile, and other forms of feedback. AR also brings about an interactive experience, but aims to supplement the real world, rather than creating an entirely artificial environment.

4. TYPES OF AUGMENTED REALITY a.

b. c.

d.

Surface: The most understandable form of reality that is augmented. The input would be screen, floors, walls, etc that respond to the touch providing virtual real time information or collaboration.

Pattern: The AR system performs simple pattern recognition on a shape, marker or face and replaces it with a static or moving element.

Outline: It is where hand, body or eye outline is picked up and seamlessly merged with the virtual object e.g, pick up a 3D object which doesn’t exist because the system is tracking the hand outline.

Location: Based on detailed GPS or triangulation location and position/view of the camera/device, the AR system overlays

 Wipro Technologies

Page 4 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

information precisely over buildings and people as we move through real space.

We will discuss about Pattern based Augmented Reality in this paper.

5. APPLICATIONS OF AR IN MANUFACTURING

a. Design and Prototyping  Checking of assembly space and possible collisions between different components.  Packaging analysis of a power train at the front wheel.  Design, Aesthetics and usability validation.  Examination of quantity and positions of welding points and bolts. b. Production and Shop Floor Management    

AR can highlight a dangerous area around a virtual fire.

AR based Assembly Line management and machinery space allocation.

Visualization of virtual robot and machinery in plant-environment.

Viewing maintenance records for a machine.

c. Supply Chain and Distribution  AR based order picking for material and spare parts.  AR based drop ship route planning, distribution tracking and performance.  Trucker’s route assistance and Delivery details (Item details, Quantity, etc) d. Usage and Training  BMW’s AR glasses, which assist mechanics in performing maintenance.  User training for assembly in case of DIY products, machinery setup guide, trouble shooting help using AR e. EOL Management  Layar based AR mapping of Recycling facilities.  Information related to pickup timings.

 Wipro Technologies

Page 5 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

6. MOBILE BASED AUGMENTED REALITY With the advent of IPhone and Android operating system, Augmented Reality applications can be created on mobile devices having these platforms.

Design aspects of Mobile based AR: a. Tracking the location of user device: For tracking solutions like Hybrid Tracking and Sensor Net Tracking have been used. b. Deciding where the calculations need to be done for combining the virtual and real worlds: Computations were earlier done on a remote server.

However with the increased capability of mobile devices, most of the computation is done on the mobile device itself.

Mobile based AR systems can be classified on the basis of tracking mechanism as follows: a. Location based: The tracking of the input happens on the basis of latitude and longitude of the location. Layar is one of the popular applications supporting Location based AR on mobile devices. b. Image based: The input would be tracked with an image. The virtual content mapped to the input image would be rendered on the mobile AR browser.

Fig 1: Mobile Augmented Reality

7. DEVELOPMENT PLATFORM We will be discussing the development of image based AR applications in the following topics.

The development platform consists of the following: Hardware: 1. A smart phone with the below configuration is required: a. IPhone: It should at least have iOS4. b. Android based: HTC Desire or Motorola Tablet version. ďƒ“ Wipro Technologies

Page 6 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

c. Built in Camera with at least 3MP. d. Touch screen. e. Connected to network with a good bandwidth.

2. A server machine a. The server machine should be accessible on internet.

Software: 1. Junaio 1.5 browser should be installed on the mobile device.

2. Apache server should be installed on the server machine.

3. Zend Framework should be installed on the server machine.

8. JUNAIO : APPLICATION FRAMEWORK Junaio is one of the development platforms for creating mobile based AR applications. Junaio provides an AR browser which needs to be installed on IPhone or Android mobile.

Junaio framework supports both location-based and image-based AR. The image based AR is referred to as Junaio Glue.

Junaio framework helps in achieving the following objectives with an image based tracking: 1. The input being an image, the output can be rendered as a 2D image.

2. The input being an image, the output can be a 3D content.

3. The input being an image, the output can be an audio or video file.

ďƒ“ Wipro Technologies

Page 7 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

Fig 2: Junaio Architecture How Junaio Works: 1. Channel: As a first step, a channel needs to be created on Junaio server.

2. Point of Interest (poi): Every input is considered as a point of interest. For such POI, virtual content could be a 2D, 3D or audio/video file.

3. Callback URL: The channel created above will contain the URL of the server where the virtual content resides.

The request for POI is sent from mobile device to the Junaio server which is in turn forwarded to the external server. The virtual content is routed back to the mobile device via the Junaio server.

Steps to develop an AR application: 1. The application can be developed using PHP programming language.

2. An account needs to be created at junaio.com. An API key is provided. This key will be used for authentication.

3. Register a channel: When creating a channel, channel type needs to be selected either as location based or image based (junaio glue).

4. Create a Tracking file: A tracking XML file has to be created for the input image.

5. Provide the API key in the application configuration file.

6. Create Callback API: A POI file needs to be created which contains the tracking file path and poi. The POI is identified with it’s id and contains the reference to virtual content. ďƒ“ Wipro Technologies

Page 8 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

7. Publish the POI on the external server.

8. Provide the site url as callback URL in the channel created above.

9. Validate the channel.

Run the Application: 1. Open Junaio browser on the mobile device.

2. Login to your Junaio account.

3. Click on Browse Channels.

4. Select your channel.

5. Point your mobile towards the input image for which the tracking XML was created.

6. Virtual content mapped to this image will be rendered on the mobile browser.

9. CHALLENGES OF MOBILE BASED AR a. High network bandwidth on mobile devices is required. This is possible with 3G technology which is still not so easily available in India. b. The virtual content needs to be always prepared and updated on the server.

10. APPENDIX

<?php echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?> <results trackingurl=\"http://www.junaio.com/publisherDownload/tutorial/ tracking_tutorial.xml_enc\"> <poi id=\"1\" interactionfeedback=\"none\"><name><![CDATA[metaio Man]]></name> <author><![CDATA[metaio]]></author> <translation>0.0,0.0,0.0</translation> <o>0.0,0.0,0.0</o> <mime-type>model/md2</mime-type> <mainresource><![CDATA[http://www.junaio.com/publisherDownload/ tutorial/metaioman.md2_enc]]></mainresource> <thumbnail>http://www.junaio.com/publisherDownload/tutorial/ic on.jpg</thumbnail> <icon>http://www.junaio.com/publisherDownload/tutorial/icon.jp g</icon> <route>false</route> <force3d>true</force3d>

ď&#x192;&#x201C; Wipro Technologies

Page 9 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

<s>1</s> <behaviours> <behaviour type=\"click\"><length>6</length><node_id>close_up</nod e_id> </behaviour> <behaviour type=\"idle\"><length>0</length><node_id>idle</node_id> </behaviour> </behaviours> <customizations/> <resources><resource>http://www.junaio.com/publisherDownload/t utorial/metaioman.png</resource></resources> </poi> </results>";

?>

11. REFERENCES Mobile Based Augmented Reality http://www1.cse.wustl.edu/~jain/cse574-10/ftp/reality/index.html 2. Mobile Based Augmented Reality http://graphics.cs.columbia.edu/projects/mars/mars.html 3. Junaio http://www.junaio.com 1.

ď&#x192;&#x201C; Wipro Technologies

Page 10 of 11

WHITE PAPER

MOBILE BASED AUGMENTED REALITY

About the Authors Arvind Chikkalavalasa is a Module Lead working for JSSTK project in Boeing Account.

About Wipro Technologies Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Company globally. Wipro provides comprehensive IT solutions and services (including systems integration, IS outsourcing, package implementation, software application development and maintenance) and Research & Development services (hardware and software design, development and implementation) to corporations globally.

Wipro's unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringent Quality Processes of SEI and Six Sigma. Š Copyright 2002. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission from Wipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are the property of their respective owners. Specifications subject to change without notice.

ď&#x192;&#x201C; Wipro Technologies

Page 11 of 11