Augmented Reality
-1-
PUBLICATION Journal
Lee, I. J., Chen, C. H., & Chang, K. P. (2016). Augmented reality technology combined with three-dimensional holography to train the mental rotation ability of older adults. Computers in Human Behavior, 65, 488-500.
Conference Chang, K. P., & Chen, C. H. (2015, August). Design of the Augmented Reality Based Training System to Promote Spatial Visualization Ability for Older Adults. In International Conference on Virtual, Augmented and Mixed Reality (pp. 3-12). Springer International Publishing. Chang, K. P., Huang, Y. W., Hsueh, S. Y., Chen, Y. T., Huang, S. N., Chen, C. H., & Chien, S. F. (2014, October). Hidden lion: a location based app game of sword lion searching. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play (pp. 323-326). ACM. LANGUAGE TOEFL iBT
101 ( R26 S21 R27 W27 )
SKILL & EXPERTISE Interaction
Unity / Arduino / Processing
Rendering
Solidworks / Keyshot / AutoCAD
Visual Design
Adobe Photoshop / Illustrator / Indesign
Programming
Flash/ C / C # / html5 / actionscript
Administration
Microsoft Office / SPSS
KUOPING CHANG RESEARCH INTEREST
interaction and industrial designerr ckuoping@gmail.com +886 921 649 041
Augmented Reality / Human Computer Interaction / Interaction Design / 3D Printing / Communication Design / Web Design / User Interface Design / Product Design / Ergonomics / Human factors / Emotional Design EDUCATION Scholarship Program
Design for Interaction, TU delft, the Netherlands, 2015-2016
Master
Ergonomics & Interaction Design, NCKU, Taiwan, 2016
Bachelor
Industrial Design, NCKU, Taiwan, 2013
WORKING EXPERIENCE Administrator National Taiwan Museum of Fine Arts, Taiwan, 20163D drafter
Ninomae Creative Technology Corp., Taiwan, 2016
Author
Oranje Express, 2015-2016
Research Assistant
Develop the App to Maintain the Spatial Visualization Ability for Elderly by Using Augmented Reality Technology, MOST, 2013-2015
Instructor
Augmented Reality Project, NCKU Ergolab, 2016 Design Sketching, NCKU Design Camp, 2013
Intern
Dot-IDesign, Taiwan, 2012 U10 Product design, Taiwan, 2012
Project
Commercial Catalogue Design, AIOS tech, Taiwan, 2014 Technology Design for SCiB battery, TOSHIBA, Japan, 2013 GUI Design, ITRI, Taiwan, 2012 Product Design, TAYOHYA Gourmet, China, 2012
INDEX
Augmented Reality Goodlock
3D hologram, leap motion, mental rotation, elderly
ARotation
manual control, app, spatial visualization, elderly
APP Design Hidden Lion
iBeacon, location based service, sword lion, tourist
Arduino Happy Scanner Night Owl
RFID, processing, happiness, carbon footprint
Adobe Flash, human computer interaction, assistant
3D Modeling Ready to Bake
contextual inquiry, baking, mental disabled
Part.1
Augmented Reality
For the video: https://goo.gl/b1Bmb9
Goodlock
3D Hologram Based Mental Rotation Training Tool
-6-
Augmented Reality
Goodlock
3D Hologram Based Mental Rotation Training Tool for the Elderly
Master Thesis Design, Accomplished in the 2nd year of Graduate School of Industrial Design, NCKU
Mental rotation, one of cognition abilities, is related with a sense of spatial direction and perception. However, the decline of this ability results in poor spatial awareness on the elderly, further influencing their lives. Goodlock, which applies 3D hologram technology, is designed as a mental rotation training tool for the elderly. The use of augmented reality(AR) and leap motion of Goodlock could enhance the training effect and trainers' interests and thus improve the sense of spatial direction of the elderly. Also, compared with conventional mental rotation stimuli which are abstract, the training content of Goodlock, which is inserting keys into different lock holes, is more interesting and easily understandable. Goodlock is implemented in public spaces such as senior citizen centers to provide another training option rather than physical fitness for elderly to use and play.
-7-
Goodlock
Decline of
Mental Rotation Ability
Mental Rotation Ability
a cognition ability of spatial psychology Mental rotation, one of spatial cognition abilities, is defined as an ability to mentally rotate a 2D or 3D object in an imaginative space. In a more professional term, it refers to the mental manipulation of spatial information to determine how a given spatial configuration appears if they are being transformed[1]. Human beings are endowed with this ability to help orient themselves in physical space and navigate from place to place. For instance, map reading, way finding and direction judgement are all involved with the use of mental rotation ability [2][3][4]. Hence, mental rotation ability has a significant influence in one's daily life.
mentally rotate an object in space
-8-
Augmented Reality lateral prefrontal cortex
However, mental rotation ability, just as most cognition abilities, will also decline with human's increasing age. This fact has been corroborated by numerous neuroscience studies. Under computerized tomography, lateral prefrontal cortex and hippocampus volume, which have a stake in cognition perception and the ability of processing spatial information, reveal a significant decreasing pattern on older adults[5]. Also, older adults tend to show a low white matter integrity in their brain, suggesting that they have poor performance on spatial memory and attention [6].
hippocampus
Therefore, it might be tough for the elderly to mentally comprehend what the objects will be after they are spatially transformed. Since mental rotation ability has so much to do with one's daily life, the decline of it will result in the poor sense of direction and environmental psychology, causing the elderly people to have spatial problems and low spatial awareness in real life [7][8][9].
-9-
Goodlock
motivation one:
Weaknesses of Current Training In fact, the decline of mental rotation can be delayed by undergoing considerable mental rotation training[10]. Usually conducted in 2D presentation modes, this training requires participants to judge whether two stimuli are the same or mirrored [11], and then write down their answers on paper. Yet, this test-like training might be inappropriate for the elderly due to the following reasons:
cognitive load spatial training in 2D m e d u i m g e n e ra t e s mental load.
interaction
there is no much sensory interaction involved in the training.
paper test
this training is testing rather than gaming.
boringness
the training, similar to testing, makes trainers boring.
-10-
Augmented Reality
motivation two:
Brain Fitness
No in Senior Citizen Center Although many community and senior citizen centers have been established and serve as well-organized places for the local elderly to socialize and exercise, few cognitive training facilities are provided. The physical fitness training is still the main service in senior citizen centers; for instance, treadmills, stationary bikes and steppers are the most common equipments. Since cognition and mental rotation
ability are also important to the life of the elderly, to design and implement a mental rotation training system in the senior citizen centers is essential and innovative. In this light, the elderly will have more opportunities to take cognition training in usual time, and gradually prevent the cognitive and mental rotation from declining in the future.
-11-
Goodlock
Augmented Reality Effect 3D hologram is a projection system which displays virtual images inside its pyramid-like glass structure. The virtual images produced by reflection and overlapped in real space are vivid and dynamic as if they completely exist in real world, creating an augmented reality(AR) visual effect. The AR effect presents correct spatial information and promotes a concept of visualization which is important to mental rotation ability[12]. Moreover, AR can effectively reduce the cognitive load and help facilitate trainers' learning interests in the training[13].
motivation three:
Strengths
of 3D Hologram
-12-
Augmented Reality
Direct Manipulation In some 3D hologram projects, viewers are allowed to manipulate the virutal images through the use of a gesture sensor-Leap Motion. The gesture control of direct manipulation easily engages the viewers during interaction and promotes a great immersion for the viewers. Also, the use of gesture is proved to facilitate mental rotation process since trainers can easily predict how the mental presentations look after they are rotated and helps reduce the cognitive load produced from mental rotation process [14][15].
360 Perspective With the strength of 360 degree perspective of 3D hologram, viewers can observe the images from any angle rather than from only the front side of it. This advantage allows not only the front viewers but everyone to enjoy the visual effect. Therefore, 3D hologram will be a good installation settled in public spaces like senior citizen centers.
-13-
Goodlock
Goal of Goodlock x 3D Hologram x manipulation x gamification x senior center
The goal is to integrate 3D hologram, augmented reality, gamification and manipulation into one system which will be implemented in senior centers as a mental rotaiton training device. In this way, senior citizens have other training options rather than physical fitness and thus the decline of spatial cogitition can be delayed successfully.
-14-
Augmented Reality The first part of the research was to analyze gestures older adults used when confronting mental rotation te st s . T h e ge st u re s were examined by guessability theory [16] and the most representative ones were selected. These representative gestures were applied in the final system as the input for direct manipulations.
-15-
Goodlock
The second part of the research was to generate new training content based on the gestures defined in the first part of the research. A design workshop was held and five interaction designers were invited in an effort to contribute the ideas of training games. Final outcome, which was scored as the highest by the designers, was called Goodlock, which was a set of key and lock. Players rotate the key in a correct angle to make it pass through the hole on the lock. The key-lock components in Goodlock are different from t ra d i t i o n a l m e n ta l ro ta i o n stimuli and thus can be easily apprehended by older adults.
-16-
Augmented Reality
Solidworks was used to construct the form of the key and lock, and 3ds MAX was used to attach textures and color to the models. To differentiate the levels by difficulties, three types of key: iron keys, bronze keys and gold keys were made. The more expensive material the key was, the more complicated the shape on the key would be.
Gesture control was done by the use of leap motion. Three types of control were implemented: Rotation, which was to rotate one's palm around his wrist; Translation, which was to move one's hand with hand open; Standby, which was to make a fist to stop any motion of the key.
-17-
Goodlock
The game was implemented in Unity3D, a game development engine by programming in C#. The game rule was to make the key pass through the correct hole to touch the first collider and further touch the second collider to move into next level.
The split screen was made in order to create 360 degree perspectives in the pyramid hologram. It was achieved by creating cameras in one scene. So viewers were able to see three perspectives simultaneously in the screen canvas.
-18-
Augmented Reality
The biggest challenge for hardware implementation was to create the pyramid form. The requirement of a perfect pyramid is to make each side and plane lie at an angle of 45 degree. This depended on accurate geometry calculation and prototype testing. After the form was done, cover th e pyrami d wi th s em i transparent film to improve the reflection. The figure shown below is the steps of creating the standard threesided pyramid.
-19-
Goodlock
Final refinement of the device was done by fortifying its intensity, polishing the pyramid, and painting the supportive structure in black. Painting the structure in black was a way to prevent itself from being reflected inside the pyramid. The reflection of unnecessary images rather than the game would disturb the view and distract the viewers. By doing the refinement, the device looked more in a unity; also, this could improve the quality of reflection and thus made the augmented images more vivid and real.
A supportive structure made from wood and stainless steel was used as a platform to demonstrate the pyramid and also for the convenience of transporting.
-20-
Augmented Reality
One strength of 3D hologram is the augmented effect, which, by the study of cognition science, is beneficial to the understanding of spatial relationship. Another strength is 360 degree perspectives. It makes the game playing not limit to only the players right in front of the device but open to everyone around the device. The game of Goodlock is more appealing to older adults since it is tranformed with a more contextual and interesting rules other than traditional mental rotation stimuli.
3D hologram
Mental Rotation
Training System
-21-
Goodlock
-22-
Augmented Reality
Game Scenes Scene 1-2 Opening scene of Goodlock. Three types of key (iron, bronze and gold) indicting three levels (novice, medium and advance) would be presented in this scene to inform players of level difference. In addition, a practice session was included in this scene allowing players to try gesture control.
Scene 3-8 Different scenes in iron key level. First, players needed to reply the hole on the lock they think matched the shape of the key (scene 4). Then, they were able to insert the key into the hole they answered. If the key passed through the lock, a green correct circle would appear on the lock; o t h e r w i s e t h e key wo u l d b e blocked by the lock until the player rotated the key to the correct angle to pass it through the lock.
Scene 9-10 The medium level (bronze key) and the advance level (gold key). Basically the game rules were the same in these two levels, but the only difference was the complexity the key was. A more complex key meant more corners on the shape, so playsers might take more effort distinguishing the same shape or the mirrored shape.
-23-
Goodlock
Evaluation
pretest & posttest design To test the training effect and older adults' preference on Goodlock, an evaluation of pretestposttest design (A-B-A design) was set up. In a total of 14 older adults with average age of 67.21 were recrutied. During the intervention phase, they received Goodlock training 12 times within 1.5 months. During the pretest and posttest phase, they were given mental rotation paper tests for 8 times. In each time, the error rate was recorded so the result of training effect would be clear if we compared the mean error rate of pretest and posttest. In the setting, participants sit in front of the 3D hologram to play with Goodlock. A researcher also sit beside the participant to observe the whole playing process.
-24-
Augmented Reality
Outcome The paper test consisted of 2D and 3D mental rotation tasks. The error rates of the two types of tasks were recorded individually. As seen from the figure below, the mean error rate of 2D mental rotation tasks decreased significantly in the posttest. The error rate of most 3D tasks also
decreased. The improvement of their performance showed a remarkable training effect of Goodlock. In addition, the participants enjoyed the play of Goodlock and replied a postivie feedbacl toward this training device.
-25-
Goodlock
Vision in the future Create a Community Network for the Elderly to do Brain Fitness Nowadays, senior centers play an important role for older adults. They serve as a friendly place for older adults to study, exercise and socialize. The installation of Goodlock in senior centers will enrich the value of senior centers because it is an innovative cognition training system which utilizes 3D hologram technology, augmented reality and motion sensor. It is expected that more and more centers can install Goodlock so that older adults have a new training option of "brain fitness" other than typical physical fitness.
More importantly, one strength of 3D hologram is 360 degree perspectives, which allows not only one player but many audience to get involved in the training session. They are more likely to discuss the game, share ideas, interact with other people while training, increasing the socialization function of senior citizen centers.
-26-
Augmented Reality
Reference [1] Salthouse, T. A., Babcock, R. L., Skovronek, E., Mitchell, D. R., & Palmon, R. (1990). Age and experience effects in spatial visualization. Developmental Psychology, 26(1), 128. [2] Lawton, C. A. (1994). Gender differences in way-finding strategies: Relationship to spatial ability and spatial anxiety. Sex roles, 30(11-12), 765-779. [3] Parsons, T. D., Courtney, C. G., Dawson, M. E., Rizzo, A. A., & Arizmendi, B. J. (2013). Visuospatial processing and learning effects in virtual reality based mental rotation and navigational tasks Engineering Psychology and Cognitive Ergonomics. Understanding Human Cognition (pp. 75-83): Springer. [4] Malinowski, J. C. (2001). Mental rotation and real-world wayfinding. Perceptual and Motor Skills(0031-5125 (Print)). [5] Bird, C. M., & Burgess, N. (2008). The hippocampus and memory: insights from spatial processing. Nat Rev Neurosci, 9(3), 182-194. doi: 10.1038/nrn2335 [6] Hedden, T., & Gabrieli, J. D. E. (2004). Insights into the ageing mind: a view from cognitive neuroscience. Nat Rev Neurosci, 5(2), 87-96. [7] Beni, R. d., Pazzaglia, F., & Gardini, S. (2006). The Role of Mental Rotation and Age in Spatial Perspective-Taking Tasks: When Age does not Impair Perspective-Taking Performance. Applied Cognitive Psychology, 807-821. [8] Jansen, P., & Heil, M. (2010). Gender differences in mental rotation across adulthood. Exp Aging Res, 36(1), 94-104. doi: 10.1080/03610730903422762 [9] Lee, S., & Kline, R. (2011). Wayfinding Study in Virtual Environments: The Elderly vs. the Younger-aged Groups. International Journal of Architectural Research, 5(2), 63-76. [10] Hertzog, C., Kramer, A. F., Wilson, R. S., & Lindenberger, U. (2008). Enrichment effects on adult cognitive development: Can the functional capacity of older adults be preserved and enhanced? Psychological Science in the Public Interest, 9(1), 1-65. [11] Shepard, R. N., & Metzler, J. (1971). Mental Rotation of Three-Dimensional Objects. Science, 171, pp. 701-703. [12] Kaufmann, H., & Schmalstieg, D. (2003). Mathematics And Geometry Education With Collaborative Augmented Reality. Computers & Graphics, 37(3), 339-345. [13] Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology Instruction Cognition and Learning, 1, 323-357. [14] Chu, M., & Kita, S. (2011). The nature of gestures' beneficial role in spatial problem solving. Journal of Experimental Psychology(1939-2222 (Electronic)). [15] Chu, M., & Kita, S. (2012). The role of spontaneous gestures in spatial problem solving Gesture and Sign Language in Human-Computer Interaction and Embodied Communication (pp. 57-68): Springer. [16] Wobbrock, J. O., Morris, M. R., & Wilson, A. D. (2009). User-defined gestures for surface computing. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
-27-
For the video: https://goo.gl/8Hpky1
ARotation
-28-
Augmented Reality
ARotation
Augmented Reality Based Spatial Visualization Training Game for the Elderly ARotation, which is an APP version of Goodlock, is also a spatial visualization training tool designed for the elderly. Similar to mental rotation ability, spatial visualization is one of spatial cognition abilities which will decline with human's increasing age, resulting in much inconvenience to elderly's life. However, compared with Goodlock which is a fixed installation in public centers, ARotation is implemented as an APP which allows the elderly to train their visualization ability through tablets and smartphones, increasing the portability and accessibility of training. Therefore, it will be easier and more convenient for the elderly to do cognition training at all times. Project of Interface Design, Accomplished in the 1st year of Graduate School of Industrial Design, NCKU
-29-
ARotation
Spatial Visualization & Mental Rotation what's between these two spatial cognition abilities Spatial visualization ability is the ability to mentally m a n i p u l ate 2 - d i m e n s i o n a l , 3-dimensional and 4-dimensional figures(Lohman, 1996). It is typically measured with simple cognitive tests. According to Lohman's definition(Lohman, 1988), these tests are mental rotation, paper folding, block d e s i g n , p a p e r fo r m b o a rd , surface development and
-30-
mechanical principle. In other words, mental rotation is one of spatial visualization abilities which centers on rotation of an object in an imaginative space(McGee, 1979). They share the similar pattern of cognitive operation and also the similar function of spatial concepts. Hence, these two abilities relate to each other.
Augmented Reality
Goal of ARotation
delay the decline of visualization ability
1.Elderly Friendly to provide a visualization training option for the elderly
2.Portability to enhance the accessbility and portability of training. So users can do the training without the restriction of specific places.
3.AR technology to apply interactive technology to enhance the training engagement and the digital interaction
4.Manual Control to activate motor operation in human's brain which is beneficial to visualization. Also, this is easier for the elderly to play and use.
-31-
ARotation
Augmented Reality
AR is the real environment whose elements are augmented by computer-generated sensory input. Viewers can see the virtual figures through the devices that are equipped with camera. When the camera is targeting toward marker patterns, the virtual figures will appear from the patterns in the screen of the device.
SPATIAL INFORMATION AR provides correct spatial information and faciliates spatial cognition. People can better realize spatial relationship in AR rather than in 2D platforms.
REDUCE COGNITIVE LOAD AR increases germane load and reduces extraneous load. By training in AR environment, AR can help users reduce intrinsic load; therefore, the overall load is down (Shelton & Hedley, 2004).
-32-
ENHANCE IMMERSION With the features of immediate interaction and digital manipulation with the virtual figures, AR can enhance users' engagement and interests when they are in the training.
Augmented Reality
Design Process In order to create user centered game and controller, an experiment was conducted to collect older adults' behaviors when they were handling visualization problems. Next, transform these behaviors into the design of controller and games.
-33-
ARotation
Idea Generation
-34-
Augmented Reality
Implementation create Augmented Reality by coding in C# in Unity 3D
-35-
ARotation
The game was implemented in Unity3D and tested in laptop and APP version.
The form and the arrangement of the controller were also improved in order to achieve fluent use.
Evaluation Process
Older adults were recruited to test the usability of ARotation.
This section summarizes the process of implementing the software of ARotation and reveals how the improvement of the game and the manual controller were done. Last but not the least, evaluate the training effect of ARotation by single subject research.
-36-
Augmented Reality
There were 3 phases in the evaluation. In the 1st phase, 8 elderly participants were tested their baseline performace of spatial visualization by taking paper tests. In the 2nd phase, they received ARotation. In the 3rd phase, they were taken the paper tests again to see if they made an improvement or not.
-37-
ARotation
3D Printing model making A reflection device was made to redirect the camera route of the tablet, and a stand was provided to keep the tablet in vertical angle. These two items were done by 3D printing, which saved quite a lot of cost and time.
The material used to print was ABS. The time for implementation took around 6 hours.
A reflection device whose angle was adjustable could be equipped on the tablet.
playing / training process User rotates the cube marker to control the virtual figure that is above the virtual floor.
Next, press the round marker which presents a red virtual button in the screen, and the figure will fall down
-38-
By reflection, the camera route was redirected toward the ground so the markers could be scanned.
Open Sources
Augmented Reality
ARotation will be open sources accessible through cloud sharing. Older adults and their families can download the marker pictures from ARotation websites and print them out. The original 3D file of iPad stand is also available. Therefore, older adults can directly start training at all times!
Reference [1] Lohman, David F. "Spatial ability and g." Human abilities: Their nature and measurement 97 (1996): 116. [2] Lohman, D. F. (1988). Spatial abilities as traits, processes, and knowledge. [3] McGee, M. G. (1979). Human spatial abilities: Psychometric studies and environmental, genetic, hormonal, and neurological influences. Psychological bulletin, 86(5), 889. [4] Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology Instruction Cognition and Learning, 1, 323-357.
-39-
Part.2
APP Design
For the video: https://goo.gl/K9a583
Hidden Lion
-42-
APP Design
Hidden Lion An Location Based APP Game of Sword Lion Searching
Hidden Lion is a Location Based Service(LBS) APP game which is related to sword lion culture in Anping, Taiwan. We develop a storyline and theme that has a connection with the local culture. While playing the interactive games at each sword lion sites with this APP, visitors can find sword lions and experience the background story of each sword lion. Furthermore, Hidden Lion creates a service system in Anping. From the support from Anping district office, more and more visitors can come here and play the Hidden Lion. After completing all the interactive games, the visitors will receive a coupon of sword lion model coloring from Sword Lion School as a reward, which gradually forms a business cycle and culture connection. Project of Human Computer Interaction, Accomplished in the 1st year of Graduate School of Industrial Design, NCKU
-43-
Hidden Lion
Sword Lion Statue
Tainan is a place full of traditional culture and atmosphere in Taiwan, which is also the reason why many visitors are attracted to come here and travel. Among all activities, the most special traveling activity in Tainan is Sword Lion Searching in Anping. In the past, Anping was a crowded region near the sea; therefore, local people built sword lion statues in front of their houses to keep bad evils away. Sword lion statues are the stone plates installed on the wall. The appearance of sword lions is different from each other, representing different protecting abilities and stories. Nowadays these sword lion statues decay with time and weather and remain few intact in Anping.
Different sword lions have different protecting capabilities. However, sword lions decay with time and remain only few intact nowadays.
-44-
APP Design
Sword Lion Searching
discover remaining sword lions in Anping Recently, with the promotion of Anping District Office, many travelers are attracted to come here to find out remaining sword lions and experience the cultural stories of these protectors.
Activity Theory In order to analyze sword lion searching more specifically, a research method: Activity Theory is adopted which is clear for us to collect data and details of sword lion searching.
According to the interview and observation, most travelers like to share pictures and experience on social media and their personal blogs.
-45-
Hidden Lion
Problems of sword lion searching
However, current sword lion searching has some problems. First, exisitng searching maps have too many versions, which may confuse visitors and make them hard to locate lions. Second, sword lion searching results in disturbance to the locals. Third, visitors do not realize the cultural story of sword lions.
Vision of our project
We are devoted to creating Hidden Lion which is a LBS APP helpful for sword lion searching that includes interactive games to make visitors learn the sword lion story in a more interesting way.
-46-
APP Design
SERVICE SEEKER
Mrs. Hsueh loves to travel to other places in the weekends with her families. She likes Tainan for its cultural value and fascinating history. She always has fun with her son during each traveling not only for teaching the local culture but for improving their relationship.
-47-
Hidden Lion
Dramatic Elements
save sword lions from the evil's invasion in Anping
In order to make the APP similar to a game, we create the game story. Since Anping is an island in the past and sword lions fend off evils, a plot “Anping is attacked by evil spirits� is created. Powerful lions are imprisoned, so players need to locate and play interactive games to free them. Sword lions will be more powerful throughout the game by defeating evils and bringing peace in Anping again.
Formal Elements
We follow the formal elements to plan the game procedures. The detailed information and functions of Hidden Lion are listed through 4W1H questions. Hidden Lion is suitable for everyone to use. The basic reqirement in each procedure is shown in the left table.
Dramatic Arc
-48-
APP Design
APPLE iBeacon
iBeacon is a location sensor to detect the distance between iphone and iBeacon itself. When the players touch the sensing area, an interactive game will be on.
Location Based APP
Implementation GOOGLE GPS
Our system uses the google map service as the main guide view and connects to the Parse database for getting the annotation data.
ELECTRONIC COMPASS
We set up compass functions in the APP so that user's walking direction is detected.
MULTITOUCH
Mapping function is used in X i Pe n g Pe n g ' s m i s s i o n , s o multitouch coding is required.
IMAGE RECOGNITION
P l ay e rs t u r n o n c a m e ra t o recognize sword lions so that the game will get started.
-49-
Hidden Lion
Island of Sword Lion
A. empire city The most powerful sword lions are mainly located in this area.
B. military camp In the past, soldiers and general lived here. Many historic sites are also in this zone.
C. harbor zone This area is near the canal and Anping harbor.
D. ghost town -50-
This zone is named Ghost Town due to many tombs located here.
APP Design
Empire City
Basic Function of the APP Since the whole area is too large, we only implement the empire city area in the APP. Here we show the detailed map view of empire city. Icons on the right side are the basic functions users can use to help them in searching process such as the location of sword lion, tourist spots, other players, restaurants, mayor 's home and temples.
sword lion site
other players
the location of sword lions in empire city area
the location of other p l aye rs w h o a l s o have Hidden Lion A P P. P l a y e r s c a n invite them to join the searching
tourist spots
restaurant
the location of famous tourist spots in Anping
Anping is famous for its delicious snack. The location of these restaurants is also noted
mayor's home
temple
mayor serves as a searching instrutor so players can ask him for help
temple is also where user can ask for the searching help
-51-
Hidden Lion
Once upon a time, sword lions lived peacefully in the sword lion island.
Suddenly, a devil invaded and destroyed the island. Some lions were even imprisoned.
Luckily, one little lion escaped from the attack safely.
He was devoted to bringing peace back again and saved his lion friends.
First, the lion introduces the background story and asks players for help.
Then, the player selects t h e re g i o n w h e re h e wants to challenge.
Press the camera icon, and the interface shows the introduction of the tourist spot.
Press the noodle icon, and the interface shows the introduction of the famous restaurant.
P re s s t h e swo rd l i o n icon, and the interface shows the location and introduction of the lion.
If the player discovers any sword lion in empire city, he can press the catch button.
Once the button is pressed, the player is able to scan the lion discovered at the spot.
For example, the player can target the Yu Jian Er sword lion to start the game.
-52-
APP Design
The story of Yu Jian Er: once upon a time, he was resting.
H o we ve r, a g ro u p o f ghosts was approaching.
They attacked the resting Yu Jian Er sword lion.
Yu Jian Er was defeated and even got imprisoned.
Fortunately, Yu Jian Er left a magic star and awaited the player to save him.
Next, the game introduces how to finish Yu Jian Er mission.
The player should run the route of the star to release Yu Jian Er.
Once the mission is completed succeddfully, a congratulations page will show.
Bagua & Magic Star Bagua models with each iBeacon inside are placed individually at five corners in a yard in front of Yu Jian Er statue. The player completes the mission by running the star route to touch the bagua models.
-53-
Hidden Lion
T h e sto r y o f X i Pe n g Peng: Once upon a time, Xi Peng Peng was strolling along a street.
Suddenly, weather got bad and a water devil appeared.
Xi Peng Peng decided to fight against the devil.
However, the devil was too strong and Xi Peng Peng was hurt greatly.
Unfortunately, Xi Peng Peng was imprisoned.
First, the game will d e m o n s t ra t e h o w t o finish the mission.
T h e p l aye r m a p s t h e screen back and forth to remove bubbles.
Once the mission is completed succeddfully, a congratulations page will show.
Mapping on the screen The player needs to map the screen back and forth to finish the mission: remove the bubbles and clear the screen.
-54-
APP Design
In the achievement page, it shows how many missions the player has accomplished.
If all missions in one region are completed, t h e p l ay e r r e c e i v e a coupon.
Lion Models Mock Up If the player received a coupon, he can go to the sword lion school in exchange of sword lion model making and coloring in lower price. It is expected that the player enjoys the model mocking up process and is impressed with cultural story of sword lions by playing Hidden Lion.
-55-
Hidden Lion
conference of game competition in Toronto Hidden Lion was also published in CHI PLAY 2014 in Toronto Canada. We were invited to attend CHI PLAY conference and introduced Hidden Lion APP in Canada. I n a d d i t i o n , H i d d e n L i o n p ro j e c t wa s presented in the exhibition held by the Fundation of Historic City Conservation and Regeneration. We were devoted to cooperating with local governments and promote its into real markets.
-56-
APP Design
Through Hidden Lion, which has a storyline and theme that connects the local culture, we can link the government, sword lion promotion organization, local restaurants and stores. We not only make it an ideal medium for exploring cultural history b u t a l s o a p e r fe c t a p p ro a c h fo r revitalizing local economic.
Future Goal of
the Service System -57-
Part.3
Arduino Projects
Happyscanner
-60-
Arduino
Happy Scanner
An Interface Design of Card Checking Machines to Motivate Postive Emotions for Train Commuters Happy Scanner, which is an interface design of OV chipkaart checking machines, shows how eco-friendly you are by displaying how many trees you haved saved and how much carbon footprint you have reduced through taking public transportations. In most of train stations, when the machines are checked by shipkaart, the interface will only show how much money the cards still have. This information is necessary, but it might result in negative feelings for the card owner(commuters) even though taking public transportation is such a contributive act. Hence, the new interface turns to show the information about reduced carbon footprint and trees saved by the journey. Commuters will realize their contribution to the environment and feel more positive by taking public transportations. Project of Design for Happiness, Accomplished in the 1st year of Master Program of Design for Interaction, TU Delft
-61-
Happyscanner
Public transportation is well-organized in the Netherlands. Not only does Holland boast an extensive railway network, but the country also owns a large population of train commuters. With the bustle and hustle of city, these commuters suffer a lot from intense daily schedules. Every time when they enter stations and scan their easy cards, the interface on checking machines always show the decreasing amount of money that the card owner still has. This is such a negative information to increase the commuters' worries and anxiety. Before they start a journey, they do not expect how amazing the journey will be, but feel worried about how much the account balance is and decide if they need to top on or not.
-62-
Arduino
Tension & Anxiety how people feel in current societies
-63-
Happyscanner
However, taking public transportation is such an eco-friendly act to contribute to the natural environment. By replacing vehicles with trains, each commuter can help reduce around 75 percent of the emission of carbon dioxide which is amost equal to the growth of an intact tree. Hence, in order to reduce the commuters's negative emotions, it is even better to inform them of their contribution of taking public transportations, which, indirectly facilitates their happiness and postiveness.
-64-
Arduino
Happiness ,Contribution how people should feel in current societies
-65-
Happyscanner
Framework for
Positive Design
Three ingredients of positive design: design for pleasure, design for personal significance, and design for virtue. Pleasure indicates that a person is enjoying the moment. The focus of personal significance is on one's personal goals and aspirations. Virtuous behaviour means to behave honourably and philanthropically.
-66-
Arduino
Translation The more journey the commuters travel by trains, the more carbon footprint they help reduce. Also, it means the more tree the commuters help save.
design for pleasure If the communters are informed of their contribution to the environment, they are immersed in the pleasure of protecting the earth.
design for personal significance Taking a longer journey by train will reduce more carbon footprint and save more trees, creating a goal that the communters want to reach a certian amount of the reduction of the emission of carbon dioxide.
design for virtue Being eco-friendly and protecting the environment is a virtuous behavior in current societies.
-67-
Happyscanner
BEFORE the Scan
Rather than only shows the number of account balance, the new design of interface also presents an appreciation to the commuters by showing the number of saved trees. The commuters are able to know how their traveling would contribute to environment
-68-
OV chipkaart is also transformed in e-paper material, which can immediately record and show the account balance on the back of the card, which is clear for the commuters to retrieve their traveling information.
Arduino
AFTER
the Scan
-69-
Happyscanner
Calculation of Carbon Footprint Two journeys are taken as examples to illustrate the idea of the new interface. One is the busiest journey which starts from Amsterdam to Rotterdam. If the jouney is carried by trains, around 1.7 trees can be recovered. Another may be the longest jouney in Holland which starts from Groningen to Maastricht. 6.7 trees can be recovered if communters take this journey by trains.
-70-
Arduino
-71-
Happyscanner
Implementation Happyscanner is done with the help of Arduino and Processing. Arduino is responsible for RFID reader, which allows users to scan the card so the machine will make some feedbacks. On the other hand, processing is used for the display, the change of the interface after the RFID receives the data. The figure above shows all components used in the implementation of happy scanner.
The connection of the components is presented in the figure on the right. These components are all well-organized and stored in the model of OV chipkaart checking machine. When a user scans his card on the machine, the monitor will show the design of the new interface to demonstrate the idea.
-72-
Arduino
-73-
Happyscanner
Presentation in the Ministry of Education in Taiwan It was an honor to present the project of happyscanner to Taiwaneses Ministry of Education in October, 2016. This project was done in a course called Design for Happiness taken in TU delft, the Netherlands. Since I had won the scholarship to study abroad in Holland, I was invited to present what I learned and introduce the design courses that are different from the ones taught in Taiwan. The presentation was successful and the project drew lots of attention from the audience and the press. Happyscanner and I were even reported in print media, which was really a honorable experience.
-74-
Arduino
-75-
For the video: https://goo.gl/JuyudV
Night Owl
-76-
Arduino
Night Owl A Smart and Innovative Computer Using Assistant
Project of Simulation in Ergomomics Design, Accomplished in the 1st year of Graduate School of Industrial Design, NCKU
Night Owl is a useful computer using assistant to protect computer users from incorrect behaviors of using computers. It combines with Arduino and other sensors to achieve the human computer interaction. First, when the user is too closed to the screen, Night Owl will enlarge the information on the window so the user can see the texts clearly rather than approach the computer. Second, if the light in the environment is weak, the additional light source on Night Owl will shine to provide enough lighting. Third, if the user has used computers for a long time, the software of Night Owl will make sound to remind the user to take a break and replenish water. With Night Owl, users are able to use computers in correct bahavors and free from physical illness.
-77-
Night Owl
Without a doubt, with the advanced technological development, computers have always been necessay items in current times. In spite of providing entertainment, computers also function as working or learning tools for most people such as engineers, researchers, and students. These people are required to ke e p co n c e nt rat i n g on the screen and focus on the work for a long time without appropriate rest and break, gradually resulting in some physical illness like eyestrain, stiff shoulder, carpal tunnel syndrome and so on.
Protential Problems of
Using Computer
rate of using computer
problems caused by using computer
some statistics data about using computer are collected. For instance, 35 % of people require to use computers when working. These people are mainly composed of young population aging from 25 to 34, which shows the importance of computer nowadays.
However, if users do not keep correct behaviors when using computer, many physical problems will occur such as stiff shoulder and myopia.
-78-
Arduino
Behavior Observation
be too close to the screen
neck pain
Indeed, computers are essential for us. However, many ignore incorrect behaviors of using computers. This is also why so much physical illness occur. First of all, an observation was conducted to see how people work with computers. Without any previous notification, these subjects were being observed when using the computer. They were all graduate school students who usually used notebooks. Each of them was being observed for two hours or so. With the observation, we could realize the incorrect behaviors of using computers, and the reason why these behaviors would happen.
incorrect posture
subject no support for elbow
hunchback
graduate school students PHD students , all notebook users
location ergo lab, department of ID, NCKU
procedure
be too close to the screen
they were observed without previous notification. Each of them was observed for around 2 hours
goal to realize the incorrect behaviors of using computer. Additionally, the reason why the behavior occurs would also be understood
hunchback
result too close to the screen, neck pain, incorrect posture, hunchback, without taking a rest and so on
neck pain
without rest
-79-
Night Owl
Result of
Observation do not turn on the light all subjects did not turn on the light when using computer
do not take any rest
too close to the screen
incorrect posture
most people were used to finishing their assignments without any break
if users wanted to see something tiny, they would approach the screen
when users felt tired, they might have the incorrect posture when working
hunchback
no support for elbow
this incorrect behavior often followed by approaching the computer
some people ignored to support their elbow when using computers.
neck pain this problem often followed by keeping focus on the screen without enough rest
-80-
Arduino
Expected Goal
Considerate Computer
enlarge the window computer can detect the distance between itself and the user. If the user is approaching, indicating he wants to see tiny text, the window will enlarge.
additional light
emotional element
provide lighting if the light in the environment is weak.
This assistant also serves as little helper that enhances user's comfort.
time reminder make sound to remind the user to take a break if he has worked for a long time.
-81-
Night Owl
implemented by
Arduino
& Flash
Arduino is a small processor that deals with the electrical signals and functions. It is very suitable to be used for implementing the expected functions mentioned before.
Photoresistor it can sense the light current in the existing space.
Light-Emitting Diode it is used for output use because it will shine after receiving signals.
Resistance resistance can control the amount of current flowing through the coil.
Adobe Flash
Supersonic Sensor
this software which will be connected to Arduino is adopted to show the scaling effect of window.
it can detect the distance between itself and the right front obstacle.
-82-
Arduino
Connection between sensors and software
-83-
Night Owl
Kitten
computer using assistant best comfort when working at midnight
in order to incorportate emotional elements into design, a kitten puppet which included the sensors and electric wire was created. This adorable computer using assistant would make users comfortable and warm when they worked overnight. time reminder this little cat was a software that got started after the user installed the kitten puppet on the computer. It functioned as the time reminder.
photoresistor On the top of the kitten. Since it was sensitve to the light, the top of the puppet was the best position for it.
supersonic sensor On the face of the kitten. The feature of this sensor was two little cylinders which were similar to eyes of the kitten.
light emitting diode On the neck of the kitten. The shining LED served as the necklace on the kitten's neck.
-84-
Arduino
install the kitten puppet get the Night Owl kitten and put it on the computer.
start the software
after the kitten is installed well, the Night Owl software will get started.
enlarge screen
if the user is too close to the screen, window will enlarge. Thus, user can clearly see the content and return to the correct position on the seat.
additional light
when the light in the environment is weak, the necklace on the kitten will shine serving as an additional light source.
time reminder
if the user has worked over 30 mins, the virtual kitten will make sound to remind the user to take a break.
-85-
Night Owl
Correct
Keep Behavior when using computers
By Night Owl, all users can keep correct behaviors when using computer for the work. This project makes me realize how to study user behaviors and use more profound methods to analyze the interaction between human and computers. Through the implementaion of Arduino and Adobe Flash, the interaction effect is easily achieved. In the future, computer users will get to know the correct way of using computers and prevent physical illness from happening.
-86-
Part.4
3D Modeling
For the video: https://goo.gl/FiyXUK
Ready to Bake
-88-
3D Modeling
Ready to Bake Baking Utensil for Mentally Disabled Bakers
Ready to Bake is a set of baking utensil designed for the mentally disabled people who are bakers. Due to cognition deficiency of the bakers, some tasks in baking process which involved with abstract notions such as measuring time, weight, and amount of the ingredient might be tough for them. In order to help them work on baking smoothly, these new utensils are designed with a creative principle: aligning with referential objects, which means mentally disabled bakers will receive concrete feedbacks to ensure they work on the procedures successfully. In this light, they can finish baking with more efficiency as well. Project of Graduation Design, Accomplished in the 4th year of Department of Industrial Design, NCKU
-89-
Ready to Bake
Mentally Disabled People & Sheltered Bakery Due to the deficit in learning social rules and problem solving skills, finding a stable job and living independently are difficult mentally disabled people. To help support their own living capabilities, one solution is sending them to sheltered houses. Shelterd houses, which are managed by local governments, provide them with job training and skills. In Taiwan, most sheltered houses are bakeries. Instructors in the houses teach them baking, cooking and also selling their handmade products. However, even if mentally disabled people receive intructions from the professional instructors, some problems are still there: They may be too dependent on the instructors, making instructors too exhausted to take care each worker very well. In addition, without the instruction, some of them will not be able to handle things on their own.
-90-
3D Modeling
Field Study Process 1.focus on one sheltered house f i rst , we v i s i te d m a ny sheltered houses to realize the working environment. We chose Zhanyi Backery to keep the observation. Around eight mentally disable people and two instructors worked in this backery.
2.observe and discover problems we kept visiting and observing at Zhanyi B a c k e r y. D u r i n g t h e observation, we realized the baking process and the tools mentally disabled people used for working. In addition, We also recorded how they interacted with o t h e r b a ke r s a n d t h e instructors.
3.discuss with the instructors We discussed with the instructors to realize the more closed and detailed information about mentally disabled people. Besides, we shared our design ideas with them and discussed together.
4.design and develop After defining problems, we came up with more ideas and kept designing the best solutions to help them deal with their current working problems.
-91-
Ready to Bake
1.WEIGHING bakers start baking by weighing ingredients such as flour, sugar, butter and egg. The tools they use are traditional or digital scales and weighing bowls.
Baking
Process
main baking procedures in the Zhanyi Sheltered Bakery, Tainan
2.WHISKING they put all ingredients together into a larger bowl to mix them. Next, the raw paste will be made. The tools they use are larger bowl and a whisk.
3.DIVIDING
4.STUFFING
bakers begin to divide the stuffing such as red bean paste into equal amount. Usually they will use digital scales to finish this work.
bakers fill the paste with the stuffing and make them into the raw bread. They will not use any tool in this step.
-92-
3D Modeling
Main Problems of Baking Process Weighing They cannot realize the meanings of each figure on the screen of the scale since they are too abstract for them to comprehend.
5.TOASTING bakers put the raw bread on the stainless plate and send to the oven to bake the bread.
Whisking They are unable to catch the accurate time to finish mixing. Besides, the current whisk is hard for them to hold well to mix the sticky and thick ingredients.
Dividing It is difficult for them to divide the paste in equal volume. The only solution is to use the scale to weigh the paste one by one.
-93-
Ready to Bake
Project Vision
Apply to more shelterd workplaces It is always promising to develop our design further in the future. After the implementation of our design in Zhanyi Backery, we search for other resources like patent and design competitions and try to get contact with departments of social welfare of local governments. With the support from the governments, it is expected that more incorporations and social charities would collaborate with us and provide us with financial support to apply our design to more sheltered workplaces in Taiwan.
-94-
3D Modeling
Idea Generation
scale, whisk and other assistance
Design ideas are based on different steps in baking process. The ideas of scale focus on weighing. The ideas of whisk are for whisking. The ideas of other assistant tools are for minor details in baking process. For instance, a handle design to keep balance when whisking or an equippment to help hold the weighing bowl stably.
-95-
Ready to Bake
Modeling Mock Up
material:ABS, aluminium; method: CNC, 3D printing
The main material the products are made of is ABS. Another material used is aluminium. This material is used on the base of the scale, the wire of the whisk and the ladle part of the divider. The whisk and the divider are made through 3D printing, and the body of the scale is made by CNC.
-96-
3D Modeling
Baking Scale The interface of baking scale is a recipe ring whic h s hows the appropriate amount of every ingredient. Hence, what bakers do is to weigh ingredients to make the indicator correctly point to the colorful location which is where their precise weight should be. In this light, bakers just need to recognize the clear colorful icons rather than conventional and tiny numbers.
1.put the recipe ring on the scale
2.point to the original zero
3.finish the weighing process
First, select a suitable recipe ring according to the feature of mixed ingredients. Next, put the recipe ring on the surface of the scale.
Place the weighing bowl on the scale, and the light will remind bakers to press the button to make the indicator rotate to original zero.
Lastly, weigh the ingredients to make the indicator rotate to their corresponding locations.
easy to hold A large space between the body and the base is easy for bakers to hold the scale and move it less strenuously.
-97-
Ready to Bake
Baking Guidiebook The content of baking guidebook shows the procedure and tools in baking process. The recipe rings are also stored in it. Before baking, bakers read it to realize what they need to prepare and take out the recipe ring to start weighing. Baking guidebook is made of plastic that is convenient to be cleaned.
baking procedure and guidance Mentally disabled bakers sometimes forget the detailed procedures and requirements in baking so that the instructors have to remind them vey often. With this book, bakers can check the necessary items and the procedures on their own.
recipe rings The weight of every ingredient is marked on the recipe rings in colorful and remarkable pattern that is clear and obvious for bakers to notice and weigh easily.
-98-
3D Modeling
Whisk & Counter
Counters are used as timers that make sound to help bakers catch the finish time. Counter labeled 1 is used for mixing simple ingredients that take little time to finish mixing, while counter labeled 3 is for handling sticky paste that may be more time-consuming. Bakers can attach the counter to the whisk when mixing the ingredients so that they can realize when to stop mixing.
1.attach the counter First, choose a suitable counter according to the feature of the mixed ingredients. Then, attach the counter to the whisk
2.catch the finish time Next, bakers can realize when to stop mixing by listening to the sound made by the counter.
-99-
Ready to Bake
Divider
A Divider is composed of two handtools: a slicer and an ingredient tube. The ingredient tube is marked with bright yellow lines which equally divide the tube into three parts. Each part is equal to the amount of the bean paste that will be used in making one red bean bun. Thus, bakers are able to divide the stuffing correctly with this handy divider.
1.fill the ingredient tube with the red bean paste
2.smoothe the paste with slicer
3.divide the paste equally
First, insert the tube into the red bean paste and use slicer to help fill the tube with bean paste. With the ergonomic form of the handle, bakers feel comfortable and less strenuous to finish this procedure. Then, the ingredient tube is filled with the red bean paste.
Second, flatten the surface of the bean paste in the ingredient tube by slicer. Thus, the amount of paste is identical in the space separated by yellow lines.
Use the slicer to divide the red bean paste in equal amount by aligning with the bright yellow lines on the surface of the ingredient tube
-100-
3D Modeling
Ready to Bake Ready to Bake had been implemented in Zhanyi bakery and evaluated with the use. Positively, both the instructor and the bakers gave us supportive response. The clear and direct indications on each tool made it easier and faster for them to work smoothly. However, there is still something need to be imporved. Ready to bake is currently specialized for making red bean buns. It will be better if Ready to bake can be utilized to handle more types of bread that Zhanyi bakery offers. Hence, the next step of Ready to Bake is to develop further on its adaptability to different bread.
-101-