WristEye: An Elderly Computer Learning Assistant System with Wrist-Wearable Devices Wan-Jung Chang1,*, Yi-You Hou2, Rung-Shiang Cheng3, and Ming-Che Cheng3 Department of Electronics Engineering, Southern Taiwan University of Science and Technology, 71005 Tainan, Taiwan(R.O.C.) 2 Department of Electrical Engineering, Far East University, Tainan 74448, Taiwan(R.O.C.) 2 Department of Computer and Communication, Kun Shan University, Tainan 71070, Taiwan(R.O.C.) 3 Department of Electrical Engineering, National Cheng Kung University, Tainan 701, Taiwan(R.O.C.) 1
Abstract. This paper develops an elderly computer learning assistant system with wrist-wearable devices, designated as WristEye, which can be used to analyze the computer learning attitudes, reactions, and behaviors of elderly individuals whilst in computer learning classes. WristEye is equipped with a kinematic sensor to effectively detect the changes in the orientation and vertical acceleration of the elderly wrist and to determine the corresponding operations in learning computer, i.e., moving mouse, hitting keyboard, idle, and swing mouse. Furthermore, a remote backend server receives the detected signal from the wearable unit via a Wireless Sensor Network (WSN) and then identifies the corresponding computer learning effectiveness. The experimental results show that WristEye has a classification accuracy to recognize computer learning status of elderly individuals. Keywords: Wearable, Elderly, Learning.
1
Introduction
Due to rising life expectancies as a result of advancements in medical science and the continuing aging of the baby boomers born in the middle of the last century, the elderly populations of many countries around the world have increased significantly in recent years. For example in Japan, the percentage of the population greater than 65 years of age was 15.1% in 1996, and increased to 21.8% in 2008 [1][2]. Moreover, many countries in the world are currently planning and developing various egovernment projects (e.g., e-Japan in Japan, A Framework for Global EC in American, UK Online in E England, e-Korea in Korea, and etc) to turning them into the advanced Information Technology (IT) nationals [3]. These two worldwide demographic shifts have many significant social implications, not least of which includes an additional burden on the computer learning services for elderly populations. The computer learning effects of elderly is limited by some factors, such as educational background, the knowledge of digital information, small texts on the
monitor, unacquainted with keyboard and mouse, web page operation, and so on [4][5]. These factors always cause elderlies to feel barriers and disappoint in learning computer. Such phenomenon of leading elderlies to decrease their computer learning interest and desire is called “Computer Phobia”[6]. Morrell et al [6] present that if the instructors of teaching computer design the useful skills for using computer, it would be efficient to increase the study interests of using computers by elderly people. To resolve the Computer Phobia issue, the current study develops a sophisticated elderly computer learning assistant system with wrist-wearable devices, designated as WristEye, which employs a kinematic sensor for the continuous monitoring and reporting of an individual’s computer learning status over a Wireless Sensor Network (WSN). Utilizing an accelerometer and a gyroscope, the kinematic sensor measures the changes in the orientation and acceleration of the wrist and then communicates this data to Micro Controller Unit (MCU), where the signal is processed by a wrist recognition algorithm to determine the corresponding activities in learning computer, i.e., moving mouse, hitting keyboard, idle, and swing mouse. Furthermore, a remote backend server receives the detected signal from the MCU via a Wireless Sensor Network (WSN) and then identifies the corresponding computer learning effectiveness. Significantly, WristEye system not only detects each occurrence of computer learning status, but also records the approximate amount of time spent in performing each computer learning activity, and therefore enables a rough estimate to be made of the computer learning effectiveness of the individual. Through the use of WristEye, abnormal computer learning activities of elderly individuals can be immediately detected and appropriate actions from the instructor are followed. As a result, the proposed WristEye system is expected to be of great use to the e-learning class infrastructure vendors in implementing an elderly e-learning system. The remainder of this paper is organized as follows. Section 2 introduces the proposed WristEye system comprising the sensor unit hardware, the signal processing scheme, and the wrist recognition algorithm. Section 3 presents the results of a series of experimental trials designed to evaluate the performance of the WristEye. Finally, Section 4 draws some brief conclusions.
2
2.1
WristEye System
Overall WristEye Structure
The objective in the present study is to realize the real-time recognition of computer learning activities utilizing a small, wireless, low-power kinematic device. The system hardware of WristEye system comprises just two parts, namely a wearable unit and the backend server. The wearable unit is attached to the subject’s wrist and consists of a data acquisition board and a kinematic sensor. Whereas the ambulatory systems presented in [7-9] utilize a single triaxial accelerometer for motion sensing purposes, the kinematic sensor in WristEye makes use of information of two axes from an accelerometer and information of single axis from a gyroscope in order to increase the reliability of the movement information provided to the recognition algorithm. Fig. 1(a) presents a schematic illustration of the system
architecture. The wearable unit performs an initial pre-processing of the signals produced by the gyroscope and accelerometer to determine the wrist orientation and vertical acceleration, respectively, and then transmits this information to MCU which processes the information using a wrist recognition algorithm to classify the corresponding the computer learning activities. The wearable unit transmits the recognition information via a wireless transceiver module to the backend server through the wireless base station attached to the server. The server then processes the information to classify the computer learning effectiveness of elderly individuals.
battery
kinematic sensor
(a)
data acquisition board
(b)
Fig. 1. (a)Schematic of system architecture. (b) Photograph showing full-size view of wireless transceiver and sensor device.
2.2
Wearable Unit Hardware
Fig. 1(b) presents a full-size photograph of the wearable unit within the WristEye system. The data acquisition board includes an Atmel Atmega 128L microcontroller, 4
KB of RAM, various ICs and a wireless transceiver module, while the kinematic sensor comprises an IDG300 gyroscope [8] and an ADXL330 accelerometer [9]. The sensor unit has two main functions, namely to detect changes in the posture and movements of the wearer, and to merge the data signals of the accelerometer and gyroscope, respectively, prior to their transmission to the base station. The sensor unit is equipped with a low-voltage, low-power consumption ZigBee-compliant transceiver module (CC2420, Chipcon, TI) to facilitate its WSN communication ability with the base station.
2.3
Wrist Recognition Algorithm
Upon receiving the wrist orientation and vertical acceleration from kinematic sensor, the MCU utilizes a wrist recognition algorithm to identify the corresponding computer learning activities. The algorithm commences by applying vertical wrist acceleration threshold value to determine whether the wrist posture of the subject falls into a “moving mouse” category or a “hitting keyboard” category. For example, when
changing from a “moving mouse” to a “hitting keyboard” computer learning active, the vertical of the wrist posture exceeds the vertical threshold since the elderly invariably hit the keyboard in alphabets-by-alphabets manner, i.e., use only one hand to key-in a word in alphabets-by-alphabets. Consequently, the corresponding wrist motion is classified as a “hitting keyboard”. Conversely, when moving mouse, the wrist tends to be in a relatively horizontal position, and thus the vertical acceleration is less than the critical vertical value and the corresponding wrist motion is classified as a “moving mouse”.
(a)
(b)
Fig. 2. (a) Wrist vertical acceleration when performing Google Search leaning course. (b) Photograph showing wear of a wearable unit on the wrist.
Fig. 3. Decision-tree manner. To further distinguish the wrist motion of “moving mouse”, “hitting keyboard”, “idle”, and “swing mouse” the vertical wrist acceleration is closely examined. Fig. 2(a) illustrates the measurement signals obtained from the kinematic sensor for the wrist vertical acceleration as an object ware a wearable unit (see Fig. 2(b)) and excuses a simple computer learning course, namely Google Search, including the following execution steps: － － －
Step 1:Move the mouse until the mouse pointer is located at the icon of web explore (e.g., Chorme, Explore, etc.) and then click the icon. Move the mouse until the mouse pointer is located at the search textbox and then click the mouse. Step 2:Hit the keyboard to insert the search key word in the search textbox in alphabets-by-alphabets manner. Step 3:Move the mouse to the location of search target and then click the mouse.
The acceleration profile associated with the moving mouse active has a lower vertical acceleration than that of hitting keyboard. Thus, the wrist recognition algorithm developed in this study exploits this characteristic of the acceleration profiles to distinguish the four motions. Figure 3 illustrates the details of the posture recognition algorithm in flowchart form.
3
Experimental Results
The performance of the WristEye system was evaluated by performing a series of trials using a total of 30 elderly people with various computer abilities. During the trials, the 30 elderly people is further divided into 10 groups and each group is organized by three different kinds of computer abilities of elderly students, i.e., ďź? ďź? ďź?
Class A: The elderly student has various experiences in using computer. Class B: The elderly student only has a little experiences in using computer. Class B: The elderly student has no experiences in using computer.
Each trial of a group performed a Google Search computer learning course as described in Section 2.3. The performance of the proposed WristEye system was quantified by comparing the actual computer abilities of the elderly people with that identified by the wrist recognition algorithm (see Fig. 4). Table I summarizes the overall experimental results of the average time for each kinds of elderly students who successfully perform each steps of Google Search computer learning course. The recognition results show that Class A participants spend average less time for successfully performing each steps of Google Search computer learning course than Class B and Class C participants. This result reflects the actual computer abilities of participants and therefore demonstrates that the proposed WristEye achieves a recognition accuracies compared with the actual computer abilities of participants.
Fig. 4. Graphical user interface of PC-based monitoring system.
Table 1.
Experimental results of the average time(s) for each kinds of elderly students who successfully perform each steps of Google Search computer learning course. abilities wrist motion Time of Moving Mouse (Step 1) Time of Hitting Keyboard (Step 2) Time of Moving Mouse (Step 1)
4
Class A 4.5s±0.23s 6.2s±0.3.1s 2.7s±0.17s
Class B
Class C
4.3s±0.1s 10.8s±0.2s 3.2s±0.11s
8.2s±0.25s 18.3s±1s 7.8s±0.16s
Conclusions
This study has developed a MEMS-based monitor, called WristEye, for identifying the wrist motions of elderly individuals whilst in the computer learning classes. The proposed WristEye applies a kinematic sensor to derive more accurate vertical acceleration signals of the monitoring target for recognition of wrist posture by employing a posture recognition algorithm. These recognition results are then forwarded wirelessly in real time to a backend server for further processing via a Zigbee WSN. Upon receipt of the signals, the backend server extract the corresponding wrist motion (i.e., “moving mouse”, “hitting keyboard”, “idle”, and “swing mouse”). The experimental results have confirmed that WristEye system is capable of identifying a variety of common daily activities of computer learning for elderly individuals in real time with an excellent accuracy. Therefore, it may be considered as a suitable solution for the emerging elderly e-learning paradigm aimed at computer learning institutes located in communities.
References 1. Statistics Bureau and the Director-General for Policy Planning (Statistical Standards), Ministry of Internal Affairs and Communication Home. http://www.stat.go.jp/english/index.htm. 2. “Japan eyes robots to support aging population,” Boston Global Home. http://www.boston.com/news/world/asia/articles/2007/09/18/japan_eyes_robots_to_support _aging_population/. 3. Jupiter Communications（2000）. http://www.jup.com. 4. Pei-Yun Wang, “Trends in Media Usage and Life Style of Mature Ages－A Case Study of 2004 Shih Hsin Communication Databas”, Master Thesis, Taipei, Taiwan (R.O.C), 2006. 5. Morrell, Q.W.＆ Echt, K.V.（1996）Designing Written Instructions for Older Adults Learning to Use Computers, in：A.D. Fisk, W.A. Rogers（Eds.）,Handbook of Human Factors and the Older Adult, Academic Press, San Diego, CA. 6. Jay,T. “Computerphobia: What to do about it.” Educational Technology, vol. 21, pp. 47-48, 1981. 7. IDG 300 Gyroscope Home. http://www.invensense.com/products/idg_300.html 8. ADXL330 Accelerometer Home. http://www.analog.com/en/mems-and-sensors/imemsaccelerometers/adxl330/products/product.html. 9. S.-L. Sun et al., “Multi-sensor optimal information fusion Kalman filter,” Automatica, vol. 40, issue 6, pp.1017-1023, Jun. 2004.