Wearable Mobility Monitoring using a Multimedia Smartphone Platform by Natalie Baddour

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

3153

Wearable Mobility Monitoring Using a Multimedia Smartphone Platform Gaëtanne Haché, Member, IEEE, Edward D. Lemaire, Member, IEEE, and Natalie Baddour

Abstract—Understanding mobility is important for effective clinical decision making in the area of physical rehabilitation. Ideally, a person’s mobility profile in a nonclinical setting, such as the home or community, could be obtained. This profile would include the environment and context in which the mobility takes place. This paper introduces a novel wearable mobility monitoring system (WMMS) for an objective ubiquitous measurement of mobility. This prototype WMMS was created using a smartphone-based approach that allowed for an all-in-one WMMS. The wearable system is freely worn on a person’s belt, such as a normal phone. The WMMS was designed to monitor a user’s mobility state and to take a photograph when a change of state was detected. These photographs were used to identify the context of mobility events (i.e., using an elevator, walking up/down stairs, and type of walking surface). Validation of the proposed WMMS was performed with five able-bodied subjects performing a structured sequence of mobility tasks. System performance was evaluated by its ability to detect changes of state and the ability to identify context from the photographs. The WMMS demonstrated good potential for community mobility monitoring. Index Terms—Acceleration, cameras, mobility, monitoring, multimedia systems, wearable.

I. I NTRODUCTION

OBILITY can be defined as the ability to move independently from one point to another [1] and is essential for maintaining independence. Mobility is required to perform many activities of daily life, such as, cooking, dressing, shopping, and visiting friends. According to Statistics Canada, mobility problems are one of the issues that affect the greatest number of adults [2]. Mobility disabilities can affect an individual’s quality of life, health, productivity, and independence and also affect the lives of their family and the people around them.

Manuscript received September 5, 2010; revised December 15, 2010; accepted December 16, 2010. Date of publication March 28, 2011; date of current version August 10, 2011. This work was supported in part by the Research In Motion, by The Ontario Graduate Scholarships in Science and Technology Program, and by the Ontario Centers of Excellence. The Associate Editor coordinating the review process for this paper was Dr. Salvatore Baglio. G. Haché is with the Ottawa-Carleton Institute for Biomedical Engineering, Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada (e-mail: ghache@hotmail.com). E. D. Lemaire is with the Ottawa-Carleton Institute for Biomedical Engineering, Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada, and also with the Institute for Rehabilitation Research and Development, The Ottawa Hospital Rehabilitation Centre, Ottawa, ON K1H 8M2, Canada. N. Baddour is with the Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIM.2011.2122490

Preserving mobility is paramount for staying independent and active at home and in the community. Accurate and objective mobility assessment is required for decision making in rehabilitation medicine. Such assessments can be used to determine mobility issues outside a hospital environment, evaluate the progress made during and after rehabilitation, and enhance clinical decision making about a rehabilitation program (i.e., assistive devices, exercises, treatment, etc.). Currently, many types of mobility assessments are performed in a clinical setting and are supervised by the rehabilitation physician. These assessments include clinical tests, quantitative measures, and subjective feedback from client to patient. Although clinical mobility tests have a value, such assessment tools may not be appropriate for determining the contributing factors for independent community walking and the impact of the environment on the individual’s mobility [3], [4]. Monitoring the mobility outside a clinical setting is important because mobility in the real world is typically different from the mobility measured in the clinic [5]. Wearable mobility monitoring systems (WMMSs) are designed to be worn on the body and allow mobility monitoring in the person’s home and the community [6]. Many wearable mobility monitoring studies measure biomechanical and/or location parameters [5], [7]–[10], but most lack environmental or contextual information. In community mobility monitoring, contextual information is important since it provides insight on where, how, and on what a person is moving. A camera could provide contextual information from a person’s surrounding environment. Some wearable systems that use contextual information, such as context-aware systems [11] and life logs [12], are not meant for community mobility monitoring for people with physical disabilities. Other context-aware wearable systems use context information to better recognize activities [13]–[15], but the environmental characteristics in which activities take place are not analyzed for their impact on mobility. We propose a novel WMMS that provides unsupervised objective mobility measurements in a cost-effective way, using smartphone technology that has already achieved consumer acceptance. In addition to monitoring biomechanical parameters, our WMMS also aims to identify mobility tasks and their context. This paper uses the smartphone as the central processing hub for data capture, data processing, multimedia capture, outcome storage, and the option for wireless outcome data transmission. The novelty of this approach is the combination of biomechanical task identification methods and context identification via mobile multimedia tools.

3154

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

This WMMS is the first to combine change-of-state detection, sensor-based activity classification, and environmentalimage capture into an integrated system designed to improve the contextual information available for mobility assessment. Photographs provide contextual information that cannot be supplied by inertial sensor systems, such as using an elevator, walking on carpet or grass, or walking in a crowded room. The WMMS provides information on the context and environment in which mobility events take place, which will help identify mobility challenges in a person’s own environment. For this paper, changes of state include starting or stopping an activity, postural changes, walking on stairs, indoor and outdoor transitions, and using transportation. This paper describes the WMMS design from a system perspective, including hardware integration, data processing methods, mobility assessment, and context identification outcomes. Fig. 1.

Sensors and board for attachment to the holster.

II. M ETHODOLOGY A. WMMS Prototype Development The BlackBerry smartphone platform was chosen for the WMMS due to capability, acceptance in the health-care sector, and device/platform security. In our preliminary study, the BlackBerry platform demonstrated the multitasking, communication, and processing capabilities required for a WMMS hub [16]. BlackBerry smartphones possess the required security features, built-in global positioning system (GPS), integrated camera, video recording, Wi-Fi, Bluetooth, data encryption, adequate processing speed, and large storage capacity. Furthermore, newer models provide access to accelerometer raw data that could enable the design of an all-in-one WMMS. A mature Java environment and many secure development interfaces [i.e., application programming interference (API)] are also available with the BlackBerry devices. Based on the latest available BlackBerry Java development environment and API at the time of project inception, BlackBerry Bold 9000 was used in this study. While recently released phones have integrated accelerometers and the potential to test ambient light, the Blackberry Bold 9000 did not possess these features; therefore, an external board with mobility analysis sensors was designed. The external sensors were required because BlackBerry smartphones with all the required capabilities were not available during the project development phase (i.e., accelerometer, GPS, Wi-Fi, Bluetooth, and camera). The external board design, integrated into the phone’s holster (i.e., Smart Holster), provided a flexible approach to add other measurement sensors or tools in the future. The board was connected to the BlackBerry via Bluetooth. As shown in Fig. 1, a microcontroller CY8C27443 (Cypress Semiconductor Corporation, San Jose, CA, USA), Bluetooth Module F2M03GLA (Free2Move AB, Halmstad, Sweden), triaxial accelerometer LIS344alh (STMicroelectronics, Geneva, Switzerland), and light sensor APDS-9005 (Avago Technologies Limited, San Jose, CA, USA) were mounted on the board. A rechargeable lithium battery powered the board. This external board could continuously run

for approximately 14 h on one charge. However, the length of time that the WMMS could run without recharging was approximately 3 h with the BlackBerry’s Li-Ion 1500 mAh battery. The sensors were not put to sleep if no activity was detected. To start and stop sensor data sampling, sampling delay commands were sent from the BlackBerry to the board microcontroller. Bluetooth serial port profile (SPP) protocol was used for communication with the external board. The data from the accelerometer and the light sensor were stored in a buffer on the microcontroller. At every sampling delay, the last data packet stored in the buffer was sent to the host (BlackBerry) via SPP. A data packet (21 B) was sent to the host (BlackBerry) every 20 ms (50 Hz). In the data packet, 6 B were required for raw acceleration data and 2 B for light sensor data. The data packet-sampling rate equaled the data-sampling rate; therefore, the sampling rate for the accelerometer and light sensor was 50 Hz. All signal processing and the state detection algorithms were implemented and performed on the BlackBerry. These algorithms are subsequently discussed. Following signal processing, the outcome data for each record were stored on the BlackBerry’s 8-Gb secure-digital card. The WMMS software was developed using the BlackBerry Java Development Environment version 4.6.1.

B. Signal Processing The accelerometer and light sensor sampling rate was 50 Hz since body-fixed accelerometers placed at the waist must be able to measure acceleration with frequencies up to 20 Hz [17]. The accelerometer was calibrated to remove the directcurrent offset [17]. The calibration method involved rotating the sensor to known angles, as suggested in the manufacturer datasheet [18]. For example, under static conditions, the output from an axis pointed toward the center of the earth should be equal to 1 g. If the axis is then rotated by 180◦ , its output should be equal to −1 g. The sensitivity s and the offset o of a

HACHÉ et al.: WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM

particular axis of the sensor can be calculated using the following equations: (umax − umin ) 2 (umax + umin ) o= 2 s=

(1) (2)

where umax and umin are the maximum and minimum acceleration measured during the rotation between ±1 g. The output a of one accelerometer can then be expressed as a=

(u − o) s

(3)

where u is the uncalibrated acceleration. On the Blackberry, the calibrated acceleration data were passed through a median filter (n = 3) to remove spikes [7]. The signal was then passed through a low-pass digital filter (0.25 Hz) to separate the static component from the dynamic component [19]. A nonoverlapping sliding window of 1.02 s (51 samples) [7], [20] was used to extract signal features from the static and dynamic acceleration components and the average light sensor output. These extracted features were used as input to calculate the parameters that determined the user’s mobility state. C. Mobility Features Various mobility and activity classification variables were selected from the literature to help determine mobility changes of state. One of the largest challenges in the development of the WMMS is the integration of individual components into a coherent and functional system that meets the overall system objectives. Therefore, the decision was made to focus on the system-level development and to incorporate previously proven mobility classification variables. Preliminary evaluations, over all activities, with two able-bodied subjects were used to verify these measures and to determine the threshold values for activity classification. 1) Inclination Angle: Inclination angle was used to help classify posture [9], [21]–[23] and identify postural transition [24]. The inclination angle was calculated for every window period using the two-axis method presented by Freescale Semiconductor [25]. The static components of the acceleration, obtained from low-pass filtering (0.25 Hz), were used to calculate the inclination angle Φ using the two-argument arctangent function, i.e., GAz ◦ (4) () Φ = arctan GAy where GAz and GAy are the averaged static accelerations of the z-axis (forward) and y-axis (vertical), respectively. An offset of 180◦ was then added to the inclination angle to give a range of 0 to 360◦ . Ideally, an inclination value of 90◦ should be obtained when the person lays on their belly, a value of 180◦ when the person stands, and a value of 270◦ when the person lays on their back. The averaged inclination angle was compared with high and low thresholds (200◦ and 160◦ ) to determine if the person was in a standing position. If the person was

3155

not standing, the angle was compared with different high and low thresholds (320◦ and 250◦ ) to determine if the person was lying on their back. If both conditions were false, the position was determined to be somewhere in between. The threshold values were based on the study by Culhane et al. [23] and our preliminary observations. 2) Standard Deviation of Vertical Acceleration: Standard deviation is a well-supported measure for activity classification [17], [21]–[23], [26]. Since most daily activities can be classified by changes in vertical axis acceleration, vertical acceleration (y-axis) was used to differentiate between static and dynamic states by comparing the standard deviation of the yaxis acceleration with two thresholds (static and dynamic). This algorithm was defined as the double threshold (DT) algorithm. With a DT algorithm, if the initial state is static, the activity classification remains static until the signal crosses the dynamic threshold (0.120 g). Then, the state is set to dynamic and stays dynamic until the signal passes below the static threshold (0.075 g). When the person stands still, the standard deviation should be close to 0 g. 3) Skewness of Vertical Acceleration: The skewness value of the vertical acceleration is a time-domain feature that was used by Baek et al. [26] to differentiate walking/running from going up/down stairs. Skewness is a measure of asymmetry of the y-axis (vertical) acceleration about the average acceleration (e.g., a skewness value of 0 indicates a symmetric distribution of accelerations). Skewness of the y-axis acceleration was calculated as n skewness = (n − 1)(n − 2) i=1 n

xi − x σ

3 (5)

where n is the number of points, xi the y-axis acceleration at point i, and σ and x are the standard deviation and the mean of the y-axis acceleration signal, respectively. The DT algorithm was also applied to the skewness value but only when in a dynamic classification state. Based on preliminary works, skewness values greater than 1 were observed for stair descent [16], [26]. Skewness increased when ascending stairs, but values were less than stair descent. Similar skewness values were sometimes observed for both stair ascent and normal walking, which could result in a false positive change-of-state detection. High (0.6) and low (0.2) thresholds were chosen to detect stair descent and detect stairs ascent with minimal false positive results. 4) SMA of Three-Axis Acceleration: Signal magnitude area (SMA) is another viable activity and mobility measure [7], [9]. The SMA normalized to the length of the signal T can be calculated from  T  T T 1 |ay |dt + |az |dt (6) SMA =  |ax |dt + T t=0

t=0

where t is the time in seconds and ax , ay , and az are the acceleration of x-, y-, and z-axes, respectively. During preliminary testing, peaks occurred in the SMA signal for sitting, rising from a chair, and lying down. The SMA also

3156

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

Fig. 2. User’s state detection algorithm.

helps identify activity intensity changes that could indicate a change of state. Therefore, three thresholds were used, and three states were determined: no peak with normal intensity, no peak with increased intensity, or a peak. A DT algorithm was used to determine increases in intensity and peak detection. The “increased intensity” low and high thresholds were 0.100 and 0.190 g, respectively. Since an increase in intensity should happen when the person is moving, the algorithm verified that the person was in a dynamic state when detecting the “no peak with increase in intensity” state. The peak detection low and high thresholds were 0.100 and 0.320 g, respectively. 5) Light Intensity: The light sensor measured light intensity of the ambient environment. To detect indoor and outdoor states during the day, the DT algorithm was applied to the light intensity feature. Based on preliminary light sensor calibration, a high threshold of 1000 mV and a low threshold of 300 mV would differentiate outdoor from indoor states during the day. However, during preliminary testing while driving, many false changes of state were recorded due to the light intensity changes in the car. To remove those false changes of state, the DT

algorithm was only applied to the light intensity feature when not moving in a vehicle. 6) GPS Speed: GPS data have been used in mobility monitoring to complement motion data, improve activity recognition, and provide contextual data [5], [8], [27]–[29]. Therefore, GPS location coordinates and speed were collected and added to the WMMS output file when available (outdoors). The GPS data were extracted from the BlackBerry Bold every 9 s. For this WMMS prototype, only the speed was considered for the change-of-state detection algorithm, to detect whether the person was in a vehicle. This feature was passed through a DT algorithm, with the low and high thresholds of 1 and 7 m/s, respectively. When the speed was above the 7 m/s threshold, the state was identified as “in a vehicle.” The state stayed the same until the GPS speed measured below the low threshold. D. Determination of State and Change of State The user’s state was assessed for every data window (1.02 s) (see Fig. 2). A change of state was determined by subtracting

HACHÉ et al.: WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM

the three previous states from the current state. If the answer was different from zero for one of the subtractions, a change of state occurred. As a result of a change of state, the algorithm determined if a picture should be taken. From our camera performance test, approximately 0.7 s was required to take a picture, and another 0.9 s was required before another picture could be taken. Therefore, it was decided to wait at least two windows before taking another picture (i.e., 3 s later). Picture encoding was set to Joint Photographers Expert Group, i.e., 640 × 480 pixels, and the quality was set to normal. The memory size of a picture with this encoding was 10 to 70 KB. E. Mobility Evaluation A convenience sample of five able-bodied subjects (three males and two females; age: 36.6 ± 6.4 years old; height: 173.82 ± 13.17 cm; weight: 69.32 ± 16.09 kg) was recruited from the staff at The Ottawa Hospital Rehabilitation Center (TOHRC, Ottawa, Canada) and the community. Informed consent was obtained from all the participants. People with injuries or gait deficits were excluded. Data collection took place within the TOHRC (hallways, elevator, stairs, and Rehabilitation Technology Laboratory), outside the TOHRC on a paved pathway, and in a car driving around the Ottawa Hospital campus. The subjects were asked to wear the WMMS on their waist, with the holster attached on a belt, on their right hip, with the device pointing forward. No additional instructions were given for positioning the instrumented holster. The subjects were asked to follow a predetermined path with a series of mobility tasks. For every trial, the subjects were filmed with a digital video camera. Three trials per subject were performed. The digital camera was synchronized with the WMMS by having the subject block the light sensor with their hand for 5 s when starting data collection. A digital video was necessary to validate the change-of-state detection, to determine the change-ofstate timing, and to provide context information for validation. The predefined mobility tasks were sequentially performed. Moving from one task to another should trigger a change of state, providing a total of 38 changes of state per trial. The given sequence of tasks was as follows: 1) stand; 2) walk on level ground (25 m); 3) stand-to-sit transition; 4) sit; 5) sit-tostand transition; 6) walk on level ground (60 m); 7) stand and wait for an elevator; 8) walk to get in the elevator; 9) take the elevator to the second floor; 10) walk out of the elevator and keep walking on level ground (30 m); 11) stand and wait for the elevator; 12) walk to get in the elevator; 13) take the elevator to the first floor; 14) walk out of the elevator and keep walking on level ground (50 m); 15) ascend stairs (13 steps); 16) walk on stair intermediate landing (level ground for 1.5 m); 17) ascend stairs (13 steps); 18) walk on level ground (30 m); 19) descend stairs (13 steps); 20) walk on stair intermediate landing (level ground for 1.5 m); 21) descend stairs (13 steps); 22) walk on level ground (20 m); 23) stand-to-lie transition; 24) lie on back; 25) lie-to-stand transition; 26) walk on level ground (45 m); 27) ascend and descend a 7◦ angled ramp (5 m); 28) walk on level ground (15 m); 29) transition indoor/outdoor and keep walking outdoors on level ground (60 m); 30) transition out-

3157

door/indoor and keep walking indoors on level ground (40 m); 31) transition indoor/outdoor and keep walking outdoors on level ground (30 m); 32) stand-to-sit transition to get in a car; 33) sit in the car; 34) start car and ride around campus; 35) stop car ride; 36) sit-to-stand transition; 37) walk on level ground (30 m); 38) transition outdoor/indoor and keep walking indoors on level ground; 39) and finally stand. Changes-of-state timing from the digital video were compared with the WMMS change-of-state timestamps. Each data window was analyzed to determine if the change of state was a true or false positive, or true or false negative. The number of true and false positives and true and false negatives were used to calculate WMMS sensitivity and specificity, i.e., Sensitivity =

#TruePositives × 100 #TruePositives + #FalseNegatives (7)

Speciﬁcity =

#TrueNegatives × 100. #TrueNegatives + #FalsePositives (8)

Two research assistants independently evaluated the BlackBerry Bold images. The evaluators were asked to identify the context (i.e., stairs, elevator, ramp, floor, outdoor, etc.) from the digital images. Only the images taken due to a real change of state (true positives) were evaluated. The evaluators were given a list of context options to choose from. The evaluators were not informed of the mobility tasks represented by the images prior to evaluation. The results from the two evaluators were then analyzed to determine if the context was successfully identified from the pictures. Table II shows the various contexts that were identified for each mobility task. For the context “type of ground,” the evaluators also had to choose between floor, grass, and pavement. III. R ESULTS An overall sensitivity of 77.7% ± 2.5% and a specificity of 96.4% ± 2.2% were obtained across all the activities. Averaged sensitivity of the different change-of-state categories are given in Table I. Context identification from the photographs had an overall success rate of 72.5% ± 33%. Table II shows a summary of the results divided by type of context. IV. D ISCUSSION Understanding mobility in a nonclinical setting is important when making rehabilitation/health-care-related decisions. Our results suggest that smartphones, particularly the newer versions to come, have great potential for community mobility monitoring. By using the integrated camera, information on the context/environment in which mobility events take place can be identified. Additionally, the BlackBerry has the necessary processing power to log and process data, run algorithms, collect GPS data, and take pictures, all without data loss. Our approach of taking a photograph when a change of state occurred demonstrated that mobility tasks such as taking

3158

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

TABLE I P ERFORMANCE R ESULTS OF THE D IFFERENT C HANGE - OF -S TATE C ATEGORIES

TABLE II I MAGES C ATEGORIZATION R ESULTS FOR THE D IFFERENT T YPES OF C ONTEXT

an elevator, going up stairs, and riding in a car could be identified from the images. The type of walking surface (i.e., floor and pavement) could also be identified. From the study by Shumay-Cook et al. [30], terrain was used to differentiate older adults with mobility disabilities from people without mobility issues. Since terrain has been related to fall risk [31], terrain contextual information could be used to help in understanding the underlying causes of falls and help with fall prevention. The use of images to capture context and environment in mobility monitoring could also help to monitor activity avoidance. Mobility disability has been characterized by a reduction in the number and type of environment challenges [32]. Activity avoidance could lead to a reduction in mobility, which could lead to further deterioration in physical status and social interactions. Since the camera was pointing forward from the pelvis, the WMMS did not provide the downward angle that would be required for viewing stairs during descent. A wide-angle or

sphere lens could improve context identification by providing a larger field of view. A short video of several seconds, or multiple pictures of the same context, could help with context identification. However, preliminary testing of the BlackBerry camera showed that multiple still images could not be taken with the Bold 9000 smartphone camera (i.e., a picture could only be taken every 1.6 s). The use of short video clips could be explored in future research. Additionally, as with many cell phones, the camera performed moderately well under low-light conditions, causing images to be blurry and dark. The flash was not used for image capture since multiple camera flashes would be obtrusive and rapidly drain the battery. The camera location on the front of the pelvis could also cause dark images due to subjectâ&#x20AC;&#x2122;s clothes hiding the camera lens, although lens obstruction did not occur in this study. Our algorithm demonstrated good results at detecting changes of state caused by walking start/stop (97.4%), postural

HACHÉ et al.: WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM

change (97.8%), and walking down stairs (100%). These results were obtained using an accelerometer attached to the smartphone’s holster that was worn “like a usual phone” on the subject’s waist. The evaluation protocol provided a reallife situation where the mobility tasks were consecutively and freely performed. It is interesting to note that our approach produced similar results to other studies that used a different wearing location [22], several accelerometers [23], or where the evaluation was performed in a controlled laboratory setting executing discrete mobility tasks [9]. The WMMS poorly performed at detecting the change of state due to stair ascent and walking on a ramp. As with stair descent, skewness was used to detect stair ascent. Since literature was lacking on ramp detection using accelerometer signals, the same method was explored to detect ramp navigation. The choice of the skewness feature was based on the work by Baek et al. [26], which obtained a classification rate of 93% for upstairs and 87% for downstairs. Some differences with our methods were accelerometer location, where the accelerometer of Baek et al. was worn on the lateral side of the pelvis instead of the front, their additional signal processing and analysis features, a larger data window, and the use of a neural network. More complex algorithms could be explored since the current generation of smartphones have greatly enhanced processing power. Moreover, in older populations or individuals with mobility disabilities, a slow/almost-stopping movement could be present before attempting walking up a ramp or stairs; therefore, a picture would be taken to help in identifying the mobility task. The healthy young subjects who participated in this study had a minimal change in forward progression when transitioning from level ground to stairs or ramps. A light sensor was added to the WMMS to detect outdoor and indoor conditions. Our approach of selecting outdoor/indoor thresholds did not perform as well as anticipated. A change in light intensity level could have been a better measure instead of using fixed outdoor/indoor thresholds since changes could be detected on overcast/cloudy days. As seen in the image evaluation results, pictures could be used to detect indoors/outdoors. The GPS speed was used to detect if a person was in a vehicle. For the trials where GPS satellites were detected, the change caused by being in a vehicle was well classified. While the initiation of being in a vehicle can be identified using the camera images, WMMS classification was delayed by the 9-s sampling interval for GPS speed and the 7 m/s threshold. The main problem with GPS during evaluation was the time required to detect satellites and initiate GPS data acquisition. Based on preliminary tests, up to 30 min could be required to detect GPS satellites, depending on the exterior conditions. The BlackBerry was set to autonomous mode to detect location, which is slower but more precise than using cell-site mapping. For our WMMS, GPS speed was required for detecting vehicle riding. Cell tower-based location could be investigated since location estimation occurs faster and would work indoors and in cloudy weather, although this method is of lower precision. Improvement to the change-of-state algorithm is necessary to enhance detection of stairs ascent, ramp navigation, and indoor/outdoor conditions. Additional signal processing could be added offline to improve classification of the raw data and to

3159

compile the results into clinically relevant metrics. Automated digital images processing to automatically identify context will need to be considered if the WMMS is to be widely used in health-care patient monitoring applications. Time-motion analysis should also be performed in future studies to examine the time requirements for machine-assisted visual inspection of the reference images. For example, custom clinical mobility analysis software could allow the person analyzing the mobility data to quickly jump from image to image and view the picture and activity classification information or view images grouped by activity classification type (e.g., just walking and just stairs). The acceptable time for a technician to extract meaningful information for clinical use remains unknown. Implementing the change-of-state algorithm to the new generation of BlackBerry smartphones should be considered since new versions provide raw accelerometer data and improved camera performance. This will remove the need for the external board for the activities evaluated in this paper. However, other external sensors could be integrated into the WMMS using the new WMMS software and Bluetooth communications, such as for pressure or electromyography analyses. Future generations of our WMMS will have to improve the software to include a power-saving mode. The BlackBerry battery in this study only lasted 3 h. A larger battery is required for this type of application. The data from five able-bodied subjects demonstrated the feasibility of a smartphone-based WMMS for community mobility monitoring and therefore will be used to motivate future work in mobility monitoring. More evaluations are required to verify how well the WMMS would perform in real-life situations where walking speed, movement obstacles, and environments are constantly changing. Testing on people with mobility disabilities is part of our objectives for the next generation of this WMMS. Outcomes from people with physical disabilities are important to validate the signal processing and algorithm methods. Extracting other signal features may be required, as well as modifying calibration methods to adapt the sensitivity of the change-of-state detection for different levels of mobility (e.g., walking slower and abnormal gait pattern). Another population that could possibly benefit from this WMMS are wheelchair users. Health-care professionals would benefit by knowing how a person uses their wheelchair or scooter in the community, including the proportion of time for wheeled versus walking mobility. Having images taken while using the wheelchair could also highlight the types of environments that challenge the wheelchair user. V. C ONCLUSION Maintaining independent mobility at home and in the community plays an important role in an individual’s independence, quality of life and health, and in the lives of their family and the people around them. Nonclinical measurement of mobility and the context in which mobility events take place can help with these roles. Our WMMS approach to respond to the need for community mobility assessment tools has shown great potential. The smartphone approach provides an accessible and

3160

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

cost-effective option that can easily be implemented in society. Adding the camera to the WMMS suggested that images could help identify mobility tasks such as walking up stairs and taking an elevator, as well as identifying the type of ground and the type of vehicle. However, the limitations should be addressed to improve performance. Interesting future work exists for smartphone-based WMMS. ACKNOWLEDGMENT The authors would like to thank K. Heggie for his technical work on the external sensor board and C. Kendell, S. Doyle, and H. Wu for their assistance with data collection and analysis. The Ottawa Hospital Rehabilitation Centre is acknowledged for their facility support. R EFERENCES [1] A. E. Patla and A. Shumway-Cook, “Dimensions of mobility: Defining the complexity and difficulty associated with community mobility,” J. Aging Phys. Activity, vol. 7, no. 1, pp. 7–19, Jan. 1999. [2] Statistics Canada, Participation and Activity Limitation Survey 2006: Analytical Report, Minister Industry, Ottawa, Canada, 2007, Accessed: 25 Nov. 2009. [Online]. Available: http://dsp-psd.pwgsc.gc.ca/collection_ 2007/statcan/89-628-X/89-628-XIE2007002.pdf [3] T. Lam, V. K. Noonan, and J. J. Eng, “A systematic review of functional ambulation outcome measures in spinal cord injury,” Spinal Cord, vol. 46, no. 4, pp. 246–254, Apr. 2008. [4] R. Corrigan and H. McBurney, “Community ambulation: Environmental impacts and assessment inadequacies,” Disabil. Rehabil., vol. 30, no. 19, pp. 1411–1419, 2008. [5] M. Ermes, J. Pärkka, J. Mantyjarvi, and I. Korhonen, “Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions,” IEEE Trans. Inf. Technol. Biomed., vol. 12, no. 1, pp. 20–26, Jan. 2008. [6] C. N. Scanaill, S. Carew, P. Barralon, N. Noury, D. Lyons, and G. M. Lyons, “A review of approaches to mobility telemonitoring of the elderly in their living environment,” Ann. Biomed. Eng., vol. 34, no. 4, pp. 547–563, Apr. 2006. [7] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “Detection of daily physical activities using a triaxial accelerometer,” Med. Biol. Eng. Comput., vol. 41, no. 3, pp. 296–301, May 2003. [8] D. A. Rodríguez, A. L. Brown, and P. J. Troped, “Portable global positioning units to complement accelerometry-based physical activity monitors,” Med. Sci. Sports Exerc., vol. 37, no. 11 Suppl., pp. S572–S581, Nov. 2005. [9] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G. Celler, “Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring,” IEEE Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 156–167, Jan. 2006. [10] E. Farella, A. Pieracci, L. Benini, and A. Acquaviva, “A wireless body area sensor network for posture detection,” in Proc. 11th IEEE Symp. Comput. Commun., Cagliari, Sardinia, Jun. 2006, pp. 454–459. [11] C. Randell and H. Muller, “Context awareness by analysing accelerometer data,” in Proc. 4th Int. Symp. Wearable Comput., Atlanta, GA, Oct. 2000, pp. 175–176. [12] Y. Lee and S.-B. Cho, “Extracting meaningful contexts from mobile life log,” in Proc. Intell. Data Eng. Automated Learn., Birmingham, U.K., Dec. 2007, pp. 750–759. [13] G. H. Jin, S. B. Lee, and T. S. Lee, “Context awareness of human motion states using accelerometer,” J. Med. Syst., vol. 32, no. 2, pp. 93–100, Apr. 2008. [14] T. Choudhury, G. Borriello, S. Consolvo, D. Haehnel, B. Harrison, B. Hemingway, J. Hightower, P. Klasnja, K. Koscher, A. LaMarca, J. A. Landay, L. LeGrand, J. Lester, A. Rahimi, A. Rea, and D. Wyatt, “The mobile sensing platform: An embedded activity recognition system,” IEEE Pervasive Comput., vol. 7, no. 2, pp. 32–41, Apr./Jun. 2008. [15] U. Maurer, A. Rowe, A. Smailagic, and D. Siewiorek, “Location and activity recognition using eWatch: A wearable sensor platform,” in Proc. Ambient Intell. Everyday Life, 2006, pp. 86–102. [16] G. Hache, E. Lemaire, and N. Baddour, “Development of a Wearable Mobility Monitoring System,” in Proc. Can. Med. Biological Eng. Conf., Calgary, Canada, May 2009.

[17] C. V. C. Bouten, K. T. M. Koekkoek, M. Verduin, R. Kodde, and J. D. Janssen, “A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity,” IEEE Trans. Biomed. Eng., vol. 44, no. 3, pp. 136–147, Mar. 1997. [18] STMicroelectronics, MEMS Inertial Sensor—High Performance 3-Axis ±2/±6g Ultracompact Linear Accelerometer, LIS344ALH Datasheet, Rev.3, STMicroelectronics, Geneva, Switzerland, 2008. [19] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “Accelerometry: Providing an integrated, practical method for long-term, ambulatory monitoring of human movement,” Physiol. Meas., vol. 25, no. 2, pp. R1–R20, Apr. 2004. [20] S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, D. Howard, K. Meijer, and R. Crompton, “Activity identification using body-mounted sensors—A review of classification techniques,” Physiol. Meas., vol. 30, no. 4, pp. R1–R33, Apr. 2009. [21] P. H. Veltink, H. B. J. Bussmann, W. De Vries, W. L. J. Martens, and R. C. Van Lummel, “Detection of static and dynamic activities using uniaxial accelerometers,” IEEE Trans. Rehabil. Eng., vol. 4, no. 4, pp. 375–385, Dec. 1996. [22] G. M. Lyons, K. M. Culhane, D. Hilton, P. A. Grace, and D. Lyons, “A description of an accelerometer-based mobility monitoring technique,” Med. Eng. Phys., vol. 27, no. 6, pp. 497–504, Jul. 2005. [23] K. M. Culhane, G. M. Lyons, D. Hilton, P. A. Grace, and D. Lyons, “Longterm mobility monitoring of older adults using accelerometers in a clinical environment,” Clin. Rehabil., vol. 18, no. 3, pp. 335–343, Mar. 2004. [24] B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, C. J. Bula, and P. Robert, “Ambulatory system for human motion analysis using a kinematic sensor: Monitoring of daily physical activity in the elderly,” IEEE Trans. Biomed. Eng., vol. 50, no. 6, pp. 711–723, Jun. 2003. [25] Freescale Semiconductor, Application Note 3461, Rev 2 Tilt Sensing using Accelerometers Sensors, pp. 2–4. [26] J. Baek, G. Lee, W. Park, and B. -J. Yun, “Accelerometer signal processing for user activity detection,” in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Berlin, Germany: Springer-Verlag, 2004, pp. 610–617. [27] J. S. Frank and A. E. Patla, “Balance and mobility challenges in older adults: Implications for preserving community mobility,” Amer. J. Prev. Med., vol. 25, no. 3 Suppl 2, pp. 157–163, Oct. 2003. [28] P. J. Troped, M. S. Oliveira, C. E. Matthews, E. K. Cromley, S. J. Melly, and B. A. Craig, “Prediction of activity mode with global positioning system and accelerometer data,” Med. Sci. Sports Exerc., vol. 40, no. 5, pp. 972–978, May 2008. [29] G. MacLellan and L. Baillie, “Development of a location and movement monitoring system to quantify physical activity,” in Proc. Conf. Human Factors Comput. Syst., 2008, pp. 2889–2894. [30] A. Shumway-Cook, A. E. Patla, A. Stewart, L. Ferrucci, M. A. Ciol, and J. M. Guralnik, “Environmental demands associated with community mobility in older adults with and without mobility disabilities,” Phys. Ther., vol. 82, no. 7, pp. 670–681, Jul. 2002. [31] J. Stokes and J. Lindsay, “Major causes of death and hospitalization in Canadian seniors,” Chronic Dis. Can., vol. 17, no. 2, pp. 63–73, 1996. [32] A. Shumway-Cook, A. Patla, A. Stewart, L. Ferrucci, M. A. Ciol, and J. M. Guralnik, “Environmental components of mobility disability in community-living older persons,” J. Amer. Geriatrics Soc., vol. 51, no. 3, pp. 393–398, Mar. 2003.

Gaëtanne Haché (M’08) received the Bachelor of Applied Science in electrical engineering from Université de Moncton, Moncton, NB, Canada, in 2001 and the M.A.Sc. degree in biomedical engineering from the University of Ottawa, Ottawa, ON, Canada in 2010. She worked for six years as an Engineer in the high-tech sector. She is currently working with the Department of Biomedical Engineering, Children’s Hospital of Eastern Ontario, Ottawa.

HACHÉ et al.: WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM

Edward D. Lemaire (M’09) received the B.Sc. degree in kinanthropology and the M.Sc. degree in biomechanics from the University of Ottawa, Ottawa, ON, Canada, in 1984 and 1988, respectively, and the Ph.D. degree from the University of Strathclyde, Glasgow, U.K., in 1998. He is a Clinical Researcher with the Institute for Rehabilitation Research and Development, The Ottawa Hospital Rehabilitation Centre, Ottawa, and an Associate Professor with the Faculty of Medicine, University of Ottawa. He has extensive experience with research involving technology to enhance mobility for people with physical disabilities, particularly in the areas of prosthetics and orthotics, computedaided technologies, and telerehabilitation applications. Dr. Lemaire is the President of the Canadian National Society of the International Society for Prosthetics and Orthotics.

3161

Natalie Baddour received the B.Sc. (Physics) degree from the Memorial University of Newfoundland, St. John’s, NL, Canada in 1994, the M.Math. degree from the University of Waterloo, Waterloo, ON, Canada in 1996, and the Ph.D. degree in mechanical engineering from the University of Toronto, Toronto, ON in 2001. Following postdoctoral work at the University of Toronto and the University of Bath, Bath, U.K., she joined the Department of Mechanical Engineering, University of Ottawa, Ottawa, ON, where she is currently an Associate Professor. Her research interests include mathematical methods and algorithms, with applications to dynamics, vibrations, and biomedical engineering.