CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 1 of 6
Comparison of Accuracy, Precision and Response Time with Pen Input Tablet and Wireless Mouse in Two Different Positions Animesh Tripathi ISE 210, San Jose State University, San Jose, CA
1
This is study aims to understand the effects on accuracy, precision and response time in basic point and click task. The accuracy, precision and response time are compared with different relative position of the mouse and pen input tablet (PIT) with respect to the screen. In this study with 6 participants we discovered that overall accuracy with PIT was higher compared to a mouse. Precision was better with a mouse, and the average response time varied though having other internal consistency. INTRODUCTION The study is designed to understand how people perform on tasks of accuracy, precision and response time using a pen input table and a wireless mouse. An experiment was conducted with (x) participants where they play a game to test their accuracy and precision skills where the mouse and pen input tablet was kept in two different position relative to the screen. RELATED WORK Kotani, K., & Horii, K. (2003) found Pen input tablet have relatively smaller learning period when compared to a mouse. Overall load at flexor digitorum superficialis and extensor digitorum was reduced by 5 to 10 percent when the pen-tablet was used for the tasks. Müller, Tomatis, & Läubli, T. (2010) conducted an experiment in which participants using the a mouse outperformed consistently for 5 days, where as in the research study conducted by Kotani and Horii the participant’s PIT performance surpassed mouse’s performance on the 3rd day. Both of these research leaves room for the assumption where the devices were positioned and task were not specifically designed to measure accuracy precision and response time. However, Müller, Tomatis, & Läubli, T. (2010) did assess the performance of dgitorum superficialis and extensor digitorum in their tasks. Both research teams find no significant effects on the trapezoidal muscle while using either pen or mouse. Cook & Kothiyal (1998) studied the effected of mouse position with respect to the relative distance with the keyboard and consequent muscular load on neck, shoulder and arms of the user. To explore the overall effectiveness of how well user would perform using either one of the tool, Vogel and Balakrishnana (2010)
found that participants took longer to complete tasks on a PIT compared to mouse. It was attributed to a reduction in accuracy in the corner of the tablet. Other issues were noted in the research, Precision, occlusion, egronomical issues, and several other limiting factors. Due to a lack of intuitive left and right click mechanism on a PIT longer input times were reported. Similarly, a tap using a pen is both a click and position input for the pen, this does not occur in the mouse. A double click on the mouse is very different from a double click on a tablet input. While at the same time the PIT reported a higher accuracy and clarity with handwriting input. Second issue was reported of occlusion, when the software recognizes the palm as a source of input causing inaccurate input of the task. Ergonomically, pen input table would be used in 3 dimensional space for a majority of the tasks. Whereas a mouse is predominantly is used in two dimensions, meaning expect for click with finger, the mouse is moved only in X and Y coordinate. For a pen input tablet a tap on the pen has to be done in the Z axis, adding extra load on the write. Mouse when used on a table top have the whole empty desk space as a platform to interaction. A mouse can be quickly moved to a corner of the screen while by moving the mouse as far as we like, however a pen input tablet is bound by the physical dimension of the tablet space making the corners of a GUI hard to hit. Vogel and Balakrishnana (2010) reported that in case of direct pen input table such as the ones artists use, a lack of keyboard is felt by users. While mouse users would reach out to the keyboard to for a quick undo (Crtl+Z) a pen user would have to reach out to additional steps to complete the same task. When looking at accuracy and precision Mizobuchi and Yasumura (2004) conducted an experiment comparing tapping and circling as two mode selections. They found that there were no significant different between the error rates of the two modes. They also
CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 2 of 6
noted that tapping took significantly less time when compared to circling the object. When using a graphics tablet Kim et al (2015) reported 40% increase in accuracy for selection of curved target. They also reported a gradual decrease in the selection error over 15 weeks of use, however mouse proved to be the least accurate in selecting complicated images. The comparison of response time between mouse and PIT requires more examination.
participant reported having any prior experience in using the pen input tablet.
METHOD
A quite room within the university premise was chosen where the participant would complete the task at one at a time.
Design of the experiment The experiment was designed to assess user’s accuracy, precision and response time using an online game. http://www.aimbooster.com/ we can measure the accuracy and precision of the participants. While at the same time be able to control the exact size of the target, speeds of appearance of the target can be controlled. The games would be played using the mouse and pen input tablet in two different positions. Position A next to the laptop to the side (fig. 1a) and Position B in front of the screen in the light of sight (fig. 1b). After each session with the input device at one position they were requested to fill out a NASA TLX questionnaire reporting their mental and physical workload.
Fig 1A. Arrangement of Mouse at Position A and Pen Input table at Position B
Fig 1B. Arrangement of Mouse at Position B and Pen Input table at Position A
Participants 6 participants volunteered to participate in the study. Average age of the participants were 29.1 ranging from 24 to 45 (3 female and 3 male). Only one participant used the magic mouse regularly, while rest had little to no prior experience with the equipment. Two participants reported for regular gaming activities with a mouse however not with an Apple’s magic mouse. None of the
Equipment • • •
MacBook Pro (Retina, 15-inch, Late 2013) Wacom Intuos Small Touch and Input Pen Apple’s Magic Mouse-Wireless
Environment
Procedure At the beginning of the session the participants were asked some preliminary questions about their previous experience with equipment. They informed that at any point they wished to take a break after completing a task they were free to do so. Firstly, the participants were allowed to play all three games using the mouse at the position to gain familiarity with the mouse as well as the experimenter giving a brief description of how all games were played. Once comfortable the participants were requested to play the games of accuracy, precision and response time with the mouse on position A. In the game of accuracy 1.5 targets of 70 pixels in size would appear on the screen every second. The participant would then have to reach out using the input device and click to hit the target before it disappeared. If the participant would miss hitting a target before it disappeared a life point would be automatically be deducted out of 10. If they ran out of lives the game was over, else the game was set to last only 1 min. In the game of precision, 10-pixel target would appear on the screen for 2 seconds and the participant would have to reach in using the input device and click at the target to score a point. This game had no life point, instead was only meant to last for a minute. They were free to trade precision over number of target as there was no penalty of missing a target. The game recorded the average response time of hitting target and thus were used in auxiliary analysis of the performance. For simple response time, the game was to be played for a minute where a simple 50-pixel target would appear at the center of the screen. Participant was requested to click on the target as soon as it appeared, however, the time interval within which that targets appeared were random. After playing all three games, participants were then requested to complete the NASA Task Load Index
CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 3 of 6
questionnaire, using the following dimensions they reported their workload of playing the games using a specific device on designated position, Mental Demand (MD): How much mental and perceptual activity was required (e.g. thinking, deciding, calculating, remembering, looking, searching, etc) Physical Demand (PD): How much physical activity was required (e.g. pushing, pulling, turning, controlling, activating, etc) Temporal Demand (TD): How much time pressure did the participants feel due to the rate of pace at which the tasks or task elements occurred. Performance (P): How successful they thought they were in accomplishing the goals of the task set by the experimenter. How satisfied they were with their performance in accomplishing aforementioned goals. Effort (E): How hard they had to work (mentally and physically) to accomplish the reported level of performance. Frustration (F): How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent they felt during the tasks at certain position of the task? Then, the mouse was replaced with the pen input table for position A, and process was repeated again. Once again experimenter permitted the participant to play using the pen input tablet to gain comfort using the device to play the same games. The three game were played in the same order as the first time with followed with the NASA TLX questionnaire Once completed, the mouse was then placed at Position B and same three games were played, followed by self reporting their workload in the NASA TLX questionnaire while keeping in mind the new position of the device. Finally, the mouse was replaced with pen input tablet at Position B and same process was repeated again.
5.
NASA TLX Measure
6 Standard workload measures were recorded. (MD, PD, TD, E, F P)
Table 1. The list of variable considered for the experiment
Data Analysis A total of 21 different measures had been recorded from all three games and workload index measurement. This was a combination of playing with two input devices in two different position. The average performance of target hit by both devices in both position in accuracy game was recorded. Similarly, the average target hit for both devices in both positions was recorded for precision and recorded in an excel sheet. Participant’s average response time were documented and trends analyzed. The self report on how they performed for workload was also taken on average. RESULTS Accuracy and Precision Data Analysis Accuracy(Avg)
Precision(Avg)
Position A with Mouse
Position and Device
24.71
26.85
Position A with Pen
35.38
23.42
Position B with Mouse
19.71
26
Position B with Pen 42.85 18.85 Table 2. Average performance of subjects on Accuracy and Precision games.
When analyzed on average, participant performed marginally better at precision task with mouse in both positons A and B with an average score of 26 target per minute. On accuracy their performance with a pen input table was higher in both position. On position B the difference in average of target hit with a pen was of 24 hits.
Graph 1. The graph of average performance of subjects on Accuracy and Precision games.
Variables Ite m 1.
Variable
Description
Target Hit (70 pixel)
2.
Target Hit (10 pixel)
3.
Average Response Time for Precision Simple Response Time
Target for assessing accuracy were set this large Target assessing precision were only 10 pixel big Average time the participant took in locating and hitting the target A single target (50 pixel) would appear randomly at the screen and participant had to respond quickly and hit the target.
4.
Assessing the response time participants had on the precision task, that participants on average took lesser response time overall using a pen input table as compared to a mouse on both position. There is a drop of 243.93 ms by using a pen in position A and 220.81 ms drop in response time on position B. Participants performed better with a pen input tablet as compared to the mouse. Graph 2. The graph of average performance of subjects on Accuracy and Precision games.
CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 4 of 6
Simple Response Time Data Analysis The simple response time for participants showed the following trends. There is an increase in simple response time during when using a Pen input tablet as compared to mouse. However, there is an overall decrease in response time when both instruments were used in position B. (Table 3) Graph 3. The graph of average performance on simple response time game
After each game session at a particular position with one device the participants were requested to report their assessment of how much work load did they experience on the following dimensions. Physical Demand, Mental Demand, Temporal Demand, Performance, Effort and Frustration. Each was rated on a 10-point scale, which was completed while keeping in mind what they experienced during the games.
Workload Data Analysis: NASA Task Load Index
Graph 4. The graph of self report on workload using NASA TLX
Mental Demand: The participants reported their first attempt with a PIT the highest on mental demand (76.6) in position A followed by their first attempt with a mouse the second highest (63.3) on the same position. Physical Demand: PIT on average was rated the highest on physical demand followed by using the mouse on position B. Mouse on position A and PIT on position B was rate equally on average at 38.3. Temporal Demand: Both PIT and Mouse were on average rated the highest at 83.3, meaning participant felt the highest amount of pressure while using both the devices equally. Performance: The participants reported their performance with a PIT the highest on average. Reporting an average performance level of 51.6 to 47.5 with a PIT on both position. Meaning they felt they did equally well on both position. Effort: Participants report high level of effort using a PIT at any position. They reported position B less effortful for using a mouse. The variability of their response are much higher for Mouse at Position A when compared to PIT at position B. Frustration: PIT at position A was reported the most frustrating session 65.8, followed by position B 58.3. DISCUSSION There is important distinction of how people perceived versus how they actually perform on task. The performance of the participants on simple task of hit merely point target showed initial trend which is not in agreement to the research already conducted. Vogel and Balakrishnan (2010) reported reduction in response time
in doing task with a PIT versus a mouse, however in our research we can see a decrease of +200 ms in the response time for doing precise task, such as hitting a 10pixel target that appears only for 2 seconds. In their research they attribute accuracy in the corners of the PIT as one of the reasons for decline in response time. We observed the participant’s performance on the contrary where amongst all measure of accuracy using PIT at position B has the highest level of accuracy. The performance of participants on simple response time was difference from their performance in the precision game. Their response time rose with a PIT in both positions, though marginal its important to note. Here the participant reported having difficulty in assessing when the click had taken place. This was due to the fact PIT registers the left click via a tap and this meaning after resetting the wrist position to mid air after the initial tap can vary. Meaning the wrist has to flex even further than before the tip of the pen makes contact and a tap is registered. This problem would not occur in the mouse with shape allowing for a resting position for the index finger the act of clicking handed of to the device rather than the time it take for the write to travel. This specific trend is different from the trend in the precision game as the appearance of target has a rhythm to it. As a target would appear every 2 seconds randomly all over the playing canvas. This allows for quick taps between two targets with relative ease. At this point how ever the mouse proves to be barrier as whole arm muscle has to be involved into completing the task, where as a simple bend of the wrist can accomplish the task for the pen given the finger joints add the extra space of mobility within the area of the current position. For this reason, it
CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 5 of 6
can be safely stated why task requiring fine motor skills such as drawing on a computer are better accomplished using a PIT rather than a mouse, since there are more ranges of motion available in between the wrist and finger. Though, it is important to note that since all the participants had never used a PIT before their self report on what the workload was much different. In their self report since they reported high level of mental, temporal, physical demands leading them to assume a poor level of performance and feeling higher levels of frustration. Due to this it can be assumed that people who would start to use a PIT would to feel frustrated and can potentially lead to them not choosing to use a PIT method of interacting with an interface. However, we have to be cautious of creating over reaching conclusion about the possible trends of the study and its potential implications for making a use case in favor of PIT. There are simply other factors that reduce the likelihood of using a PIT for most of daily work. Occlusion is constant barrier to PIT and users experience with accuracy. In our study couple of time the game had to be reinitiated as the tablet surface would recognize the bottom of the palm as tap, causing user to force themselves to hover over the tablet to play the game. Though now small and ergonomical, the form is to big to carry around when compared to a standard computer mouse. Another shortcoming that has to be acknowledged that a very particular set of sample participated for the study. These were users who had no or little prior experience with both the device. And there were only 6 participants for this reason making any statistically significant claims about the populations would be incorrect. Thus requiring larger sample size to understand greater trends. Finally, when comparing position two different positions of practice effect can be observed on the general trends as performances and self report of workload decrease overtime. Though at the same time it can be seen that participant’s performance with a shift of position had very different results. The accuracy rose as the PIT was moved from position A to position B, but not precision. Performance on precision was consistent irrespective of the position. When comparing simple response time, the performance improved on both PIT and Mouse. CONCLUSION In conclusion, we can say that there in an increase in accuracy when using a Pen Input Tablet directly in the line of sight of the screen, and has higher level of accuracy overall when compared to a mouse. Precision
remains consistent on both positon (in front of the screen and right next to it) while overall precision remains higher when compared to simple tasks using a PIT. It would take longer to respond using a PIT than a mouse. New users would assumer their performance to worst using a PIT when compared to Mouse even though on certain aspects their performance might better. For greater trends more studies have to conducted with larger sample size to conclusively speak about the trends reported in this research. ACKNOWLEDGEMENTS I would like to thank Dr. Dan-Nathan Roberts for his guidance and support during this research. His quick insights made the whole process much more enjoyable. I would like to acknowledge and thank all the participants who very generously donated their time to patiently to participate in this research. REFERENCES Buxton, W. (1990). A three-state model of graphical input. Paper presented at the Human-Computer Interaction-INTERACT, , 90 449-456. Cook, C. J., & Kothiyal, K. (1998). Influence of mouse position on muscular activity in the neck, shoulder and arm in computer users. Applied Ergonomics, 29(6), 439-443. doi:http://dx.doi.org.libaccess.sjlibrary.org/10.10 16/S0003-6870(98)00008-8 Kim, H., Kim, H., Yoon, I., Jung, C., & Hwan, Y. (2015). User performance differences between graphics tablet and mouse in graphic applications: Focus on controllability and accuracy. Kotani, K., & Horii, K. (2003). An analysis of muscular load and performance in using a pen-tablet system. Journal of Physiological Anthropology and Applied Human Science, 22(2), 89-95. Müller, C., Tomatis, L., & Läubli, T. (2010). Muscular load and performance compared between a pen and a computer mouse as input devices. International Journal of Industrial Ergonomics, 40(6), 607-617. doi:http://dx.doi.org.libaccess.sjlibrary.org/10.10 16/j.ergon.2010.08.004 Van Galen, G. P., & de Jong, W. P. (1995). Fitts' law as the outcome of a dynamic noise filtering model of motor control. Human Movement Science,
CLASS PROJECT: NOT INTENDED FOR PUBLICATION
Class project Only -Not Intended for Publication or Peer review Page 6 of 6
14(4–5), doi:http://dx.doi.org/10.1016/01679457(95)00027-3
539-571.
Vogel, D., & Balakrishnan, R. (2010). Direct pen interaction with a conventional graphical user interface. Human–Computer Interaction, 25(4), 324-388.
Fitzmaurice, G. W., Balakrishnan, R., Kurtenbach, G., & Buxton, W. A. S. (1999). An exploration into supporting artwork orientation in the user interface. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: The CHI Is the Limit. Pittsburgh.