USABILITY TESTING Microsoft Word 2007 Usability Study Tianjie Li Simple Version
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
1. OVERVIEW The usability testing of Microsoft word 2007 contained 1540 micro-tasks and based on the deficiencies of Nielsen’s methodology, it revised the usability testing and explored the relationship between the number of participants and usability problems. 2. TESTING PROCESS 2.1. Preparation View the material samples in Appendix 1. TABLE 1 Preparation NO
MATERIAL
USER
USAGE
OBJECTIVES
00
Destination
Both testers
Destination documents described the
Avoided testers’ description
document
and
tasks which participants should finish.
to the participants which
participants
View the samples in
might be unclear, orienting statement.
01
Testing
testers
script
02
Material
participants
files
Testing script which should only be
Made up for the testers’ lack
viewed by the testers contained all the
of professional skills in the
methods to finish a task.
testing
Meanwhile,
software
and
the script highlighted the key points
meanwhile, gave the testers
where testers should observe carefully
prompts of observing points
and inquiry the participants properly.
which
It
monitoring
also
underscored
some
special
made
the
testers
participants
situations which might occur.
easily.
Material files involved the pictures, data
Divided a whole task into
documents or any other stuff which
several
might be used during the tests.
individual testing files. improved
micro-tasks the
with This
experience
when participants failing in one of them and beginning another one. 03
Record
testers
form
The
testers
recorded
some
basic
information or problems discovered when observing the users’ operations and revised it with the participants after a whole task.
04
Data entry form
testers
Count the errors and problems during a whole
task
objectives,
through plans,
four
facets,
implements
and
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
evaluations. 2.2. Target users TABLE 2 Target users USER
FEATURES
TYPE Novice
Didn’t acquire most functions of word 2007. The specific characteristics were they didn’t
users
know the function and how to use it. The problems and errors focused on concepts.
Normal
Could undertake the basic functions of the software, but couldn’t edit long writings through
users
advanced functions.
The specific characteristics were their problems and errors mostly
appeared in plans and implements and they started to have trials when they didn’t know the function. Expert
The typical operation was to use shortcut keys to finish tasks. They were skilled in most tasks
users
and usually accomplished in a short time with few errors or trials.
A specific characteristic was
that they could realize their mistake after an error.
3. RESULTS View the data in Appendix 2.
PICTURE 1 The relationship between the usability problems discovered by users
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
THE NUMBER OF USABILITY PROBLEMS DISCOVERED BY THREE TYPES OF USERS (DIFFERENT SAMPLING) NUMBER OF USABILITY PROBLEMS
900.00
875
800.00 700.00 600.00 519
500.00
novice users common users
400.00
353
300.00 254
200.00
expert users overall
100.00 0.00 0
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 THE NUMBER OF PARTICIPANTS
FIGURE 1 The number of usability problems discovered by three types of users
THE RATIO OF USABILITY PROBLEMS DISCOVERED BY THREE TYPES OF USERS (DIFFERENT SAMPLING) RATIO OF USABILITY PROBLEMS
100.00%
100.00%
90.00% 80.00% 70.00% 59.31%
60.00% 50.00%
40.34%
40.00%
novice users common users expert users
30.00%
29.03%
20.00%
overall
10.00% 0.00% 0
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100105
THE NUMBER OF PARTICIPANTS FIGURE 2 The ratio of usability problems discovered by three types of users
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
THE TREND OF THE RELATIONSHIP BETWEEN INCREMENT OF USABILITY PROBLEMS AND INCREMENT OF PARTICIPANTS 50 40
novice users 30
common users expert users
20
overall 10 0 0
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 THE NUMBER OF PARTICIPANTS FIGURE 3 The trend of the increment
THE NUMBER OF USABILITY PROBLEMS IN COGNITIVE STAGE THE NUMBER OF USABILITY PROBLEMS
RATIO OF USABILITY PROBLEMS
60
novice users
common users
expert users
108 207 12 125 5 0 0 memory
304
10 0 0 comprehension recognition
201 discovery
119 21 14 others
FIGURE 4 The number of usability problems in cognitive stage
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
THE NUMBER OF USABILITY PROBLEMS
THE NUMBER OF USABILITY PROBLEMS IN LEARNING STAGE novice users
58
common users
expert users
1 197
149 195 294 217 association
perception
7 8 proficiency
FIGURE 5 The number of usability problems in learning stage
THE NUMBER OF USABILITY PROBLEMS
THE NUMBER OF USABILITY PROBLEMS IN ACTION STAGE novice users
common users
expert users
6 298
136 458
plans
39 50 implements
112 16 11 evaluations
FIGURE 6 The number of usability problems in action stage
4. APPENDIX 4.1. Appendix 1 Back to Preparation.
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
PICTURE 2 Destination document of a task
PICTURE 3 Testing script of a task
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
PICTURE 4 Testing script of a task ver2.0
PICTURE 5 Material file of a task
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
PICTURE 6 Record form
PICTURE 7 Data entry form
4.1.
Appendix 2
Back to view RESULTS. TABLE 1 The data of usability problems discovered by users (A) Sampling number
Usability problems (novice user)
Usability problems (common user)
Usability problems (expert user)
number
ratio
number
ratio
number
ratio
1
67
7.70%
39
4.42%
22
2.53%
2
120
13.77%
69
7.85%
49
5.57%
3
160
18.26%
94
10.72%
71
8.16%
4
191
21.85%
115
13.19%
89
10.17%
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
5
217
24.84%
135
15.41%
107
12.22%
6
241
27.56%
152
17.40%
123
14.09%
7
262
29.90%
167
19.09%
137
15.71%
8
281
32.12%
180
20.57%
151
17.27%
9
297
33.90%
193
22.04%
162
18.57%
10
312
35.70%
204
23.32%
173
19.74%
11
326
37.30%
216
24.64%
182
20.78%
12
340
38.89%
225
25.74%
191
21.77%
13
352
40.28%
235
26.83%
199
22.75%
14
364
41.60%
244
27.87%
206
23.50%
15
375
42.87%
252
28.84%
212
24.21%
16
385
44.04%
260
29.75%
218
24.87%
17
395
45.20%
267
30.54%
223
25.46%
18
404
46.20%
274
31.30%
227
25.92%
19
415
47.38%
281
32.14%
231
26.36%
20
423
48.34%
287
32.85%
234
26.75%
21
430
49.18%
294
33.64%
238
27.16%
22
438
50.11%
300
34.28%
240
27.48%
23
446
51.00%
306
34.99%
243
27.77%
24
453
51.82%
311
35.60%
245
28.04%
25
460
52.59%
317
36.18%
247
28.26%
26
467
53.37%
322
36.77%
249
28.51%
27
473
54.10%
327
37.33%
251
28.65%
28
479
54.79%
331
37.85%
252
28.80%
29
485
55.48%
336
38.45%
253
28.92%
30
491
56.15%
341
38.95%
254
29.03%
31
497
56.83%
345
39.40%
N/A
N/A
32
503
57.49%
349
39.86%
N/A
N/A
33
508
58.06%
353
40.34%
N/A
N/A
34
514
58.72%
N/A
N/A
N/A
N/A
35
519
59.31%
N/A
N/A
N/A
N/A
TABLE 2 The data of usability problems discovered by users (B) Stage
novice
Nu
users
m. rati
common
Mem
Compreh
Recog
Disco
Oth
Associ
Perce
Profici
Objec
ory
ension
nition
very
ers
ation
ption
ency
217
294
8
41.81%
56.65%
0
304
0
201
14
0.00
58.57%
0.00%
38.73
2.70
%
%
125
21
o
%
Nu
0
207
0
1.54%
Plan
Imple
Evalua
tives
s
ments
tions
N/A
458
50
11
N/A
88.2
9.63%
2.12%
5% 149
197
7
N/A
298
39
16
For questions or details please contact litianjie901001@gmail.com.
Click www.tianjieli.com to view Tianjie Li’s online portfolio.
users
m. rati
0.00
58.64%
0.00%
35.41
5.95
%
%
42.21%
o
%
expert
Nu
5
108
10
12
119
58
users
m. 1.97
42.52%
3.94%
4.72%
46.8
22.83%
rati o total
% 5
55.81%
1.98%
N/A
10
338
154
11.05%
4.53%
2% 1
0.39%
195
N/A
6
136
112
76.77%
N/A
2.36
53.54%
44.09%
225
139
5% 619
84.4
% 424
492
210
N/A
762
For questions or details please contact litianjie901001@gmail.com.