leveraging physical actions to interact with digital surfaces mark hancock university of waterloo
downhill
back-country (telemark)
downhill bindings attached at rear
telemark bindings detatched at rear
falling over is a good thing!
providing rich multi-touch manipulation together with the appropriate visual feedback can enable virtual objects to be used in a meaningful way, as tools we can use these virtual tools to leverage physical actions to interact with digital surfaces
part i
part ii
part iii
multi-touch manipulation + visual feedback = virtual tools embodied interaction
part iv
part i: multi-touch manipulation
how do I interact with these objects? pick them up? turn them over?
how do I use them like real objects?
technique one: one-finger interaction
one finger = two DOF input dedicated areas
can use on any one-touch display
technique two: two-finger interaction
two fingers = four DOF input dedicated translate area
separate and simultaneous actions
technique three: three-finger interaction
three fingers = six DOF input dedicated fingers
comparative study
twelve participants six male, six female
three tasks passing, docking, puzzle
results
people are faster with more touches
Task Completion Time (s)
25 passing
20
docking
15 10 5 0 one-finger
two-finger
three-finger
people prefer more touches
Agree
7 6 5 4
Disagree
3 2 1 I found the technique easy to use
one-finger
the object reacted as I expected it to
two-finger
I could easily move an object to where I wanted
three-finger
Agree
7 6 5 4
Disagree
3 2 1 I found the technique difficult to control
one-finger
I found it difficult I found it difficult to to turn objects in roll objects over the plane
two-finger
three-finger
people perform many simultaneous actions
Time Spent Touching (s)
12 10 8 6 4 2 0 one-finger
two-finger
three-finger
Time Spent Touching (s)
12 10 8 6 4 2 0 one-finger
two-finger
three-finger
Time Spent Touching (s)
12 10 8
passing docking
6 4 2 0 one-finger
two-finger
three-finger
Time Spent Touching (s)
12 10 8
translations planar rotations spatial rotations
6 4 2 0 one-finger
two-finger
three-finger
people are faster with more touches people prefer more touches
people perform many simultaneous actions
observation: “why is the object turning when I just move it?�
part ii: perception
perspective projection rays
center of projection (COP)
picture plane
point of view
center of projection
perception study
factor: POV-COP discrepancy
no discrepancy
medium discrepancy
large discrepancy
factor: projection type perspective
parallel
factor: motion parallax
factor: motion parallax
no parallax
parallax
task
results
80
parallel
70
perspective
Mean Error (°)
60 50 40 30 20 10
0
motion parallax discrepancy
none
med
large
none
med
large
mean error up to 60째
part iii: virtual tools
the dawn of digital tables
application: sandtray therapy
cooperative design face-to-face meetings iterative design via distance collaboration face-to-face feedback session interviews & mock therapy
design considerations narrative potential associative medium digital extensions facilitate interpretation
feedback construction storytelling actions arrangement
part iv: embodied interaction research in the Touchlab (and other labs)
Urp (Underkoffler & Ishii, 1998)
measuring embodiment stephanie mikulecky (ucalgary) miguel nacenta (st. andrews, scotland) sheelagh carpendale (ucalgary)
cloth interaction kimberly mikulecky (ucalgary) john brosz (ucalgary) sheelagh carpendale (ucalgary)
collaborative multi-touch navigation joseph shum (uwaterloo)
lego star wars
call of duty black ops
Shear Shear
Ripple
above the surface dmitry pyryeskin (uwaterloo) jesse hoey (uwaterloo)
understanding automation betty chang (uwaterloo) stacey scott (uwaterloo)
playing with data adam bradley (uwaterloo) sheelagh carpendale (ucalgary)
downhill
back-country (telemark)
with a better understanding of human perception together with interaction that uses our hands and bodies, we can leverage physical actions to better interact with digital surfaces
students:
adam bradley betty chang arezoo irannejad rebecca langer dmitry pyryeskin joseph shum kimberly mikulecky (ucalgary)
stephanie mikulecky (ucalgary)
acknowledgements: NSERC
Games Institute SurfNet GRAND NCE
thank you mark.hancock@uwaterloo.ca