The Chameleon Project Tina Gonsalves, Nadia Berthouze, Matt Iacobini
Abstract ! "#$!%&'!()!*#&+!%,*&+*!-,$+$.*%*&(.!&+!*(!/&+01++!*#$!21&3/&.4! ()! *#$! 5#%'$3$(.! 6,(7$0*! 8#**-9::;;;<*&.%4(.+%3=$+<0(': 0#%'+$3$0*),%'$>?<#*'@<!A*!;&33!2$!21&3*!(=$,!*;(!B$%,+!8?>>CD ?>E>@F! %./! *#,(14#! *$.! -,(*(*B-$+! ;&*#! %! 0,(++D/&+0&-3&.%,B! 4,(1-!()!%.!%,*&+*F!+(0&%3!.$1,(+0&$.*&+*F!$'(*&(.!.$1,(+0&$.*&+*F! %))$0*&=$! 0('-1*$,! +0&$.*&+*+F! *$0#.(3(4&+*+F! #1'%.! 0('-1*$,! &.*$,%0*&(.! +0&$.*&+*+! %./! %! 01,%*(,<! "#$! -,(7$0*! &.=$+*&4%*$+! !"#$ %&'#(!')&$ *+,(-.!'+(%$ +*$ #/+!'+(.0$ &+(!.1'+(2$ -3.4'(1$ %**$.*&(.! *(! #(;! ;$! &..%*$3B! %./! 0(.*&.1(1+3B! +B.0#,(.&G$! ;&*#! *#$! )%0&%3! $H-,$++&(.+F! =(&0$+! %./! -(+*1,$+! ()! (*#$,+! 2B! 1.0(.+0&(1+3B! &.)$0*&.4! $%0#! (*#$,! ;&*#! (1,! $'(*&(.+<! ! A*! 5+!"$*+00+4%$.(-$&3'!'6,#%$!"#$%&'#(!')&$/#!"+-2$!",%$&3#.!'(1$ %&'#(!')&$ .(-$ .3!'%!'&$ 3#%#.3&"$ 73+-,&'(1$ (#4$ /+-#0%$ !+$ 5#$ ,%#-$'($%&'#(!')&$#87#3'/#(!%$.(-$(#4$4.9%$!+$#87#3'#(&#$.3!:$$ "#$!-,$+$.*%*&(.!;&33!2$!+*,10*1,$/!%+!)(33(;+9!"#$!-,(7$0*!;&33! 2$!&.*,(/10$/F!(*#$,!,$3$=%.*!,$+$%,0#!%./!%,*!;(,I!/&+01++$/F! *#$.! *#$! *$0#.&0%3! -%,*+! ()! *#$! -,(7$0*! ;&33! 2$! /$+0,&2$/F! &.031/&.4!*#$!21&3/&.4!()!*#$!=&/$(!/%*%2%+$+F!*#$!)%0$D,$%/&.4! *$0#.(3(4BF! *#$! =&/$(! $.4&.$! %./! *#$! J$'(*&(.%3! %34(,&*#'+K! *#%*! /,&=$! *#$! =&/$(! $.4&.$<! A.! *#$! )(33(;&.4! +$0*&(.! *#$,$! ;&33!2$!%!/&+01++&(.!%2(1*!*#$!%1/&$.0$L+!$H-$,&$.0$F!%+!+$$.! ),('! (2+$,=%*&(.+! %./! *#$! )$$/2%0I! %./! $=%31%*&(.! +$++&(.+F! .(-$ )(.009$ !"#$ 73#%#(!.!'+($ 4'00$ #(-$ 4'!"$ .$ -'%&,%%'+($ +($ ! *#$!0(33%2(,%*&(.<
138
S E C O N D N AT U R E Issue No.3, March 2010
Sensing You M$0$.*!/$=$3(-'$.*+!&.!3%2(,%*(,B!+0&$.0$!%./!=&+1%3&G%*&(.!*$0#.(3(4B! #%=$!2$$.!$++$.*&%3!*(!,$=$%3&.4!*#$!2&(3(4&0%3!2%+&+!()!$'(*&(.+F!3$%/&.4! 0(4.&*&=$!%./!%))$0*&=$!+0&$.0$+!*(!%,,&=$!%*!%!'(,$!/$*%&3$/!1./$,+*%./&.4! ()!#(;!$'-%*#$*&0!%./!$'(*&(.%3!,$+-(.+$+!%,$!1+$/!2B!#1'%.!2$&.4+! !+$ *.&'0'!.!#$ '(!#37#3%+(.0$ !3.(%.&!'+(%$ ;<.!)#0-$ =>>?2$ @3'!"$ ?AABC:$D"'%$ I.(;3$/4$! &+! 2$4&..&.4! *(! 2$! $'2$//$/! &.*(! *$0#.(3(4BF! I.(;.! %+! N%))$0*&=$! 0('-1*&.4L! 8O%3&(12B! ?>>PF! 6%.*&0! ?>>QF! R&%.0#&DR$,*#(1G$! ?AABC:$E3!'%!%$ .3#$ (+4$ ".3(#%%'(1$ !"'%$ 3#%#.3&"2$ #870+3'(1$ .(-$ 5,'0-'(1$ N$'(*&(.%33BL!&.*$,%0*&=$!%,*!&.+*%33%*&(.+!%23$!*(!$.4%4$!&.!%.!$'(*&(.%3! &.*$,%0*&(.!3((-!;&*#!*#$&,!%1/&$.0$!8S,&4#*!?>>QF!O#1*!?>>C@ "#$!!"#$%&%'()*+',%-.!&.=$+*&4%*$+!*#$!+(0&%3!,(3$!-3%B$/!2B!$'(*&(.%3! $H-,$++&(.+!%./!*#$!*,%.+)$,!'$0#%.&+'!()!$'(*&(.+!*#%*!'$/&%*$!+(0&%3! &.*$,%0*&(.<! M$+$%,0#! #%+! +144$+*$/! *#%*! (=$,! $&4#*B! -$,0$.*! ()! #1'%.! 0(''1.&0%*&(.!&+!$.0(/$/!&.!)%0&%3!$H-,$++&(.F!2(/B!'(=$'$.*+!%./!*(.$! ()!=(&0$F!%./!*#%*!*#$+$!.(.D=$,2%3!0#%..$3+!%,$!-%,*&013%,3B!&'-(,*%.*!*(! 0(''1.&0%*$!$'(*&(.+!%./!)$$3&.4+!8T$++!?>>E@<!"#$+$!0#%..$3+!%,$!%*! *#$!2%+&+!()!*#$!$'(*&(.%3!0(.*%4&(.!-#$.('$.(.F!*#%*!&+!N*#$!*$./$.0B! *(!%1*('%*&0%33B!'&'&0!%./!+B.0#,(.&G$!)%0&%3!$H-,$++&(.+F!=(0%3&G%*&(.F! -(+*1,$+F!%./!'(=$'$.*+!;&*#!*#(+$!()!%.(*#$,!-$,+(.!%./F!0(.+$U1$.*3BF! !+$&+(F#31#$#/+!'+(.009G$;<.!)#0-$=>>?C:$D"'%$7"#(+/#(+($'%$&+/70#82$ '(F+0F'(1$ 3.!'+(.0'!92$ '(%!'(&!2$ .(-$ &+(-'!'+(#-$ 3#.&!'+(%$ ;@3'!"$ ?AABC:$ V%,&(1+! +*1/&$+! #%=$! +#(;.! *#%*! $'(*&(.%3! 0(.*%4&(.! *%I$+! -3%0$! ()*$.! &.!.(,'%3!/%&3B!3&)$!#&4#3&4#*&.4!(1,!+1+0$-*&2&3&*B!*(!$'(*&(.%3!0(.*%4&(.! ;H.(I#!!.$=>J>2$D./'#!!+$?AAJ2$<#%%$?AA=2$K.!+$?AAL2$M#,/.(($?AAA2$ W&#%3&.$0!?>>CF!R(1,4$(&+!?>>C@< E$ 0'N#09$ (#,3+0+1'&.0$ %!3,&!,3#$ .!$ !"#$ 5.%'%$ +*$ !"'%$ 7"#(+/#(+($ '%$ !"#$ '&,,(,!.$1,(.!+B+*$'!XV%.!/$,!Y%%/!?>>QZF!*#$!+B+*$'!%*!*#$!2%+&+!()! &'&*%*&(.<! T(;$=$,F! #1'%.+! /(! .(*! %3;%B+! ,$+-(./! *(! %.! $'(*&(.%3! $H-,$++&(.! ;&*#! %.! $H%0*3B! $U1&=%3$.*! $'(*&(.%3! $H-,$++&(.<! [*#$,! #F+0,!'+(.39$ 73'(&'70#%$ /.9$ 1,'-#$ %+&'.0$ '(!#3.&!'+(:$E($ #873#%%'+($ +*$ %.4$,! '%B! 1.0(.+0&(1+3B! *,&44$,! %.! $H-,$++&(.! ()! +%/.$++! %+! %! )(,'! ()! $'-%*#B<! \./$,+*%./&.4! *#$! /B.%'&0+! *#%*! %,$! %*! *#$! 2%+&+! ()! *#&+! $'(*&(.%3! 3((-! %./! #%,.$++&.4! *#$'! *(! 0,$%*$! $'(*&(.%3! %;%,$.$++! &+! (.$!()!*#$!%&'+!()!*#$!-,(7$0*<
Issue No.3, March 2010 SECOND NATURE
139
Creating an emotional visual databases "#$!-,(7$0*!&.*$,%0*+!;&*#!&*+!%1/&$.0$!2B!/&+-3%B&.4!=&/$(+!()!J$'(*&(.%3! $H-,$++&(.! -(,*,%&*+K! /%*%2%+$! 0,$%*$/! 2B! %,*&+*F! "&.%! Y(.+%3=$+! 8;;;< !'(.1+(%.0F#%:&+/C:$O,&"$+*$!"#$&,33#(!$%&'#(!')&$#87#3'/#(!%$#870+3'(1$ #873#%%'+($.(-$#/+!'+($,%#$P.,0$QN/.(G%$=>LA%$F'%,.0$-.!.5.%#$+*$%!.!'&$ *.&'.0$#873#%%'+(%$3#73#%#(!'(1$#/+!'+(.0$%!.!#%$;QN/.($?A=AC:$E($.'/$+*$ *#$!%,*&+*L+!.$;!0(,-1+!()!)%0&%3!$'(*&(.!$H-,$++&(.!=&/$(+!1*&3&G$/!&.!*#$! !"#$%&%'()*+',%-.!;%+!*(!2$!'(,$!/B.%'&0!%./!%$+*#$*&0F!$.4%4&.4!%./! #/+!'+(.009$73+5'(1:$R3,(+$EF#35#&N2$(#,3+%&'#(!')&$&+00.5+3.!+3$+($!"#$ !"#$%&%'()*+',%-.!%4,$$+!*#%*!*#&+!&+!&'-(,*%.*<
Most work which has been done on understanding emotions has been static. For example, much research has been done on how the brain responds to particular emotional expressions. The Chameleon Project is interesting because it brings in a dynamic element. This brings it one step closer to real social interactions. Experience with this project may very well lead to ideas which can be incorporated into our own work. 1 [=$,!*#$!3%+*!B$%,!%./!%!#%3)F!"&.%!Y(.+%3=$+!#%+!%+I$/!=(31.*$$,+!(=$,! !"#$4+30-$;#F#39-.9$7#+70#2$.&!+3%2$F'%,.0$.3!'%!%C2$!+$5#$)0/#-$#873#%%'(1$ #/+!'+(%:$ K"#$ '(-,&#-$ !"#%#$ #/+!'+(.0$ %!.!#%$ ,%'(1$ F.3'+,%$ !#&"('6,#%$ &.031/&.4!0,$%*&.4!+0$.%,&(+!&.!*#$!+*1/&(!;#$,$!*#$!$'(*&(.+!%,$!,$%0*&(.+! *(!+*%4$/!$=$.*F!%./!$'-3(B&.4!03%++&0%3!-+B0#(%.%3B*&0%3!*$0#.&U1$+!+10#! %+! $.0(1,%4&.4! *#$! =(31.*$$,+! *(! &'%4&.$! /&))$,$.*! -$,+(.%3! $'(*&(.%3! %&#(.3'+%$*3+/$!"#'3$7.%!$.(-$!+$3#S#(.&!$!"#/$.%$'*$'($!"#$73#%#(!:$K,5T#&!%$ ;$,$! +#(*! &.! %! +*1/&(! +-%0$! ;&*#! %! .$1*,%3! 23%0I! 2%0I4,(1./! %./! ;&*#! +&'-3$!3&4#*&.4<
Figure 1: Tina Gonsalves, collage of past facial emotion expression databases used in !"#$%&#'"($)*&#*%(!&+,#$!(-(#%".+,#%/(01)2%3(425*.#%!123(6257#%3(6+"82#%$ Figure 2: Portrait 05, video stills of sad expression channel from the facial emotion expression database of the Chameleon Project.
"&.%! Y(.+%3=$+! #%+! .(;! +#(*! *#&,*B! -(,*,%&*+F! %./! &.*$./+! *(! +#((*! +*&33! /+3#:$ E*!#3$ !"#$ %"++!2$ !"#$ .3!'%!$ 43'!#%$ !+$ !"#$ %,5T#&!%$ !+$ -'%&,%%$ !"#$ $H-$,&$.0$<! [.$!()!*#$!+127$0*+!,$-(,*$/9
The whole process from entering the darkened studio, to being confronted with a camera that stood only half a meter away
140
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
141
from my face, I instantly thought would promote a perhaps less sincere and maybe uncomfortable response; but !" #$%" $&$'()" *+" ,-)" .+#" ($%/01" &1" recalling of these particular emotionally %/2-/,3$-*" (4(-*%" 5-6$4(0()7" $-)" .+#" moments of embarrassment were soon ousted. 2 E(+!"#3$ 7.3!'&'7.(!$ 3#7+3!%$ .5+,!$ !"#$ $H-$,&$.0$!&.!*#$!+*1/&(9!
...in the attempt to recreate the emotion, *.(" 8((0/-2%" 9++)()" :$3;<" !" 8(0*" =5/*(" moved This intimate relationship with a camera lens was a new experience for me and I found its scrutiny a great challenge. I learned that while it becomes possible to represent a range of emotions, the sudden and immediate proximity of some, particularly sadness and fear, was potent and very real.3 RB! *#$! $./! ()! *#$! -,(7$0*! *#$! %,*&+*! %&'+! !+$ %"++!$ '($ E*3'&.2$ E%'.2$ Q,3+7#$ .(-$ !"#$ E/#3'&.%:$D"#$.3!'%!$43+!#$'($"#3$%!,-'+$(+!#%$ &.!W%,0#!?>>C!;#&3$!;(,I&.4!&.!5%.%/%9
“....It’s taken a while to get comfortable asking people to evoke emotions. It’s been exhausting, because it feels so personal. I have sourced some artists, some actors, and some people off the street! It’s been a varied response, Figure 3: Portrait 08, video stills of angry expression channel from the facial emotion expression database of the Chameleon Project.
142
S E C O N D N AT U R E Issue No.3, March 2010
ranging from deep deep crying for half an hour to more laughter and very light expression. It’s been hard to watch people cry and stand over the other side of the camera documenting it. For some, sadness has been very close to the surface, and recent events such as loss make sadness the easiest to access. 4 U"#3#$.%$V+(%.0F#%G$(+!#%$'($E73'0$?AA>2$4+3N'(1$'($P.3'%$
It’s harder to coax Parisians to reveal emotions… I need to spend more time getting to know participants, to develop a more trusting relationship. I am asking them to give a lot, and it’s hard without a closeness. The studio time needs to be much longer than it was in Canada…5’
9$!&#%/(&8$($)*&#*%2.("*%&2/#*%(:*7$5(*; (&8$(,2&2<2!$ "#$!$'(*&(.%3!)%0&%3!$H-,$++&(.+!()!*#$!%1/&$.0$!%,$!1+$/!%+!%4$.0B!&.! *#$!!"#$%&%'()*+',%-.F!/,&=&.4!*#$!&.*$,%0*&=&*B!()!*#$!!"#$%&%'()*+',%-.<! "#$,$)(,$F!&*!;%+!&'-(,*%.*!*#%*!*#$!/&4&*%3!=&/$(!-(,*,%&*+!0,$%*$/!2B!*#$! %,*&+*!$3&0&*!%.!%1*('%*&0!=&+&23$!$'(*&(.%3!)%0&%3!$H-,$++&(.!*#,(14#!*#$! '$0#%.&+'!()!$'(*&(.%3!0(.*%4&(.!XE>Z!)(,!*#$!-%,*&0&-%*&.4!%1/&$.0$+<! D"'%$4.%$!#%!#-$'($!"#$0.5$59$M.-'.$R#3!"+,I#$.(-$"#3$3#%#.3&"$%!,-#(!2$ W%**!A%0(2&.&!),('!*#$!\.&=$,+&*B!5(33$4$!()!](./(.!A.*$,%0*&(.!5$.*$,<! E!$!"#$!'/#$+*$!"#$)3%!$#F.0,.!'+(2$!"#$!"#$%&%'()*+',%-.!;%+!.(*!B$*!)133B! *,(&!'+(.0$%+$.$WU'I.3-$+*$XIY$;O.,0%59$=>>BC$!97#$+*$%&#(.3'+$4.%$,%#-$ *(!&.=$+*&4%*$!*#$!U1$+*&(.+< ^3$=$.!=(31.*$$,!(2+$,=$,+!;$,$!,$0,1&*$/!*(!=&$;!*#$!=&/$(!-(,*,%&*+<!"#$! (2+$,=$,+!,%*$/!*#$!$'(*&(.%3!&.*$.+&*B!()!*#$!=&/$(!-(,*,%&*<!"#$!(2+$,=$,L+! *.&'.0$#873#%%'+(%$4#3#$)0/#-$4"'0#$4.!&"'(1$!"#$7+3!3.'!%:$$Z($.$%#7.3.!#-$ +-%0$F!%!-$,+(.!8%!,%*$,@!;%+!1+$/!*(!(2+$,=$!%./!,%*$!*#$!)%0&%3!$H-,$++&(.+! ()!*#$!=&$;$,!%+!*#$B!;%*0#$/!*#$!-(,*,%&*+<!"#$!+B+*$'!I$-*!%!,$0(,/!()! *#$! 3&+*! ()! =&/$(+! -,$+$.*$/! *(! *#$! (2+$,=$,F! ()! *#$! (2+$,=$,L+! $'(*&(.%3! 3#%7+(%#%$;':#:2$"'%["#3$*.&'.0$#873#%%'+(C$.(-$+*$!"#$3.!#3$0.5#0%:$E*!#3$!"#$ $H-$,&'$.*F! %33! *#$! (2+$,=$,+! ;$,$! 4&=$.! %! '13*&-3$D0#(&0$! )(,'! %2(1*! *#$&,! $H-$,&$.0$<! "#$! U1$+*&(..%&,$! ,$+13*+! +#(;$/! *#%*9! +&H! (2+$,=$,+! )$3*! *#$! +*&'131+! $'(*&(.+_! C! (2+$,=$,+! )$3*! *#$! /$+&,$! *(! ,$+-(./! *(! *#$!
Issue No.3, March 2010 SECOND NATURE
143
%!'/,0,%$ #/+!'+(%\$ )F#$ +5%#3F#3%$ *#0!$ !".!$ !"#9$ 4#3#$ '(!#3.&!'(1$ 4'!"$ .$ 7#3%+($+3$0++N'(1$.!$.$7#3%+(:$E&&+3-'(1$!+$!"#$3.!#3G%$&0.%%')&.!'+(2$!#($ (2+$,=$,+!,$%0*$/!=&+&23B!/1,&.4!*#$!$H-$,&'$.*!2B!/&+-3%B&.4!%!+$,&$+!()! /&))$,$.*!$'(*&(.%3!$H-,$++&(.+!&.!,$+-(.+$!*(!*#$!=&/$(!-(,*,%&*+<
=#/+5$(J-(=52)$/52<(*; (9#%2(K*%!2.L$!(#%&$52"&#%/(7#&8(&8$(=2"#2.($)*&#*%(5$2,#%/(#%&$5;2"$(( M=2"$G$%!$N(*; (&8$(C82)$.$*%(D5*E$"&(,$L$.*:$,(<O(P2%2(0.(42.#*+<O(2&(&8$(?Q9(?$,#2( Lab Affective Computing group, version 2, March 2009
=#/+5$(>-(?*"1(+:(*; (@#A25,(*; (BA(!$&(+:(;*5(C82)$.$*%(D5*E$"&F
Figure 6: Frame Grab of video engine of the Chameleon Project developed in Max MSP, prototype 06, March 2008
! "#$+$! ,$+13*+! +#(;$/! *#%*! *#$! =&+1%3! /%*%2%+$! 0,$%*$/! 2B! *#$! %,*&+*! ;%+! %23$!*(!*,&44$,!$'(*&(.%3!0(.*%4&(.!&.!*#$!%1/&$.0$<!"#&+!2,(14#*!1+!*(!*#$! .$H*!+*%4$_!*(!&.*$4,%*$!%./!*$+*!*#$!$'(*&(.!,$0(4.&*&(.!*$0#.(3(4B<
G$%!#%/(&8$(H+,#$%"$I!(0)*&#*%!(( "#$! )%0&%3! $'(*&(.! ,$%/&.4! *$0#.(3(4B! *#%*! '(.&*(,+! *#$! %1/&$.0$! &+! 2$&.4! /$=$3(-$/! 2B! 0(33%2(,%*(,+! M%.%! ^3! O%3&(12B! 8#**-9::;$2<'$/&%<'&*< #-,[]N.0'+,59[C2$^+,%%#*$_.%"#*2$.(-$E5-#03."/.($O."/+,-2$0#.-$59$`+%.0'(-$ P'&.3-$ ;Q0$ _.0'+,59$ ?AAaC:$D"'%$ %9%!#/$ &0.%%')#%$ !"#$ #/+!'+(.0$ %!.!#$ +*$ !"#$ %1/&$.0$!;&*#&.!%!4,(1-!()!+&H!$'(*&(.+!8#%--BF!.$1*,%3F!+%/F!%.4,BF!/&+41+*$/F! +1,-,&+$/@<!"#$+$!$'(*&(.%3!+*%*$+!%,$!I.(;.!%+!J1.&=$,+%3!$'(*&(.+K!%./!%,$! ,%#-$.%$!"#9$!"#$+(#%$/+%!$%&'#(!')&$3#%#.3&"#3%$".F#$&.!#1+3'I#-:$E($.--#-$ 0(.*&.1%3!0#%33$.4$!#%+!2$$.!*(!/$=$3(-!*#$!*$0#.(3(4B!*(!;(,I!1./$,!/%,I$,! 3&4#*&.4!+0$.%,&(+F!;&*#!'(=&.4!)%0$+F!%./!=%,B&.4!/&+*%.0$<
144
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
145
,%*#$,!*#%.!$'(*&(.%3!0(.*%4&(.<! D"#%#$ )(-'(1%$ 5#&./#$ !"#$ 5.%#$ 0'(#$ *+3$ !"#$ &+-#$ 3#%7+(%#:$ R3,(+$ EF#35#&N$(+!#%d$
Although we use the mind reading technology to classify the expression, we don’t have any information on the intensity or subtlety of the expression, or the context of the expression. For example, someone can be crying because they are happy or sad. At some level it can be modeled into a code, but we are still working with simple models at this time. We are not able to capture the full range of emotions. 6 =#/+5$(R-(9#%2(K*%!2.L$!3(Q%#&#2.(9$"8%#"2.()*"1(+:(*; (&8$( Chameleon Project, prototype 10, 2007 ! "#$! )%0&%3! $'(*&(.! ,$%/&.4! +B+*$'! +$./+! *#&+! &.)(,'%*&(.! *(! *#$! =&/$(! $.4&.$! *#%*! #%+! 2$$.! 21&3*! 2B!"&.%! Y(.+%3=$+F! `$))! W%.F! ^=%.! M%+I(2! %./! 5#,&+*&%.! "(-).$,<! "#$! =&/$(! $.4&.$! &'-3$'$.*+! %.! $'(*&(.%3! %34(,&*#'!+(!*#$!=&/$(!-(,*,%&*!,$+-(.+$!*(!*#$!%1/&$.0$!&+!$'-%*#&0%33B! .773+73'.!#2$0#.-$59$%+&'.0$(#,3+%&'#(!'%!2$b"3'%$@3'!":$@3'!"$"97+!"#%'I#-$ %2(1*!#(;!$'(*&(.%3!$H-,$++&(.+!%,$!$H0#%.4$/!2$*;$$.!+(0&%3!4,(1-+F! 21&3/&.4!*#&+!&.*(!%.!%34(,&*#'!*#%*!.$*;(,I+!=&/$(!$.4&.$+F!)%0$!,$%/&.4! !#&"(+0+19$.(-$.,-'#(&#$/#/5#3%:$@3'!"G%$/+-#0$73#-'&!%$!"#$73+5.5'0'!9$ ()! #(;! %! -$,+(.! ;&33! ,$+-(./! *(! %! +*&'131+! ;&*#&.! *#$! 4,(1-! ()! +&H! ,('F#3%.0$#/+!'+(%:$D'(.$V+(%.0F#%2$M.-'.$R#3!"+,I#$.(-$O.!!$Z.&+5'('2$ !#%!#-$ @3'!"G%$ #/+!'+(.0$ !3.(%*#3#(&#$ "97+!"#%'%$ '($ !"#$ 0.5+3.!+39:$ @+3$ W<.779Y2$ WK.-Y$ .(-$ WM#,!3.0Y$ %!'/,0'2$ +5%#3F#3%$ !#(-#-$ !+$ 3#.&!$ 4'!"$ !"#$ %./#$ #/+!'+(:$ K.-(#%%$ %!'/,0'$ .&"'#F#-$ .$ %'8!9$ 7#3$ &#(!$ 3#%7+(%#$ +*$%.-(#%%:$WE(1#3Y$%!'/,0'$#0'&'!#-$/.'(09$.$(#,!3.0$3#.&!'+(:$D"'%$'%$'($ %00(,/%.0$!;&*#!-,$=&(1+!+*1/&$+!8R(1,4$(&+!?>>C@!*#%*!+#(;$/!*#%*!&.! *#$! 0%+$! ()! .$4%*&=$! $'(*&(.F! %! '&'&0,B! ,$+-(.+$! 0%.! 2$! &.#&2&*$/! (,! $3&0&*!%!0(1.*$,D$H-,$++&(.!;#$.!(2+$,=$,!%./!$H-,$++$,!/(!.(*!2$3(.4!*(! !"#$%./#$%+&'.0$13+,7:$Wc'%1,%!Y$.(-$WK,373'%#Y$%!'/,0'$#0'&'!#-$(+!$+(09$ .$1*,%3! 21*! %3+(! #%--B! $H-,$++&(.+<! !"#,(14#(1*! *#$! $H-$,&'$.*! *#$,$! ;%+!%!#&4#!),$U1$.0B!()!T%--B!$H-,$++&(.+F!-,(2%23B!/1$!*(!%'1+$'$.*!
146
S E C O N D N AT U R E Issue No.3, March 2010
U+3N'(1$ 4'!"$ EF#35#&N2$ 4#$ .3#$ (+4$ !39'(1$ !+$ #8!#(-$ +($ !"'%2$ 5,'0-'(1$ '$'(,B! %./! /$+&,$! )(,! '&'&0,B! &.*(! *#$! %34(,&*#'<! "#,(14#! '(/$3&.4! '$'(,B!&.*(!*#$!%34(,&*#'F!;#&0#!3$%/+!$%0#!/&4&*%3!-(,*,%&*+!()!*#$!=&+1%3! /%*%2%+$!*(!$H-$,&$.0$!J'((/+K!%./F!(=$,!3(.4$,!$H#&2&*&(.!*&'$+F!/$=$3(-! %!J*$'-$,%'$.*KF!2(*#!0(.+*%.*3B!%))$0*&.4!*#$!;%B+!*#$!+B+*$'!,$+-(./+<!! D"#$&+(%!.(!$0#.3('(1$+*$!"#$%9%!#/$.04.9%$'(e,#(&#$!"#$F'-#+$#(1'(#:$@+3$ $H%'-3$F!(.!%!2,&4#*!+1..B!/%B!&.!R,&4#*(.F!'(+*!-$(-3$!$.*$,$/!*#$!+-%0$! U1&*$!#%--B<!"#&+!'((/!()!#%--&.$++!+-,$%/!*(!*#$!-(,*,%&*+!()!!"#$%&%'(F! %./!'(+*!()!*#$!-(,*,%&*+!/$=$3(-$/!%!#%--B!'((/!%./!*$'-$,%'$.*<!"#&+! '$%.+! *#$! $'(*&(.%3! %,0#&*$0*1,$! ()! !"#$%&%'(! &+! 0(.+*%.*3B! 0#%.4&.4! %./!+#&)*&.4F!'10#!3&I$!#1'%.+!&.*$,,$3%*&(.+F!!"#$%&%'(!#%+!*#$!/$+&,$! *(! 2$! 0(.*%4&(1+F! 21*! %3;%B+! +#%-$/! ;&*#! '$'(,B! ()! -%+*! &.*$,%0*&(.+<! "#$! '(/$3+! *,&44$,$/! *#$! =&/$(! -(,*,%&*+! &.! %! ;%B! *#%*! &+! ,$-,$+$.*%*&=$! +*$ "+4$ #/+!'+(.0$ &+(!.1'+($ 4+3N%$ '($ -.9S!+S-.9$ 0'*#:$ $ @+3$ %&'#(&#2$ !"'%$ $.4&.$!#%+!*#$!-(*$.*&%3!*(!2$0('$!%!3&=$!/%*%D0%-*1,&.4!*((3!*(!%.%3B+$! *#$!*,%.+)$,!()!$'(*&(.+!2$*;$$.!-$(-3$<
98$($S:$5#$%"$(*; (#%&$52"&#%/(7#&8(&8$(C82)$.$*%(D5*E$"& "#$! !"#$%&%'() *+',%-.! %**$'-*+! *(! 21&3/! %.! $'-%*#&0! /&%3(41$! ;&*#! *#$! %1/&$.0$F! %+! *#$! %1/&$.0$! 2$0('$+! &.0,$%+&.43B! %;%,$! ()! *#$! 0(.+$U1$.0$+!()!*#$&,!$'(*&(.%3!$H-,$++&(.+<!
The visitor’s empathy becomes a powerful form of agency,
Issue No.3, March 2010 SECOND NATURE
147
whereby they become increasingly aware of and sensitive to the 3+-%(=5(-3(%"+8 "*.(/6"/-*(6$3*/+-%<">./%"(-2(-)(6%"$"?6+:0(&$*/3" of ethics, in that the visitor to the gallery must consider the degree to which they feel responsible for the subject’s changing emotional states... (Tofts 2008) V+(%.0F#%$4+3N#-$4'!"$&,3.!+32$<#0#($K0+.($!+$&,3.!#$!"#$!"#$%&%'() *+',%-.<!Y(.+%3=$+!21&3*!*#$!;(,I!&.!*$.!-,(4,$++&(.+!*(!1./$,+*%./!*#$! 0('-3$H&*&$+! ()! *#$! -,(7$0*F! 21&3/! %! +*,(.4$,! 0(33%2(,%*&=$! /&%3(41$F! 0,$%*$!%!=&+1%3!3%.41%4$!%./!,$01,,&.4!'&3$+*(.$+!)(,!%33!0(33%2(,%*(,+F! (--(,*1.&*&$+! )(,! ),$U1$.*! *$+*+! ()! *#$! &.*$,%0*&=&*B! %./! *(! *$+*! *#$! ,(21+*.$++! ()! *#$! *$0#.(3(4B<! ! "#$! ;(,I! #%+! 2$$.! $H#&2&*$/! ()*$.F! ;&*#! '(+*! $H#&2&*&(.! =$.1$+! -,(=&/&.4! %! 0#%.0$! *(! $=%31%*$! %1/&$.0$! '(!#3.&!'+(:$@+3$K0+.(2$)(-'(1$F#(,#%$!".!$&.($#/53.&#$!"'%$73+&#%%$".%$ 73+F#-$-'*)&,0!$.!$!'/#%d$
@6*"%?$3(%"*(-)"*+"3+&&/%%/+-"30($-",-/%.()"#+6;"$-)"%3/(-3(" spaces like robust interactive and interpretive work. Gonsalves well presented iterations of a process fall between the gaps and it is no coincidence that digital galleries and spaces are the ones that support the project” 7 K+$*.32$!"#$P3+T#&!$".%$5##($70.&#-$'($4.'!'(1$3++/%$%,&"$.%$f('F#3%'!9$ b+00#1#$+*$H+(-+(G%$<+%7'!.0$@+9#32$&.*g[5.3$F#(,#%$%,&"$.%$!"#$H+(-+($ K&'#(&#$O,%#,/G%$c.(.$b#(!#3$.(-$H+(-+(G%$ZbE$5.32$1.00#3'#%$%,&"$.%$ H'1"!"+,%#$ .(-$ @.53'&.$ '($ R3'1"!+(2$ .(-$ /,%#,/%$ '(&0,-'(1$ H+(-+(G%$ M.!,3.0$ <'%!+39$ O,%#,/:$ ;"!!7d[[444:!'(.1+(%.0F#%:&+/[&./F'-#+: #*'3@!"#$!%,*&+*!+$$+!*#$+$!+-%0$+!%+!+('$;#%*!0(.*,(33$/F!B$*!%33(;!'(,$! *3##-+/$ !".($ !"#$ %&'#(!')&$ 0.5+3.!+392$ 53'(1'(1$ !"#$ %&'#(!')&$ %!,-9$ +*$ +(0&%3!$'(*&(.+!(.$!+*$-!03(+$,!*(!,$%3!+(0&%3!&.*$,%0*&(.+<
=#/+5$(T(M&*:(5#/8&N-(98$(C82)$.$*%(D5*E$"&3(#%!&2..2&#*%(L#$7(*; (:5*&*&O:$(UV-(?#)#"1#%/( 0)*&#*%2.(C*%&2/#*%3(W#/8&8*+!$(X43(YUUZ =#/+5$(Z(M<*&&*)(5#/8&N-(98$(C82)$.$*%(D5*E$"&3(Q%!&2..2&#*%(L#$7(*; (:5*&*&O:$(U[-( ?#)#"1#%/(0)*&#*%2.(C*%&2/#*%(#%(G*"#2.(K5*+:!F(H;&$5(6257#%-(C*%&$):*525O( 0S:5$!!#*%!3(\2&+52.(]#!&*5O(?+!$+)3(X43(YUUZ
148
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
149
=#/+5$(^U-(98$(C82)$.$*%(D5*E$"&3(Q%!&2..2&#*%(L#$7(*; (:5*&*&O:$(U[3(?#)#"1#%/(0)*&#*%2.( C*%&2/#*%(#%(G*"#2.(K5*+:!F(X%#L$5!#&O(C*..$/$(*; (W*%,*%(]*!:#&2.(=*O$53(X43(YUUZ
E!$F#(,#%$%,&"$.%$H'1"!"+,%#2$#F.0,.!'+($%#%%'+(%$4#3#$-#%'1(#-$59$D'(.$ V+(%.0F#%2$.(-$",/.($&+/7,!#3$'(!#3.&!'+($%&'#(!'%!%$M.-'.$R#3!"+,I#2$ O.!!$Z.&+5'('$.(-$_'/$R9#3%:$E,-'#(&#%$4#3#$'(F'!#-$!+$'(!#3.&!$4'!"$!"#$ 4+3N:$D"#9$4#3#$!"#($'(!#3F'#4#-$!+$)(-$+,!$/+3#$.5+,!$!"#'3$#87#3'#(&#$ +*$ !"#$ 4+3N:$D"#9$ 4#3#$ .0%+$ .%N#-$ !+$ )00$ +,!$ .($ W#/+!'+(.0$ &+(!.1'+($ +0%3$! U1$+*&(..%&,$K<!"#$! =&/$(! &.*$,=&$;! ;%+! *,%.+0,&2$/F! '%*0#$/! *(! $'(*&(.%3! 0(.*%4&(.! +0%3$! U1$+*&(..%&,$+<! "#$! *$%'! 1+$/! Y,(1./$/! "#$(,BF!%!*$0#.&U1$!;#$,$!I$B!-(&.*+!%,$!$H*,%0*$/!),('!*#$!*,%.+0,&2$/! &.*$,=&$;+! %./! 4,(1-$/! &.*(! +&'&3%,! 0(.0$-*+! &.! (,/$,! *(! '%I$! *#$'! /+3#$ 4+3N.50#:$ @3+/$ !"#%#$ &+(&#7!%2$ &.!#1+3'#%$ 4#3#$ *+3/#-$ 4"'&"$ ;$,$! %33(;$/! 1+! &.+&4#*! &.*(! +&'&3%,! 0(.0$,.+! %./! ,$+-(.+$+<! "#,(14#! *#$! $=%31%*&(.! +$++&(.+! ;$! 3$%,.! )%0*(,+! *#%*! +#%-$! *#$! .$H*! &*$,%*&(.! +*$ !"#$ 4+3N:$ @'(#3$ -#!.'0%2$ %,&"$ .%$ !"+%#$ !".!$ 5,'0!$ .(-$ 53+N#$ .$ *##0'(1$ ()! $.4%4$'$.*F! &''$,+&(.F! 0(..$0*&(.! %./! $'(*&(.%3! 0(.*%4&(.F! %,$! ,$=$%3$/!%./!%))$0*!&/$%+!()!.%,,%*&=$!%./!/&+-3%B!()!*#$!.$H*!-,(*(*B-$!()! *#$!!"#$%&%'()*+',%-.<
Figure 11: The Chameleon Project, photos of audience #%&$52"&#%/(7#&8(&8$('5!&( iteration of prototype 07: Integration of Facial 0)*&#*%(P$"*/%#&#*%( 9$"8%*.*/O(2&(&8$(62%2( Center, Science Museum, London, 2009
@##-5.&N$%,11#%!%$!".!$!"#$!"#$%&%'()*+',%-.!-,(=(I$/!*#$!%1/&$.0$!*(! *#&.I!%2(1*!$'(*&(.%3!&.*$,%0*&(.<!
150
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
151
=#/+5$(^Y-(98$(C82)$.$*%(D5*E$"&3(:8*&*!(*; (2+,#$%"$()$)<$5(#%&$52"&#%/(7#&8(&8$(!$"*%,( #&$52&#*%(*; (:5*&*&O:$(UR-(Q%&$/52&#*%(*; (=2"#2.(0)*&#*%(P$"*/%#&#*%(9$"8%*.*/O(2&( W#/8&8*+!$3(_5#/8&*%3(X4(YUUZ =#/+5$(^V-((98$(C82)$.$*%(D5*E$"&3(:8*&*!(*; (2+,#$%"$()$)<$5(#%&$52"&#%/(7#&8(&8$( &8#5,(#&$52&#*%(*; (:5*&*&O:$(UR-(Q%&$/52&#*%(*; (=2"#2.(0)*&#*%(P$"*/%#&#*%(9$"8%*.*/O3( G+:$58+)2%(0S8#<#&#*%(2&(P?Q9(/2..$5O3(?$.<*+5%$3(H+!&52.#2(YUUZ
=#/+5$(^>-(98$(C82)$.$*%(D5*E$"&3(Q%!&2..2&#*%(L#$7(*; (:5*&*&O:$(UZ-((Q):.$)$%&#%/(?**,( 2%,(9$):$52)$%&!F(=2<5#"23(X43(YUUZ(M:8*&*-(D8#..#:(C255N =#/+5$(^J-(98$(C82)$.$*%(D5*E$"&3(Q%!&2..2&#*%(L#$7(*; (:5*&*&O:$(UZ-((Q):.$)$%&#%/()**,( 2%,(&$):$52)$%&!F(=2<5#"23(X43(YUUZ(M:8*&*-(D8#..#:(C255N
MD8*&*-(?251(H!812%2!O3(C*:O5#/8&(P?Q9(K2..$5ON 152
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
153
A$6*/3/?$-*"BCDE!"%5??+%("/*"/%"=5/*("$-"/-*(-%("(F?(6/(-3("$-)"!" will probably think about it later….’ A$6*/3/?$-*" BGD" HI.(" J*.(" 4/)(+" ?+6*6$/*K" #$%" :(/-2" =5/*(" 9/6*$*/+5%<">.("8((0/-2"!".$)"/-%/)("#$%"0/;(".$4/-2"$"3+--(3*/+-" with someone that you had met in a bar or something…’ A$6*/3/?$-*"BLD"HI"!"#$%"30+%("*+"*.("3.$6$3*(6<"M("#$%"=5/*("5?" 86+-*"$-)"/-"&1"8$3(<"@-)"*$0;/-2"=5/*("0+#"$-)"=5/*("/-*/&$*(01E"" Subject 09: ‘I was thinking of some sad things that happened to me, when [...the digital portrait] was sad for a while, it felt like a long time, and it reminded me of some things’. Participant 10: ‘I didn’t like it when he looked sad and I didn’t know why’.8 ‘If you laugh, then the person laughs back. I was impressed by one of the interactions, as the software recognised, as expressed by the portrait, that there had been a recent death affecting the participant. This was in fact true, and that was a very unnerving moment...’ 9 @##-5.&N$ .0%+$ %,11#%!%$ !".!$ 4"#($ .,-'#(&#%$ '(!#3.&!$ 4'!"$ !"#$ %9%!#/2$ -$(-3$! %,$! 03$%,3B! &.*,(+-$0*&.4F! %./! *#&.I&.4! %2(1*! #(;! *(! 2$#%=$F! %+! *#$B!'%B!/(!&.!%!'(,$!*,%/&*&(.%3!+(0&%3!0(.*$H*<!
Participant 07:“I was changing my facial expression from very sad and angry to laughing. It was interesting because sometimes when they were sad and telling you how disappointed they were, you #(6("3.$-2/-2"1+56"(F?6(%%/+-"*+".$??1"%+"/*"#$%"=5/*("%*6$-2(7" laughing when they were sad or angry”…. Participant 09:“If someone was crying in a room, you wouldn’t really laugh at them. So, I was doing something I wouldn’t normally do. You normally empathize, so your face would be as =#/+5$(^[(M.$;&N`(98$(C82)$.$*%(D5*E$"&3(:8*&*!(*; (2+,#$%"$()$)<$5(#%&$52"&#%/(7#&8(&8$('5!&( #&$52&#*%(*; (:5*&*&O:$(UZ-(Q):.$)$%&#%/()**,(2%,(&$):$52)$%&!F(=2<5#"23(X43(YUUZ
154
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
155
(=5$001"%$)<"A0$1/-2"$2$/-%*"/*"#$%"=5/*("/-*6/25/-2"$%"/*%"%+&(*./-2" you just don’t do. Participant 12: “…I was looking at him and he turned around, it was strange, I asked myself why he would do that, it never happened to me, so I was looking at him to understand”. 10
The collaboration of the Chameleon Project "&.%!Y(.+%3=$+!2$3&$=$+!*#%*!*#$!$'(*&(.+!$H-,$++$/!%./!'(.&*(,$/!&.! 3%2(,%*(,&$+! /(.L*! ()*$.! 0(,,$3%*$! *(! *#$! $'(*&(.+! *#%*! )(,'! *#$! )%2,&0! ()!(1,!$=$,B/%B!3&=$+<!T$,!-%+*!;(,I!#%+!2$$.!%!+1+*%&.$/!&.=$+*&4%*&(.! ()! #(;! *(! )1,*#$,! 1./$,+*%./! $'(*&(.! %./! 0,$%*$! *((3+! *#%*! %33(;! 1+! *(! )1,*#$,! $H-3(,$! $'(*&(.+! &.! '(,$! .%*1,%3&+*&0! +0$.%,&(+F! ;&*#! '(,$! -(*$.*!+*&'13&<!
Studio notes in March 2008: There seem to be limited emotions being explored, visually underwhelming databases being used, and the non-ecological settings such as the lab to test responses... using small groups of subjects with narrow representation, what does the knowledge that science is building about emotions actually mean? ” 11 R$0%1+$! ()! *#$! ),$$/('! &.*,(/10$/! 2B! *#$! &.=(3=$'$.*! ()! %.! %,*&+*F! *#$! %&'#(!')&$&+00.5+3.!+3%$+*$!"#$!"#$%&%'()*+',%-.!%,$!3&2$,%*$/!*(!$H-$,&'$.*! &.! ;%B+! 2$B(./! *#$! #&4#3B! 0(.*,(33$/! $H-$,&'$.*+! *#%*! *#$B! %,$! 1+1%33B! ,$+*,&0*$/!*(<!"&.%!Y(.+%3=$+!+$$+!*#$!'(/$3!()!*#$!!"#$%&%'()*+',%-.!%+! 2$&.4!%/%-*%23$!)(,!+0&$.0$!$H-$,&'$.*+<!W10#!,$+$%,0#!&.*(!$'(*&(.%3! ,$+-(.+$+!*%I$+!-3%0$!&.!*#$!3%2(,%*(,B<!"#&+!1.)%'&3&%,!$.=&,(.'$.*!'%B! 0%1+$!%.H&(1+.$++!&.!*#$!+127$0*<!S&*#!5#%'$3$(.F!*#$!%&'!&+!*(!0,$%*$! %!'(,$!$0(3(4&0%3!%./!$.4%4&.4!$H-$,&'$.*%3!*((3!*(!'(.&*(,!$'(*&(.+F! +(!*#%*!-%,*&0&-%.*+!0%.!$H-$,&$.0$!$'(*&(.%3!$H0#%.4$+!'(,$!.%*1,%33BF! 03(+$,!*(!(1,!/%BD*(D/%B!+(0&%3!&.*$,%0*&(.+<!W10#!,$+$%,0#!&.*(!$'(*&(.%3! ,$+-(.+$+! 1+$! 1./$,;#$3'&.4! &'%4$,B<!"#$! !"#$%&%'() *+',%-.$ K9%!#/$ -,(=&/$+! %! +*&'131+! +$*! *(! 0%-*1,$! '(,$! %1*#$.*&0! $'(*&(.+! *(! 0,$%*$! (#4$ /+F'(1$ '/.1#$ -.!.5.%#%$ *+3$ %&'#(&#%:$ D"#$ %&'#(!')&$ &+00.5+3.!+3%$ %4,$$! *#%*! *#&+! ;&33! 2$! &.)(,'%*&=$! &.! 4$.$,%*&.4! .(=$3! ,$+$%,0#! &/$%+!
156
S E C O N D N AT U R E Issue No.3, March 2010
%./! &.! /$=&+&.4! )(0%3! +*1/&$+! *(! 0,$%*$! %! *((3! *(! #$3-! -$(-3$! ;#(! /(.L*! .%*1,%33B!1./$,+*%./!$'(*&(.%3!&.*$,%0*&(.+F!+10#!%+!-$(-3$!;&*#!%1*&+'F! /$-,$++&(.F!(,!%3BH*#&'$%F!%!0(./&*&(.!&.!;#&0#!*#$!-$,+(.!&+!1.%23$!*(! /$+0,&2$!$'(*&(.+!&.!;(,/+<!5#%'$3$(.!&+!'$,4&.4!*#$!2(1./%,&$+!()!*#$! K&'#(&#[E3!$&+00.5+3.!'+(:$
Sloan writes: ‘This is incredibly exciting, at a point when some scientists still, in spite of claims to the contrary, have a tendency to see art as a visualisation or interpretive representation of their work’ Chris Frith sees the cross disciplinary collaboration as liberating: ‘This project has developed far beyond what I would dare to do in the carefully controlled experiments that we are restricted to. But the end result will provide us with marvelous tools for doing new experiments.’ Hugo Critchley, who has been working with the artist for the last ,4("1($6%7"#6/*(%D"H@%"*.("/-%*$00$*/+-")(4(0+?%7"*.("8(():$3;"86+&" audience and one’s own subjective experience will be informative in generating research ideas and focal studies. The most compelling aspect of this is the capacity to engage’. He also sees the artist as a conduit for more cross disciplinary collaborative opportunities ‘The interdisciplinary collaboration has enriched my interest and understanding of emotional research beyond neuroscience & psychophysiology, developing links across disciplines arts humanities engineering etc that are still active, and developing as research collaborations.’ Rosalind Picard, who is mentoring the building of the face reading technology, is interested in its use for people with autism: ‘ As I watch (the Chameleon ProjectK7"&1"&/-)",00%"#/*."#$1%"#("3$-".(0?" people to learn from the interactions portrayed. These are scripts with naked emotion, uncovered, and whether ugly or beautiful, they are hard to turn the eyes from. Here is an engaging palate for helping people who don’t naturally understand emotional interactions, and who want to deepen their ability to do so’.
Issue No.3, March 2010 SECOND NATURE
157
Nadia Berthouze agrees: ‘I see Chameleon as a source of ideas for the creation of digital environments conducive to patients becoming aware of their emotional states’. 12 "#$!!"#$%&%'()*+',%-.!-,$+$.*+!%!4$.1&.$!%./!,%,$!0(33%2(,%*&(.!%0,(++! *#$! 2(1./%,&$+! ()! %,*+! %./! +0&$.0$! -,(=&/&.4! 0#%33$.4&.4! %./! ,$=$%3&.4! .$;!'(/$3+!)(,!$H-$,&'$.*%*&(.!*#,(14#!%,*!&.+*%33%*&(.+F!,$+$%,0#!-%-$,+F! .(-$(+F#02$/+3#$-9(./'&$/+-#0%$*+3$%&'#(!')&$3#%#.3&"$$!".!$'(&3#/#(!.009$ ,$=$%3! *#$! $'(*&(.%3! $H0#%.4$F! '&'&0,B! %./! 0(.*%4&(.! %0,(++! +(0&%3! 13+,7%:$D"#$)(.0$#8"'5'!'+($+*$!"#$4+3N$4'00$!.N#$70.&#$'($?A=A:$
H"1%*7.$,/$)$%&! The work has been core-supported by the Wellcome Trust, The Australia Council for the Arts, The Australian Network for Art and Technology Synapse Residency, Arts Council England, Lighthouse Brighton, Banff New Media Institute, University College London, MIT Media Lab, Solent University Rapid Prototyping Lab, Brighton and Sussex Medical School, Fabrica, SCAN and Dana Center @ the Science Museum.
0%,%*&$! 1. Email excerpts between Tina Gonsalves and collaborators between March 2009-August 2009 for the development of the catalogue H;&$5(6257#%-( C*%&$):*525O(0S:5$!!#*%! at the Natural History Museum. 2 Email excerpts between Tina Gonsalves and subject. 3 Email excerpts between Tina Gonsalves and subject. 4 Excerpts taken from artist’s journal 5 Excerpts taken from artist’s journal 6 Email excerpts between Tina Gonsalves and collaborator. 7 Email excerpts between Tina Gonsalves and collaborator. 8 Taken from video transcripts of subjects taking part in evaluation of prototype 07 at Lighthouse, Brighton, March 2007 9 Taken from blog, http://transjuice.org/page20.htm 10 Taken from video transcripts of subjects taking part in evaluation of prototype 07 at Lighthouse, Brighton, March 2007 (8&&:-aa777F&#%2/*%!2.L$!F"*)a chamselectframe02.htm) select evaluation. 11 Taken from Artists’ studio notes, March 2009 12 Email excerpts between Tina Gonsalves and collaborators between March 2009-August 2009 for the development of the catalogue H;&$5(6257#%-( C*%&$):*525O(0S:5$!!#*%! at the Natural History Museum.
158
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
159
Biographies
" N+6"+4(6"$")(3$)(7">/-$"O+-%$04(%".$%":((-"5%/-2"*.("95/)"$-)"&$00($:0(" medium of video to explore complex emotional landscapes. Gonsalves is currently working with world-leaders in psychology, neuroscience and emotion computing in order to research and produce moving image artworks that respond to your emotions. Poetic installation video works, mobile and wearable technology works respond to pulse, sweat, voice and emotional expressions. She has been awarded numerous Artist in Residence programs including The Banff New Media Institute in Canada, the Centre for Contemporary Art in Prague, Asialink Residency in Bangkok, Thailand, (Pro)duction residency at Artsway, UK, the Advanced Institute of Media Arts and Sciences residency in Japan and Arts Council England’s International Fellowship based in London. She is currently honorary artist in resident at the Institute of Neurology at UCL in London, visiting artist at the Media Lab at The Massachusetts Institute of Technology in Boston, USA and artist in residence at Nokia Research Labs, Finland. Works by Gonsalves have been screened/exhibited extensively. URL: 8&&:-aa777F&#%2/*%!2.L$!F"*)
" N6+&"R$**"!$3+:/-/E%":$3;26+5-)"/-"&$3./-("4/%/+-7"$-)"@6*/,3/$0"!-*(00/2(-3(" Software Engineering studied at the University of Edinburgh and applied in a &50*/)/%3/?0/-$61"3+-*(F*"8+6"&/36+95/)/3%"$-)"-$-+*(3.-+0+21"$*"*.("S+-%/20/+" Nazionale delle Ricerche in Bologna, Matt Iacobini moved his research focus towards a more humanistic direction by studying Human Computer Interaction in the MSc course at the UCL Interaction Centre, and writing a research based thesis within the Chameleon project under the supervision of Dr Nadia BianchiBerthouze and Tina Gonsalves. At the moment he is dividing his time between research projects collaborating with scientists from UCL, MIT and Microsoft T(%($63.7"$-)"$??01/-2".5&$-"3+&?5*(6"/-*(6$3*/+-"*(3.-/=5(%"*+")(4(0+?/-2" applications in the private sector.
The premise of Nadia Berthouze’s research is that affect, emotion, and subjective experience should be factored into the design of interactive technology. Indeed, for technology to be truly effective in our social network, it should be able to adapt to the affective needs of each user group or even each individual. The aim of Berthouze’s research is to create systems/software that can sense the affective state of their users and use that information to tailor the interaction process. Body movement appears to be a promising medium for this goal: it supports cognitive processes, regulates emotions, and mediates affective and social communication. Berthouze is currently pursuing three lines of research looking at body movement as a medium to induce, 6(3+2-/'("$-)"&($%56("*.("=5$0/*1"+8 "(F?(6/(-3("+8 ".5&$-%"$-)"/-"?$6*/350$601" of humans interacting and engaging through/with technology. She is trying to identify the various factors that affect the recognition process, including crosscultural differences and task context. Finally, she is looking onto the existence of dialects in affective body movement communication, including avatar%?(3/,3")/$0(3*%<"P(6*.+5'("#$%"$#$6)()"$"Q"1($6%"!-*(6-$*/+-$0"R$6/("S56/(" Reintegration Grant started in 2006 to investigate these issues in the clinical context and in the gaming industry.
160
S E C O N D N AT U R E Issue No.3, March 2010
Issue No.3, March 2010 SECOND NATURE
161
References Bianchi-Berthouze, N. Kleinsmith, A. “A categorical approach to affective gesture recognition”, Connection Science, 15 (4), 259-269, 2003 Bourgeois. P, Hess. U. “The impact of social context on mimicry”, Biological Psychology, 77 (3), 343-352, 2008. Camurri, A. Ricchetti, M. Trocca, M. “EyesWeb - Toward Gesture and Affect Recognition in Dance/Music Interactive Systems”, Q000(Q%&$5%2&#*%2.(C*%;$5$%"$( on Multimedia Computing and Systems, 1999 Ekman, P. “Paul Ekman Group LLC” 8&&:-aa777F:2+.$1)2%F"*)/ last accessed March 2010 Frith, F. Wolpert, D. “The Neuroscience of social Interaction. Decoding, !&/*$*/-27"$-)"!-95(-3/-2"*.("@3*/+-%"+8 "U*.(6%V<"UF8+6)"W-/4(6%/*1"A6(%%7" 2003. van der Gaad, C. Minderaa, R.B. Keysers, C. “Facial expressions: What the mirror neuron system can and cannot tell us”, Social Neuroscience, 2(3-4), 179-222, 2007 Gilroy, S.W. Cavazza, M.R. Chaignon, S.M. Mäkelä, M. Niranen, E. André, T. Vogt, J. Urbain, M. Billinghurst, H. Seichter, M. Benayoun, 2, “E-tree: emotionally driven augmented reality art”, International Multimedia Conference Proceeding of the 16th ACM international conference on Multimedia, 945-948, 2008 M$*,(0)<"X7"S$3/+??+<"Y7"T$?%+-<"Y<"ZX&+*/+-$0"S+-*$2/+-V7"P$L#$7(*; (:$5!*%2.#&O( 2%,(!*"#2.(:!O"8*.*/OF(^>F(0)*&#*%(2%,(!*"#2.(<$82L#*5 (pp. 151-177). Newbury Park, CA: Sage, 1992 Hess. U, Blairy, S. “Facial mimicry and emotional contagion to dynamic emotional 8$3/$0"(F?6(%%/+-%"$-)"*.(/6"/-95(-3("+-")(3+)/-2"$3356$31V7"International Journal of Psychophysiology, 40 (2), 129-141, 2001 Kaliouby, R. and Robinson P,”Real-Time Vision for HCI” , chapter ~Realtime Inference of Complex Mental States from Facial Expressions and Head Gestures, 181-200. Spring-Verlag, 2005. Khut, G, “Interactive Art, Embodiment and Participation” 8&&:-aa/$*5/$18+&F"*)a last accessed March 2010 [5-)=4/%*7"[<U<"ZN$3/$0"XRO"6($3*/+-%"*+"8$3/$0"(F?6(%%/+-%D"$"3$%("+8 "8$3/$0" emotional contagion?” Scandinavian Journal of Psychology, 36, 130-141, 1995. Lanzetta. L.T, Englis, B.G. “Expectations of Cooperation and Competition and Their Effects on Observers’ Vicarious Emotional Responses”, Journal of Personality and Social Psychology, 56 (4), 543-554, 1989
Maulsby, D. Greenberg, S. Mander, R. “Prototyping an Intelligent Agent through Wizard of Oz”, Proc HC?(GQKC]Q(C*%;F(]+)2%(=2"&*5!(#%(C*):+&#%/(GO!&$)!, ACM Press, 277-284, 1993 Mihalinec, R. Stevanovic, K. “Emotion Recognition and its Aesthetic Interpretation”, Q000(JU&8(Q%&$5%2&#*%2.(GO):*!#+)(*%(0W?HP, 2, 491-494, 2008 Neumann, R. Strack, F. “Mood contagion: The automatic transfer of mood between persons”, Journal of Personality and Social Psychology, 79 (2), 211-223, 2000 Pantic, M A. Pentland, A. Nijholt, T.S. Huang, “Human Computing and Machine Understanding of Human Behavior: A Survey”, H5&#'"2.(Q%&$..#/$%"$(;*5(]+)2%( Computing, 47-71, 2007 Sato, W. Yoshikawa, S. “Spontaneous facial mimicry in response to dynamic facial expressions”, Cognition, 104(1), 1-18, 2007 Tamietto, M. de Gelder, B. “Emotional Contagion for Unseen Bodily Expressions: Evidence from Facial EMG”, Proceeding of the FG 2008 meeting in Amsterdam , 2008 Tofts, D. “Tina Gonsalves: Unleashing Emotion”, Artlink, vol 28 no 2 \6/2.*7"@<"]./-;0(7"X<"[/--(1<"@<""Z@0*(6"X2+D"S+&?5*(6"T(9(3*/+-%"+8 ".5&$-" Emotions”. Proceedings of the [&8(6#/#&2.(H5&(C*%;$5$%"$, 2005
162
Issue No.3, March 2010 SECOND NATURE
S E C O N D N AT U R E Issue No.3, March 2010
163