Does Accurate Facial Expression Recognition Influence

Dwell Time?

Katie Kozlowski, Kali DeBorde, Ayesha Ferozpuri, Karen Salazar, and Peyton Stites Faculty Sponsor: Dr. Laurie Hunter, Department of Psychology

Abstract

The current study explored whether accuracy of recognition of facial expression of emotion differentiates individual’s eye-tracking patterns, via the eye-tracking metric of dwell time (visit duration). We argue visit duration provides more detailed information for evaluating which features of the face are used more often for processing emotional faces. Visit duration is greater for the middle and lower areas of the faces as compared to the upper and non-core areas of faces, but it is unclear whether this focus leads to greater accuracy (Hunter, Roland, & Ferozpuri, 2019).

Thirty-five undergraduate students from a small liberal arts university in the Mid-Atlantic viewed an 80-slide presentation of faces, taken from CAFE child facial set (LoBue & Thrasher, 2014) and NimStim adult facial set (Tottenham et al., 2009) while measures of eye-tracking were documented using Tobii X3-120. Participants were instructed to label the faces with the appropriate emotion. A median split on accuracy of recognition for each emotion was utilized to divide participants into high accuracy and low accuracy.

Five, one for each emotion (anger, fear, happy, neutral, sad), 2 (Accuracy: high or low) X 2 (age of face) X 4 (AOI) ANOVAs were conducted to assess visit duration. For anger, fear, happy, neutral, and sad, accuracy of recognition did not explain differences in visit duration among the four AOIs. As suggested in previous literature (Hunter, Roland & Ferozpuri 2019), the lower and middle AOI had greater visit durations than upper or non-core AOIs. The non-core area of interest (all areas of the face other than eyebrows, eyes, and nose/mouth) had greater visit duration among individuals with lower accuracy scores compared to individuals with higher accuracy scores. In addition, as with previous studies (Barabanschikov, 2015; Dinkler et al., 2018) the chosen stimuli were ones which had the highest ratings of accuracy from the original validation study, thus stimuli utilized in this study may not be sensitive enough to detect potential differences. Facial expression stimuli with lower accuracy rating (i.e., less intensity) would be a prudent investigation, and would more likely mirror real world interactions.

Kali DeBorde is a senior Psychology major with a minor in Childhood Studies at Christopher Newport University. In the fall, she will be pursuing her Master’s in speech-language pathology at Old Dominion University.

Ayesha Ferozpuri is a Christopher Newport University alumna with a Bachelor of Science in Neuroscience and Psychology. She currently works at Solstice East, a residential treatment center for adolescent girls, as a personal development mentor. In the fall, she will be pursuing her Masters in rehabilitation and mental health counseling at Virginia Commonwealth University.

Karen Salazar is a rising Junior at Christopher Newport University with a Neuroscience major and a double minor in human rights and conflict resolution and psychology.

Katie Kozlowski earned her Bachelor of Science in Psychology at Christopher Newport University. During her time at CNU, she was a research assistant in the Facial Processing and Recognition Lab and plans to work in the data analysis field.

Peyton Stites earned her Bachelor of Science in Psychology at Christopher Newport University and took part in Emotion/Recognition research. She plans to continue her education and obtain her Master in Occupational Therapy.

Does Accurate Facial Expression Recognition Influence Dwell Time?

Kali DeBorde, Ayesha Ferozpuri, Karen Salazar, Katie Kozlowski, Peyton Stites, & Laurie Hunter Department of Psychology

Abstract

The current study explored eye-tracking patterns and examined whether accuracy of recognition differentiated how facial expressions of emotion are processed. Specifically, we asked whether accurate recognizers focused more on the critical areas of interest. Our findings indicated no difference between the high and low accuracy groups when processing emotions. Both groups focused on the middle and lower areas of the face most often, support previous literature, thus highlighting on the importance of eyes and mouth when processing emotions.

Methodology

● The participants were instructed to label the faces with the appropriate emotion.

● A median split on accuracy of recognition for each emotion was utilized to divide participants into high accuracy and low accuracy.

Findings

Results

Five 2x2x4 mixed model ANOVAs (2 Accuracy of recognition, high versus low); 2 ages: adult and children & 4 AOIs: lower, middle, noncore, and upper) were conducted for anger, fear, happy, neutral, and sad.

● Of importance to the current study, no interactions among AOIs and accuracy were obtained for any of the emotions.

o F(3, 387) = 0.467, p>0.05.

● Hunter, Roland, and Ferozpuri (2019) found that visit duration is greater for the middle and lower areas of the face in comparison to upper and non-core areas of the face.

● The mouth (low AOI) is necessary for the recognition of happiness and the eye (middle AOI)/brow (upper AOI) is necessary for the recognition of sadness (Beaudry et al., 2014).

● The lower and middle AOI have greater visit duration compared to upper or non-core AOI (Hunter, Roland, and Ferozpuri, 2019).

Importance of the Research Participants

● 138 undergraduate students from a small liberal arts university in the Mid-Atlantic.

● An 80-slide presentation of faces was taken from CAFE child facial set (LoBue & Thrasher, 2014) and NimStim adult facial set (Tottenham et al., 2009) while measures of eye-tracking were documented using Tobii X3-120.

(Tottenham et al., 2009)

References

Barabanschikov, V.A. (2015). Gaze dynamics in the recognition of facial expressions of emotion. Perception, 44(8-9), 1007-1019. doi: 10.1177/0301006615594942.

Beaudry, O., Roy-Charland, A., Perron, M., Cormier, I., Tapp, R. (2014). Featural processing in recognition of emotion facial expressions. Cognition and Emotion, 28(3), 416-432. Hunter, L.S., Roland, L. & Ferozpuri, A. (2019). Emotional expression processing and depressive symptomatology: Eye-tracking reveals differential importance of lower and middle facial areas of interest. Manuscript under revision, Depression Research and Treatment.

LoBue, V., & Thrasher, C. (2015). The child affective facial expression CAFE) set: Validity and reliability from untrained adults. Frontiers in Psychology, 5 1-8.

Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., & Nelson, C. (2009). The NimStim set of facial expressions: Judgements from untrained research participants. psychiatry Research, 168(3), 242-249. doi: 10.1016/j.psychres.2008.05.006 o F(3,387) = 1.470, p>0.05. o F(3, 387) = 0.795, p>0.05. o F(3, 387) = 0.223, p>0.05. o F(3, 387) = 0.257, p>0.05.

Interpretation

● The ability to accurately recognize emotion did not have an effect on a participant's ability to process faces

● The lower and middle AOI have greater visit duration in comparison to the upper and non-core AOI, for each of the five emotions

● For this study, facial emotion stimuli were chosen specifically for their ease of recognizability.

○ Stimuli with lower recognizability scores, arguably more difficult to process, may provide an ability to detect differences in processing for various emotions.