Context Sensitivity of EEG-Based Workload Classification Under Different Affective Valence

Page 1

IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, VOL. 11, NO. 2, APRIL-JUNE 2020

327

Context Sensitivity of EEG-Based Workload Classification Under Different Affective Valence €ler , Josef Faller, Tanja Krumpe , Thorsten O. Zander, Sebastian Grissmann , Martin Spu Augustin Kelava, Christian Scharinger , and Peter Gerjets Abstract—State of the art brain-computer interfaces (BCIs) largely focus on detecting single, specific, often experimentally induced or manipulated aspects of the user state. In a less controlled, more naturalistic environment, a larger variety of mental processes may be active and possibly interacting. When moving BCI applications from the lab to real-life applications, these additional unaccounted mental processes could interfere with user state decoding, thus decreasing system efficacy and decreasing real-world applicability. Here, we assess the impact of affective valence on classification of working memory load, by re-analyzing a dataset that used an affective N-back task with picture stimuli. Our analyses showed that classification of working memory load under affective valence can lead to good classification accuracies (> 70 percent), which can be further improved via data integration over time. However, positive as well as negative affective valence resulted in decreased classification accuracies, when compared to the neutral affective context. Furthermore, classifiers failed to generalize across affective contexts, highlighting the need for user state models that can account for different contexts or new, context-independent, EEG features. Index Terms—Brain-computer interface (BCI), electroencephalography (EEG), working memory load (WML), affective valence (VAL), emoback

Ç 1

INTRODUCTION

1.1

Brain-Based Adaptive Systems HE electroencephalogram (EEG) represents an interesting method for the evaluation of human machine interaction [1] by enabling the design of brain-based adaptive human-machine interaction systems. Such adaptive systems could allow us to tailor future workspaces to the individual needs of users. Passive brain-computer interfaces (BCIs) represent a good example for such adaptive systems. Passive BCIs utilize mental state monitoring to establish an implicit control loop and therefore do not rely on awareness of the user about this communication. Such an implicit control loop could use additional context information about the user to potentially improve a given human-machine

T

S. Grissmann is with the LEAD Graduate School & Research Network, University of T€ ubingen, T€ ubingen 72072, Germany. E-mail: sebastian.grissmann@lead.uni-tuebingen.de. M. Sp€ uler and T. Krumpe are with the Wilhelm-Schickard-Institute for Computer Science, University of T€ ubingen, T€ ubingen 72076, Germany. E-mail: {spueler, krumpe}@informatik.uni-tuebingen.de. J. Faller is with the Laboratory for Intelligent Imaging and Neural Computing, Columbia University, New York, NY 10027. E-mail: josef.faller@gmail.com. T.O. Zander is with the Team PhyPA, Berlin Institute of Technology, Berlin 10623, Germany. E-mail: tzander@gmail.com. A. Kelava is with the Hector Research Institute of Education Science and Psychology, University of T€ ubingen, T€ ubingen 72072, Germany. E-mail: augustin.kelava@uni-tuebingen.de. C. Scharinger and P. Gerjets are with the Leibniz-Institut f€ ur Wissensmedien, T€ ubingen 72076, Germany. E-mail: {c.scharinger, p.gerjets}@iwm-kmrc.de.

Manuscript received 20 Mar. 2017; revised 6 Oct. 2017; accepted 22 Oct. 2017. Date of publication 22 Nov. 2017; date of current version 29 May 2020. (Corresponding author: Sebastian Grissmann.) Recommended for acceptance by A. A. Salah. Digital Object Identifier no. 10.1109/TAFFC.2017.2775616

interaction. Passive BCIs have been previously applied to adaptive learning environments [2] and even the detection of covert user states like bluffing in a game context [3].

1.2 Challenges When Moving Out of the Lab Most current passive BCIs are used in lab environments and focus on narrow user states like working memory load (WML) [4] or affective valence (VAL) [5]. Therefore, such systems might not be able to provide stable performance outside the lab, since real world environments tend to evoke a combination of multiple basic mental processes [6]. Furthermore, many measures used to infer mental states from the ongoing EEG are known to have many-to-many relations, meaning that several physiological variables are associated with multiple psychological constructs [7]. Previous work suggests that contextual changes, as expected to arise in natural environments, could be related to spontaneously occurring decrements in passive BCI performance [8], [9]. To identify such context-sensitivity in EEG features used for passive BCI classification, we investigated a combination of two important mental states used in passive BCIs: WML and VAL. 1.3 Working Memory Load (WML) Automatic detection of WML has already been studied extensively for brain-based adaptive systems [2], [10], [11]. Working memory is usually defined as the interaction of short term memory structures that handle different representational codes with controlled attentional processes [12], [13]. Content in short term memory is thought to be maintained via parietal brain areas like the intraparietal sulcus, while the central attentional control system is assumed to

1949-3045 ß 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See ht_tp://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Khwaja Fareed University of Eng & IT. Downloaded on July 06,2020 at 01:54:16 UTC from IEEE Xplore. Restrictions apply.


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.