Ryan murdoch proposal

Developing a Virtual Reality Neurofeedback Environment to Aid and Teach Meditation and Mindfulness

MSc Research Proposal

Ryan Murdoch Glasgow School of Art Digital Design Studio

Abstract

This proposal outlines the possibilities of combining current technology and research to create an educational experience; developed to use neurofeedback in an accessible and immersive environment. A combination of virtual reality and brain-computer interfacing would work to create a virtual world that would react in real-time to the user’s cognitive processes, specifically meditation. Meditation and mindfulness have been shown to have massive benefits with regards to mental health and well-being, an area of growing concern, with 1 in 4 people experiencing mental health problems and ž of those people receiving no treatment. The gamification of neurofeedback therapy could allow easier access to this experience, which psychological and medical studies have shown to be both enjoyable and beneficial.

i

Contents 1 Introduction ......................................................................................................................................... 1 1.1 Problem Statement ....................................................................................................................... 1 1.2 Proposed Solution ......................................................................................................................... 2 1.3 Definitions and Concepts .............................................................................................................. 2 1.3.1 EEG ......................................................................................................................................... 2 1.3.2 BCI .......................................................................................................................................... 2 1.3.3 Neurofeedback....................................................................................................................... 3 1.3.4 Meditation and Mindfulness.................................................................................................. 3 2. Completed Work ................................................................................................................................. 4 2.1 Overview ....................................................................................................................................... 4 2.2 Hardware and Software ................................................................................................................ 4 2.2.1 The Neurosky Mindwave Mobile ........................................................................................... 4 2.2.2 Arduino Boards ...................................................................................................................... 5 2.2.3 Max MSP ................................................................................................................................ 6 2.3 Configuring the Hardware............................................................................................................. 6 2.3.1 Configuring Arduino One ....................................................................................................... 6 2.3.2 Configuring the BlueSMiRF Silver........................................................................................... 7 2.3.3 Configuring Arduino Two ....................................................................................................... 7 2.4 Developing the Dynamic Music System in Max MSP .................................................................... 8 2.4.1 Overview and Approach......................................................................................................... 8 2.4.2 Arduino2Max ......................................................................................................................... 8 2.4.6 Developing MESS ................................................................................................................... 9 2.4.7 Creating a Dynamic Soundscape .......................................................................................... 10 2.4.8 MESS’s Potential Use in Games ........................................................................................... 10 2.5 Implementing BCI Data in Game Engines ............................................................................... 10 2.6 Testing the Systems .................................................................................................................... 12 2.6.1 Overview .............................................................................................................................. 12 2.6.2 Setting .................................................................................................................................. 12 2.6.3 Participants .......................................................................................................................... 12 2.6.4 Ethics and Participant Consent ............................................................................................ 12 2.6.6 MESS Testing ........................................................................................................................ 13 2.6.7 Possible Issues with Testing ................................................................................................. 13 2.7 Results ......................................................................................................................................... 13 2.7.1 MESS System Usability Score ............................................................................................... 13

ii

2.7.2 A Look at Collected MESS Qualitative Data ......................................................................... 13 3. Proposed Work ................................................................................................................................. 14 3.1 Overview ..................................................................................................................................... 14 3.2 Shortcomings of previous work .................................................................................................. 14 3.3 Development............................................................................................................................... 14 3.3.1 Using the Unity Game Engine and VR .................................................................................. 14 3.3.2 Adding Voice Recognition .................................................................................................... 14 3.3.3 Designing a Meditative Aesthetic ........................................................................................ 14 3.4 Testing ......................................................................................................................................... 15 3.5 Threats ........................................................................................................................................ 15 4. Summary ........................................................................................................................................... 15 References ............................................................................................................................................ 16 Appendix 1 Collected Results ................................................................................................................ 18 Appendix 2 ............................................................................................................................................ 19 Appendix 3 ............................................................................................................................................ 20

iii

1 Introduction This proposal analyses existing research surrounding virtual reality and BCI and their uses within medical and psychological fields. It outlines an approach for creating an immersive neurofeedback simulation to improve user’s ability and understanding with regards to meditation and mindfulness. The intention is to use the medium of gaming to create an intuitive, immersive and usable tool that individuals can utilise to both familiarize themselves with and practice mindfulness meditation using neurofeedback. Though this study is rooted in psychological and clinical research the aim is to make a game; a well-polished and interactive experience that users can use independently.

A continuation of completed work regarding the sonification of EEG signals in real-time using Neurosky’s Mindwave EEG headset and Max MSP software. This research intends to a) further develop this work to include visualization and sonification of EEG signals in a truly immersive environment utilizing the Unity game engine and virtual reality displays, and b) to explore the usefulness of this kind of neurofeedback in clinical therapy and self-therapy.

1.1 Problem Statement Mental health is a serious problem and has recently been identified as the largest burden of disease in the UK, counting for 28% of the total burden, considerably more than cancer and heart disease (each representing 16%). Despite affecting one in four people in the UK annually, mental health receives only 5.5% (£115 million) of UK health research spending.

[1] Meditation and mindfulness have been shown to be useful coping techniques, which can address a plethora of mental health conditions such as ADHD and anxiety, and benefiting over all psychological well-being [2, 3]. However these techniques are not easy to learn, and understanding such cognitive functions can be difficult. A solution to this is neurofeedback, allowing users to receive real-time feedback on cognitive actions.

Neurofeedback has been proven to be beneficial for mental health, spanning from the novel diagnosis of Alzheimer’s [4] to treatment for ADHD [5]. Neurofeedback provides a noninvasive, medication free treatment with no negative side effects. In fact, the experience is often described as enjoyable. However, neurofeedback is predominantly restricted to clinical use, often requiring a therapist to calibrate and adjust systems to work with a particular patient or problem.

Virtual reality has been shown to be beneficial when helping individuals become meditative, even using their immersive properties for pain relief [6]. Research into various combinations of these technologies and techniques continues to be carried out, with increasing interest in mindfulness, self-regulation and awareness through novel technology [7]. However, attention to game design, mechanics and overall immersion are secondary.

1

1.2 Proposed Solution Designing an educational game that teaches users how to be mindful and meditative using immersive neurofeedback could provide a useful tool, with which users can learn and practice being meditative in their own time. Utilising affordable consumer technology, a system could be created that would not need to be restricted to a hospital or therapist's office. An interactive and useful experience, designed around user experience and usability, which would allow for users to effectively learn and practice meditation in a safe, unintrusive, and enjoyable environment.

Developing a complete experience, with narration and explanations of its workings would allow users to interact with neurofeedback individually and uniquely, in an experience that would put little strain on the already thinly spread mental health treatment sector.

1.3 Definitions and Concepts 1.3.1 EEG EEG or electroencephalogram is a method of using electrodes placed on the scalp to measure electrical brain activity, pioneered by Hans Berger in 1924. When neurons in the brain communicate information to one another the small electrical signals can be detected, together some 10^6 neurons create a measureable reading of a few microvolts that can be monitored this way. Despite being filtered by the meninges, skull, and scalp, there is no latency in this method of monitoring [8].

EEG is a particularly useful technique in that it is not intrusive, and ‘has become one of the most useful tools in the diagnosis of epilepsy and other neurological disorders’ [9]. Various methods of EEG analysis exist including Hjorth, power spectrum and spectral centroid analysis, these methods take the frequency of the EEG and extrapolate readings from them often using FFT or similar approaches.

EEG has some downfalls however; eye movement, eye blink and heartbeat can all cause artefacts in the EEG readings many modern methods of EEG monitoring use algorithms to account for this kind of bodily noise that is introduced.

1.3.2 BCI From the dawn of computers to the current day computers and processing have taken massive leaps. Computers once as big as rooms can now be housed inside a mobile phone.

Despite making massive leaps in terms of computer processing since its initial conception, the way we as users interact with machines has changed very little; there is still reliance on peripheral devices such as the mouse and keyboard to control and interface with machines.

Brain Computer Interfacing means to provide new ways for users to communicate with computers, by utilising EEG or ERP signals machines can be trained to react to the specific brain functions of the user. The possibilities of BCI where explored by Vidal in 1973 in his paper ‘Towards Direct Brain-Computer Communication’ [10], seeing the potential of BCI in future technology Vidal asked if “observable electrical brain signals can be put to work as carriers of information in man-computer communication?”. Suggesting that BCI could be used in the future to control “prosthetic devices or spaceships”.

2

However these concepts are being realized today, in ‘A Novel BCI-Controlled Pneumatic Glove System for Home-Based Neurorehabilitation’ [11] Coffey discusses development of prosthetic hand for rehabilitation, importance is given that this can be achieved inexpensively and with consumer technology. This being just one example of many BCI systems being developed today. The recent presence of EEG monitoring headsets available to consumers such as the Epoc Emotiv or the Neurosky Mindset used in this study has resulted in an increase in popularity of this kind of research. With brainwave monitoring no longer being restricted to neurology labs and hospitals, BCI is becoming a keen area of study across many disciplines including game development and music generation.

1.3.3 Neurofeedback Neurofeedback is the process of giving real time indications to a user of their cognitive functions, such as attention or meditation. Using data from user EEG readings, information can be visualised or sonified to allow the user see how their efforts effect these autonomous functions. Importantly it can allow users to interact and learn about functions of the mind that are uniquely experienced and hard to articulate.

Neurofeedback therapy has been found to be effective in assisting the treatment and management of a range of mental health problems such as ADHD [5].

1.3.4 Meditation and Mindfulness Meditation is the practice of focusing in an attempt to increase awareness of the present moment, to combat stress and to assist relaxation. Mindfulness meditation focuses on a calm awareness, Bishop et al. [12] gives a succinct description: ‘‘We see mindfulness as a process of regulating attention in order to bring a quality of nonelaborative awareness to current experience and a quality of relating to one’s experience within an orientation of curiosity, experiential openness, and acceptance. We further see mindfulness as a process of gaining insight into the nature of one’s mind and the adoption of a de-centred perspective... on thoughts and feelings so that they can be experienced in terms of their subjectivity (versus their necessary validity) and transient nature (versus their permanence).’’

Mindfulness meditation has been found to assist with stress [13], anxiety [3] and even pain relief [14].

Quantifying mindfulness can be difficult and currently more research is investigating ways in which biofeedback methods may do so. In the current study proposal, meditation will be measured using the EEG data utilised in the proposed system wherein Alpha and Theta brainwaves can show variance in levels of meditation. In addition, the Freiburg Mindfulness Inventory (FMI), a questionnaire developed and extensively tested to measure mindfulness [15] will be implemented.

3

2. Completed Work 2.1 Overview The early systems created are intended to demonstrate the potential of sonification of neurofeedback. They are fairly simplistic in nature. This approach to design was chosen to reduce the overall complexity of the systems and hopefully result in less difficulty creating efficient prototypes. The two main hardware components that operate within the system are Neurosky’s ‘Mindwave Mobile’ and two of Arduino’s Uno boards. Max MSP was utilised as the software used to create the audio system. The hardware communicates with the software via serial data, as detailed in the following sections.

The Meditation Enhancing Soundscape System (MESS) was developed to provide users with effective audio neurofeedback with regards to their meditation. An additional system utilising Neurosky’s ‘attention’ eSense value was also created, giving users audio feedback based on their attentiveness.

2.2 Hardware and Software 2.2.1 The Neurosky Mindwave Mobile Neurosky produce reliable headsets for measuring EEG brainwaves. Outside of novel consumer uses these products have been used by developers from a series of different fields as a reliable source of EEG data.

The Neurosky Mindwave Mobile is a wireless headset that uses one monopolar electrode on the forehead and an electrode clip that functions as the null electrode, attached to the earlobe.

These electrodes use EEG to read and analyse user brain activity. The ‘ThinkGear’ technology used inside the headset uses algorithms to determine the user’s level of attention and meditation. These ‘eSense’ values (measured between 0 and 100) are then sent via Bluetooth to a PC or device.

The research into the validity of this technology has been researched by Katona [16] among others. In Katona’s evaluation of the Mindwave it was concluded that: “On the basis of the results, the information obtained by the processed brain waves can be used in several research areas, for instance medical research, multimedia applications, games etc.”

4

The Neurosky Mindwave Mobile, source: Mindwave Mobile Quick Start Guide

2.2.2 Arduino Boards Arduino develop open-source easy to use microprocessors, the user has the ability to program these small computers. The board can then work with data and communicate electronically and digitally with other devices. Two Arduino Uno boards where used to process and communicate data in this system.

Arduino layout diagram taken from ‘Learning JavaScript and Arduino programming with Johnny-Five’

5

2.2.3 Max MSP Max MSP is a modular visual programming language used to create interactive media. It has been used by several sound designers, performers and developers in the past because of its flexibility and its user friendly UI. For these reasons it was chosen as the primary software for testing the capabilities of the BCI system.

2.3 Configuring the Hardware 2.3.1 Configuring Arduino One As two Arduinos where used, each with a different function within the system, the Arduinos will be referred to as Arduino One and Arduino Two. The function of Arduino One was to take the eSense values being sent out by Bluetooth, process that data and display it using LED’s. To communicate with the Bluetooth headset a Bluetooth dongle, the BlueSMiRF Silver, was used.

This method was outlined by the Neurosky Developers guide, the diagram below shows the Arduino configuration. It is worth noting that the Arduino’s TX pin should not be connected when using the UNO.

Arduino One layout, www.developer.neurosky.com

The BlueSMiRF is powered by the Arduino and communicates with the Mindwave headset using Bluetooth. The Bluetooth data is sent to the Arduino from the BlueSMiRF via serial using its TX or transmit pin. The Arduino receives this serial data through its RX pin, from here the code uploaded to the Arduino process the eSense data. The Arduino displays the eSense value (the value used is determined by the code) using a series of ten LED lights. This scales from one LED lit; representing that the headset is properly connected. To ten lit LEDs showing the user has reached ‘100’ for the chosen eSense value.

6

2.3.2 Configuring the BlueSMiRF Silver The BlueSMiRF module can be loosely connected to the Arduino and powered up. Once this is done it can be connected to a PC as a Bluetooth device. Once connected terminal software is used to access the BlueSMiRF and issue it commands.

CoolTerm software used to access and program the BlueSMiRF.

Using a PC to identify the Mindwave’s MAC address the module can be programmed to search and connect to the Mindwave. The appropriate BaudRate and authentication settings can be configured to allow the devices to communicate with one another. It is worth noting that, though not included in the developer documentation, the authentication level or ‘sa’ should be set to 0. Making the BlueSMiRF only discoverable by the headset but not setting the authentication level this way results with an undiscoverable module that you cannot reprogram to accept the headset.

Once this is done the Bluetooth module and the headset can be paired, after this connection is established the readings are sent from the headset to the module and so into the system. The connections on the module where soldered to insure a good signal.

2.3.3 Configuring Arduino Two Having configured the first Arduino the system now has a means to read and display different eSense values. The signals from Arduino One’s digital pins light up the LED array on a bread board. The signal is then taken from the breadboard into Arduino Two’s corresponding digital pins. When this is done the eSense value can be communicated to Arduino Two, the code uploaded to this second Arduino allows it to send serial data via USB to the systems software. Arduino Two requires this USB connection to communicate however Arduino One can run without USB from a power source. Having this configured the system now displays the chosen eSense value and communicates it via USB to the Max MSP patch.

7

Completed system hardware.

2.4 Developing the Dynamic Music System in Max MSP 2.4.1 Overview and Approach Max MSP is used as software to test the effectiveness of the system, once the data was in Max MSP the patches began being developed. The challenge was to recreate the studied approaches game sound designers use in Max. A familiar approach often used in middleware software such as FMOD, vertical re-orchestration was chosen as the solution.

A simple Max patch mimics the fading in and out of layered stems or tracks of a particular looped section of music. Creating a sense of swelling and falling music, however instead of utilising conventional in game events and parameters this system uses user EEG. The attention or meditation eSense levels act as the parameter that’s value dictates the reorchestration of the dynamic game music.

2.4.2 Arduino2Max Arduino2Max is a Max patch available online as an open-source resource on Arduino’s ‘Playground’. This patch developed by Daniel Jolliffe based on a patch by Thomas Ouellet Fredericks and was intended to allow art students easy integration of their projects with Max. This patch is one of many solutions for reading serial data into Max, but it was found to be 8

the most effective. It allows the user to select a COM or USB port, once connected it represents the data on its UI. The UI displays the Arduino’s pins, displaying their current value. It is worth noting that one connection on the patch needs to be changed based on the operating system being used.

Arduino2Max’s easy to use UI.

A version of the MESS patch in Max MSP

2.4.6 Developing MESS The Meditation Enhancing Soundscape System was created to explore the capabilities of the headsets meditation eSense value. It similarly uses four stem tracks simultaneously looped and re-orchestrated. The challenge with MESS was purposely reengineering the system to reduce the number of tracks heard as mediation value increases. Instead of gates automated live.gain objects where used, automating the level of the music to varying degrees as the meditation level increases. This was to test different various approaches to creating systems of this kind. The transitions in MESS are not as smooth as the fade in transitions of AGRO, though as gates are not used the MESS system is able to fade tracks out.

9

2.4.7 Creating a Dynamic Soundscape Similarly the music and soundscape of MESS where created in Pro Tools and performed live, in an effort to give the music a more human feel. At total meditation the user hears only white noise. White noise is often used in meditation to create an audio blanket over other background noises, creating a more uniform and less distracting audio environment. It also works simply to let the user know they achieved a high level of meditation. As meditation lowers a loop of lapping waves is gradually introduced, again in a hope to add to a tranquil and meditative soundscape. A final two tracks; a pad and a peaceful piano track are introduced. Being fully heard when the user isn’t meditating.

2.4.8 MESS’s Potential Use in Games Using a system like MESS games aiming to create relaxed or pleasant virtual worlds could monitor a player’s level of meditation and use dynamic music to react to the user’s state.

Extreme player immersion is considered to be like a ‘state of Zen’, so perhaps levels of meditation could give some insight into player immersion.

2.5 Implementing BCI Data in Game Engines Though the systems developed for this study utilised Max MSP, which was in a sense mimicking the nature of dynamic audio systems in game engines. Further work was carried out in order to establish the possibilities of using the same systems and data inside of real game engines. Using the Unity game engine and a plugin for Unity called Uniduino.

Uniduino allows Unity to communicate with the host PC’s serial ports, by doing this it can send and receive data from and to the Arduinos connected. Data is sent to Unity in the form of an integer for each pin on the connected Arduino, a 0 represent an off or LOW pin and a 1 representing an on or HIGH pin reading.

10

A simple test was carried out to demonstrate the system functioning within Unity, far from a real game it is instead just a cube that can be rotated when the player’s attention value is high enough.

Uniduino does most of the work establishing a connection, all that is required is for it to be set up to communicate with the correct COM port. When the connection is established the values can be used in real time within the Unity. A short piece of simple code in the form of a Unity C# script allows these values to interact with the game world, in this instance a floating rotating cube in 3D space.

This additional work shows that the systems created could be utilised inside game engines. This data could then be used to interact with almost every aspect of a game and would not be limited to audio and music. As demonstrated in the spinning block test the system can work to utilise other parts of game engines such as the physics which were applied to the 3D cube using simple code.

This data could also be used to allow players to interact with game mechanics or alter the visual content of the game. The implementation of these early systems could create near endless possibilities for users to interact with the virtual world of games.

Simple single page of code used in the floating cube demonstration

11

2.6 Testing the Systems 2.6.1 Overview Similar to approaches to other aspects of this study, a focus was placed on simplicity. The aim of the tests was to establish the successful functioning and usability of the system. In essence: do the systems work as intended? By giving the subjects simple tasks to carry out whilst using each system, it was hoped the subjects would be able to pay attention to how the audio feedback responded to their thoughts and cognitive activity. The participants were asked to consider how responsive, and easy to use the system was, before filling out a System Usability Scale questionnaire and were encouraged to write down any thoughts they had on the system in the areas provided. This allowed a robust source of quantitative and qualitative data with which to assess the systems.

2.6.2 Setting The tests were carried out in the author’s study/office room where the system was originally developed. The tests where all carried out between 12am and 6pm over the course of four days to allow for similar lighting conditions at the time of testing. The room was well lit and the area is fairly isolated from extraneous sounds which could cause distractions during testing. Participants tested the system one at a time and were observed during the experiment.

They spent the duration of the test seated in a comfortable office chair at a computer desk.

The audio playback from the computers M-Audio speakers was set a standardized volume to allow a similar acoustic environment for testing.

It was noted during testing that as I was more familiar with some of the participants than others; some participants therefore were likely to be more familiar with my study than others.

This could perhaps cause different results as a newer environment may result in higher attention levels, where as a familiarity with the test environment may be conducive of a more relaxed state in subjects.

2.6.3 Participants Twenty participants were used in the study aged between 18 and 30. As could be expected of this demographic, participants had varying, but generally good, experience and knowledge of technology. The participants had varying familiarity with meditation and where made up of seventeen males and three females, a more even distribution of men and women could have been more useful for this study. However it is felt that brain activity, especially at the kind of general levels utilised here, are not effected by gender and are innately human allowing for the success of EEG in medical and psychological fields.

2.6.4 Ethics and Participant Consent Each participant was issued a consent form with their questionnaires, the consent form informed users of what they would be asked to do and what potential risks there were. As far as risks the headset can cause some discomfort to the earlobe and head if worn for long periods of time but is totally safe. It was felt to be important to acknowledge that though the headset monitored brain activity it in no way directly effects or interacts with participant brain activity.

12

The participants were told they could discontinue the experiment at any time, that they had the right to withdraw their results, and that their results were going to be kept confidential.

They were also told that they could request access to this study and any further research carried out using their results.

2.6.6 MESS Testing The system was initialized and the participant was seated. The participant was simply asked to meditate, being meditative was described as relaxing and trying to clear your thoughts, aided by breathing deeply and closing of the eyes. Participants were asked to try and relax in order to bring the music in MESS to the quiet white noise generated at high levels of meditation. This exercise was carried out for five minutes. After this the subjects were given a further five minutes to experiment with the system. After this they were asked to fill out a System Usability Scale questionnaire (SUS) [17].

2.6.7 Possible Issues with Testing Though measures were taken to control variables the test could still have been a more controlled environment. Though the study used is quiet perhaps an alternative setting would have been better. Though the system is portable, a laptop or computer would have to have the appropriate software installed and configured and in the face of time restraints testing where the system was being developed was more efficient.

The participant group consisted mostly of employees from a local hotel, most of the participant’s knew the author to varying degrees and this may have caused biased results. The participants were encouraged to be honest with their answers.

Literature around SUS has suggested that question 8 ‘I found the system very cumbersome to use’ could cause difficulty with participants. During the test many subjects were unsure of what was meant by ‘cumbersome’. It would perhaps have been better to amend cumbersome to awkward as is sometimes done to make the questionnaire more user friendly.

2.7 Results 2.7.1 MESS System Usability Score An average score was taken from the twenty participant scores, this resulted in a SUS score of 85/100 for the meditation based system. This system would be consider a band A and is an extremely high score. Suggesting an extremely accessible system. Comparing the additional learnability and usability data to the attention based system MESS scored a 63/100 for learnability and a 69/100 for usability. Suggesting this system was even easier to use and become familiar with than the attention based system which also scored highly.

2.7.2 A Look at Collected MESS Qualitative Data Overall participants seemed to find MESS easier to use, which was not only reflected in the SUS score awarded to the system but also in participant statements. Participants felt that this system was “a lot better than previous test” (regarding system A, the attention based system) and that generally this system was easier to influence. Participants seemed to find the experience relaxing and an audio representation of the user’s meditation level allowed using the system to be an informative and ‘reflective’ experience for users.

13

3. Proposed Work 3.1 Overview Previous work showed that users felt that the meditation eSense was easier to ‘use’ when attempting the neurofeedback. Moving forward and away from just sonified neurofeedback towards a full immersive experience in VR, meditation was chosen to be used individually.

Though it only utilizes some of the headsets data, it can provide a simplicity to the system and a means of focusing on developing a game centred on meditation and the positive effects that could have on users.

3.2 Shortcomings of previous work Previous work succeeded in showing the effectiveness of the Neurosky headset with sonified neurofeedback. It also displayed that the eSense data could be used in the Unity game engine in real time. However with only audio feedback the system did little to show the potential of the headset or its use in immersive neurofeedback environments and was far from a game.

The system could also be simplified and improved, for instance one Arduino could be used instead of two. However it is felt that additional Arduinos add flexibility to the system.

3.3 Development Moving forward, the aim is to focus on developing an effective and unique VR experience utilising the Neurosky headset. A user focused means of neurofeedback, an interactive experience with which users can learn about meditation and its benefits.

3.3.1 Using the Unity Game Engine and VR As shown in the completed work section the system works with the Unity game engine. Unity is a powerful tool, reactive audio and music can be created using FMOD middleware, programmed to respond dynamically to the users meditation. Now using the Unity game engine visualization of the neurofeedback is possible, a reactive 3D environment can be created allowing users to see their cognitive actions effect the world around them. With this immersive form of neurofeedback we aim to effectively teach individuals about meditation.

3.3.2 Adding Voice Recognition Allowing players to relax and interact with the experience mostly with their meditation, voice commands will be added to the system to allow players to be hands free. Any interaction needed in the game will be able to be carried out by voice commands, such as 'simulation start' or 'simulation end'. This can be achieved by using voice recognition software that can recognise and send select pre-programmed voice commands to unity as serial data.

3.3.3 Designing a Meditative Aesthetic Careful attention will be paid to design a visual environment that is engaging and rewarding for the user to engage with. This will be done by creating a reactive and captivating 3D environment utilising assets modelled in 3ds Max. Creating a familiar room environment in which the user can follow a tutorial and become accustomed to using their meditation to move objects. Proceeding through the tutorial completing tasks and utilising the voice commands eventually the player will be able to start the simulation. The walls and furniture will slowly disappear to reveal a new Zen garden like environment as the player meditates. Creating an environment that visually represents the 14

player’s action of meditating, alongside reactive music and audio feedback all displayed in virtual reality.

3.4 Testing Currently intentions for testing, based on research in related fields, consists of testing two groups of participants. Testing two participants groups of twenty; half practiced meditators and half novice, will allow the impact and usefulness of the system to be studied.

Testing both groups with FMI questionnaires before they interact with the systems will allow for an initial measure of their mindfulness. They will then be asked to use the system once a week for twenty minutes for three weeks. After using the neurofeedback system to practice mindfulness meditation over the three weeks a second round of questionnaires will be issued to see if the participants ability to utilise mindfulness has improved.

3.5 Threats It is possible that the VR headset could interact with the Neurosky headsets EEG readings negatively effecting the system. Creating an effective and immersive multisensory experience will also require the creation of a great deal of assets, this has to be considered as well as the potential for 'feature creep' with continued development of the system.

4. Summary By creating an effective and immersive neurofeedback environment in virtual reality players can plug in and play an enjoyable and educational experience. By utilising the powerful developing technologies of today, and the user-focused design of video games, new ways of interacting with educational media can be created. This research intends to take all of these somewhat novel influences to create a truly useful and beneficial experience. It is possible, in an increasingly busy and digital world to create a relaxing experience through which everyone can learn mindfulness meditation and its benefits.

15

References 1. Mentalhealth.org 2015. Fundamental Facts About Mental Health. [ONLINE] Available at:https://www.mentalhealth.org.uk/sites/default/files/fundamental-facts-15.pdf.

[Accessed 20 July 2016] 2. Zylowska, L., Ackerman, D.L., Yang, M.H., Futrell, J.L., Horton, N.L., Hale, T.S., Pataki, C. and Smalley, S.L., 2008. Mindfulness meditation training in adults and adolescents with ADHD a feasibility study. Journal of Attention Disorders, 11(6), pp.737-746.

3. Hofmann, S.G., Sawyer, A.T., Witt, A.A. and Oh, D., 2010. The effect of mindfulness-based therapy on anxiety and depression: A meta-analytic review. Journal of consulting and clinical psychology, 78(2), p.169.

4. Elgendi, M., Dauwels, J., Rebsamen, B., Shukla, R., Putra, Y., Gamez, J., ZePing, N., Ho, B., Prasad, N., Aggarwal, D. and Nair, A., 2014. From auditory and visual to immersive neurofeedback: application to diagnosis of Alzheimer’s disease. In Neural Computation, Neural Devices, and Neural Prosthesis (pp. 63-97). Springer New York.

5. Gevensleben, H., Holl, B., Albrecht, B., Vogel, C., Schlamp, D., Kratz, O., Studer, P., Rothenberger, A., Moll, G.H. and Heinrich, H., 2009. Is neurofeedback an efficacious treatment for ADHD? A randomised controlled clinical trial. Journal of Child Psychology and Psychiatry, 50(7), pp.780-789.

6. Tong, X., Gromala, D., Choo, A., Amin, A. and Shaw, C., 2015, August. The virtual meditative walk: an immersive virtual environment for pain self-modulation through mindfulness-based stress reduction meditation. InInternational Conference on Virtual, Augmented and Mixed Reality (pp. 388-397). Springer International Publishing.

7. Shaw, C., Gromala, D. and Song, M., 2010. The meditation chamber: towards selfmodulation. Metaplasticity in virtual worlds: aesthetics and semantics concepts. IGI Publishing, pp.121-133.

8. Woodman, G.F., 2010. A brief introduction to the use of event-related potentials in studies of perception and attention. Attention, Perception, & Psychophysics, 72(8), pp.2031-2046.

9. Miranda, E. and Brouse, A., 2005, May. Toward direct brain-computer musical interfaces.

In Proceedings of the 2005 conference on New interfaces for musical expression (pp. 216219). National University of Singapore.

10. Vidal, J.J., 1973. Toward direct brain-computer communication. Annual review of Biophysics and Bioengineering, 2(1), pp.157-180.

11. Coffey, A.L., Leamy, D.J. and Ward, T.E., 2014, August. A novel BCI-controlled pneumatic glove system for home-based neurorehabilitation. InEngineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE (pp. 3622-3625).

IEEE. 12. Bishop, S.R., Lau, M., Shapiro, S., Carlson, L., Anderson, N.D., Carmody, J., Segal, Z.V., Abbey, S., Speca, M., Velting, D. and Devins, G., 2004. Mindfulness: A proposed operational definition. Clinical psychology: Science and practice, 11(3), pp.230-241.

13. Grossman, P., Niemann, L., Schmidt, S. and Walach, H., 2004. Mindfulness-based stress reduction and health benefits: A meta-analysis. Journal of psychosomatic research, 57(1), pp.35-43.

16

14. Zeidan, F., Grant, J.A., Brown, C.A., McHaffie, J.G. and Coghill, R.C., 2012. Mindfulness meditation-related pain relief: evidence for unique brain mechanisms in the regulation of pain. Neuroscience letters, 520(2), pp.165-173.

15. Walach, H., Buchheld, N., Buttenmßller, V., Kleinknecht, N. and Schmidt, S., 2006. Measuring mindfulness—the Freiburg mindfulness inventory (FMI).Personality and Individual Differences, 40(8), pp.1543-1555.

16. Katona, J., Farkas, I., Ujbanyi, T., Dukan, P. and Kovari, A., 2014, January. Evaluation of the NeuroSky MindFlex EEG headset brain waves data. InApplied Machine Intelligence and Informatics (SAMI), 2014 IEEE 12th International Symposium on (pp. 91-94). IEEE. 17. Brooke, J., 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), pp.4-7.

18. Sas, C. and Chopra, R., 2015. MeditAid: a wearable adaptive neurofeedback-based system for training mindfulness state. Personal and Ubiquitous Computing, 19(7), pp.1169-1182.

19. Kizony, R. and Katz, N., 2003. Adapting an immersive virtual reality system for rehabilitation. The Journal of Visualization and Computer Animation,14(5), pp.261-268.

17

Appendix 1 Collected Results

18

Appendix 2

19

Appendix 3

20