A Multimodal Approach Exploiting EEG to Investigate the Effects of VR Environment on Mental Workload

Virtual reality (VR) is a technology that allows users to experience multisensory and interactive environments that simulate real or imaginary scenarios. The effect of different VR immersive technology on mental workload (MWL), i.e., the amount of resources required to perform a task, is still debated; however the potential role of EEG in this context was never exploited. This paper aims to investigate the effects on MWL of performing a cognitive task in a VR environment in two conditions characterized by different degrees of immersion using a multimodal approach which combines well-assessed subjective evaluations of MWL with physiological EEG measures. A cognitive task based on the n-back test was proposed to compare the performance and MWL of participants who used either a head-mounted display (HMD) or a desktop computer to present the stimuli. The task had four different complexity levels (n = 1 or 2 with either visual or visual and audio stimuli). Twenty-seven healthy participants were enrolled in this study and performed the tasks in both conditions. EEG data and NASA Task Load indeX (NASA-TLX) were used to assess changes in objective and subjective MWL, respectively. Error rates (ERs) and reaction times (RTs) were also collected for each condition and task level. Task levels had significant effects on MWL, increasing subjective measures and decreasing performance, in both conditions. EEG MWL index have shown a significant increase especially if compared to rest. Different degrees of immersion did not show significant differences neither in individual's performance nor in MWL as estimated by subjective ratings. However, HMD reduced the EEG-derived MWL in most conditions indicating a lower cognitive load. In conclusion, HMD may reduce the cognitive load of some tasks. The reduced level of MWL, as depicted by the EEG MWL index, may have implications for the design and future evaluation of VR-based applications.