Next Article in Journal
Retrieving Sun-Induced Chlorophyll Fluorescence from Hyperspectral Data with TanSat Satellite
Previous Article in Journal
A Hybrid Deep Learning-Driven SDN Enabled Mechanism for Secure Communication in Internet of Things (IoT)
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Workplace Mental State Monitoring during VR-Based Training for Offshore Environment

Centre for Intelligent Signal and Imaging Research (CISIR), Electrical and Electronics Engineering Department, Universiti Teknologi PETRONAS, Bandar Seri Iskandar 32610, Malaysia
Petroliam Nasional Berhad, Kuala Lumpur 50088, Malaysia
Author to whom correspondence should be addressed.
Sensors 2021, 21(14), 4885;
Received: 22 April 2021 / Revised: 22 May 2021 / Accepted: 3 June 2021 / Published: 18 July 2021
(This article belongs to the Section Wearables)


Adults are constantly exposed to stressful conditions at their workplace, and this can lead to decreased job performance followed by detrimental clinical health problems. Advancement of sensor technologies has allowed the electroencephalography (EEG) devices to be portable and used in real-time to monitor mental health. However, real-time monitoring is not often practical in workplace environments with complex operations such as kindergarten, firefighting and offshore facilities. Integrating the EEG with virtual reality (VR) that emulates workplace conditions can be a tool to assess and monitor mental health of adults within their working environment. This paper evaluates the mental states induced when performing a stressful task in a VR-based offshore environment. The theta, alpha and beta frequency bands are analysed to assess changes in mental states due to physical discomfort, stress and concentration. During the VR trials, mental states of discomfort and disorientation are observed with the drop of theta activity, whilst the stress induced from the conditional tasks is reflected in the changes of low-alpha and high-beta activities. The deflection of frontal alpha asymmetry from negative to positive direction reflects the learning effects from emotion-focus to problem-solving strategies adopted to accomplish the VR task. This study highlights the need for an integrated VR-EEG system in workplace settings as a tool to monitor and assess mental health of working adults.

1. Introduction

Stress response can be seen as a built-in alarm for the body to detect deviations from the balanced condition known as homeostasis, and further activate neural activities to increase arousal and alertness [1]. Although its effects may not be easily visible, prolonged stressful conditions can lead to physical and mental health deterioration. The increase in adults facing high level of work-related stress are observed throughout the world [2,3], more so in the recent COVID-19 pandemic due to perceived stress and financial uncertainties [4]. Aside from the conventional approaches used in psychotherapy for diagnosis and healthcare, researchers are exploring physiological signals of stress responses to help in providing objective evaluations of a person’s mental health. The electroencephalography (EEG) is the most used brain-monitoring method for mental stress because of its portability, high temporal resolution, non-invasive, and low cost advantages [5]. The EEG brain waves (0.5–45 Hz) are strongly associated with different brain responses towards stress and have been used to detect stress [6,7,8].
However, monitoring the EEG signals in an actual workplace setting to assess mental health and stress levels is not feasible as this can disrupt the working routines. With virtual reality (VR), immersive 3D environments based on any real-life scenarios can be developed to provide interactive experience for the users. Rapid advancement in sensor technologies has allowed simpler and cheaper setup that integrates EEG with the VR technology. It has been used in various settings such as healthcare [9,10,11,12], oil production [13,14], workplace [15] and education [16]. With the integration of EEG, researchers have studied various brain functions such as attention [17,18], cognitive load [19,20], motor intention and planning [11], emotional processing [21,22], sense of immersion [23,24], engagement [25] and relaxation [26,27,28]. Although VR has been used with the EEG for meditation and mind therapy purposes, it has not been used as an integrated system to monitor mental health in workplaces.

Motivation of Study

Workplace stress is unique and depends on the setting and challenges faced by employers within the company. The flexibility of VR to emulate different real-life scenarios makes it an excellent tool to assess mental states and performances of workers, especially in workplaces with high-risk conditions such as offshore facilities. Offshore workers are amongst working adults that face high-level of stress during their working hours [29]. Moreover, the severe impact of COVID-19 pandemic on the offshore industry has changed the standard operating procedures in these facilities [30,31]. The lack of expertise or carelessness in handling the machinery within an oil well facility can lead to accidents and severe injury. Prior to operating at remote locations, workers must undergo extensive training to operate offshore oil and gas production facilities.
Leading oil and gas companies have adopted the VR technology [32] in their training and operations [33]. Conventional training methods for offshore operations involve direct interaction between trainers and trainees where courses are given on how to choose proper tools and operate complex machines under varying conditions. Theoretical and conceptual training such as classroom and e-learning courses lack hands-on training, whereas live-training during actual working hours are costly, high risk and not applicable for dangerous operations. Research has been done to develop VR-based offshore training with different modules such as safety handling [34], crane training [35,36], fire incidents [37,38] and hazard-based emergency response training [39]. Integration of VR technology in the chemical industry has been observed to improve training time and cost [14,40], and time for facility recovery and startup after shutdown [41].
In this paper, we evaluate three different mental states induced when performing a stressful task within a VR offshore environment. We hypothesize that the changes in mental state when performing the VR task of a working environment can reflect specific behaviours which is measurable using EEG. The mental states observed are the discomfort felt due to the VR itself, and changes in stress and concentration levels when performing the task. Findings from this paper contributes to further efforts to integrate VR and EEG to monitor mental health in stressful workplace. The VR offshore environment is developed based on four different training modules, and is assessed on user-experience using questionnaires and performance scores. Section 2 details the developed VR-offshore environment and its four modules, Section 3 describes the testing procedures conducted to assess the VR performance and mental states using EEG device, followed by results and discussion in Section 4 and Section 5.

2. VR-Based Training for Offshore Environment

More research is incorporating the VR technology into mental health applications such as in resilience [42], emotion regulation [43], and mindfulness [25,44] interventions. This paper presents the changes of mental states in response to a stressful task performed in a VR-based offshore environment. The VR environment is developed based on four different training scenarios of operating the oil well facilities using the Unity 3D software. Microsoft Visual Studio is used to program the interactions within the environment. The HTC Vive headset is used to display the VR environment and its controllers are enabled to allow hand-movement operations. The HTC Vive is calibrated using Steam VR engine to ensure the efficiency of training during the simulation and provide better immersive-effect in the user. The following section presents the four VR-training modules developed for offshore training.

2.1. Virtual Environment for Offshore Training

Figure 1 presents the startup scene of the VR environment. The scene begins with the simulated event of arriving at the offshore platform through a helicopter. Once the user arrives on the platform, the controller can be used to choose and teleport to the preferred training area. Each modules begin with a list of instructions and several tools that will be used during the task. The tools can be chosen and operated using the HTC Vive controllers.

2.1.1. Module 1—Oil Sampling

Oil sampling is an essential routine activity for offshore workers where they are required to collect oil samples for laboratory analysis. Careless sampling or contamination in the sample container can lead to inaccurate results and losses as the whole routine also involves transporting and testing of the sample. Module 1 (as shown in Figure 2) requires the trainee to close the inlet valve and lock it to stop the oil flow into the tank, place a sampling bottle under the tap, open tap until sufficient amount of oil is drained, close tap and clean with sponge and lastly pickup the sampling bottle.

2.1.2. Module 2—Confined Space Inspection

Working at the offshore facilities often involved performing task within confined spaces such as silos and tanks. This module introduces the trainee to procedures of entering and exiting a tank. Similar to Module 1, trainee must first close the inlet valve to stop the tank process flow. Before climbing the tank, the sensor power board needs to be switched off. For the climbing movement, the trainee must touch the ladder to trigger the climbing animation to the top of tank in the VR environment. The VR environment inside the tank is built to emulate dark surrounding of a small space (as shown in Figure 3). Trainee will enter the tank and inspect for damages inside the tank.

2.1.3. Module 3—Pipeline Valve Rotation

Managing offshore pipelines is also a crucial operation to ensure safe transportation of oils and gas. In this module, the trainee is required to observe any smoke leakage from the pipeline and rotate the valve in clockwise direction for at least 30 turns using the controller (as shown in Figure 4). The leakage must be contained within a time frame of 35 s, with the speedometer attached to the pipeline is set to reduce at a rate of 2 unit/s to indicate drop of pressure. Scores in Module 3 is based on the number of valve rotation the participant was able to perform within 35 s or until no smoke leakage was observed in the VR environment.

2.1.4. Module 4—Crane Training

This module allows trainee to experience handling the crane while moving large cabins without any collision. When situated within the crane driving cab, the trainee will control the position of the lifting hook, pick up the cabin using the hook and place it on the allocated area (as shown in Figure 5). There are two cabins that need to be moved. During this module, coin scores are given based on the success of placing the two cabins in the allocated areas (eight coins per area). During the movement of the cabin, any collision with nearby obstacles results in −5% reduction from a 100% score. The number of coins collected indicate proper placement of cabin and the percentage score indicates the task was performed at minimal collision with any nearby obstacles.

3. Mental State Monitoring during VR Training

Twenty one participants are recruited to test the developed VR environment for offshore training. Inclusion criteria for the participants are healthy male and female adults, aged 18 or above, and had normal or corrected-to-normal vision. The participants are students from the Universiti Teknologi Petronas (UTP) studying different engineering programs. This study is done under the ethical approval from the Universiti Pendidikan Sultan Idris Ethics Committee (2020-0121-01). Figure 6 presents the flow of experiments done in this study. Module 1 and 2 are used to assess the experience of participants when undergoing the VR training, and Module 3 and 4 are used to assess the task performance of participants. The mental states of participant are monitored using MITSAR-EEG-202-31 device during the Module 4 training.

3.1. User Experience and Performance Assessments

In the feedback assessment, participants are asked to choose the most relevant response from Table 1 they experienced after performing the tasks in Module 1 and Module 2. The responses reflect five aspects of user experience from the developed VR environment and can reveal the advantages and drawbacks of the developed VR in emulating offshore operations.
Performance of participants in Module 3 and 4 are evaluated in percentage from the coins collected and ability to avoid obstacles when moving the cabin using crane. There are two cabins with each having a designated placement area with 8 coins floating on the surface. Each coin carry 6.25%, giving 100% score when all 16 coins are collected. For the collision performance, participants began with 100% and receive −5% deduction if they collided with nearby obstacles when moving the cabin using crane.

3.2. Mental State Assessment Using EEG

Figure 7 illustrates the experimental setup for the EEG recording and the VR headset. The EEG signals are collected using a 32-channel EEG cap with common reference setting and at 500 Hz sampling rate. Impedance for all channels is maintained below 10 kΩ during EEG data recording. Figure 8 illustrates the electrode channels used for this study.
The whole EEG recording session is divided into six segments as shown in Figure 9. Firstly, the EEG is recorded at resting with eyes-close without wearing the VR headset for five minutes. The participant is then given the VR headset and asked to rest with eyes-open for two minutes. During the eyes-open resting, a black screen is displayed on the VR. The VR trials in this session are given two conditions where the participant must complete the crane training in Module 4 within a 5 min time limit and without colliding with any nearby obstacles. Once the second VR trial is completed, the VR headset is removed and participant is asked to rest with eyes-close for five minutes. The duration of the whole EEG recording is approximately 24 min.

EEG Analysis

The EEG signals are re-referenced to Cz and filtered using band pass filter between 4 and 45 Hz. Delta band (0.5–4 Hz) is not analysed as activity below 5 Hz is prone to gross movement artifacts [20]. Independent component analysis is done using the Infomax algorithm (runica, ‘extended’) in the MARA plugin [45] for EEGLAB toolbox [46] to remove motion artefacts, with automatic artefact rejection probability > 95%. Signals are segmented into 30 s epochs to observe the average mental state in each segment (Figure 8).
In this study, only 10 channels were selected for analysis as shown in Table 2. Using the FIR filter, the EEG signals are decomposed into theta (4–8 Hz), alpha (8–13 Hz) and beta (13–30 Hz) bands. Theta band in the temporo-parietal areas has been associated with discomfort and disorientation when using the VR headset [47]. Alpha and beta bands reflect the conscious states of the brain, with increasing stress results in decreased alpha and increased beta activities [48,49]. For the present analysis, both bands are further decomposed into sub-bands: low-alpha (8–10 Hz), high-alpha (10–13 Hz), low-beta (13–18 Hz) and high-beta (18–30 Hz). Each sub-bands reflect different brain responses as shown in Table 1. Relative power for each frequency band is calculated using Equation (1) [50,51].
P o w e r r e l a t i v e = i n B a n d 2 i n T o t a l 2
Within the alpha band, alpha asymmetry (AAS) between the right and left frontal areas is a significant indicator for stress [53,54]. We calculated the AAS between the right (Fp2,F4,F8) and left (Fp1,F3,F7) frontal areas using Equation (2) [54]. Brouwer et al. highlighted the direction of AAS reflects approaches of stress coping [53]. The AAS ranges between [−1, 1], with negative values reflecting higher level of stress in the right frontal areas.
A A S = R i g h t L e f t R i g h t + L e f t

4. Results

A total of 21 participants were recruited among UTP students for this study. None of the participants had any experience of working on the oil rig platform. Table 3 presents the characteristics of participants recruited for this study.

4.1. User Experience Assessment

Figure 10 presents the percentage of participants based on the most relevant response they experienced when performing the VR modules. As reflected in the green pie slices, 74% gave positive responses on their VR experience with regards to the usability, engagement and sense of presence. The training modules were easy to understand, and motivated participants for future training, with one participant volunteered to perform the training again. The confined space inspection training in Module 2 was able to induce a sense of claustrophobic as participants felt that the training helped them to face their fear of confined space. However, there were participants that felt the VR environment had poor sense of immersion when performing the training. Participants that felt motion sickness during the VR training also informed researchers that they had no prior experience of using the VR headset.

4.2. Performance Assessment

In Module 3 and 4, the performance of participants improved throughout the five sessions. This is reflected by the increasing performance scores of rotating the pipeline valve and moving cabins using the crane in Figure 11. As more training were given to the participants, they were able to use the VR controller to rotate the valve with more ease, avoid collision with nearby obstacles and place the cabin correctly within the designated placement areas and collecting more coins.

4.3. Mental State Assessment

The mental states were assessed for Module 4 where the crane training must be completed within a duration of 5 min by collecting all coins and avoid any collision with nearby obstacles when moving and placing the cabins. These conditions were placed to induce stress in participants as the task must be completed correctly and efficiently. Relative power of the 30 s epochs in each segment is calculated to observe the changes of mental states in the three frequency bands.

4.3.1. Theta Activity

Changes in the theta activity are observed in Figure 12a, with decreased relative power of theta during the VR trials. During the second 2 min rest with eyes-open, the theta activity slightly increased but did not reach to the level of theta before VR trial 1. Once the VR trials were completed, the theta activity increased close to the relative power before VR headset is placed on participant. The discomfort and disorientation of VR headset has been negatively associated with relative power of theta [47].

4.3.2. Alpha Activity

With reference to Figure 12, alpha band had the highest relative power in the first and last segments where participant is not wearing the VR headset and in relaxed states. Similar to the theta activity, Figure 12b also shows decreased in alpha activity once the VR headset was worn, and it further decreased during the VR trials. However, unlike the theta activity, both alpha and the sub-bands had comparable relative powers during the eyes-open resting segments. Higher alpha activity indicates a calm and relaxing state, whilst the decrease in this frequency band reflects increased level of stress [48,55]. Both alpha sub-bands had similar changes throughout the six segments suggesting no differences in the level of alertness during relaxed states.

4.3.3. Beta Activity

In the beta frequency band, there is an increase of activity during the VR trials as shown in Figure 12c. This is also observed in the high-beta sub-band but not the low-beta sub-band, suggesting increasing mental concentrations but with high emotional intensity during the VR trials [52]. Further analysis of the high-beta sub-band is done using the EEGLAB to extract the topomap of the 32-channel brain activity during resting and VR trials, as shown in Figure 13. The topomap allows us to observe the brain areas that are affected within the high-beta frequency in response to changes in concentration under intense emotions. Increased brain activity is observed in the temporal and parietal brain areas during the VR trials, which is not present during the 2 min resting with eyes-open.

4.3.4. Alpha Asymmetry

The AAS gives an indication of the dominant frontal areas in response to stress, with positive value reflects high alpha activity in the right frontal area. In Figure 12d, AAS values are close to zero during resting for both eyes-open and eyes-close conditions showing similar alpha activity between the right and left frontal areas. However, AAS decreased further in VR trial 1 showing a dominance of left frontal areas, and deflected to positive value in VR trial 2. Direction of the AAS is associated with the strategies taken to cope with stress, with negative asymmetry reflect withdrawal strategies and positive asymmetry for approach strategies [53].

5. Discussion

Workplace stress is one of the causes of mental health problems among adults [2,56]. Self-reports of mental health and work-related stress are often under-reported due to concern of societal stigma and backlash of employment [57]. Aside from professional help, conventional approaches of managing mental health usually depend on personal resources or social supports. We propose the integration of VR and EEG system as a tool to manage mental health in stressful workplaces. The VR technology allows the monitoring of mental health of adults in virtual environment of their workplace. This study aims to evaluate the mental states when performing stressful tasks within a VR offshore environment.

5.1. Advantages and Drawbacks of VR

The developed VR modules were able to engage users to learn basic operations of offshore facilities. In addition to the positive feedbacks on usability, engagement and sense of presence of the VR modules, the learning effects is reflected by the improvement in the performance scores. The ease-of-use and learning motivation observed from the VR offshore training suggest that this technology is a good platform to train offshore workers on operations within the oil well environments. The scenarios can be programmed to mimic accidents or high-risk situations to train workers, without the need for a real-structure location.
However, these scenarios need to be more immersive and realistic enough to the real-environment. Higher level of immersion and sense of presence have been observed to elicit similar brain activity to the responses in real-life setting [58]. In a VR-based fire extinguishing training, the level of realism during the VR experience improve the interaction and sense of presence in users [38].
With the increase level of realism in a stressful VR environment, precaution must be taken to ensure extreme stress is not induced during the VR experience. Fadeev et al. reported that VR with extreme stress scenarios can elicit autonomic and EEG responses due to high emotional stress [21]. Saghafian et al. also suggested the need to measure realism with emotions within a stressful VR experience [38]. Motion sickness, disorientation and discomfort are also potential stressors when experiencing the VR [47,59]. Measuring these aspects of the VR using questionnaires has the limitation of inaccuracy as the user experiences are reported after the VR session [59]. Using the EEG signals, Liao et al. developed a prediction model to detect motion sickness in VR. The integration of a VR-EEG system that emulates workplace with complex or high risk conditions can ensure a safer VR environment with objective monitoring tools to measure both mental states and user experience.

5.2. Presence of Stress

Brain activity in different frequency bands evinces specific brain functionalities [60,61]. Changes in the mental states observed in this study corroborate the presence of stress due to physical discomfort and conditional tasks during the VR trials. Theta frequencies are known as slower brain rhythms and are prominent during unconscious states such as drowsiness [62] and relaxed [63]. Heo et al. observed a negative association between relative theta power in the left temporo-parietal area and discomfort felt during VR gaming [47]. In our study, the decrease level of relative theta power observed between the VR trials suggest disorientation and discomfort when wearing the VR headset [47]. Although these unfavourable spatial effects remained during the 2 min rest between the two VR trials, the increase in relative power of alpha in this segment suggest an increase of relaxed state.
Reduction in alpha activity is a strong indicator of stress as brain activity in this frequency band reflects relaxed state of consciousness [48,61]. Higher relative power in the alpha band compared to theta and beta bands during resting reflects its dominant role during relaxed consciousness [62], and has commonly been used as the neuromarker for stress response state [6,8,23,25]. Change in the alpha activity from high to low indicates increasing alertness and attention to external stimuli, as observed during the VR trials. The deflection of AAS from the zero-baseline also reflects the brain responses in coping with stress [53].
Beta frequency band reflects brain activities with faster oscillation, and has been associated with concentration [64], alertness [65], and attention [66,67,68]. Changes in the beta activity over the sensorimotor cortex has been associated with sensorimotor control and peripheral muscular activity [69]. In our study, the increase of beta activity during the VR trials is dominantly due to the increase of high-beta sub-band (18–30 Hz). This sub-band has been associated with higher level of concentration but with increasing emotional intensity such as anxiety [52] and agitation [68]. Errico et al. used the ratio of high-beta to low-beta sub-bands to calculate alertness index [22] in a study on emotion reactions within the VR. The alertness index correlated positively with distress, and negatively to emphatic interest and empathy. In this study, the findings of beta activity reflects the brain responses involving sensorimotor processes under high level of stress when performing the VR trials. The stress is induced through the VR crane training by placing the conditions of achieving perfect scores within a 5-min duration.

5.3. Learning under Stress

Improvement in the performance scores is a marker to show learning occurring throughout the VR sessions. This is also shown from the change of AAS from negative to positive between the two VR trials. According to Brouwer et al., negative AAS indicates withdrawal-coping strategies whilst positive AAS indicates problem-solving strategies when facing stress [53]. The positive AAS observed in VR trial 2 reflects the change in strategies to complete the task successfully. This change is also observed in the reduction of high-beta activity in the topomap for VR trial 2 near the temporal and parietal brain areas. The decrease in high-beta activity shows lesser anxiety felt by participant as compared to the first trial. Both of these brain areas correspond to sensory processing in the brain where parietal lobe is known for spatial and visual perception, and temporal lobe for sequencing and organization [60]. The shift of high-beta activity from left to right posterior area between the VR trial topomaps may reflect improved error detection and complex decision-making during the second VR trial [60].

6. Conclusions

In conclusion, this study shows that the changes in mental states due to discomfort, stress and concentration induced when performing VR tasks in an offshore environment can be monitored within the theta, alpha and high-beta frequency bands. The integration of VR and EEG to emulate workplace environments is a viable tool to monitor mental health. In settings where the workers are under high risk conditions or involved in complex routines, the use of EEG in identifying potential stressors can ensure the development of safer and effective VR environments. The mental states monitored when performing tasks in the virtual workplace can also be used to improve learning and performance through VR training modules. The proposed approach of integrating VR-based workplace environments with mental state monitoring can be implemented in various settings such as healthcare, office workplace and high-risk industries to train employees on their working skills.

Author Contributions

Conceptualization, S.S.A.A.; methodology, R.A.H., S.S. and N.N.A.; software, S.S. and N.N.A.; validation, S.S.A.A., and M.N.A.; formal analysis, R.A.H., S.S.A.A. and Y.H.; investigation, R.A.H., S.S. and N.N.A.; resources, R.A.H., S.S.A.A. and Y.H.; data curation, M.N.A.; writing—original draft preparation, R.A.H., S.S. and N.N.A.; writing—review and editing, R.A.H., S.S.A.A. and Y.H.; visualization, R.A.H., Y.H. and S.S.A.A.; supervision, S.S.A.A.; project administration, S.S.A.A.; funding acquisition, S.S.A.A. All authors have read and agreed to the published version of the manuscript.


This research was funded by Ministry of Higher Education Malaysia (MoHE) under Higher Institution Centre of Excellence (HICoE) Scheme awarded to Center for Intelligent Signal and Imaging Research (CISIR) grant 015MA0-050 and Yayasan Universiti Teknologi PETRONAS grant 015LC0-235.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of Sultan Idris Education University (2020-0121-01, approved on 30-September-2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data are not publicly available due to restrictions presented in informed consent. The data presented in this study are available on request from the corresponding author.


The authors would like to acknowledge and thank Dr Tang Tong Boon for his helpful advice and supervision on mental state analysis.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Ursin, H.; Eriksen, H.R. The cognitive activation theory of stress. Psychoneuroendocrinology 2004, 29, 567–592. [Google Scholar] [CrossRef]
  2. Kassim, M.S.A.; Ismail, A.; Ismail, R. A Review of Occupational Stress Prevalence and its Predictors among Selected Working Populations in Malaysia. Malays. J. Public Health Med. 2018, 18, 1–6. [Google Scholar]
  3. Fink, G. Stress: Concepts, Cognition, Emotion, and Behavior; Handbook of Stress Series; Academic Press: Salt Lake City, UT, USA, 2016; Volume 1. [Google Scholar]
  4. Varma, P.; Junge, M.; Meaklim, H.; Jackson, M.L. Younger people are more vulnerable to stress, anxiety and depression during COVID-19 pandemic: A global cross-sectional survey. Prog.-Neuro-Psychopharmacol. Biol. Psychiatry 2021, 109, 110236. [Google Scholar] [CrossRef]
  5. Sharma, N.; Gedeon, T. Objective measures, sensors and computational techniques for stress recognition and classification: A survey. Comput. Methods Programs Biomed. 2012, 108, 1287–1301. [Google Scholar] [CrossRef]
  6. Vanitha, V.; Krishnan, P. Real time stress detection system based on EEG signals. Biomed. Res. Comput. Life Sci. Smarter Technol. Adv. 2016, 2016, S271–S275. [Google Scholar]
  7. Marshall, A.C.; Cooper, N.R. The association between high levels of cumulative life stress and aberrant resting state EEG dynamics in old age. Biol. Psychol. 2017, 127, 64–73. [Google Scholar] [CrossRef]
  8. Naqvi, S.F.; Ali, S.S.A.; Yahya, N.; Yasin, M.A.; Hafeez, Y.; Subhani, A.R.; Adil, S.H.; Saggaf, U.M.A.; Moinuddin, M. Real-Time Stress Assessment Using Sliding Window Based Convolutional Neural Network. Sensors 2020, 20, 4400. [Google Scholar] [CrossRef]
  9. Giannopulu, I. Neuroscience, Robotics and Virtual Reality: Internalised vs Externalised Mind/Brain; Springer: Cham, Switzerland, 2018; Volume 1. [Google Scholar]
  10. Saposnik, G.; Cohen, L.G.; Mamdani, M.; Pooyania, S.; Ploughman, M.; Cheung, D.; Shaw, J.; Hall, J.; Nord, P.; Dukelow, S.; et al. Efficacy and safety of non-immersive virtual reality exercising in stroke rehabilitation (EVREST): A randomised, multicentre, single-blind, controlled trial. Lancet Neurol. 2016, 15, 1019–1027. [Google Scholar] [CrossRef][Green Version]
  11. Calabrò, R.S.; Naro, A.; Russo, M.; Leo, A.; Luca, R.D.; Balletta, T.; Buda, A.; Rosa, G.L.; Bramanti, A.; Bramanti, P. The role of virtual reality in improving motor performance as revealed by EEG: Randomized clinical trial. J. Neuroeng. Rehabil. 2017, 1–16. [Google Scholar] [CrossRef] [PubMed]
  12. Hafeez, Y.; Ali, S.S.A.; Mumtaz, W.; Moinuddin, M.; Adil, S.H.; Al-Saggaf, U.M.; Yasin, M.A.M.; Malik, A.S. Investigating Neurofeedback Protocols for Stress Mitigation: A Comparative Analysis of Different Stimulus Contents. IEEE Access 2019. [Google Scholar] [CrossRef]
  13. binti Sulaiman, S.; Ali, S.S.A.; Adil, S.H.; Ebrahim, M.; Raza, K. Virtual Reality Training and Skill Enhancement for Offshore Workers. In Proceedings of the 2020 International Conference on Computational Intelligence (ICCI), Bandar Seri Iskandar, Malaysia, 8–9 October 2020; pp. 287–292. [Google Scholar] [CrossRef]
  14. Holm, R.; Priglinger, M. Virtual Reality Training as a Method for Interactive and Experience-based Learning. In Proceedings of the SPE Intelligent Energy International Conference and Exhibition, Amsterdam, The Netherlands, 25–28 February 2008. [Google Scholar] [CrossRef]
  15. Grajewski, D.; Górski, F.; Hamrol, A.; Zawadzki, P. Immersive and Haptic Educational Simulations of Assembly Workplace Conditions. Procedia Comput. Sci. 2015, 75, 359–368. [Google Scholar] [CrossRef][Green Version]
  16. Khedher, A.B.; Jraidi, I.; Frasson, C. Tracking Students’ Mental Engagement Using EEG Signals during an Interaction with a Virtual Learning Environment. J. Intell. Learn. Syst. Appl. 2019, 11. [Google Scholar] [CrossRef][Green Version]
  17. Wan, B.; Wang, Q.; Su, K.; Dong, C.; Song, W.; Pang, M. Measuring the Impacts of Virtual Reality Games on Cognitive Ability Using EEG Signals and Game Performance Data. IEEE Access 2021, 9, 18326–18344. [Google Scholar] [CrossRef]
  18. Magosso, E.; De Crescenzio, F.; Ricci, G.; Piastra, S.; Ursino, M. EEG Alpha Power Is Modulated by Attentional Changes during Cognitive Tasks and Virtual Reality Immersion. Comput. Intell. Neurosci. 2019, 2019, 7051079. [Google Scholar] [CrossRef][Green Version]
  19. Dan, A.; Reiner, M. EEG-based cognitive load of processing events in 3D virtual worlds is lower than processing events in 2D displays. Int. J. Psychophysiol. 2017, 122, 75–84. [Google Scholar] [CrossRef]
  20. Tremmel, C.; Herff, C.; Sato, T.; Rechowicz, K.; Yamani, Y.; Krusienski, D.J. Estimating Cognitive Workload in an Interactive Virtual Reality Environment Using EEG. Front. Hum. Neurosci. 2019, 13, 401. [Google Scholar] [CrossRef]
  21. Fadeev, K.A.; Smirnov, A.S.; Zhigalova, O.P.; Bazhina, P.S.; Tumialis, A.V.; Golokhvast, K.S. Too Real to Be Virtual: Autonomic and EEG Responses to Extreme Stress Scenarios in Virtual Reality. Behav. Neurol. 2020, 2020, 5758038. [Google Scholar] [CrossRef]
  22. D’Errico, F.; Leone, G.; Schmid, M.; D’Anna, C. Prosocial Virtual Reality, Empathy, and EEG Measures: A Pilot Study Aimed at Monitoring Emotional Processes in Intergroup Helping Behaviors. Appl. Sci. 2020, 10, 1196. [Google Scholar] [CrossRef][Green Version]
  23. Kosunen, I.; Salminen, M.; Järvelä, S.; Ruonala, A.; Ravaja, N.; Jacucci, G. RelaWorld: Neuroadaptive and Immersive Virtual Reality Meditation System. In Proceedings of the 21st International Conference on Intelligent User Interfaces, Sonoma, CA, USA, 7–10 March 2016; ACM: New York, NY, USA, 2016; pp. 208–217. [Google Scholar] [CrossRef]
  24. Rose, T.; Nam, C.S.; Chen, K.B. Immersion of virtual reality for rehabilitation-Review. Appl. Ergon. 2018, 69, 153–161. [Google Scholar] [CrossRef] [PubMed]
  25. Amores, J.; Richer, R.; Zhao, N.; Maes, P.; Eskofier, B.M. Promoting relaxation using virtual reality, olfactory interfaces and wearable EEG. In Proceedings of the 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Las Vegas, NV, USA, 4–7 March 2018; pp. 98–101. [Google Scholar] [CrossRef][Green Version]
  26. Perhakaran, G.; Yusof, A.M.; Rusli, M.E.; Yusoff, M.Z.M.; Mahalil, I.; Zainuddin, A.R.R. A Study of Meditation Effectiveness for Virtual Reality Based Stress Therapy Using EEG Measurement and Questionnaire Approaches. In Innovation in Medicine and Healthcare 2015; Chen, Y.W., Torro, C., Tanaka, S., Howlett, R.J., Jain, L.C., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 365–373. [Google Scholar]
  27. Gu, G.; Frasson, C. Virtual Sophrologist: A Virtual Reality Neurofeedback Relaxation Training System. In Brain Function Assessment in Learning; Frasson, C., Kostopoulos, G., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 176–185. [Google Scholar]
  28. Prpa, M.; Cochrane, K.; Riecke, B.E. Hacking Alternatives in 21st Century: Designing a Bio-Responsive Virtual Environment for Stress Reduction. In Pervasive Computing Paradigms for Mental Health; Serino, S., Matic, A., Giakoumis, D., Lopez, G., Cipresso, P., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 34–39. [Google Scholar]
  29. Bastos, I.L.M.; Faria, M.G.d.A.; Koopmans, F.F.; Alves, L.V.V.; Mello, A.S.d.; David, H.M.S.L. Risks, injuries, and illnesses among professionals working on offshore platforms: An integrative review. Rev. Eletr. Enferm. 2020, 22. [Google Scholar] [CrossRef]
  30. Farmer, M. How the Covid-19 Coronavirus Is Affecting the Offshore Industry. 2020. Available online: (accessed on 22 March 2021).
  31. Wells, C.R.; Townsend, J.P.; Pandey, A.; Moghadas, S.M.; Krieger, G.; Singer, B.; McDonald, R.H.; Fitzpatrick, M.C.; Galvani, A.P. Optimal COVID-19 quarantine and testing strategies. Nat. Commun. 2021, 12, 356. [Google Scholar] [CrossRef]
  32. GlobalData. How Leading Oil and Gas Companies Are Adopting Virtual Reality. 2020. Available online: (accessed on 22 March 2021).
  33. GlobalData. Virtual Reality in Oil and Gas: Key Trends Revealed. 2020. Available online: (accessed on 22 March 2021).
  34. Perez, B.Z.; Marin, M.M.; Perez, E.I. Developing a Virtual Environment for Safety Training. In Proceedings of the Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007), Cuernavaca, Mexico, 25–28 September 2007; pp. 545–550. [Google Scholar] [CrossRef]
  35. George, A.K.; McLain, M.L.; Bijlani, K.; Jayakrishnan, R.; Bhavani, R.R. A Novel Approach for Training Crane Operators: Serious Game on Crane Simulator. In Proceedings of the 2016 IEEE Eighth International Conference on Technology for Education (T4E), Mumbai, India, 2–4 December 2016; pp. 116–119. [Google Scholar] [CrossRef]
  36. Dong, H.; Xu, G. An Expert System for Bridge Crane Training System Based on Virtual Reality. In Proceedings of the 2010 International Conference on Artificial Intelligence and Computational Intelligence, Sanya, China, 23–24 October 2010; Volume 3, pp. 30–33. [Google Scholar] [CrossRef]
  37. Koo, M.W.; Ha, S.; Cha, J.H.; Cho, D.Y. Fire Incident Training for Offshore Worker Using Virtual Reality. In Proceedings of the International Ocean and Polar Engineering Conference, San Francisco, CA, USA, 25–30 June 2017; pp. 886–890. [Google Scholar]
  38. Saghafian, M.; Laumann, K.; Akhtar, R.S.; Skogstad, M.R. The Evaluation of Virtual Reality Fire Extinguisher Training. Front. Psychol. 2020, 11, 3137. [Google Scholar] [CrossRef]
  39. Duncan, R. The Potential Contribution of Augmented and Virtual Reality to the Oil and Gas Industry. Int. J. Manag. Appl. Res. 2015, 2, 112–120. [Google Scholar] [CrossRef][Green Version]
  40. Xu, X.; Li, Z.; Wang, L.; Yao, S.; Fang, X. Interactive Visual Reality of the Offshore Hoisting Operation and Numerical Modeling. Int. J. Pattern Recognit. Artif. Intell. 2018, 32, 1855012. [Google Scholar] [CrossRef]
  41. Patle, D.S.; Manca, D.; Nazir, S.; Sharma, S. Operator training simulators in virtual reality environment for process operators: A review. Virtual Real. 2019, 23, 293–311. [Google Scholar] [CrossRef]
  42. Pusey, M.; Wong, K.W.; Rappa, N.A. Resilience interventions using interactive technology: A scoping review. Interact. Learn. Environ. 2020, 1–16. [Google Scholar] [CrossRef]
  43. Montana, J.I.; Matamala-Gomez, M.; Maisto, M.; Mavrodiev, P.A.; Cavalera, C.M.; Diana, B.; Mantovani, F.; Realdon, O. The Benefits of emotion Regulation Interventions in Virtual Reality for the Improvement of Wellbeing in Adults and Older Adults: A Systematic Review. J. Clin. Med. 2020, 9, 500. [Google Scholar] [CrossRef][Green Version]
  44. Bruggeman, K.J. Creating Biofeedback-Based Virtual Reality Applications to Enhance Coherence of Mindfulness Practice. Ph.D. Thesis, The Ohio State University, Columbus, OH, USA, 2019. [Google Scholar]
  45. Winkler, I.; Haufe, S.; Tangermann, M. Automatic Classification of Artifactual ICA-Components for Artifact Removal in EEG Signals. Behav. Brain Funct. 2011, 7, 30. [Google Scholar] [CrossRef][Green Version]
  46. Delorme, A.; Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 2004, 134, 9–21. [Google Scholar] [CrossRef][Green Version]
  47. Heo, J.; Yoon, G. EEG Studies on Physical Discomforts Induced by Virtual Reality Gaming. J. Electr. Eng. Technol. 2020, 15, 1323–1329. [Google Scholar] [CrossRef]
  48. Hoffmann, E. Brain Training against Stress; Technical report; Mental Fitness & Forskningscenter: Copenhagen, Denmark, 2005. [Google Scholar]
  49. Hafeez, Y.; Ali, S.S.A.; Malik, A.S. Neurofeedback training content for treatment of stress. In Proceedings of the 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES), Kuala Lumpur, Malaysia, 4–8 December 2016; pp. 133–137. [Google Scholar] [CrossRef]
  50. Hillard, B. Analysis of Eeg Rhythms Using Custom-made Matlab Applicationfor Processing of Data Collected during Neurofeedback Training in Adhd Subjects. Master’s Thesis, University of Louisville, Louisville, KY, USA, 2011. [Google Scholar]
  51. Wang, Y.; Sokhadze, E.M.; El-Baz, A.S.; Li, X.; Sears, L.; Casanova, M.F.; Tasman, A. Relative Power of Specific EEG Bands and Their Ratios during Neurofeedback Training in Children with Autism Spectrum Disorder. Front. Hum. Neurosci. 2016, 9, 723. [Google Scholar] [CrossRef] [PubMed]
  52. Tarrant, J.; Viczko, J.; Cope, H. Virtual Reality for Anxiety Reduction Demonstrated by Quantitative EEG: A Pilot Study. Front. Psychol. 2018, 9, 1280. [Google Scholar] [CrossRef][Green Version]
  53. Brouwer, A.M.; Neerincx, M.A.; Kallen, V.; van der Leer, L.; ten Brinke, M. EEG Alpha Asymmetry, Heart Rate Variability and Cortisol in Response to Virtual Reality Induced Stress. J. Cybertherapy Rehabil. 2011, 4, 27–40. [Google Scholar]
  54. Al-shargie, F.; Tang, T.B.; Badruddin, N.; Kiguchi, M. Towards multilevel mental stress assessment using SVM with ECOC: An EEG approach. Med Biol. Eng. Comput. 2017, 56, 125–136. [Google Scholar] [CrossRef] [PubMed]
  55. Lehrer, P.M.; Woolfolk, R.L.; Sime, W.E. (Eds.) Principles and Practice of Stress Management; The Guilford Press: New York, NY, USA, 2007. [Google Scholar]
  56. Thye, T.S.L.L. Work Stress and Mental Health. Available online: (accessed on 22 March 2021).
  57. Kwon, K.A.; Jeon, S.; Jeon, L.; Castle, S. The role of teachers’ depressive symptoms in classroom quality and child developmental outcomes in Early Head Start programs. Learn. Individ. Differ. 2019, 74, 101748. [Google Scholar] [CrossRef]
  58. Penn, R.A.; Hout, M.C. Making Reality Virtual: How VR “Tricks” Your Brain. Front. Young Minds 2018, 6. [Google Scholar] [CrossRef][Green Version]
  59. Liao, C.Y.; Tai, S.K.; Chen, R.C.; Hendry, H. Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device. IEEE Access 2020, 8, 126784–126796. [Google Scholar] [CrossRef]
  60. Jatoi, M.A.; Kamel, N. Brain Source Localization Using EEG Signal Analysis; CRC Press: Boca Raton, FL, USA, 2018. [Google Scholar]
  61. Kamel, N.; Malik, A.S. (Eds.) EEG/ERP Analysis; CRC Press: Boca Raton, FL, USA, 2015. [Google Scholar]
  62. Louis, E.K.S.; Frey, L.C. (Eds.) Electroencephalography (EEG): An Introductory Text and Atlas of Normal and Abnormal Findings in Adults, Children, and Infants; American Epilepsy Society: Chicago, IL, USA, 2016. [Google Scholar]
  63. Mari-Acevedo, J.; Yelvington, K.; Tatum, W.O.; Levin, K.H.; Chauvel, P. Chapter 9—Normal EEG variants. In Handbook of Clinical Neurology; Elsevier: Amsterdam, The Netherlands, 2019; Volume 160, pp. 143–160. [Google Scholar]
  64. Kovacevic, N.; Ritter, P.; Tays, W.; Moreno, S.; McIntosh, A.R. ‘My Virtual Dream’: Collective Neurofeedback in an Immersive Art Environment. PLoS ONE 2015, 10, 1–18. [Google Scholar] [CrossRef]
  65. Kamiński, J.; Brzezicka, A.; Gola, M.; Wróbel, A. Beta band oscillations engagement in human alertness process. Int. J. Psychophysiol. 2012, 85, 125–128. [Google Scholar] [CrossRef]
  66. Vernon, D.; Egner, T.; Cooper, N.; Compton, T.; Neilands, C.; Sheri, A.; Gruzelier, J. The effect of training distinct neurofeedback protocols on aspects of cognitive performance. Int. J. Psychophysiol. 2003, 47, 75–85. [Google Scholar] [CrossRef]
  67. Kiiski, H.; Bennett, M.; Rueda-Delgado, L.M.; Farina, F.R.; Knight, R.; Boyle, R.; Roddy, D.; Grogan, K.; Bramham, J.; Kelly, C.; et al. EEG spectral power, but not theta/beta ratio, is a neuromarker for adult ADHD. Eur. J. Neurosci. 2019, 51, 2095–2109. [Google Scholar] [CrossRef] [PubMed]
  68. Hafeez, Y.; Ali, S.S.A.; Faraz, S.; Moinuddin, M.; Adil, S.H. Effect of Neurofeedback 2D and 3D Stimulus Content On Stress Mitigation. In Proceedings of the 2019 IEEE Student Conference on Research and Development (SCOReD), Bandar Seri Iskandar, Malaysia, 15–17 October 2019; pp. 289–293. [Google Scholar] [CrossRef]
  69. Vukelić, M.; Gharabaghi, A. Oscillatory entrainment of the motor cortical network during motor imagery is modulated by the feedback modality. NeuroImage 2015, 111, 1–11. [Google Scholar] [CrossRef] [PubMed]
Figure 1. (a) Startup interface of the VR-training module and (b) the initial scene of arriving at offshore platform.
Figure 1. (a) Startup interface of the VR-training module and (b) the initial scene of arriving at offshore platform.
Sensors 21 04885 g001
Figure 2. Module 1 Oil sampling.
Figure 2. Module 1 Oil sampling.
Sensors 21 04885 g002
Figure 3. Module 2 Confined space inspection: (a) Oil tank with ladder. (b) View when standing on top of tank. (c) Dark surrounding when inspecting inside of tank.
Figure 3. Module 2 Confined space inspection: (a) Oil tank with ladder. (b) View when standing on top of tank. (c) Dark surrounding when inspecting inside of tank.
Sensors 21 04885 g003
Figure 4. Module 3 Pipeline Valve Rotation: (a) Smoke leakage from pipeline. (b) Rotating valve using controller.
Figure 4. Module 3 Pipeline Valve Rotation: (a) Smoke leakage from pipeline. (b) Rotating valve using controller.
Sensors 21 04885 g004
Figure 5. Module 4 Crane training: (a) Transporting into crane driving cab. (b) Adjusting the lifting hook. (c) Lifting cabin whilst avoiding obstacles. (d) Example of accident during cabin lifting.
Figure 5. Module 4 Crane training: (a) Transporting into crane driving cab. (b) Adjusting the lifting hook. (c) Lifting cabin whilst avoiding obstacles. (d) Example of accident during cabin lifting.
Sensors 21 04885 g005
Figure 6. Flow of experiment.
Figure 6. Flow of experiment.
Sensors 21 04885 g006
Figure 7. Experimental setup with MITSAR-EEG and HTC Vive headset and controllers.
Figure 7. Experimental setup with MITSAR-EEG and HTC Vive headset and controllers.
Sensors 21 04885 g007
Figure 8. Electrode channels for the MITSAR-EEG-202-31. Circled electrode channels are used for EEG analysis.
Figure 8. Electrode channels for the MITSAR-EEG-202-31. Circled electrode channels are used for EEG analysis.
Sensors 21 04885 g008
Figure 9. Six segments of the whole EEG data recording.
Figure 9. Six segments of the whole EEG data recording.
Sensors 21 04885 g009
Figure 10. User experience during Module 1 and 2.
Figure 10. User experience during Module 1 and 2.
Sensors 21 04885 g010
Figure 11. Percentage scores achieved by participants during training in Module 3 and 4 for five sessions.
Figure 11. Percentage scores achieved by participants during training in Module 3 and 4 for five sessions.
Sensors 21 04885 g011
Figure 12. Changes of (a) theta, (b) alpha and (c) beta relative powers, and (d) alpha asymmetry throughout the six segments.
Figure 12. Changes of (a) theta, (b) alpha and (c) beta relative powers, and (d) alpha asymmetry throughout the six segments.
Sensors 21 04885 g012
Figure 13. Topomap in high-beta sub-bands during (a) eyes-open resting (b) VR-trial 1, (c) eyes-open resting, and (d) VR trial 2. Increase in high-beta activity observed in the posterior areas during VR trials compared to eyes-open resting.
Figure 13. Topomap in high-beta sub-bands during (a) eyes-open resting (b) VR-trial 1, (c) eyes-open resting, and (d) VR trial 2. Increase in high-beta activity observed in the posterior areas during VR trials compared to eyes-open resting.
Sensors 21 04885 g013
Table 1. How do you feel after experiencing the training modules using the VR?
Table 1. How do you feel after experiencing the training modules using the VR?
User Experience Response
UsabilityThe training is easy to understand
EngagingExcited to try another virtual training
PresenceIt helps me to overcome my claustrophobia
DisorientationI got motion sickness from the VR headset
ImmersiveThe experience is not immersive enough
Table 2. EEG brain activity with respective stress responses.
Table 2. EEG brain activity with respective stress responses.
EEG WaveBrain AreaElectrodeStress ResponseSub-Band
Theta [47]Temporo-parietalTP7,TP8Discomfort, disorientation
Alpha [52]FrontalFp1,Fp2,F3,Relaxed and calmLow
F4,F7,F8Relaxed and alertHigh
Beta [52]ParietalP3,P4ConcentrationLow
Concentration and anxietyHigh
Table 3. Characteristics of participants.
Table 3. Characteristics of participants.
CharacteristicN = 21
Age27 ± 5.90
Gender9 males, 12 females
Tertiary Education FieldEngineering
Normal or corrected-to-normal visionYes
Neurological and physiological complicationNo
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Abu Hasan, R.; Sulaiman, S.; Ashykin, N.N.; Abdullah, M.N.; Hafeez, Y.; Ali, S.S.A. Workplace Mental State Monitoring during VR-Based Training for Offshore Environment. Sensors 2021, 21, 4885.

AMA Style

Abu Hasan R, Sulaiman S, Ashykin NN, Abdullah MN, Hafeez Y, Ali SSA. Workplace Mental State Monitoring during VR-Based Training for Offshore Environment. Sensors. 2021; 21(14):4885.

Chicago/Turabian Style

Abu Hasan, Rumaisa, Shahida Sulaiman, Nur Nabila Ashykin, Mohd Nasir Abdullah, Yasir Hafeez, and Syed Saad Azhar Ali. 2021. "Workplace Mental State Monitoring during VR-Based Training for Offshore Environment" Sensors 21, no. 14: 4885.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop