Figure - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
The use of serious game tools in training of medical professions is steadily growing. However, there is a lack of reliable performance assessment methods to evaluate learner’s outcome. The aim of this study is to determine whether functional near infrared spectroscopy (fNIRS) can be used as an additional tool for assessing the learning outcome of v...
Context in source publication
Citations
... Neuroergonomic approaches provide perspectives for enhancing the efficacy and configuration of interrelations between humans and systems focusing on everyday life situations [96][97][98]. Investigations into assessing cognitive workload serve as a driving force behind research on neuroergonomic professional training in critical settings [27], such as surgeons operating in a VR environment (see [99,100] for review), pilots, and air traffic controllers [39,101]. Similarly, we applied the neuroergonomics method to provide insights into the relationship between brain function and behavioral outcomes. ...
Spatial cognition plays a crucial role in academic achievement, particularly in science, technology, engineering, and mathematics (STEM) domains. Immersive virtual environments (VRs) have the growing potential to reduce cognitive load and improve spatial reasoning. However, traditional methods struggle to assess the mental effort required for visuospatial processes due to the difficulty in verbalizing actions and other limitations in self-reported evaluations. In this neuroergonomics study, we aimed to capture the neural activity associated with cognitive workload during visuospatial tasks and evaluate the impact of the visualization medium on visuospatial task performance. We utilized functional near-infrared spectroscopy (fNIRS) wearable neuroimaging to assess cognitive effort during spatial-reasoning-based problem-solving and compared a VR, a computer screen, and a physical real-world task presentation. Our results reveal a higher neural efficiency in the prefrontal cortex (PFC) during 3D geometry puzzles in VR settings compared to the settings in the physical world and on the computer screen. VR appears to reduce the visuospatial task load by facilitating spatial visualization and providing visual cues. This makes it a valuable tool for spatial cognition training, especially for beginners. Additionally, our multimodal approach allows for progressively increasing task complexity, maintaining a challenge throughout training. This study underscores the potential of VR in developing spatial skills and highlights the value of comparing brain data and human interaction across different training settings.
... Studies using fNIRS reported increased PFC activation with heightened WM task difficulty (17,18). Aksoy and colleagues evaluated PFC activation during a healthcare training course with VR, finding decreased fNIRS signals with increased familiarity and practice, potentially indicating decreased CL (19). However, no studies comparing CL across different VR devices using fNIRS have been reported. ...
... 1) and burping (HMD: 1; laptop: 1) that were equally felt in both conditions. With the HMD condition, more than half of the participants felt general discomfort (13), fatigue (12), eyestrain (19), difficulty focusing (11), difficulty concentrating (11), fullness of head (15) and blurred vision (17). With the HMD condition, all participants felt at least 2 symptoms. ...
Background: Virtual Reality (VR) is utilized in health simulations as a method for presenting clinical cases. VR experiences offer numerous advantages such as interactivity and a high level of immersion, which enhance performance compared to conventional teaching methods. The extent of immersive VR's impact on cognitive load remains insufficiently investigated. This experimental cross-over study aimed (a) to assess students' cognitive load, (b) to evaluate the usability, intrinsic motivation, and cybersickness of the system, and (c) to compare students' performance in resolving two clinical cases between immersive VR and non-immersive conditions.Method: Twenty students were included in this study. We developed two physiotherapy clinical cases (musculoskeletal and respiratory) as 360° videos. The clinical cases were randomized between exposure conditions: immersive VR using a head-mounted display (HMD) and non-immersive VR using a laptop. Performance was evaluated through multiple-choice questions, cognitive load was measured using functional near-infrared spectroscopy, and usability, intrinsic motivation, and cybersickness were assessed using the System Usability Scale, Intrinsic Motivation Inventory, and Simulator Sickness Questionnaire, respectively.Results: There was no significant difference between the scores obtained with the HMD and the laptop (p = 0.245). Results indicated a higher activation of the prefrontal cortex with the laptop condition (p = 0.007). Usability was significantly better (p = 0.005), and the number of reported side effects was lower for the computerized condition, whereas intrinsic motivation was similar.Conclusion: Immersive VR led to a lower cognitive load compared to non-immersive VR. Despite similar performance between the two exposure conditions, usability was superior, and side effects were fewer for the computerized condition.
... Briefly, fNIRS techniques can be used to reveal different spatiotemporal features of cortical brain activations between surgeons of different levels of proficiency [7][8][9][10] . In turn, these distinct cortical signatures can be leveraged to perform skill-level classification, and FLS score prediction. ...
Functional near-infrared spectroscopy (fNIRS) is a neuroimaging tool for studying brain activity in mobile subjects. Open-access fNIRS datasets are limited to simple and/or motion-restricted tasks. Here, we report a fNIRS dataset acquired on mobile subjects performing Fundamentals of Laparoscopic Surgery (FLS) tasks in a laboratory environment. Demonstrating competency in the FLS tasks is a prerequisite for board certification in general surgery in the United States. The ASTaUND data set was acquired over four different studies. We provide the relevant information about the hardware, FLS task execution protocols, and subject demographics to facilitate the use of this open-access data set. We also provide the concurrent FLS scores, a quantitative metric for surgical skill assessment developed by the FLS committee. This data set is expected to support the growing field of assessing surgical skills via neuroimaging data and provide an example of data processing pipeline for use in realistic, non-restrictive environments.
... References N Mental workload Barre et al., 2019;Bektaş et al., 2021;Guna et al., 2019;Iqbal et al., 2021;Kalantari et al., 2021;Kia et al., 2021;Lacoche et al., 2022;Lim et al., 2022;Madathil & Greenstein, 2017;Makransky, Terkildsen, et al., 2019;Peterson et al., 2018;Tian et al., 2021;Tremmel et al., 2019;Wagner Filho et al., 2019;Wu et al., 2021 15 Physical workload and muscle fatigue Barre et al., 2019;Chihara & Seo, 2018;Iqbal et al., 2021;Lee & Han, 2018;Lim et al., 2022;Penumudi et al., 2020;Recenti et al., 2021 7 Physiological responses (stress) Aksoy et al., 2019;Barreda-Angeles et al., 2020;Bzd u skov a et al., 2021;Guna et al., 2019;Higuera-Trujillo et al., 2017;Kafri et al., 2021;Khenak et al., 2020;Kia et al., 2021;Lemmens et al., 2022;Peterson et al., 2018 10 Visual fatigue Alhassan et al., 2021;Elias et al., 2019;Hirota et al., 2019;Jacobs et al., 2019;Munsamy et al., 2020;Souchet, Philippe, et al., 2022;Szpak et al., 2019;Turnbull & Phillips, 2017;Yoon et al., 2020 9 Posture stability Bzd u skov a et al., 2021; Dennison & D'Zmura, 2018;Munafo et al., 2017;Sinitski et al., 2018;Widdowson et al., 2021 5 Usability Bektaş et al., 2021;Bracq et al., 2019;Kaminska et al., 2022;Kourtesis et al., 2019;Lacoche et al., 2022;Madathil & Greenstein, 2017;Pallavicini et al., 2019;Penumudi et al., 2020;Sun et al., 2015;Wagner Filho et al., 2019, 2020Yu et al., 2019 11 Cybersickness Barreda-Angeles et al., 2020;Bektaş et al., 2021;da Silva Marinho et al., 2022;Guna et al., 2019;Hussain et al., 2021;Lacoche et al., 2022;Lee et al., 2021;Lin et al., 2015Lin et al., , 2019Liu et al., 2020;Munafo et al., 2017;Szpak et al., 2019;Wibirama et al., 2020;Widdowson et al., 2021;Widyanti & Hafizhah, 2022;Yildirim, 2020 15 Presence Khenak et al., 2020;Lemmens et al., 2022;Magalhães et al., 2021;Maneuvrier et al., 2020;Pallavicini et al., 2019;Sun et al., 2015;Wagner Filho et al., 2020 7 Figure 4. Percentage of studies based on type of technique. ...
... The results showed that automated usability assessment of VR applications might be feasible and extremely encouraging (Kaminska et al., 2022). fNIRS was used to assess the learning outcomes of VR-based learning modules and MWL (Aksoy et al., 2019;Kia et al., 2021). EEG is a neuroimaging method that records the electrical activity of neurons that oscillate simultaneously during cognitive functions. ...
A variety of human factors/ergonomics (HFE) problems have been studied by researchers and developers in VR environments. This systematic review aimed to summarize important HFE issues and classify the validated instruments used to quantify them in virtual reality environments. The most representative electronic databases for this review (2013-2022) were searched for original articles. The results showed that aspects, such as cybersickness, visual fatigue, mental workload, performance, spatial presence, and usability were the most relevant HFE issues assessed, whereas some aspects, such as physical workload, posture, stress, and discomfort, were consider less often. Previous studies have neglected some human factors and ergonomic issues, such as physical ergo-nomics, stress, and aftereffects, such as fatigue and human error. In virtual environments, presence was an emerging human factor compared to real environments. Most techniques were unidimen-sional and subjective. Future studies should focus on more factors and risks associated with HFE by emphasizing objective techniques and multidimensional subjective methods.
... Functional near infrared spectroscopy (fNIRS) is a non-invasive portable brain imaging technology that measures hemodynamic changes within the cerebral cortex in response to sensory, motor, and cognitive stimuli [37][38][39]. Previous studies have associated fNIRS measures with varying workload levels during cognitive tasks [39][40][41][42][43][44]. Furthermore, studies with fNIRS have shown a significant reduction in the activation within the PFC with increasing familiarity with the task at hand and improved performance, thus providing a quantitative measurement of trainees' expertise development [45]. ...
... Furthermore, studies with fNIRS have shown a significant reduction in the activation within the PFC with increasing familiarity with the task at hand and improved performance, thus providing a quantitative measurement of trainees' expertise development [45]. Research has shown that mental workload is sensitive to task-related features such as difficulty level, the order of the tasks (e.g., blocked vs. random practice), and the type of task (e.g., learning vs. transfer task) [39,43,44]. ...
Spatial visualization ability (SVA) has been identified as a potential key factor for academic achievement and student retention in Science, Technology, Engineering, and Mathematics (STEM) in higher education, especially for engineering and related disciplines. Prior studies have shown that training using virtual reality (VR) has the potential to enhance learning through the use of more realistic and/or immersive experiences. The aim of this study was to investigate the effect of VR-based training using spatial visualization tasks on participant performance and mental workload using behavioral (i.e., time spent) and functional near infrared spectroscopy (fNIRS) brain-imaging-technology-derived measures. Data were collected from 10 first-year biomedical engineering students, who engaged with a custom-designed spatial visualization gaming application over a six-week training protocol consisting of tasks and procedures that varied in task load and spatial characteristics. Findings revealed significant small (Cohen’s d: 0.10) to large (Cohen’s d: 2.40) effects of task load and changes in the spatial characteristics of the task, such as orientation or position changes, on time spent and oxygenated hemoglobin (HbO) measures from all the prefrontal cortex (PFC) areas. Transfer had a large (d = 1.37) significant effect on time spent and HbO measures from right anterior medial PFC (AMPFC); while training had a moderate (d = 0.48) significant effect on time spent and HbR measures from left AMPFC. The findings from this study have important implications for VR training, research, and instructional design focusing on enhancing the learning, retention, and transfer of spatial skills within and across various VR-based training scenarios.
... Monitoring cerebral blood oxygenation during training allowed researchers to assess the effectiveness of emergency-situation training. 60 Efficacy can be measured not only for therapy and training but also for prescription medication for cognitive disorders. Simultaneous VR and fNIRS monitoring was used to monitor brain oxygenation changes in students with attention deficit hyperactivity disorder (ADHD) in a classroom setting. ...
The sustained growth of digital healthcare in the field of neurology relies on portable and cost-effective brain monitoring tools that can accurately monitor brain function in real time. Functional near-infrared spectroscopy (fNIRS) is one such tool that has become popular among researchers and clinicians as a practical alternative to functional magnetic resonance imaging , and as a complementary tool to modalities such as electroencephalography. This review covers the contribution of fNIRS to the personalized goals of digital healthcare in neurology by identifying two major trends that drive current fNIRS research. The first major trend is multimodal monitoring using fNIRS, which allows clinicians to access more data that will help them to understand the interconnection between the cerebral hemodynamics and other physiological phenomena in patients. This allows clinicians to make an overall assessment of physical health to obtain a more-detailed and individualized diagnosis. The second major trend is that fNIRS research is being conducted with naturalistic experimental paradigms that involve multisensory stimulation in familiar settings. Cerebral monitoring of multisensory stimulation during dynamic activities or within virtual reality helps to understand the complex brain activities that occur in everyday life. Finally, the scope of future fNIRS studies is discussed to facilitate more-accurate assessments of brain activation and the wider clinical acceptance of fNIRS as a medical device for digital healthcare.
... Gaze-based interaction shows a lower mental workload than bimanualGupta et al. (2020) N back tasks -2 levels of difficulty, with or without the help of a virtual agentMental workload is higher when tasks indications by the agent or interactions when those interactions are too far from what users are accustomed to as well(Wismer et al. 2018;Bernard et al. 2019;Baceviciute et al. 2021). Those results seem moderated by expertise within VR and the task demands(Aksoy et al. 2019;Luong et al. 2019;Armougum et al. 2019). For instance, outside of VR, when time and load on the resources are high, humans hit the maximum resource allocation capacity(McGregor et al. 2021). ...
This narrative review synthesizes and introduces 386 previous works about virtual reality-induced symptoms and effects by focusing on cybersickness, visual fatigue, muscle fatigue, acute stress, and mental overload. Usually, these VRISE are treated independently in the literature, although virtual reality is increasingly considered an option to replace PCs at the workplace, which encourages us to consider them all at once. We emphasize the context of office-like tasks in VR, gathering 57 articles meeting our inclusion/exclusion criteria. Cybersickness symptoms, influenced by fifty factors, could prevent workers from using VR. It is studied but requires more research to reach a theoretical consensus. VR can lead to more visual fatigue than other screen uses, influenced by fifteen factors, mainly due to vergence-accommodation conflicts. This side effect requires more testing and clarification on how it differs from cybersickness. VR can provoke muscle fatigue and musculoskeletal discomfort, influenced by fifteen factors, depending on tasks and interactions. VR could lead to acute stress due to technostress, task difficulty, time pressure, and public speaking. VR also potentially leads to mental overload, mainly due to task load, time pressure, and intrinsically due interaction and interface of the virtual environment. We propose a research agenda to tackle VR ergonomics and risks issues at the workplace.
... On the other hand, functional nearinfrared spectroscopy (fNIRS) offers a recording of cortical activity in a natural mobility setting with higher spatial resolution than EEG, less susceptibility to motion artifacts and electrical noises, portability, and lightweight characteristic. These advantages substantiate the great potential for the combination of VR-HMD and fNIRS, which has been recently demonstrated in a bisection task (Seraglia et al., 2011), the assessment of prospective memory (Dong et al., 2017;Dong et al., 2018), the processing of racial stereotypes , performance monitoring during training (Hudak et al., 2017), and a neurofeedback system to support attention (Aksoy et al., 2019). However, the feasibility and ecological validity of using fNIRS to measure fear response during virtual fear exposure is still unexplored. ...
This Research Topic is composed of 11 accepted papers: seven dedicated to original research, a perspective, a mini review and two opinion pieces, and are dedicated to various themes and perspectives. These contributions address the multi-faceted nature of non-clinical BCIs, ranging from ethical ramifications of these neurotechnologies, applications to the arts, education, communication, wellbeing, and sports to the readiness of BCI deployment for gaming.
... Biosensors measuring stress (such as GSR/EDA), cognitive load (theta/alpha ratio of EEG), arousal (pupil tracker), or facial expression could help identify the reactions of the users to better quantify the different effects the AR technology has which in turn will better serve its educational promises. Lastly, the use of functional near-infrared spectroscopy to monitor the activation of different brain areas during cognitive tasks using pairs of light sources and detectors placed on the scalp has been used for the evaluation of cognitive tasks using VR (Aksoy et al., 2019) and ...
The purpose of this review was to identify the different augmented reality (AR) modalities used to teach anatomy to students, health professional trainees, and surgeons, and to examine the assessment tools used to evaluate the performance of various AR modalities. A scoping review of four databases was performed using variations of: (1) AR, (2) medical or anatomical teaching/education/training, and (3) anatomy or radiology or cadaver. Scientific articles were identified and screened for the inclusion and exclusion criteria as per Preferred Reporting Items for Systematic Reviews and Meta‐Analyses with extension for scoping reviews guidelines. Virtual reality was an exclusion criterion. From this scoping review, data were extracted from a total of 54 articles and the following four AR modalities were identified: head‐mounted display, projection, instrument and screen, and mobile device. The usability, feasibility, and acceptability of these AR modalities were evaluated using a variety of quantitative and qualitative assessment tools. Within more recent years of AR integration into anatomy education, the assessment of visuospatial ability, cognitive load, time on task, and increasing academic achievement outcomes are variables of interest, which continue to warrant more exploration. Sufficiently powered studies using validated assessment tools must be conducted to better understand the role of AR in anatomical education.
... On the other hand, functional nearinfrared spectroscopy (fNIRS) offers a recording of cortical activity in a natural mobility setting with higher spatial resolution than EEG, less susceptibility to motion artifacts and electrical noises, portability, and lightweight characteristic. These advantages substantiate the great potential for the combination of VR-HMD and fNIRS, which has been recently demonstrated in a bisection task (Seraglia et al., 2011), the assessment of prospective memory (Dong et al., 2017;Dong et al., 2018), the processing of racial stereotypes (Kim et al., 2019), performance monitoring during training (Hudak et al., 2017), and a neurofeedback system to support attention (Aksoy et al., 2019). However, the feasibility and ecological validity of using fNIRS to measure fear response during virtual fear exposure is still unexplored. ...
To enable virtual reality exposure therapy (VRET) that treats anxiety disorders by gradually exposing the patient to fear using virtual reality (VR), it is important to monitor the patient's fear levels during the exposure. Despite the evidence of a fear circuit in the brain as reflected by functional near-infrared spectroscopy (fNIRS), the measurement of fear response in highly immersive VR using fNIRS is limited, especially in combination with a head-mounted display (HMD). In particular, it is unclear to what extent fNIRS can differentiate users with and without anxiety disorders and detect fear response in a highly ecological setting using an HMD. In this study, we investigated fNIRS signals captured from participants with and without a fear of height response. To examine the extent to which fNIRS signals of both groups differ, we conducted an experiment during which participants with moderate fear of heights and participants without it were exposed to VR scenarios involving heights and no heights. The between-group statistical analysis shows that the fNIRS data of the control group and the experimental group are significantly different only in the channel located close to right frontotemporal lobe, where the grand average oxygenated hemoglobin Δ[HbO] contrast signal of the experimental group exceeds that of the control group. The within-group statistical analysis shows significant differences between the grand average Δ[HbO] contrast values during fear responses and those during no-fear responses, where the Δ[HbO] contrast values of the fear responses were significantly higher than those of the no-fear responses in the channels located towards the frontal part of the prefrontal cortex. Also, the channel located close to frontocentral lobe was found to show significant difference for the grand average deoxygenated hemoglobin contrast signals. Support vector machine-based classifier could detect fear responses at an accuracy up to 70% and 74% in subject-dependent and subject-independent classifications, respectively. The results demonstrate that cortical hemodynamic responses of a control group and an experimental group are different to a considerable extent, exhibiting the feasibility and ecological validity of the combination of VR-HMD and fNIRS to elicit and detect fear responses. This research thus paves a way toward the a brain-computer interface to effectively manipulate and control VRET.
