No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Psychophysiological responses to changes in workload during simulated air traffic control
Abstract
In this investigation, eight Air Force air traffic controllers (ATCs) performed three scenarios on TRACON (Terminal Radar Approach Control), a computer-based air traffic control (ATC) simulation. Two scenarios were used each with three levels of difficulty. One scenario varied traffic volume by manipulating the number of aircraft to be handled and the second scenario varied traffic complexity by manipulating arriving to departing flight ratios, pilot skill and mixture of aircraft types. A third scenario, overload, required subjects to handle a larger number of aircraft in a limited amount of time. The effects of the manipulations on controller workload were assessed using performance, subjective (TLX), and physiological (EEG, eye blink, heart rate, respiration, saccade) measures. Significant main effects of difficulty level were found for TRACON performance, TLX, eye blink, respiration and EEG measures. Only the EEG was associated with main effects for the type of traffic. The results provide support for the differential sensitivity of a variety of workload measures in complex tasks, underscore the importance of traffic complexity in ATC workload, and support the utility of TRACON as a tool for studies of ATC workload.
... 1-back. In accordance with previous research (Brookings et al., 1996;Grissmann et al., 2017;Puma et al., 2018), frontal theta activity was sensitive to MWL changes. The electrode Fz outperformed the electrodes F3 and F4 in the ability to differentiate the n-back levels. ...
... This aligns with previous, ambiguous results on the beta band. While it sometimes is considered the best indicator for MWL (Charbonnier, Roy, Doležalová, et al., 2016), others discussed if beta reflected types of cognitive processing instead of load (Brookings et al., 1996). As stated above, the beta band is also influenced by time-on-task effects (Roy et al., 2013). ...
... Parietal beta band power did not vary with either concept, which aligns with the heterogeneous literature on the frequency band (for MWL e.g. Brookings et al., 1996;Dussault et al., 2004; for MF e.g. Trejo et al., 2015). ...
Assessing the pilot’s cognitive state is of increasing importance in aviation, especially for the development of adaptive assistance systems. For this purpose, the assessment of mental workload (MWL) is of special interest as an indication when and how to adapt the automation to fit the pilot’s current needs. Thus, there is a need to assess the pilot continuously, objectively and non-intrusively. Neurophysiological measurements like electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are promising candidates for such an assessment. Yet, there is evidence that EEG- and fNIRS-based MWL measures are susceptible to influences from other concepts like mental fatigue (MF), and decrease in accuracy when MWL and MF confound. Still, there are only few studies targeting this problem, and no systematic investigation into this problem has taken place. Thus, the validity of neurophysiological MWL measures is not clear yet. In order to undertake such a systematic investigation, I conducted three studies: one experiment in which I investigated the effects of increasing MWL on cortical activation when MF is controlled for; a second experiment in which I examined the effects of increasing MF on cortical activation when MWL is controlled for; and a further comparative analysis of the gathered data. In order to induce MWL and MF in a controllable and comparable fashion, I conceived and used a simplified simulated flight task with an incorporated adapted n-back and monitoring task. I used a concurrent EEG-fNIRS measurement to gain neurophysiological data, and collected performance data and self-reported MWL and MF. In the first study (N = 35), I induce different four levels of MWL by increasing the difficulty of the n-back task, and controlled for MF by means of randomization and a short task duration (≤ 45 minutes). Higher task difficulty elicited higher subjective MWL ratings, declining performance, increased frontal theta band power and decreased frontal deoxyhaemoglobin (HbR) concentration. Furthermore, fNIRS proved more sensitive to tasks with low difficulty, and EEG to tasks with high difficulty. Only the combination of both methods was able to discriminate all four induced MWL levels. Thus, frontal theta band power and HbR were sensitive to changing MWL. In the second study (N = 31), I. I induced MF by means of time on task. Thus, I prolonged the task duration to approx. 90 minutes, and controlled for MWL by using a low but constant task difficulty derived from the first experiment. Over the course of the experiment, the participants’ subjective MF increased linearly, but their performance remained stable. In the EEG data, there was an early increase and levelling in parietal alpha band power and a slower, but steady increase in frontal theta band power. The fNIRS data did not show a consistent trend in any direction with increasing MF. Thus, only parietal alpha and frontal theta band power were sensitive to changing MF. In the third study, I investigated the validity of two EEG indices commonly used for MWL assessment, the Task Load Index (TLI) and the Engagement Index (EI). I computed the indices from the data of the two experiments, and compared the results between the datasets, and to single band powers. The TLI increased with increasing MWL, but was less sensitive than theta band power alone, and varied slightly with increasing MF. The EI did not vary with MWL, and was not sensitive to gradually increasing MF. Thus, neither index could be considered a valid MWL measure. In sum, neurophysiological measures can be used to assess changes in MWL. Yet, frontal HbR was the only measure sensitive to MWL that did not also vary with MF, and further research is needed to conclude if this finding holds true under different task characteristics. Thus, the tested EEG and fNIRS measures are only valid indications of MWL when confounding effects of MF are explicitly controlled for. I discuss further influences on the tested EEG and fNIRS measures, possible combinations with other data sources, and practical challenges for a neurophysiological MWL assessment. I conclude that neurophysiological measures should be used carefully outside the laboratory, as their validity will likely suffer in realistic settings. When their limitations are understood and respected, they can help to understand the cognitive processes involved in MWL, and can be a valuable addition to an MWL assessment.
... Proceedings of the International Conference on Industrial Engineering and Operations Management Sao Paulo, Brazil, April 5 -8, 2021 Roscoe (1992) v v v Wilson (1993) v v Backs and Seljos (1994) v v v Brookings et al. (1996) v v Veltman and Gailllard (1998) v v v Wientjes et al. (1998) Fournier et al. (1999) v v Bernadi et al. (2000) v v Houtven et al. (2002) v v Veltman (2002) v v Fairclough et al. (2005) v Beda et al. (2007) v v v Schleifer et al. (2008) v v Troubat et al. (2008) v Roscoe 1992;Fournier et al. 1999;Veltman and Gailllard 1996;Veltman and Gailllard 1998;Wang et al. 2015;Fairclough et al. 2008;Bruna et al. 2018;Karavidas et al. 2010 Non spesific domain Backs and Seljos 2002;Wientjes et al. 1998;Bernadi et al. 2000;Houtven et al. 2002;Beda et al. 2007;Troubat et al. 2008;Vlemincx et al. 2010;Hogervorst et al. 2014;Kodesh and Kizony 2014;Kuehl et al. 2015;Nagasawa and Hagiwara 2016;Bruder et al. 2019 12 Wilson 1993;Grassmann et al. 2016;Backs and Seljos 1994;Brookings et al. 1996;Fournier et al. 1999;Fairclough et al. 2005;Mehler et Respiratory is a physiological process mainly related to the exchange of O2 and CO2 from body tissues to the air. Respiration is measured using two elastic belts which placed around the chest and abdomen. ...
... Proceedings of the International Conference on Industrial Engineering and Operations Management Sao Paulo, Brazil, April 5 -8, 2021 Roscoe (1992) v v v Wilson (1993) v v Backs and Seljos (1994) v v v Brookings et al. (1996) v v Veltman and Gailllard (1998) v v v Wientjes et al. (1998) Fournier et al. (1999) v v Bernadi et al. (2000) v v Houtven et al. (2002) v v Veltman (2002) v v Fairclough et al. (2005) v Beda et al. (2007) v v v Schleifer et al. (2008) v v Troubat et al. (2008) v Roscoe 1992;Fournier et al. 1999;Veltman and Gailllard 1996;Veltman and Gailllard 1998;Wang et al. 2015;Fairclough et al. 2008;Bruna et al. 2018;Karavidas et al. 2010 Non spesific domain Backs and Seljos 2002;Wientjes et al. 1998;Bernadi et al. 2000;Houtven et al. 2002;Beda et al. 2007;Troubat et al. 2008;Vlemincx et al. 2010;Hogervorst et al. 2014;Kodesh and Kizony 2014;Kuehl et al. 2015;Nagasawa and Hagiwara 2016;Bruder et al. 2019 12 Wilson 1993;Grassmann et al. 2016;Backs and Seljos 1994;Brookings et al. 1996;Fournier et al. 1999;Fairclough et al. 2005;Mehler et Respiratory is a physiological process mainly related to the exchange of O2 and CO2 from body tissues to the air. Respiration is measured using two elastic belts which placed around the chest and abdomen. ...
... The RR parameters of respiratory are often used. Brookings et al. (1996) state that the increased respiration rate may have been a direct effect of the increase of metabolic demands required to perform the task. Furthermore, the RR is the most useful of the respiratory measures for MWL measurement (Roscoe 1992). ...
... Additionally, the interaction between traditional airspace users and UAVs must be considered, as it can further complicate the controllers' task of ensuring the safe and efficient operation of both sides. Integrating UAVs in a shared airspace poses challenges such as interaction adaptation, controlled information sharing, and continuous monitoring and adaptation (Anisetti et al., 2020), which align with the increased workload and complexities faced by controllers due to the implementation of new technologies (Brookings, Wilson, & Swain, 1996;Debernard, Vanderhaegen, & Millot, 1992;Tattersall, Farmer, & Belyavin, 1991). ...
... Furthermore, the coordination, communication, and management of UAVs within the non-segregated airspace add to the complexity of ATC, requiring additional effort from controllers (Al-Mousa et al., 2019). The distinctive characteristics of UAVs, such as their varied speeds and altitudes, further contribute to this complexity and can lead to increased stress and additional responsibilities for controllers (Brookings et al., 1996). The innovative technology of UAVs also brings technological changes and procedures to ATC systems, which can be a source of stress (Finkelman, & Kirschner, 1980;Liu, Feng, & Zeng, 2019;Tomic, & Liu, 2017). ...
The integration of Unmanned Aerial Vehicles (UAVs) into non-segregated airspace presents both opportunities and challenges for air traffic control (ATC). The aim of the study is to explore the perspectives of air traffic controllers on the current and anticipated challenges, workload, stress factors, performance errors, and mitigation strategies related to UAV integration. The sample consisted of 213 air traffic controllers in Türkiye. UAV operations have been available in Türkiye not only for military purposes but also for purposes such as forest fires, earthquakes, security, and others for a long time, and these UAV operations are provided with air traffic services (ATS) by air traffic controllers. The results show that air traffic controllers are concerned about mid-air collisions due to UAV technology limits and regulatory gaps, along with managing risks and unique flight characteristics. Addressing technology limitations, regulatory ambiguity, and other factors necessitates a comprehensive strategy. Solutions must prioritize collision avoidance systems, clear communication guidelines, and defined no-fly zones. It is recommended that future studies focus on the comprehensive impact of UAVs on air traffic operations and the development of regulations.
... This results in fewer but more focused eye movements. Another physiological sensor used in the study, the EEG, found that alpha bandpower decreased when MWL increased, a trend that aligns with findings from previous studies (Brookings et al., 1996;Fairclough and Venables, 2006;Jaquess et al., 2018). For example, memory retrieval and arithmetic tasks were shown to decrease alpha bandpower (Harmony et al., 1999). ...
... Future research should investigate factors affecting sensor performance in naturalistic settings, such as motion artifacts impacting EEG readings (Kappel et al., 2017). Moreover, as has been demonstrated in past literature (Brookings et al., 1996;Jaquess et al., 2018), the relationship between MWL and physiological metrics is complex. Some metrics have a linear relationship, while others have a non-linear relationship. ...
Mental workload (MWL) is a crucial area of study due to its significant influence on task performance and potential for significant operator error. However, measuring MWL presents challenges, as it is a multi-dimensional construct. Previous research on MWL models has focused on differentiating between two to three levels. Nonetheless, tasks can vary widely in their complexity, and little is known about how subtle variations in task difficulty influence workload indicators. To address this, we conducted an experiment inducing MWL in up to 5 levels, hypothesizing that our multi-modal metrics would be able to distinguish between each MWL stage. We measured the induced workload using task performance, subjective assessment, and physiological metrics. Our simulated task was designed to induce diverse MWL degrees, including five different math and three different verbal tiers. Our findings indicate that all investigated metrics successfully differentiated between various MWL levels induced by different tiers of math problems. Notably, performance metrics emerged as the most effective assessment, being the only metric capable of distinguishing all the levels. Some limitations were observed in the granularity of subjective and physiological metrics. Specifically, the subjective overall mental workload couldn't distinguish lower levels of workload, while all physiological metrics could detect a shift from lower to higher levels, but did not distinguish between workload tiers at the higher or lower ends of the scale (e.g., between the easy and the easy-medium tiers). Despite these limitations, each pair of levels was effectively differentiated by one or more metrics. This suggests a promising avenue for future research, exploring the integration or combination of multiple metrics. The findings suggest that subtle differences in workload levels may be distinguishable using combinations of subjective and physiological metrics.
... Also, it has been reported that the relationship between the saccade amplitude and the fixation duration differentiates the focal and ambient visual processing during scene perception (Unema et al., 2005). In addition, blink rate can be an indicator of sustained attention (Chan and Chen, 2004), and a decrease in blink duration and blink rate reflects an increase in cognitive demand (Veltman and Gaillard, 1998;Brookings et al., 1996;Fairclough et al., 2005). ...
... (see Figure 6 (d)). Based on the previous findings of Veltman and Gaillard (1998) and Brookings et al. (1996), the blink rate and duration results suggest that even when learning the same materials, participants experienced a higher level of cognitive load with random edits. The longer blink duration for individual edits than for random and professional edits yields a possibility that each individual's attention-based edits could decrease the level of cognitive load while learning. ...
Recently there has been a surge in demand for online video-based learning, and the importance of high-quality educational videos is ever-growing. However, a uniform format of videos that neglects individual differences and the labor-intensive process of editing are major setbacks in producing effective educational videos. This study aims to resolve the issues by proposing an automatic lecture video editing pipeline based on each individual's attention pattern. In this pipeline, the eye-tracking data are obtained while each individual watches virtual lectures, which later go through multiple filters to define the viewer's locus of attention and to select the appropriate shot at each time point to create personalized videos. To assess the effectiveness of the proposed method, video characteristics, subjective evaluations of the learning experience, and objective eye-movement features were compared between differently edited videos (attention-based, randomly edited, professionally edited). The results showed that our method dramatically reduced the editing time, with similar video characteristics to those of professionally edited versions. Attention-based versions were also evaluated to be significantly better than randomly edited ones, and as effective as professionally edited ones. Eye-tracking results indicated that attention-based videos have the potential to decrease the cognitive load of learners. These results suggest that attention-based automatic editing can be a viable or even a better alternative to the human expert-dependent approach, and individually-tailored videos have the potential to heighten the learning experience and effect.
... However, [99] points out that whereas alpha suppression is present in increasingly difficult mathematical calculations, it may not be a valid feature to distinguish major workloads from other tasks. Other authors also pointed to variations in the activity of beta [100] and theta [101], [102] bands. Additionally, [91] pointed out significant differences between veteran and novice workers in the beta and gamma bands, as well as in the brain activity asymmetry index. ...
... Among them, heart rate in terms of beats per minute and heart rate variability in both the time and frequency domains are the most popular metrics for assessing mental workload. The heart rate increases accordingly to task complexity [98], [100], [107]. Similarly, several studies [108]- [110] have shown a reduction in the interbeat interval as the mental workload demands increase. ...
The purpose of this study is to explore the measurement of human factors in the workplace that can provide critical insights into workers’ well-being. Human factors refer to physical, cognitive, and psychological states that can impact the efficiency, effectiveness, and mental health of workers. The article identifies six human factors that are particularly crucial in today’s workplaces: physical fatigue, attention, mental workload, stress, trust, and emotional state. Each of these factors alters the human physiological response in a unique way, affecting the human brain, cardiovascular, electrodermal, muscular, respiratory, and ocular reactions. This paper provides an overview of these human factors and their specific influence on psycho-physiological responses, along with suitable technologies to measure them in working environments and the currently available commercial solutions to do so. By understanding the importance of these human factors, employers can make informed decisions to create a better work environment that leads to improved worker well-being and productivity.
... Warm et al., 2008). Performance deterioration in fieldwork, such as air traffic (Brookings et al., 1996;Warm et al., 2008) or power transmission control (Brookings et al., 1996;Small et al., 2014;Warm et al., 2008) can be fatal. Increasing number of errors and reaction times over time were observed also during car driving (Wascher et al., 2016). ...
... Warm et al., 2008). Performance deterioration in fieldwork, such as air traffic (Brookings et al., 1996;Warm et al., 2008) or power transmission control (Brookings et al., 1996;Small et al., 2014;Warm et al., 2008) can be fatal. Increasing number of errors and reaction times over time were observed also during car driving (Wascher et al., 2016). ...
Vigilance refers to the ability to maintain attention and to remain alert to stimuli in prolonged and monotonous tasks. Vigilance decrement describes the decline in performance in the course of such sustained attention tasks. Time-related alterations in attention have been found to be associated with changes in EEG. We investigated these time-on-task effects on the basis of changes in the conventional EEG spectral bands with the aim of finding a compound measure of vigilance.
148 healthy adults performed a cued Go/NoGo task that lasted approximately 21 min. Behavioural performance was examined by comparing the number of errors in the first and last quarters of the task using paired t-test. EEG data were epoched per trial, and time-on-task effects were modelled by using multiple linear regression, with frequency spectra band power values as independent variables and trial number as the dependent variable.
Behavioural performance decreased in terms of omission errors only. Performance of the models, expressed by predicted R-squared, was between 0.10 and 0.27, depending on the particular task condition. The time-on-task EEG spectral changes were characterized by broad changes in the alpha and frontal changes in the beta and gamma bands.
We were able to identify a set of EEG spectral features that predict time-on-task. Our output is considered to be a measure of vigilance, reflecting the allocation of mental resources for the maintenance of attention.
... While subjective ratings have the advantage of being easy to implement, they do not capture dynamic workload fluctuations during a task (Antonenko, Paas, Grabner, & van Gog, 2010). Based on this shortcoming, several other physiological variables have been proposed to be particularly suited to track mental workload online, including electrodermal activity (Kohlisch & Schaefer, 1996;Reimer & Mehler, 2011), eye movements (Brookings, Wilson, & Swain, 1996;Veltman & Gaillard, 1998) or pupil size (Hampson, Opris, & Deadwyler, 2010;Porter, Troscianko, & Gilchrist, 2007), heart rate (Vogt, Hagemann, & Kastner, 2006) or heart rate variability (Aasman, Mulder, & Mulder, 1987;P. A. Hancock, Meshkati, & Robertson, 1985), as well as electroencephalographic (EEG) parameters (for review : Antonenko, Paas, Grabner, & van Gog, 2010;Brouwer et al., 2012). ...
... However, due to the variety of experimental settings applied to examine workload using different physiological methods, it is difficult to judge which variable might be the most valid to assess workload precisely (Hogervorst, Brouwer, & van Erp, 2014). Nevertheless, the few studies that directly compared different physiological methods concluded that EEG parameters are among the most appropriate tools (Brookings et al., 1996;Christensen, Estepp, Wilson, & Russell, 2012;Hogervorst et al., 2014;Taylor, Reinerman-Jones, Cosenzo, & Nicholson, 2010). In fact, Hogervorst and colleagues (2014) compared classification accuracies of different sensors tracking workload in tasks with varying workload levels. ...
Simultaneous interpretation is a complex task that is assumed to be associated with a high workload. To corroborate this association, we measured workload during three tasks of increasing complexity: listening, shadowing, and interpreting, using electroencephalography and self-assessments in four groups of participants with varying experience in simultaneous interpretation. The self-assessment data showed that professional interpreters perceived the most workload-inducing condition, namely the interpreting task, as less demanding compared to the less experienced participants. This higher subjectively perceived workload in non-interpreters was paralleled by increasing frontal theta power values from listening to interpreting, whereas such a modulation was less pronounced in professional interpreters. Furthermore, regarding both workload measures, trainee interpreters were situated between professional interpreters and non-interpreters. Since the non-interpreters demonstrated high proficiencies and exposure in their second language, too, our findings provide evidence for an influence of interpretation training on experienced workload during simultaneous interpretation.
... 13,14 Traditionally, major concerns have been, for example in understanding the capacity of people in safety-critical job roles like air traffic control 15,16 and designing shift work based on people's ability to maintain focus and to recover effectively from fatigue. 17 It is perhaps not surprising that these use cases have inspired early research into studying air traffic controllers with neurotechnology 18,19 and that industry has seen initial examples of applying neurotechnology in safety helmets, 1 to increase profits. The basis for many subjective and behavioural measures of cognitive activity in HCI research comes from human factors and cognitive psychology research. ...
People are increasingly eager to know more about themselves through technology. To date, technology has primarily provided information on our physiology. Yet, with advances in wearable technology and artificial intelligence, the current advent of consumer neurotechnology will enable users to measure their cognitive activity. We see an opportunity for research in Human-Computer Interaction (HCI) in the development of these devices. Neurotechnology offers new insights into user experiences and facilitates the development of novel methods in HCI. Researchers will be able to create innovative interactive systems based on the ability to measure cognitive activity at scale in real-world settings. In this paper, we contribute a vision of how neurotechnology will transform HCI research and practice. We discuss how neurotechnology prompts a discussion about ethics, privacy, and trust. This trend highlights HCI’s crucial role in ensuring that neurotechnology is developed and utilised in ways that truly benefit people.
... Data collected using inertial measurement units (IMUs), such as head micromovements and body posture changes, and neurophysiological measures, such as electroencephalography signal (EEG), can allow us to overcome the drawbacks of subjective measures by objectively assessing the user's cognitive states in real time during the execution of a task. The application of EEG and its reliability has been already well explored by the authors in a variety of laboratory and realistic settings by evaluating human-robot interaction in training assessments, driving, and air traffic control [36][37][38][39][40]. Although commercial and cheap EEG devices are available on the market, their correct usage requires specialized personnel to be able to check the correct position of the sensors (electrodes located exactly over the corresponding brain area) and quality of the EEG signals to achieve the results [41,42]. ...
Vigilance refers to the capability of humans to respond accordingly to relevant and unpredictable tasks and surrounding environment changes over prolonged periods of time. Identifying vigilance decrements can, therefore, have huge and vital impacts on several operational environments in which a simple slip of mind or a deficit in attention can bear life-threatening and disastrous consequences. Several methodologies have been proposed to assess and characterize vigilance, and the results have indicated that the sole measure of performance and self-reports are not enough to obtain reliable and real-time vigilance measure. Nowadays, monitoring head and body movements to obtain information about performance in daily activities, health conditions, and mental states has become very simple and cheap due to the miniaturization of inertial measurement units and their widespread integration into common electronic devices (e.g., smart glasses, smartwatches). The present study aimed to understand the relationship between head micromovements and body posture changes to vigilance decrease while performing the psychomotor vigilance task. The results highlighted that head micromovements can be employed to track vigilance decrement during prolonged periods of time and discriminate between conditions of high or low vigilance.
... Thus, EEG activity varies depending on local conditions, which enhances its precision in evaluating neurophysiological behaviors in human subjects [36]. In a study conducted in 1996, Brookings et al. [37] employed EEG to investigate variations in cognitive workload and stress levels that may not be discernible by alternative measurement methods. ...
The health of city residents is at risk due to the high rate of urbanization and the extensive use of electronics. In the context of urbanization, individuals have become increasingly disconnected from nature, resulting in elevated stress levels among adults. The goal of this study was to investigate the physical and psychological benefits of spending time in nature. The benefits of touching real grass and artificial turf (the control activity) outdoors with the palm of the hand for five minutes were measured. Blood pressure and electroencephalography (EEG) as well as State-trait Anxiety Inventory (STAI) scores, and the semantic differential scale (SDM) were used to investigate psychophysiological responses. Touching real grass was associated with significant changes in brainwave rhythms and a reduction in both systolic and diastolic blood pressure compared to touching artificial turf. In addition, SDM scores revealed that touching real grass increased relaxation, comfort, and a sense of naturalness while decreasing anxiety levels. Compared to the control group, the experimental group had higher mean scores in both meditation and attentiveness. Our findings indicate that contact with real grass may reduce physiological and psychological stress in adults.
... Proponents of the first type tend to believe that the right brain is emotionally negatively oriented, while the left brain is responsible for all positive emotions. However, proponents of the second type argue the same, while separating the ability to process from the ability to experience emotional information [41][42][43] . In experiments with differently emotionally coloured patterns, it was found that the more active area in the process of experiencing emotions was largely predetermined by the experimental conditions. ...
The study was conducted in the Republic of Altai, one of the Russian regions, with the participation ofstudents of different ethnic and gender backgrounds. The main idea was to clarify the anthropoestheticpreferences when choosing a sexual partner; however, in this work, attention is paid to a characteristicfeature of the drawn face – its asymmetry. One of the original projective psychodiagnostic methods was used– face drawing, which made it possible to determine the coefficient of asymmetry of its structural elementsand the manifestation of emotions in relation to a potential sexual partner. The algorithm for obtaining a facepattern, drawing reference (cephalometric) points, and a module for calculating asymmetry was describedin detail. In addition, the authors made an attempt to show the connection between asymmetry and the spaceoccupied by facial structural elements with the emotional background.
... One important point to note about EBR is that fatigue is often associated with higher spontaneous EBR (Brookings et al., 1996), indicating that the interpretation of EBR data needs to be done with due care to the task being performed and the sustained nature of the activity being undertaken. Changes in EBR can be traced to the operation of the underlying neural system. ...
Eye-tracking and physiological measurements are central to research concerned with evaluating user experience in the context of human-computer interaction. Eye-tracking is especially valuable for understanding where people’s attention is being deployed during interface search and interaction, thereby providing insights into factors that hinder the usability of computer-based technologies. Physiological measurements, in contrast, are attuned to indexing the affective aspects of user experience (e.g., levels of arousal) as well the cognitive workload associated with an interaction task. This chapter will enable readers to gain a detailed understanding of a range of eye-tracking and physiological measurements and how they are interpreted when evaluating user experience. The value of these measurement techniques will be illustrated through studies drawn from both pioneering and recent research. The chapter will also consider the limitations of eye-tracking and physiological measurements in usability research as well as ways to mitigate such limitations. The final section of the chapter will discuss key trends and directions in the use of such measurements in user experience studies, including the use of eye-movement traces to elicit retrospective reports of interface problems, and the potential for automated identification and categorisation of eye-movement and physiological patterns that are diagnostic of interaction difficulties.
... In the present study, the subscales were analyzed separately rather than calculating a single workload score. The NASA-TLX scale is a reliable measure of subjective workload [91][92][93]. The internal validity for the five dimensions (mental demand, temporal demand, performance, effort, and frustration) used in the present study was fairly high, with Cronbach 's alpha ranging from α = 0.796 to 0.896. ...
Most of the studies on the effect of trait positive affect (PA) and cardiovascular activity have focused on heart rate (HR) and blood pressure (BP) rather than heart rate variability (HRV). However, trait PA might sustain homeostasis for the autonomic system (ANS) by reducing activity in the sympathetic system (SNS) and increasing the activity in the parasympathetic system (PNS). A common index for the PNS is the vagal tone measured indirectly through HRV. The present study assessed whether trait PA influences cardiovascular response to various stress tasks by monitoring participants’ HRV measured by RMSSD (root mean square of successive differences) along with HR and interbeat interval (IBI). A total of 54 participants performed various cognitive tasks and Trier Social Stress Tasks while their vital signs were monitored, and trait PA was measured with PANAS. The cognitive tasks included both high- and low-stress tasks, including fatigue-inducing 20 min Stroop tasks. The results showed overall higher HRV as measured by RMSSD for participants who have higher levels of trait PA, indicating more PNS activity compared with low-trait-PA individuals, particularly at the end of the task performance during the fatigue induction. High-trait-PA individuals also had a lower HR during the fatigue-inducing task and a higher IBI. The results support previous work by further indicating a more adaptive response and consequently better emotional regulation for high-trait-PA individuals in a complex, prolonged task setting.
... Subjective measures rely on self-report, such as questionnaires or visual analogue scales, and are easy to administer but prone to bias. It is recommended to use subjective measures in conjunction with objective measures when assessing fatigue in the workplace [28,29]. ...
Human involvement in manufacturing is indispensable. The performance and reliability of the human element directly impacts the success of a system. The complex interplay between fatigue, workload, and human performance necessitates recognition of the importance of human factors in manufacturing to achieve optimal outcomes. The efficiency of employees in production systems can be influenced by a number of factors, including exhaustion, cognitive and physical abilities, and the amount of time available for tasks. In general, these factors can have an impact on human reliability, which in turn can affect the reliability of production systems. A key objective of workload research is to identify and analyze potential performance declines, along with their associated factors. The pursuit of successful outcomes in an operation is not enough, as the negative impacts of such outcomes on the system or personnel must also be minimized. Once a possible issue or decrease has been determined, corrective measures are put in place to address such instances. Such foresight serves to prevent the costs that may arise from making process modifications, and to optimize resource utilization. This study conducts a comprehensive review of related literature, focusing on simulation integration. A taxonomy of fatigue and workload is used to contextualize and assess these factors. Methodologies and techniques are described, and evaluations of specific assessment techniques are provided, along with discussions of available information. This research makes a persuasive case for the importance of evaluating fatigue and workload among operators.
... Maybe if humanity does not use the possibility of objective measurement of personality traits, then the moment will come when people with deviant behavior can use modern media and military-technical means to achieve subjective goals that will lead to the death of humanity. Currently, psychophysiological control is used for individuals (Brookings et al, 1996;Bobrov et al., 2021) who work in hazardous and important industries (drivers, dispatchers, plant operators, etc.) but do not use psychophysiological control for politicians who adopt more significant solutions. The current subjective social system of natural selection for social leaders may not be enough to make adequate decisions in the modern world during the 4th industrial revolution. ...
Metrology approach to the personality traits is proposed, similar to the measurement of physical quantities of the International System of Units (SI). The analysis was made of open database of 500 subjects test including psychophysiological responses (PPR) to 48 multifactorial stimuli tied to multiple intelligences and personality vices consistently presented by Blitz Judgment program. Various equations for calculating of the current psychophysiological state (PPS) and PPR according to microvibration data of a human head, measured by vibraimage technology, are considered. Mathematical criteria are proposed for assessing the correctness of PPS and PPR calculating based on the closeness of PPR distribution to the normal distribution and the maximum correlation between PPR for stimuli, that are tied to multiple intelligences and personality vices. Based on the results of PPR analysis, it is proposed to include into the International System of Units (SI) new quantities and units – the bit (information) and the vibra (PPR) used to measure personality traits. Definitions are given for new units of personality traits measurement, based on the combination of the principles of psychology, cybernetics and physics within the framework of single metrological approach to a person as biological, cybernetics and physical object.
... Я допускаю, что если человечество не использует возможность объективного измерения характеристик личности, то настанет момент, когда люди с девиантным поведением могут использовать современные СМИ, ИКТ и военно-технические средства для достижения субъективных целей, которые приведут к гибели человечества. В настоящее время широко используется психофизиологический контроль лиц (Brookings et al., 1996; Бобров и др., 2021), которые работают на опасных производствах (водители, диспетчеры, операторы АС и т.д.), но не применяется психофизиологический контроль для политиков, принимающих более значимые решения. Сложившейся системы естественного отбора общественных лидеров может не хватить для принятия адекватных решений в современном мире во время 4-й промышленной революции. ...
Предложен подход к измерению характеристик личности, аналогичный измерению физических величин метрической системы измерения, лежащей в основе Международной системы единиц СИ. Проведен анализ открытой базы данных психофизиологической реакции (ПФР) 500 испытуемых на 48 многофакторных стимулов, имеющих привязку к способностям и порокам личности и последовательно предъявляемых программой Профайлер+. Рассмотрены различные формулы расчета текущего психофизиологического состояния (ПФС) и ПФР по данным микровибраций головы человека, измеряемым технологией виброизображения. Предложены математические критерии оценки корректности вычисления ПФС и ПФР на основании близости плотности ПФР к нормальному распределению и максимальной корреляции между ПФР на стимулы, имеющие привязку к способностям и порокам личности. По результатам анализа ПФР предложено введение в систему СИ новых информационно-физических величин и единиц измерений - бит и вибра, используемых для измерения характеристик личности. Даны определения новым единицам измерения характеристик личности, основанные на совмещении принципов психологии, кибернетики и физики в рамках единого метрологического подхода к человеку как биологическому, кибернетическому и физическому объекту.
... Ultimately, OSA gradually leads to daytime sleepiness, decreased consciousness, cognitive dysfunction, impaired vigilance, impaired work performance, and the overall deterioration of the quality of life (Brookings et al., 1996;Ferini-Strambi et al., 1940;Mitler et al., 1988). ...
Oxygen saturation (SpO2 )-based parameters are more strongly linked to impaired daytime vigilance than the conventional diagnostic metrics in patients with obstructive sleep apnea (OSA). However, whether the association between SpO2 -based parameters and impaired daytime vigilance is modulated by sex, remains unknown. Hence, we investigated the interplay between sex and detailed SpO2 -based metrics and their association with impaired vigilance in patients with OSA. The study population consisted of 855 (473 males, 382 females) patients with suspected OSA who underwent overnight polysomnography and psychomotor vigilance task (PVT). The population was grouped by sex and divided into quartiles (Q1-Q4) based on median reaction times (RTs) in the PVT. In addition to conventional diagnostic metrics, desaturation severity (DesSev), fall severity (FallSev), and recovery severity (RecovSev) were compared between the sexes and between the best (Q1) and worst (Q4) performing quartiles by using cumulative distribution functions (CDFs). Additionally, sex-specific covariate-adjusted linear regression models were used to investigate the connection between the parameters and RTs. The CDFs showed significantly higher hypoxic load in Q4 in males compared to females. In addition, the DesSev (β = 8.05, p < 0.01), FallSev (β = 6.48, p = 0.02), RecovSev (β = 9.13, p < 0.01), and Oxygen Desaturation Index (β = 12.29, p < 0.01) were associated with increased RTs only in males. Conversely, the Arousal Index (β = 10.75-11.04, p < 0.01) was associated with impaired vigilance in females. The severity of intermittent hypoxaemia was strongly associated with longer RTs in males whereas the Arousal Index had the strongest association in females. Thus, the impact of hypoxic load on impaired vigilance seems to be stronger in males than females.
... Three states of the variable were defined, with thresholds of 15 blinks per minute and less correspond to "low" blink rate, 15 to 20 blinks per minute considered as "medium", and values higher than 20 blinks per minute were grouped as "high" blink rate. These threshold values were defined based on prior studies linking blink rate levels to cognitive load (Abusharha 2017;Brookings et al. 1996;Wang et al. 2016). The node probability table for workload given blink rate is shown in Table 5. ...
Successful operation of military aviation depends on effective pilot training. The current training capabilities of the United States Air Force might not be sufficient to meet the demand for new pilots. To help resolve this issue, this study focused on developing a prototype of an adaptive virtual reality (VR) training system. The system was built leveraging the three key elements of an adaptive training system including the trainee’s performance measures, adaptive logic, and adaptive variables. The prototype was based on a procedure for an F-16 cockpit and included adaptive feedback, temporal display features, and various difficulty levels to help trainees maintain an optimal level of cognitive workload while completing their training. An experiment with 20 human participants was conducted, and a trend favoring the use of adaptive training was identified. Results suggested that adaptive training could improve performance and reduce workload as compared to the traditional non-adaptive VR-based training. Implementation of adaptive VR training has the potential to reduce training time and cost. The results from this study can assist in developing future adaptive VR-training systems.
... waves (4 to 8 Hz), alpha waves (8 to 13 Hz), and gamma waves. Higher mental stress or workload is associated with the disappearance of alpha waves and the emergence of beta waves (Nishimura and Mitsukura 2013). Some studies have also reported that alpha waves decrease and theta waves increase as work pressure or stress increases (Ajiro et al. 2009). In 1996 EEG to examine differences in levels of workload or stress that cannot be identified using other devices, observing that EEG activity differs based on exposure to different environments, making this approach more accurate for estimating neurophysiological behaviors in humans (Ajiro et al. 2009). In the present study, we aimed to investi ...
Reducing stress associated with technology and the use of electronics is a major issue among Chinese adults. However, no studies have investigated the effect of tactile stimulation of the feet. In this study, we investigated psychophysiological techniques for controlling stress by having participants touch natural materials with the sole of the foot. The study included 90 young Chinese adults with a mean (±SD) age of 21.2 ± 2.7 years. A crossover design was used to examine psychological and physiological differences between touching grass with the sole of the foot and touching wood (control) for 10 minutes. Physiological assessments included blood pressure measurements and electroencephalog-raphy, and psychological assessments included the Semantic Differential Method (SDM) and State-Trait Anxiety Inventory (STAI). We observed significant decreases in sys-tolic and diastolic blood pressures in the experimental condition compared with the control condition, along with increases in alpha and beta activities. SDM results indicated that participants were moderately comfortable, very relaxed, and experienced reduced anxiety after stimulation with grass compared with after the control condition. Mean attention and relaxation scores were also significantly higher in the experimental condition than in the control condition. Thus, our results suggest that touching grass with the sole of the foot can lower psychophysiological stress in adults.
... Based on our results, the participants spent relatively more time on processing the task and less time on search activities during suturing tasks compared to peg transfer tasks. This is consistent with the results from previous studies where saccade duration was shorter during the tasks which induced higher cognitive load [82,83], and fixation duration became longer as the task workload demand increased [84,85]. ...
Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.
... A number of factors led to our choice of EEG as a physiological measure of cognitive workload. Some studies suggest that EEG measures are more sensitive measures of different levels of workload than other physiological measures [57][58][59] . Ocular measures are a problem unless the brightness of the environment is stable and flicker-free 60 . ...
Pilots of aircraft face varying degrees of cognitive workload even during normal flight operations. Periods of low cognitive workload may be followed by periods of high cognitive workload and vice versa. During such changing demands, there exists potential for increased error on behalf of the pilots due to periods of boredom or excessive cognitive task demand. To further understand cognitive workload in aviation, the present study involved collection of electroencephalogram (EEG) data from ten (10) collegiate aviation students in a live-flight environment in a single-engine aircraft. Each pilot possessed a Federal Aviation Administration (FAA) commercial pilot certificate and either FAA class I or class II medical certificate. Each pilot flew a standardized flight profile representing an average instrument flight training sequence. For data analysis, we used four main sub-bands of the recorded EEG signals: delta, theta, alpha, and beta. Power spectral density (PSD) and log energy entropy of each sub-band across 20 electrodes were computed and subjected to two feature selection algorithms (recursive feature elimination (RFE) and lasso cross-validation (LassoCV), and a stacking ensemble machine learning algorithm composed of support vector machine, random forest, and logistic regression. Also, hyperparameter optimization and tenfold cross-validation were used to improve the model performance, reliability, and generalization. The feature selection step resulted in 15 features that can be considered an indicator of pilots' cognitive workload states. Then these features were applied to the stacking ensemble algorithm, and the highest results were achieved using the selected features by the RFE algorithm with an accuracy of 91.67% (± 0.11), a precision of 93.89% (± 0.09), recall of 91.67% (± 0.11), F-score of 91.22% (± 0.12), and the mean ROC-AUC of 0.93 (± 0.06). The achieved results indicated that the combination of PSD and log energy entropy, along with well-designed machine learning algorithms, suggest the potential for the use of EEG to discriminate periods of the low, medium, and high workload to augment aircraft system design, including flight automation features to improve aviation safety.
... These motor activities produce a low frequency, high amplitude signals, which are easily distinguishable from others related to neuronal activity. According to Brookings et al. (1996), an individual's blinking rate reduces significantly when tasks become harder. Taking advantage of the eye blinks detected by Mind Monitor, we computed the number of blinks per minute and then deducted the baseline number of blinks identified throughout the participant's previous one-minute viewing of the fish-tank video before starting the task. ...
In this paper, we investigate the extent to which we could classify task difficulty in the software testing domain, using psycho-physiological sensors. Following a literature review, we selected and adapted the work of Fritz et al. (2014) among software developers, and transposed it to the testing domain. We present the results of a study conducted with 15 professional software testers carrying out predefined tasks in a lab setting, while we collected eye tracking, electroencephalogram (EEG) and electrodermal activity (EDA) data. On average, each participant took part in a two-hour data-collection session. Throughout our study, we captured approximately 14Gb of biometric data, consisting of more than 120 million data points.
Using this data, we trained 21 naïve Bayes classifiers to predict task difficulty from three perspectives (by participant, by task, by participant-task) and using the seven possible combinations of sensors. Our results confirm that we can predict task difficulty for a new tester with a precision of 74.4% and a recall of 72.5% using just an eye tracker, and for a new task with a precision of 72.2% and a recall of 70.0% using eye tracking and electrodermal activity. The results achieved are largely consistent with the work of Fritz et al. (2014). We conclude by providing insights as to which combinations of sensors would provide the best results, and how this work could be used to enhance well-being and workflow support tools in an industry setting.
... Moreover, inadequate sleep decreases productivity, work performance, and daytime vigilance. Additionally, patients with severe obstructive sleep apnea have a higher probability of increased morbidity and risk for motor vehicle accidents (Barbe et al., 1998;Brookings et al., 1996;Körber et al., 2015;Reinerman-Jones et al., 2016). ...
Obstructive sleep apnea (OSA)‐related intermittent hypoxaemia is a potential risk factor for different OSA comorbidities, for example cardiovascular disease. However, conflicting results are found as to whether intermittent hypoxaemia is associated with impaired vigilance. Therefore, we aimed to investigate how desaturation characteristics differ between the non‐impaired vigilance and impaired vigilance patient groups formed based on psychomotor vigilance task (PVT) performance and compared with traditional OSA severity parameters. The study population comprised 863 patients with suspected OSA who underwent a PVT test before polysomnography. The conventional OSA parameters, for example, the apnea‐hypopnea index, oxygen desaturation index, and arousal index were computed. Furthermore, the median desaturation area, fall area, recovery area, and desaturation depth were computed with the pre‐event baseline reference and with reference to the 100% oxygen saturation level. Patients were grouped into best‐ and worst‐performing quartiles based on the number of lapses in PVT (Q1: PVT lapses <5 and Q4: PVT lapses >36). The association between parameters and impaired vigilance was evaluated by cumulative distribution functions (CDFs) and binomial logistic regression. Based on the CDFs, patients in Q4 had larger desaturation areas, recovery areas, and deeper desaturations when these were referenced to 100% saturation compared with Q1. The odds ratio (OR) of the median desaturation area (OR = 1.56), recovery area (OR = 1.71), and depth (OR = 1.65) were significantly elevated in Q4 in regression models. However, conventional OSA parameters were not significantly associated with impaired vigilance (ORs: 0.79–1.09). Considering desaturation parameters with a 100% SpO2 reference in the diagnosis of OSA could provide additional information on the severity of OSA and related daytime vigilance impairment.
... In this context, the user interface should only output relevant information ordered by importance [9]. Irrelevant and too much information can lead to cognitive overload and eventually to errors in the work task due to limited working memory capacity [10] of human users [11]. The goal of this article is to 1) open-up the black box of design processes that enable a transition from abstract design recommendations to concrete context-specific interface designs, 2) make usercentered design processes thus more reproducible and objects of reflections in scientific discourse, 3) evaluate a mix of methods commonly used in interface design. ...
... Measurements based on different kinds of eye movements, such as blink rate, fixation duration, pupil diameter, and saccade-related measures, have been used to derive metrics of MW [46]. The blink rate and saccadic peak velocity were found to decrease with the increase in visual and cognitive demands during the air traffic control simulated tasks [33,47]. When performing nuclear power plant operation procedures [48] and dealing with artificial surrounding created by computers [49], the pupil diameter was significantly larger with a higher MW. ...
Twelve healthy male students were recruited to investigate the physiological response to different noise exposure and mental workload (MW) conditions, while performing multi-attribute task battery (MATB) tasks. The experiments were conducted under three noise exposure conditions, with different sound pressure levels and sharpness. After adaptation to each noise condition, the participants were required to perform the resting test and the MATB task tests with low, medium, and high MW. The electroencephalogram (EEG), electrocardiogram (ECG), and eye movement data were obtained, during the periods when participants were in the resting and task taking state. The results showed that subjects’ physiological responses at rest were unaffected by noise exposure conditions. However, during the execution of MATB tasks, the elevated sound pressure level and increased sharpness were significantly correlated with increased mean pupil diameter and heart rate variability (HRV). These responses suggested that the human body defends itself through physiological regulation when noise causes adverse effects. If the negative effects of noise were more severe, this could damage the body’s health and result in a significant drop in task performance. The elevated mental demands led to increased stress on the subjects, which was reflected in a considerable increase in theta relative power. Either high or low MW was related with reduced saccade amplitude and a decrease in weighted task performance, indicating an inverted U-shaped relationship between workload level and work performance.
... Many industrial, military, medical, and educational tasks require continuous vigilance for cognitive workload, including security personnel, driving, diagnostic medical screening, and industrial and air traffic control (Brookings et al., 1996;Gill, 1996;Näsholm et al., 2014;Korber et al., 2015;Reinerman-Jones et al., 2016). These long vigils can lead to vigilance decrement and a drop in cognitive efficiency, which results in increased reaction time, error rate, and even fatal consequences (Pattyn et al., 2008); therefore, it is necessary to investigate possible vigilance decrement countermeasures. ...
A prolonged period of vigilance task will lead to vigilance decrement and a drop in cognitive efficiency. Although transcranial direct current stimulation (tDCS) can be used to improve cognitive performance following vigilance decrement, the findings in this area of study are inconsistent. This study aims to identify the neuroelectrophysiological and behavioral effects of tDCS over the left dorsolateral prefrontal cortex (DLPFC) on executive vigilance under a continuous monotonous condition. We recruited 29 participants who randomly received 30 min active or sham tDCS before the vigilance task (anode electrode at the left DLPFC, cathode electrode at the right supraorbital area). Participants completed four sessions of vigilance task and five sessions of self-report sleepiness, Oddball task, and Go/Nogo task, for a total of about 5 h. EEG was acquired in real-time throughout the experiment. Repeated measures of ANOVA were utilized to analyze the evolution of each metric with task-on-time. The results demonstrated that subjective arousal state, vigilance performance, event-related potentials (ERPs), and EEG power were significantly affected by time on task. Brain stimulation did not significantly affect the evolution of subjective and objective executive vigilance performance, but significantly modulated spontaneous activity in the alpha and beta bands across the entire brain. The continuous enhancement of the prefrontal cortex increased P2 amplitude for the Oddball task, which was associated with the enhancement of the early stage of information processing. P3 amplitude had a temporary enhancement effect, which significantly decreased following a cognitive fatigue. tDCS had a continuous enhancement effect on N2 amplitude for the Go/Nogo task, which was associated with the enhanced inhibition of distracting stimuli. Together, the current data suggest that anodal tDCS over left DLPFC possibly enhances the early stage of relevant information processing and the inhibitory control of distracting stimuli during a continuous and monotonous vigilance task.
... Pupil diameter usually combines with blink indices, fixation, saccadic and dwell time to estimate the human MWL of different tasks (Brookings et al., 1996;Van Orden et al., 2000). The eye blink is believed to be an indicator of both fatigue and workload. ...
This study proposed an eye responses-based mental workload (E-MWL) evaluation method in nuclear power plants (NPPs) when performing the task via a user interface control. The fuzzy theory was used to combine four eye response indices using the entropy weight method. Then, the E-MWL method was validated through experiments by comparison with the NASA-TLX rating and performance measures indices in two different tasks of the State Oriented Procedure (SOP) in NPP. The correlation analysis results between the NASA-TLX and eye response indices showed that four eye response indices used in this study were correlated significantly with the NASA-TLX, indicating that these indices may develop the E-MWL method. The E-MWL score results indicated that it is highly correlated with NASA-TLX and performance measures indices in two different tasks of SOP in NPP. This has proved that E-MWL is an objective method suitable for evaluating and predicting human mental workload (MWL) for interface control task in NPPs.
In today's fast‐paced society, escalating work and academic pressures have led to rising stress levels. While numerous studies have explored adolescent mental health, there has been a lack of focus on “educational stress” among Chinese students. This study sought to understand the psychological and physiological effects of educational stress in Chinese university students. We studied the impact of a 5‐min nature photography session on campus compared with a control activity of photographing urban settings near campus. Data were collected using blood pressure measurements, electroencephalography (EEG), the Semantic Differential Method (SDM), and the State–Trait Anxiety Inventory (STAI) in order to understand psychophysiological reactions. The findings from the SDM and STAI assessments indicated that students felt slightly more at ease and considerably more relaxed, had a heightened sense of naturalness, and experienced reduced anxiety after engaging in nature photography compared with urban photography. Notably, we observed that both systolic and diastolic blood pressure dropped by many values and there were noticeable EEG changes among participants. The results suggest that a brief 5‐min nature photography activity can effectively reduce mental stress in Chinese university students.
The aim of this study was to experimentally test the effects of different electromyographic-based prosthetic control modes on user task performance, cognitive workload, and perceived usability to inform further human-centered design and application of these prosthetic control interfaces. We recruited 30 able-bodied participants for a between-subjects comparison of three control modes: direct control (DC), pattern recognition (PR), and continuous control (CC). Multiple human-centered evaluations were used, including task performance, cognitive workload, and usability assessments. To ensure that the results were not task-dependent, this study used two different test tasks, including the clothespin relocation task and Southampton hand assessment procedure-door handle task. Results revealed performance with each control mode to vary among tasks. When the task had high-angle adjustment accuracy requirements, the PR control outperformed DC. For cognitive workload, the CC mode was superior to DC in reducing user load across tasks. Both CC and PR control appear to be effective alternatives to DC in terms of task performance and cognitive load. Furthermore, we observed that, when comparing control modes, multitask testing and multifaceted evaluations are critical to avoid task-induced or method-induced evaluation bias. Hence, future studies with larger samples and different designs will be needed to expand the understanding of prosthetic device features and workload relationships.
Objective:
This study aimed to examine the impact of non-driving-related tasks (NDRTs) on drivers in highly automated driving scenarios and sought to develop a deep learning model for classifying mental workload using electroencephalography (EEG) signals.
Methods:
The experiment involved recruiting 28 participants who engaged in simulations within a driving simulator while exposed to 4 distinct NDRTs: (1) reading, (2) listening to radio news, (3) watching videos, and (4) texting. EEG data collected during NDRTs were categorized into 3 levels of mental workload, high, medium, and low, based on the NASA Task Load Index (NASA-TLX) scores. Two deep learning methods, namely, long short-term memory (LSTM) and bidirectional long short-term memory (BLSTM), were employed to develop the classification model.
Results:
A series of correlation analyses revealed that the channels and frequency bands are linearly correlated with mental workload. The comparative analysis of classification results demonstrates that EEG data featuring significantly correlated frequency bands exhibit superior classification accuracy compared to the raw EEG data.
Conclusions:
This research offers a reference for assessing mental workload resulting from NDRTs in the context of highly automated driving. Additionally, it delves into the development of deep learning classifiers for EEG signals with heightened accuracy.
Working memory tasks, such as n-back and arithmetic tasks, are frequently used in studying mental workload. The present study investigated and compared the sensitivity of several physiological measures at three levels of difficulty of n-back and arithmetic tasks. The results showed significant differences in fixation duration and pupil diameter among three task difficulty levels for both n-back and arithmetic tasks. Pupil diameters increase with increasing mental workload, whereas fixation duration decreases. Blink duration and heart rate (HR) were significantly increased as task difficulty increased in the n-back task, while root mean square of successive differences (RMSSD) and standard deviation of R-R intervals (SDNN) were significantly decreased in the arithmetic task. On the other hand, blink rate and Galvanic Skin Response (GSR) were not sensitive enough to assess the differences in task difficulty for both tasks. All significant physiological measures yielded significant differences between low and high task difficulty except for SDNN.Practitioner summary: This study aimed to assess the sensitivity levels of several physiological measures of mental workload in n-back and arithmetic tasks. It showed that pupil diameter was the most sensitive in both tasks. This study also found that most physiological indices are sensitive to an extreme change in task difficulty levels.
Watchkeeping, as an essential part of a nautical officer's work, is recognized for its complexity. Indeed, it requires the development of a certain amount of experience and the mastery of numerous skills to develop the ability to make critical decisions promptly. If it is obvious that a multitude of factors affects the performance of an officer in the execution of his watch, it is still difficult to establish objectively measurable criteria. However, it is essential to be able to evaluate watch leaders to guarantee the safety of the ship but also to validate and supervise their training. In addition, a better mastery of performance evaluation would also enable the development of more efficient navigation tools, as well as a better understanding of the factors affecting the quality of watchkeeping. Thus, this work attempts to identify different evaluation techniques by considering the regulatory constraints and the expectations of professionals while also exploring the cognitive aspects of human performance.
Time delay is an inherent technical feature that cannot be eliminated during space robot teleoperation and can significantly impact operator workload. In this paper, based on predictive display technology, we investigate the change of operator workload during teleoperation by image prediction graphical interface and virtual reality prediction graphical interface. By constructing a virtual simulation experiment platform for a remote operation system, 20 participants were tested and analyzed in a fetch-place experiment; TAM, NASA-TLX, and AB-directed questionnaires were distributed for subjective measurements to understand the personal perceptions and visual preferences of the participants on their workloads, and to investigate the applicability of the predictive display technology to reduce the operator’s cognitive load. The results showed that the subjective and objective performance of the virtual reality predictive graphical interface (VR-PGI) was higher than that of the image predictive graphical interface (I-PGI) in the fetch-place operation scenario. It was found that the operator’s workload did not change significantly with the increase of spatial time delay in the predictive graphical interface, which proved the practicality of applying the predictive graphical interface to the spatial teleoperation scenario.KeywordsPredictive GraphicsTime DelayTeleoperationWorkload
Adaptive automation shall support users in a flexible way. One way to achieve this could be by monitoring cognitive states of pilots in order to anticipate an individual’s need for support. A special challenge lies in choosing methods that enable a valid measurement of the cognitive state in question since different measures are associated with distinct strengths and weaknesses. For example, practical considerations like environmental factors, wearing comfort and intrusiveness have to be considered. The objective of this paper is to provide a collection of physiological, self-report and behavioral measures that can be applied to assess mental workload in pilots, and to discuss their advantages and disadvantages for this purpose. A targeted literature search was conducted to this end. The comparisons drawn in this paper reveal that a multi-method approach is preferable to relying on a single measure. In this regard, however, there is no one-size-fits-all solution and it is strongly advised to consider the selection of appropriate measures carefully for each specific research question and application context.KeywordsMental workloadpsychophysiologyaviationpilots
Wild blueberries (Vaccinium angustifolium Ait.) are among eastern Canada's most economically important crops. Despite this, the industry faces significant labor shortages required to harvest the over 69 000 ha of wild blueberry land each year. Automation of the wild blueberry is among the leading areas of wild blueberry research. The requirement to automate several different aspects of the harvester means that at present an operator is still required in the tractor. To determine the impacts that automation features have on an operator, and to assess the potential to replace skilled operators with unskilled ones, heart and respiration rates were monitored across various automatic, semi-automatic and manual harvesting conditions. Across both years of the study, the skilled operator experienced a 13.83% decrease in average heart rate under the fully automated condition versus the fully manual condition. Similarly, the new operator experienced a 19.03% decrease in average heart rate for the same scenario. While a conclusive determination cannot be made due to the significant interaction effect, it was likewise interesting to note that the skilled operator seemed to benefit more from the automated head adjustment while the new operator seemed to benefit more from the autosteer. Respiration rate data did not yield a conclusive trend, though the highest respiration rates were seen under the fully manual harvesting condition in all but the 2022 new operator data. In all, this study lays significant groundwork in the justification of automation for addressing the skilled labour shortage and for the eventual full automation of the wild blueberry harvester.
ATC (Air Traffic Control) is considered one of the most demanding jobs. This profession is considered a job with high mental workload due to its high-stress level and great responsibility. This study designed a suitable work system to improve operator performance by measuring the mental workload and the physical environment using the NASA-TLX method and safety concept by considering variables affecting the operator’s performance. This study also searched for the impact of mental workload on the work environment, the mental workload on performance, and the work environment on performance. Questionnaires were distributed to operators, and validation and verification tests were carried out using SPSS. At the PLS method's processing stage, the variables used in this study consisted of the dependent (Y) and independent (X) variables. The dependent variables in this study were performance and the physical environment of work of the operator. Meanwhile, the independent variable was mental workload. Based on the mental load calculation, an average WWL (weighted workload) score of 80 to 90 was obtained, and the factors affecting mental workload are performance aspects and mental demand. Based on the results of structural modelling with the PLS method, there was a significant influence between mental workload on the work environment, the mental workload on performance and the work environment on operator performance. The proposed work system design used an ergonomic approach, safety and regulation of Ministry of Health to get an ergonomic work system, regulate the equal distribution of workloads, create a safe and comfortable working environment, and improve operator performance. The design focused on the ATC tower's workstations and work environments. Supervisor has accepted the design.
Introduction
A well-designed brain-computer interface (BCI) can make accurate and reliable predictions of a user's state through the passive assessment of their brain activity; in turn, BCI can inform an adaptive system (such as artificial intelligence, or AI) to intelligently and optimally aid the user to maximize the human-machine team (HMT) performance. Various groupings of spectro-temporal neural features have shown to predict the same underlying cognitive state (e.g., workload) but vary in their accuracy to generalize across contexts, experimental manipulations, and beyond a single session. In our work we address an outstanding challenge in neuroergonomic research: we quantify if (how) identified neural features and a chosen modeling approach will generalize to various manipulations defined by the same underlying psychological construct, (multi)task cognitive workload.
Methods
To do this, we train and test 20 different support vector machine (SVM) models, each given a subset of neural features as recommended from previous research or matching the capabilities of commercial devices. We compute each model's accuracy to predict which (monitoring, communications, tracking) and how many (one, two, or three) task(s) were completed simultaneously. Additionally, we investigate machine learning model accuracy to predict task(s) within- vs. between-sessions, all at the individual-level.
Results
Our results indicate gamma activity across all recording locations consistently outperformed all other subsets from the full model. Our work demonstrates that modelers must consider multiple types of manipulations which may each influence a common underlying psychological construct.
Discussion
We offer a novel and practical modeling solution for system designers to predict task through brain activity and suggest next steps in expanding our framework to further contribute to research and development in the neuroergonomics community. Further, we quantified the cost in model accuracy should one choose to deploy our BCI approach using a mobile EEG-systems with fewer electrodes—a practical recommendation from our work.
Visual attention is one of the most important mechanisms deployed in the human visual system (HVS) to reduce the amount of information that our brain needs to process. An increasing amount of efforts has been dedicated to the study of visual attention, and this chapter proposes to clarify the advances achieved in computational modeling of visual attention. First the concepts of visual attention, including the links between visual salience and visual importance, are detailed. The main characteristics of the HVS involved in the process of visual perception are also explained. Next we focus on eye-tracking, because of its role in the evaluation of the performance of the models. A complete state of the art in computational modeling of visual attention is then presented. The research works that extend some visual attention models to 3D by taking into account of the impact of depth perception are finally explained and compared.
Whether from a theoretical research perspective or a practical situation, mental loads lead to lower productivity, worse job quality, and industrial accidents, and evaluating the mental loads of workers is important for task assignment, job performance, and production safety. In this study, impact sensitivity experiments were used to induce emotions such as fear and tension in participants, thus causing them to develop mental load. Prospective time-distance estimation tests and subjective questionnaires were used to validate the induction experiment. EEG signals from three brain regions and eye movements of the subjects made up four single variables, and combinations of different brain regions and eye movements were used as combined variables. Thus, a total of 15 variables were used for the analysis of mental load states. The Relief algorithm was applied to the 15 variables for feature extraction to construct an optimized feature set. Then, 4 types of classifiers were used to identify mental load states. The combination of occipital area and eye movement variables achieved better mental load assessment results than single variables. With this combination variable, the SVM (94.9 %) and RF (94.3 %) classifiers were effective. However, the SVM classification accuracy gradually increased with the number of training sessions, while the RF classifier did not show this trend. Therefore, the SVM classifier was the best mental load classification method. The results of this study can be used to assess the mental loads of workers, reducing the number of accidents and improving productivity.
As autonomous systems technology becomes more and more powerful and present in almost all aspects of humanHumans life, it is aiding in reducing the mental strain on humansHumans that employ its benefits. An increasing area of interest to do so is in the air traffic control industry, where air traffic controllers are put under immense strain as they make split-second decisions and are responsible for the safety of hundreds, if not thousands, of individuals at any given moment. In this study, a systematic literature review of the intertwining topics of air traffic control automation and mental workload will be analyzed using tools such as Web of Science, Harzing’s Publish or Perish, VOSViewer, and MAXQDAMaxqda. In order to determine the most pertinent papers, a co-citationCocitation analysis will be completed. It was found that increased automation in the air traffic control industry has had very positive effects on air traffic controllers. It has greatly reduced the mental strain by allowing an autonomous system to make some of the decisions in the split-second time that one has to do so, allowing the autonomous system and humanHumans controllers to work hand-in-hand. Given these positive results, it is likely that the systems developed for air traffic controllers will also be applicable to other high strain work environments that require high pace decision making.
Describes an example of combining laboratory- and field-based study to develop a selection battery for field implementation. The procedure provides advantages in comparison with sole use of field-based experimentation, namely, greater control over examinee samples, collection of construct validity data, and fewer field demands for cross-validation. Two experiments were conducted that converge on development of a test battery for selection of air traffic controllers (ATCs). The lab study (
N = 112) used an ATC simulator (terminal radar approach control, or TRACON) for initial development and evaluation of the selection battery. The field study of 206 Federal Aviation Administration ATC trainees provided cross-validation data as a precursor to implementation of the battery. Implications for developing ability-based and self-efficacy-based selection measures for complex job performance are discussed, as are general issues for new selection research and application. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
By means of two-stage, nonlinear multivariate pattern recognition, electroencephalograms (EEG's) were analyzed during performance
of verbal and spatial tasks. Complex scalp distributions of theta-, beta-, and, to a lesser extent, alpha-band spectral intensities
discriminated between the two members of a pair of tasks, such as writing sentences and Koh's block design. Small EEG asymmetries
were probably attributable to limb movements and other uncontrolled noncognitive aspects of tasks. Significant EEG differences
beteeen cognitive tasks were eliminated when controls for inter-task differences in efferent activity, stimulus characteristics,
and performance-related factors were introduced. Each controlled task was associated with an approximately 10 percent reduction,
as compared with visual fixation, in the magnitude of alpha- and beta-band spectral intensity. This effect occurred bilaterally
and was approximately the same over occipital, parietal, and central regions, with some minor difference over the frontal
region in the beta band. With these controls, no evidence for lateralization of different cognitive functions was found in
the EEG.
Substantial controversy exists about ability determinants of individual differences in performance during and subsequent to skill acquisition. This investigation addresses the controversy. An information-processing examination of ability-performance relations during complex task acquisition is described. Included are ability testing (including general, reasoning, spatial, perceptual speed, and perceptual/psychomotor abilities) and skill acquisition over practice on the terminal radar approach controller simulation. Results validate and extend Ackerman's (1988) theory of cognitive ability determinants of individual differences in skill acquisition. Benefits of ability component and task component analyses over global analyses of ability-skill relations are demonstrated. Implications are discussed for selection instruments to predict air traffic controller success and for other tasks with inconsistent information-processing demands.
Twelve subjects (six female) participated in an experiment designed to separate the effects of perceptual/central and physical demands on psychophysiological measures of peripheral nervous system activity. The difficulty of a single-axis continuous manual tracking task was varied in two ways: order of control was manipulated to vary perceptual/central processing demand, and disturbance amplitude was manipulated to vary physical demand. Physiological measures were sensitive to the imposition of a task and were more sensitive to physical than to perceptual/central demands. A principal components analysis identified five factors (three of them physiological) that accounted for 83.1% of the observed variance. Perceptual/central processing demands specifically affected the component identified with sympathetic cardiovascular control, whereas physical demands were reflected in the component identified with parasympathetic cardiovascular control. This finding suggests that dissociations observed among cardiovascular measures in manual performance tasks are attributable to differential activation of the autonomic control systems.
In tasks requiring sustained attention subjects are required to maintain signal features in memory to discriminate unpredictable and infrequent target signals from frequent standards. This could be achieved by permanent, intermittent, or repeated activity. The latter ability appears associated with kinds of temporary signal context activation accompanied by increased EEG-theta. In this study, 18 male subjects had to perform a Mackworth's clock test by watching a pointer move in discrete steps in a circle for 33 min and responding to infrequent double jumps. EEG spectra were obtained from epochs with different center times from 2s before (t(o)-2s) to 2s after (t(o)+2s) signal onset. Spectral band variables were computed by means of a Principle Component Analysis (PCA). EEG-theta was higher in the t(o)-1.5s epochs for responded compared to omitted signals. The larger amount of EEG-theta in an interval preceding the detected signals may reflect the temporary activation of information belonging to features of rare signals as demanded by task instruction. This kind of contextual activation preceding the signals seems to be substantial for target detection in cases where target characteristics are not permanently active in memory.
As part of a continuing investigation of the conditions associated with operating irregularities in air traffic control (ATC), reports prepared during the investigation of 301 operating irregularities were analyzed in order to identify the factors most likely to precipitate air traffic control incidents. Operating irregularities were found to occur more frequently under conditions of moderate or low workload and normal complexity. A second objective of the work has been to develop a database system from which statistical data on operating irregularities and the various identified factors can be extracted. However, in order to enhance the integrity of the descriptive information in the database, the category structure that has been used to record the occurrence of ATC incidents is being revised to reflect the contemporary approaches to human error. A brief description of the proposed structure is included in the present paper.
The Workload Assessment Monitor (WAM) is a PC based system which has two functions: 1) It collects and provides on-line reduction of psychophysiological data, and 2) provides on-line classification of operator state (workload, fatigue, etc.). Each of the physiological measures can provide unique information concerning operator state and by combining these measures there is a pattern that has utility beyond the individual measures.
Three physiological measures of workload; heart rate, eye blink, and EEG were recorded from eight experienced A-7 attack aircraft pilots. Each pilot flew the same familiar training mission three times; one mission in the lead position of a four ship formation and the other as wing, and once in an A-7 simulator. The mission lasted approximately 90 minutes and consisted of take-off, low altitude terrain following, high G maneuvers, inflight navigational updates, weapons delivery, and a high altitude cruise to base, ending in a formation landing. The data show significant differences between simulated and actual flights for all measures. There were also significant differences between mission segments for each pilot. The heart rate data most obviously reflect the changes in workload level throughout the mission and between flight position and simulator. Blink rate and duration were sensitive to changing visual attentional demands. The EEG data showed differences between the actual flight missions and the simulator.
Air traffic control (ATC) system errors rarely occur in today's system. ATC-related aircraft accidents are even rarer events. Nevertheless, more than 90% of all the system errors that do occur stem from human mistakes in attention, judgment, and communications by controllers and their supervisors. Current levels of air traffic are already pressing the capacity of today's system. Future traffic is predicted to increase significantly, but economic and other factors place severe limits on the number of airports and runways that can be built. Resultant traffic congestion and controller work load, despite automation efforts; could increase the error rate and degrade the system's tolerance for errors. Several ATC-related aircraft accidents are summarized to illustrate the controller's changing role and the manner in which the controller interacts with pilots, other controllers, and the work environment. These cases also cite exemplary performance, as well as mistakes, by pilots and controllers, and the circumstances that fostered them, to illustrate the strengths and weaknesses of the human element in the present system. Implications for future ATC system design are discussed.
A study was conducted at the Federal Aviation Administration's Jacksonville en route air traffic control center to identify factors that contribute to airspace complexity. Direct (verbal reports) and indirect (multidimensional scaling) procedures were used to identify potential factors. A list of candidate factors was created by combining the data from the direct and indirect knowledge exploration methods. Important complexity factors were identified by determining their simple and multiple correlations with overall sector complexity as judged by a group of Traffic Management Unit personnel (flow controllers). A final list of 16 complexity factors was developed and is suggested as a reference for future research in the area. An evaluation of the knowledge extraction techniques indicated that, although little unique information was generated by the indirect procedure, it was useful for the identification of complexity factors when combined with data from direct sources. Further research to validate the identified sector complexity factors is recommended.
The Federal Aviation Administration has embarked on a major curriculum redesign effort to improve the training of en route air traffic controllers. Included in this effort was a cognitive task analysis. One component of the task analysis was an analysis of operational errors, to obtain insights into cognitive-perceptual factors contributing to controller decisionmaking error. The data suggest that a failure to maintain situation awareness is the primary cause of controller error. These results highlight the importance of the controller task 'maintain situation awareness', and are consistent with the findings of the other analyses. An approach for training situation awareness skills is presented in relation to models of expertise developed from other analyses: an expert mental model of air traffic control, and a task decomposition listing thirteen primary controller tasks. The findings and training paridigm have implications for training other complex high-performance tasks performed in a real-time, multi-tasking environment.
At present the most used and probably the most reliable methods for assessing pilot workload in flight are based on some form of subjective reporting by experienced test pilots. Unfortunately, subjective opinions are susceptible to bias and preconceived ideas and so many occasionally result in false estimates of workload. For more than fifteen years subjective reporting by pilots at RAE Bedford has been augmented by recording their heart rates. At first pilots described workload in a relatively unstructured manner but the need for some form of rating scale was soon apparent. After much trial and error and with the valuable assistance of practising test pilots a ten-point rating scale using the concept of spare capacity was developed (fig 1). The overall design is based on the Handling Qualities Rating Scale of Cooper and Harper already familiar to Bedford test pilots and sometimes used previously, though mistakenly, to rate workload. During the last eight years a number of flight trials at Bedford, including the Harrier 'ski-jump' take-off trial and the Economical Category 3 landing trials, have used pilot ratings and heart rate responses to assess workload. The rationale for using heart rate in assessing pilot workload is based on the concept of neurological arousal. Flying an aeroplane, especially during the more difficult manoeuvres, requires the pilot's brain to collect, filter and process information quickly, to exercise judgement and make decisions, and to initiate rapid and appropriate actions.
The results of a multi-year research program to identify the factors associated with variations in subjective workload within and between different types of tasks are reviewed. Subjective evaluations of 10 workload-related factors were obtained from 16 different experiments. The experimental tasks included simple cognitive and manual control tasks, complex laboratory and supervisory control tasks, and aircraft simulation. Task-, behavior-, and subject-related correlates of subjective workload experiences varied as a function of difficulty manipulations within experiments, different sources of workload between experiments, and individual differences in workload definition. A multi-dimensional rating scale is proposed in which information about the magnitude and sources of six workload-related factors are combined to derive a sensitive and reliable estimate of workload.
The respiratory responses of experienced military pilots have been studied during flight in a high performance jet aircraft. The importance and relevance of information about these responses have been reviewed, with particular emphasis in the difficulties of in-flight recording and the history of such experimentation in four specific areas of respiratory physiology: respiratory frequency and flow, added external resistance, hyperventilation and the metabolic cost of flying. In the present study, respiratory variables were measured continuously using an airborne system which recorded the output from physiological and aircraft instrumentation. In order to approach normal respiratory behaviour more closely, a specially designed low resistance breathing system was developed and used in conjunction with a modified infra-red carbon dioxide analyser. A unique feature of the latter, also specially developed, was the ability to calibrate the device during flight. Inspiratory flows (and hence volumes) and expired carbon dioxide tensions were recorded on several occasions in place of carbon dioxide analysis. Three set flight profiles (two general handling and one simulated combat) were precisely defined to allow comparability between subjects. A fourth, less structured but high workload sortie, involving air combat maneuvering (ACM), was flown on two occasions.
An experiment was carried out to investigate the hypothesis that task difficulty is reflected in changes in the topographical distribution of the ongoing EEG. Subjects had to perform three different tasks at two difficulty levels each; the Sternberg memory scanning task in an auditory and in a visual mode and a task whose performance required mainly visual scanning. Task difficulty was verified by the measurement of response times. Using a commercial Brain Electrical Activity Mapping device, EEG was recorded from 19 scalp electrodes while the subjects performed the tasks. Spectral matrices of the EEG were calculated to investigate spatial relationships in the EEG. Compared to the lower level, higher task difficulty resulted in EEG changes that led to the identification of two factors. One was the reduction of parietal and occipital alpha activity due to the amount of visual scanning and the other an increase of theta activity in the left frontal electrodes which may be associated with the amount of general mental processing.
With the use of advanced equipment, respiratory measures can unobtrusively and reliably be assessed in a variety of psychophysiological research settings. New computerized analysis techniques can break down respiration into a number of components that provide valid estimates of variations in respiratory control mechanisms in the brain stem. Thus analysed, respiratory responses may vary in at least two dimensions: (A) with regard to drive and timing aspects, and (B) with regard to the metabolic appropriateness of the respiratory response. Assessment of respiratory responses may be relevant for a broad variety of research areas, including studies of the physiological effects of mental load and stress, investigations of physiological correlates of emotions and affect, and research linking physiological responses to subjective distress and psychosomatic disorders.
Cardiac and respiratory measures can be successfully applied to "real world" environments and these measures have certain advantages over both performance and subjective measures that are typically used to monitor operator state and workload. However, because of large differences between laboratory and "real world" environments one must utilize caution in directly applying laboratory data and theories to the day-to-day world environment. While most workers are highly over-trained in their jobs, laboratory subjects are often under-trained in the cognitive tasks that are used to study cognitive activity. It is possible that a substantial portion of experimental effects reported in laboratory studies is due to learning effects. In addition, relatively small changes in cardiac and respiration measures are reported to experimental manipulations in the laboratory while a much larger range of changes are reported in "real world" environments. These differences highlight questions about laboratory/real world similarities and the need to develop a database of actual work environment data. A third area of concern is the relative lack of control over the experimental situation that is the case with most applied research. The possible confounding of changes due to cognitive and physical activity levels is a major concern and strategies for overcoming these problems are suggested. The potential for valuable contributions by cardiac and respiratory measures to applied research make overcoming these difficulties worthwhile.
This study examined the effects of memory search and related processes on both time and frequency domain components of electroencephalographic activity. More specifically, we were interested in the relationship between EEG and event-related potential (ERP) components as a function of memory load and response type. Subjects performed a semantic memory search task in which they matched word probes to category labels. Consistent with previous studies, reaction time increased and accuracy decreased with increasing memory loads. A negative component of the ERP (N400) was found to reflect semantic mismatch: N400s were larger for the nontargets than for the targets. Two ERP components were found to be reciprocally related to memory load. P300 decreased and Negative Slow Wave increased in amplitude with increases in the size of the memory set. These two ERP components were reflected by different components in a Principal Components Analysis. The power in the theta band (5-7 Hz) also increased as a function of memory load and appears to be functionally and topographically related to the Negative Slow Wave in the ERP. It is argued that both measures are jointly determined and reflect the difficulty of the conceptual operations during memory search.
Discriminant analysis techniques were used to classify 8 flight segments for 19 F4 crewmembers and to classify pilots from Weapons Systems Officers (WSOs). Heart rate and eye blink data were used as variables. Pilots and WSOs were correctly classified 84% of the time, while 92% of the flight segments were correctly classified for pilots and 89% for WSOs. The percent correct classifications of flight segments using the jackknife procedure were 69% and 68%, respectively. The advantage of collecting multiple physiological signals was demonstrated. Combined cardiac and eye blink data produced better classifications than when each was used alone. Application of this technique to the flight environment is discussed.
The relationship between EEG and vigilance was studied in subjects performing tasks of differing complexity over a 15 h period by examining the structural changes of the EEG as vigilance deteriorates. Calculation of canonical variates indicated that two directions explained the changes in vigilance, though one direction was dominant across subjects and tasks, and likely to reflect decreased arousal. The second direction indicated some evidence for a second underlying change in the EEG related to vigilance in some subjects particularly in the more complex task, and this would reflect mechanisms other than drowsiness. The relationship between EEG and vigilance was modelled by fitting a linear function of EEG variables to the proportion of missed responses. The coefficient of multiple correlation was significant for most subjects, and the relationship is likely to represent changes in arousal. The most useful discriminator of worsening vigilance common to both tasks was beta activity (14-21 Hz).
Event-related spectra of short EEG epochs were investigated in a concept formation paradigm. In this task, subjects had to learn to transform letters into Morse codes. Related to a resting state, cognitive performance in this learning task was characterized (i) by an increased mean power density (MPD) within the Theta frequency band (theta) in recordings over the frontal lobes and (ii) by a reduced Alpha (alpha) MPD in all recording sites. The performance-related increase of the theta-MPD, as obtained in left frontolateral and frontomedial recordings, separated (i) the learning task from an appropriate control and (ii) a group of successful learners from a group which performed less efficiently. MPDs of the alpha- and the delta-(Delta) frequency band did not differ between tasks and groups. A consistent finding in the learning tasks was a temporal dissociation of performance-related alpha-attenuation between parietal and frontal recordings: in the period preceding the presentation of the informative stimuli, the alpha-rhythm is attenuated especially at parietal recordings, whereas in frontal recordings, alpha-attenuation accompanied information processing and response selection.
Heart rate, oxygen consumption, and respiratory activity were recorded while 18 young male subjects performed a stressful mental arithmetic task and played a video game. Measurements were also taken while subjects undertook a graded static leg lifting exercise. Physiological activity increased as an orderly function of exercise workload. For each subject, heart rate was plotted against oxygen consumption over the various exercise loads. Knowing oxygen consumption during the psychological tasks, these regression equations allowed the prediction of heart rate, and thus the computation of “additional” heart rate as the difference between actual and predicted heart rate. Overall predicted heart rate values were significantly less than the values actually recorded during the psychological tasks. However, whereas mental arithmetic was associated with “additional” heart rates of the same order as those observed in earlier research using graded dynamic exercise, the video game elicited decidedly less “additional” heart rate in the present case. The explanation for this lay in the regression lines; static exercise was characterized by heart rate-oxygen consumption regressions which were much steeper in terms of slope, as well as somewhat lower in terms of intercept, than those observed with graded dynamic exercise.
Heart rate and various metabolic and ventilatory indices were monitored while 24 young males engaged in two psychologically challenging tasks, mental arithmetic and Raven's matrices. Each task was structured to present subjects with three levels of difficulty: easy, hard, and impossible. Measurements were also made while subjects undertook graded isotonic exercise on a bicycle ergometer; for each subject, heart rate was plotted against oxygen consumption over the various exercise loads. Knowing oxygen consumption during the psychological tasks, these regression equations permitted the calculation of expected heart rates during each task condition, and thus the computation of ‘additional heart rate’ as the difference between actual and predicted heart rate values. Cardiac activity, whether represented as additional heart rate or as the difference between resting and task heart rate levels, was sensitive to variations in difficulty level. In both tasks the easy condition elicited significantly less cardiac activity than both the hard and the impossible conditions. Changes in oxygen consumption and carbon dioxide production did not vary significantly with difficulty. Subjects' self-reports of active engagement and arousal paralleled the cardiac effects; in both tasks the easy condition was experienced as relatively unengaging and unarousing.
Heart rate, plus various metabolic and ventilatory indices, were monitored while 20 young male subjects were exposed to a video game and a stressful mental arithmetic task. Measurements were also made while subjects undertook graded isotonic exercise. All measures changed as a function of psychological challenge, and during exercise physiological activity increased as an orderly function of workload. For each subject, heart rate was plotted against oxygen consumption over the various exercise loads. For the majority of subjects the analogous data points for the video game and mental arithmetic lay reliably above the exercise heart rate-oxygen consumption regression lines. When these regression lines were used to predict heart rate values during psychological challenge, the predicted values were significantly less than the values actually recorded for both tasks; although the discrepancy between predicted and actual values was on average greater with mental arithmetic, the difference was not statistically reliable. Pre-stressor baseline conditions were also associated with heart rate levels greater than predicted, albeit to a lesser extent. Finally, while both stressors produced heart rate adjustments additional to expectancies, inter-task consistency was low.
Relationships between mental activity and EEG were studied by the method of multivariate statistical analysis. All 15 different kinds of subtests were given by a 2 min time-limit method to 60 male juvenile delinquents. To convert the EEG into numerical form, frequency analysis of the EEG (Fz-Cz lead) was carried out during performance of the subsets, and amplitude of the integrated values of 5 frequency bands (δ-, θ-, α-, β1-, β2-band) was recorded for every 5 sec. 1. 1. The θ activity during mental performance seemed to originate in the frontal midline area and tended to appear in the subtests that required continous concentration of attention. 2. 2. As the result of factor analysis of the subtests' scores, four significant factors were extracted: they could be named ability of simple arithmetic and mechanical memory (factor 1), complex arithmetic and rule identification ability (factor 2), ability of discrimination and induction (factor 3), perceptual ability of simple visual patterns (factor 4). 3. 3. Factor analysis of the EEG band scores proved that each σ-, α-, β1- and β2-band component of EEG was of one-factor structure, all the correlation coefficients being approximately +0.9 at each. 4. 4. As the result of factor analysis at the θ-band scores of EEG, two significant factors were extracted. They were interpreted concerning to thinking (factor 1) and work speed (factor 2).
The work load concept is discussed in connection with the flight task, which is considered to lead primarily to mental load. Experiments in a DC7 simulator and an AT100 (Beachcraft) simulator, in which it was tried to validate heart rate (HR), heart rate irregularity (HI) and respiratory rate (RR) as indicators of mental load are described. As a criterion the following scale was used, from least to most difficult task: rest; level flight; take off; approach. The overall differences proved to be significant at the 0.1% level. The changes in physiological variables from phase to phase showed a correlation of 0.80 with the predicted direction. The highest discriminating power according to the ω2 criterion was shown by RR, followed by HI and HR, in this order. During an experiment with parachute jumpers, in which an anxiety scale was used as a criterion, the reversed order was found.
Mental load in ATC tasks is described as the brain controlling the controller's controlling performance. A moment of conscious brain control is put forward as a unit to quantify this kind of mental load. New action programs are supposed to require conscious brain control at every step of their execution; with routine this would be less and less the case. The duration of a moment of conscious control varies according to the complexity of the control to be exercised and the number of considerations which have to be taken into account. Propositions are made on how to think about selective attention, identification and cognitive processes in terms of executing programs. A job description method is put forward in terms of such executing programs.
A new off-line procedure for dealing with ocular artifacts in ERP recording is described. The procedure (EMCP) uses EOG and EEG records for individual trials in an experimental session to estimate a propagation factor which describes the relationship between the EOG and EEG traces. The propagation factor is computed after stimulus-linked variability in both traces has been removed. Different propagation factors are computed for blinks and eye movements. Tests are presented which demonstrate the validity and reliability of the procedure. ERPs derived from trials corrected by EMCP are more similar to a 'true' ERP than are ERPs derived from either uncorrected or randomly corrected trials. The procedure also reduces the difference between ERPs which are based on trials with different degrees of EOG variance. Furthermore, variability at each time point, across trials, is reduced following correction. The propagation factor decreases from frontal to parietal electrodes, and is larger for saccades than blinks. It is more consistent within experimental sessions than between sessions. The major advantage of the procedure is that it permits retention of all trials in an ERP experiment, irrespective of ocular artifact. Thus, studies of populations characterized by a high degree of artifact, and those requiring eye movements as part of the experimental task, are made possible. Furthermore, there is no need to require subjects to restrict eye movement activity. In comparison to procedures suggested by others, EMCP also has the advantage that separate correction factors are computed for blinks and movements and that these factors are based on data from the experimental session itself rather than from a separate calibration session.
The effect of distance driven on three physiological variables taken to be indicators of fatigue was investigated on a 340 km highway circuit with eight inexperienced drivers as subjects. The physiological variables used were spectral values of heart rate variability in the 0.05-0.15Hz region (0.1 Hz HRV) supplemented by standard deviation of heart rate mean (S.D. HRV) and heart rate mean (HR). The analysis showed a significant relationship between 0.1 Hz HRV and distance driven while S.D. HRV and HR showed no direct relationship. The reason for this differential effect on the three physiological variables was discussed, and it was concluded that 0.1 Hz HRV seems to be a sensitive indicator of driver fatigue.
The effects of arithmetic task difficulty and sex of subject on parietal alpha activity (8-13 Hz) and alpha asymmetry (RH-LH/LH + RH) were investigated using a series of eyes closed self-generated tasks. Sixteen right-handed subjects (8 males, 8 females) were asked to carry out sets of successive addition and multiplication tasks divided into three levels of difficulty. In comparison to a simple counting task, more difficult tasks showed lower alpha power. All tasks except the counting condition and the successive addition of 3's task (right hemisphere only) were associated with below-baseline levels of alpha activity. When laterality ratios were considered, all addition tasks and the easy multiplication task were found to be relatively left lateralized in relation to baseline and spatial task values. A sex by task interaction, however, was detected. Males exhibited relative left hemisphere activation for the more difficult addition task and the multiplication task of medium difficulty. Females, on the other hand, showed relative left lateralization for the easy tasks and one of the more difficult addition tasks. When the harder arithmetic tasks were repeated, females showed an increase in relative left hemisphere activation for one of the more difficult multiplication tasks. Males computed the problem sets significantly faster than females. These data suggest that the effects of difficulty may interact with ability such that ability may determine the maximum level of relative hemisphere activation achieved within a given difficulty range. While no absolute differences in asymmetry were found between the sexes, females tended to show greater asymmetry differences between spatial and arithmetic tasks. Finally, no significant relationship was found between the degree of alpha asymmetry and performance measures.
EEG from 19 electrodes was used to classify which of 14 tasks each of seven subjects had performed. Stepwise discriminant analysis (SWDA) was used to classify the tasks based upon training on one half of the spectrally analyzed 1 min of data. Eighty six percent correct classification was achieved using principle components analysis (PCA) to determine the EEG bands to be used by the SWDA. Other approaches to deriving the EEG bands met with lower levels of success. The results indicate that frequency and topographical information about the EEG provides useful knowledge with regard to the nature of cognitive activity. Higher frequencies provided much of the information used by the classifier. The utility of this approach is discussed with regard to evaluating operator state in the work environment.
This study examined 20 air traffic controllers from the Rome Regional Air Control Centre for three successive work shifts: afternoon (13:00-20:00), morning (07:00-13:00) and night (20:00-07:00). The number of aircraft under control per hour was recorded as index of workload. Recordings involved subjective ratings (mood, physical fitness, fatigue) and objective measures (heart rate, vanillyl mandelic acid excretion, reaction times, critical flicker fusion, oral temperature). In addition, the subjects filled out questionnaires for personality traits (extroversion, neuroticis, anxiety) and behavioural characteristics (morningness, rigidity of sleeping habits, vigourness to overcome drowsiness). The volume of air traffic varied greatly with peaks during the day and low levels at night. Nevertheless, the heart rates of the group members showed quite constant levels in all the three shifts, irrespective of the workload. The same pattern appeared in the controllers' excretion of VMA, which remained high during both day and night shifts, regardless of the reduced workload. The subjective mood and physical fitness decreased similarly, while feelings of fatigue increased on all three shifts, particularly on the night shift. The circadian rhythm of the oral temperature showed a slight modification of the nocturnal depression during the night shift, caused by the state of awakeness and activity. However, the rhythm was not altered in its normal circadian phase, due to the fast shift rotation adopted. The psychophysiological responses were affected by personal characteristics, in particular morningness and ability to overcome drowsiness.
Psychophysiological measures are used to assess the workload of F4 Phantom aircraft pilots and weapon systems officers (WSOs) during air-to-ground training missions and during the performance of two levels of difficulty of a laboratory tracking task. The bombing range portion of the missions was associated with the highest pilot workload, while the WSO flying the aircraft was the highest workload segment for the WSOs. The pilots' data were found to have a wider range of values for the physiological measures than were found in the WSO data. The different levels of tracking task difficulty produced significant physiological effects but the range of values found for most of the flight segments were much greater. These data demonstrate that extrapolating laboratory data to the flight environment is risky at best. The various physiological measures were differentially sensitive to the different demands of the various flight segments.
Psychophysiological assessment methods
- Caldwell
ATC control and communications problems: an overview of recent ASRS data
- Morrison
Dynamic methods for assessing workload for minimum crew certification
- Speyer
Respiration and stress
- Wientjes
The frontal theta rhythm indicating motor and cognitive learning. Current trends in event-related potential research
- Lang
Towards an objective assessment of cockpit workload: I. Physiological variables during different flight phases
- Opmees
The effect of endogenous alpha on hemispheric asymmetries and the relationship of frontal theta to sustained attention
- Pigeau
PATS: Psychophysiological Assessment Test System
- Wilson