Figure 1 - uploaded by Michael D. Byrne
Content may be subject to copyright.
Source publication
Routine procedural errors are facts of everyday life but have received little empirical study and have eluded prediction. Leading frameworks for thinking about such errors have not been successful in generating predictions, either. This paper describes the desiderata for error prediction, and notes that the MHP was a step in the right direction. Be...
Context in source publication
Similar publications
Accidents analyses disclosed that critical auditory stimuli such as alarms could fail to reach awareness, with devastating consequences. Previous research indicated that it is possible to perform single trial classification to detect this inattention phenomenon using a 32-wet-electrode Biosemi EEG. Nevertheless, such systems are bulky and difficult...
Citations
... 14,15 Order Errors/Task Sequence Procedural tasks (e.g., MARCH) require an individual to follow a certain sequence of actions. 16,17 High demands on attention and low automaticity mark the early phases of skill acquisition. 18 Novices do not have easy recognition of the applicable situation, nor do they easily recall the relevant steps and order they should follow, and this need to effortfully recognize and recall leads to higher mental demands on those less-practiced. ...
Introduction
The brevity of training for soldiers and combat medics to learn how to provide treatment on the battlefield may restrict optimal performance for treating chest and airway injuries, particularly when treating female soldiers. The present study tested treatment performance on patient simulators by battlefield medic trainees to determine whether there is a need for more extensive training on chest and airway procedures on female soldiers.
Materials and Methods
Battlefield medic trainees treated male and female patient simulators in counterbalanced order. The assessment considered the effects of patient gender and order on procedures performed, particularly critical chest and airway interventions such as needle chest decompression (NCD), and considered the appropriate order of treatment tasks. Four coders rated video footage of three simulated procedures, i.e., tourniquet, chest seal (front and back application), and NCD, using a binary coding system to determine completeness and order correctness according to the Massive hemorrhage, Airway, Respiration, Circulation, and Head injury/Hypothermia (MARCH) mnemonic.
Results
Results from analysis of variance showed that when presented with a female patient first, trainees performed significantly fewer total procedures on both the female and male simulators. More experienced trainees completed significantly more procedures compared to trainees with minimal experience. Results from the binary logistic regression showed that trainees with more experience and trainees presented with the male patient simulator first performed significantly more procedures in the correct order. Finally, an examination of the NCD procedure found that trainees presented with the female patient simulator first had more errors and that trainees with less experience were less likely to perform the procedure adequately.
Conclusions
The findings suggest that treating a female patient first may lead to undertreatment of both patients. Furthermore, the observed differences in treating sensitive areas of the body (e.g., near female breasts) suggest providing greater opportunities for trainees to practice often missed or incorrectly performed procedures. Treating a female patient remains a novel experience for many trainees, such that trainees are less likely to fully treat a female patient and are less likely to treat female soldiers for the most life-threatening injuries. In fact, the initial presentation of the female patient simulator appeared to affect experienced trainees, suggesting that removing the experience of novelty and stress requires more extensive exposure and alternative training. The study’s small sample size with a wide range of trainee experience may limit the findings, which may fail to capture some study effects. Finally, the study did not request trainees’ experience treating female soldiers, so future studies should examine the extent to which experience is predictive of performance. There is a need for more interactive approaches in patient simulations to provide opportunities for practice, especially those that require the treatment of sensitive areas.
... There have been a number of studies in constructing the cognitive model of post-completion errors [5,23,27,33] and identifying factors that provoke or mitigate the occurrence of these errors [2,4,7,20,24,34]. However, most of these studies are conducted in procedural tasks rather than problem-solving tasks [19]. ...
Post-completion errors have been observed in a variety of tasks by psychologists, but there is a lack of empirical studies in software engineering. This paper investigates whether post-completion errors occur in software development and the likelihood that software developers commit this error when a post-completion scenario is presented. An experimental study was conducted in the context of a programming contest. In the experiment, a programming task specification that contained a post-completion sub-task requirement was presented to the subjects. The results showed that 41.82% of the subjects committed the post-completion error in the same way—forgetting to design and implement a software requirement which is supposed to be the last sub-task and is not necessary for the completion of the main sub-task. This percentage of subjects committing the post-completion error was significantly higher than that of subjects committing other errors. This study has confirmed that post-completion error occurs in software development and, moreover, different software developers tend to commit this error in the same way with a high likelihood at the location where a post-completion scenario is presented. Strategies are proposed to prevent post-completion errors in software development.
... There have been a number of studies in constructing the cognitive model of post-completion errors [5,23,27,33] and identifying factors that provoke or mitigate the occurrence of these errors [2,4,7,20,24,34]. However, most of these studies are conducted in procedural tasks rather than problem-solving tasks [19]. ...
Post-completion errors have been observed in a variety of tasks by psychologists, but there is a lack of empirical studies in software engineering. This paper investigates whether post-completion errors occur in software development and the likelihood that software developers commit this error when a post-completion scenario is presented. An experimental study was conducted in the context of a programming contest. In the experiment, a programming task specification that contained a post-completion sub-task requirement was presented to the subjects. The results showed that 41.82% of the subjects committed the post-completion error in the same way---forgetting to design and implement a software requirement which is supposed to be the last sub-task and is not necessary for the completion of the main sub-task. This percentage of subjects committing the post-completion error was significantly higher than that of subjects committing other errors. This study has confirmed that post-completion error occurs in software development and, moreover, different software developers tend to commit this error in the same way with a high likelihood at the location where a post-completion scenario is presented. Strategies are proposed to prevent post-completion errors in software development.
... As our work demonstrates, understanding errors at the level of the interface requires consideration of all aspects of human cognition. For this reason, cognitive modeling has been suggested as a solution (see Byrne, 2003;Gray, 2004). Those studying eye movements in reading (e.g., Rayner, 1998) have converted massive collections of data into models that may be iteratively tested and validated. ...
Postcompletion errors, which are omissions of actions required after the completion of a task's main goal, occur in a variety of everyday procedural tasks. Previous research has demonstrated the difficulty of reducing their frequency by means other than redesigning the task structure [Byrne, M.D., Davis, E.M., 2006. Task structure and postcompletion error in the execution of a routine procedure. Human Factors 48, 627–638]. Nevertheless, finding a successful strategy for mitigation of this type of error may uncover important mechanisms underlying interactive behavior. Two experiments were carried out to test visual cues for their ability to reduce the frequency of postcompletion errors in a computer-based routine procedural task. A cue that was visually salient, just-in-time, and meaningful entirely eliminated the error, whereas cues that were not as specific were ineffective. These results are beyond the predictive capability of extant error identification methods and common design guidelines but are consistent with the work of Altmann and Trafton [2002. Memory for goals: an activation-based model. Cognitive Science 26, 39–83] and Hollnagel [1993. Human Reliability Analysis, Context and Control. Academic Press, London]. Finally, a computational model developed in ACT-R is presented as a first step towards validation of the major findings from the two experiments.
... Computational cognitive architectures such as ACT-R, EPIC or SOAR provide such models (see Byrne, 2003a, for a review). Thus, a cognitive architecture, properly modified, could serve as a basis for a predictive model of human error (Byrne, 2003b). Our research pursues this approach to develop a Human Error Modeling Architecture (HEMA). ...
Although computational cognitive architectures have been applied to the study of human performance for decades no such architecture for modeling human errors exists. We have undertaken the development of a Human Error Modeling Architecture (HEMA), building on the ACT-R cognitive architecture. In developing HEMA we first set the context of what error types HEMA would handle and what overall cognitive performance process (a Framework for Human Performance) was being assumed. We then identified the cognitive functions which were failing and how they are failing when an error occurs. An analysis of these failures, in relation to the Framework, enabled us to specify a set of General Error Mechanisms. Comparison of these mechanisms to existing ACT-R mechanisms identified where ACT-R could be used, where modifications where necessary and where new mechanisms or modules would be need to develop HEMA. A conceptual design for HEMA was then proposed.
... We therefore think that this analysis could be applied quite broadly to understanding the generation and detection (or failure of detection) of errors in many rapid, fluent performances. Byrne (2003) has noted that most analyses of routine procedural errors have been based on efforts to classify naturally occurring errors rather than on hypothesizing specific mechanisms that might generate particular types of errors in specific situations. The model depicted in Figure 1 and the empirical analyses guided by that model are descriptive rather than computational. ...
Event counting depends on simple, well-learned knowledge but is effortful and error-prone. In 6 experiments, the authors examined event-counting performance, testing a model that suggests that counting is controlled by minimal goal representations coordinated with perceptual events by temporal synchrony. In Experiment 1, they examined self-paced counting with or without delays that disrupted participants' preferred pacing. In subsequent experiments, participants counted computer-paced events occurring at rhythmic or varied intervals, reporting or verifying totals. Several results support the model: Participants counted rhythmic events more accurately, made undetected undercount errors when counting rhythmic events, and made false alarms to undercount or overcount probes presented at different times. These results suggest that intentions that guide fluent counting specify parameters deictically rather than semantically and that error monitoring is implicit.
... This obviously raises the question of what kind of control structure could account for the differences between these two tasks? Accounting for the error profiles seems extremely difficult with any model at this point; generative theories of error are in their infancy at best (though that is ultimately our goal, see also Byrne, 2003). Thus, we entered into a modeling exploration with the modest goal of trying to understand what drove the step completion times. ...
A major domain of inquiry in human-computer interaction is the execution of routine procedures. We have collected extensive data on human execution of two procedures which are structurally isomorphic, but not visually isomorphic. Extant control approaches (e.g., GOMS) predicts they should have the same execution time and error rate profiles, which they do not. We present a series of ACT-R models which demonstrate that control of visual search is likely a key component in modeling similar domains.
Development of an electronic user manual/checklist for a steam-generating power plant boiler start-up is described. Human factors guidelines for development of effective instructional materials were reviewed. Subject matter experts (experienced operators at the power plant) were used to gain requisite knowledge in the procedures and tasks involved in start-up of a steam boiler. Functions of the main components in the power plant and boilers were studied in relevant literature. The new manual/checklist includes description of the system components as part of operative actions and separate actions are arranged into categorical groups, each associated with informative headings. Flowcharts are used to guide users through complex procedures and pictures of critical plant components are provided. All sentences to explain actions are simple and have fewer than 20 words. Several operators have checked and proofread the new manual multiple times and agreed that its contents are accurate and error-free.
Even when executing routine procedures with relatively simple devices, people make nonrandom errors. Consequences range from the trivial to the fatal, with Navy personnel often operating at the more extreme end of this range. This problem has received surprisingly little attention from cognitive psychologists. The research summarized here examines such errors in some detail both empirically and through computational cognitive modeling. There were several key results. First, many such errors are sensitive not just to the structure of the task but also to the layout of controls and displays, contrary to the predictions of most current task analysis frameworks. Some such errors seem to be mitigable by simple layout changes. Second, a particularly pervasive error (termed postcompletion error) was found to be highly resistant to cue-based mitigation, and though an effective cue was found, the requirements for such cues are difficult to meet in field contexts. Finally, cognitive computational models constructed using the ACT-R cognitive architecture suggested that certain interface manipulations (removing state information, adding additional extraneous controls) which appeared major would actually have limited impact on human task performance, and these predictions were validated empirically.
