Dylan Schmorrow

Soar Technology, Inc. · SoarTech

The main goal of the field of augmented cognition is to research and develop adaptive systems capable of extending the information management capacity of individuals through computing technologies. Augmented cognition research and development is therefore focused on accelerating the production of novel concepts in human-system integration and inclu...

This book constitutes the proceedings of the 15th International Conference on Augmented Cognition, AC 2021, held as part of the 23rd International Conference, HCI International 2020, held as a virtual event, in July 2021. The total of 1276 papers and 241 posters included in the 36 HCII 2021 proceedings volumes was carefully reviewed and selected fr...

This book constitutes late breaking papers from the 23rd International Conference on Human-Computer Interaction, HCII 2021, which was held in July 2021. The conference was planned to take place in Washington DC, USA but had to change to a virtual conference mode due to the COVID-19 pandemic. A total of 5222 individuals from academia, research inst...

This book constitutes late breaking papers from the 22nd International Conference on Human-Computer Interaction, HCII 2020, which was held in July 2020. The conference was planned to take place in Copenhagen, Denmark, but had to change to a virtual conference mode due to the COVID-19 pandemic. From a total of 6326 submissions, a total of 1439 pape...

This study explored the chaotic properties of human emotions as expressed in social media and its implications for attainable forecasting horizons. Three human emotional states extracted from Twitter were analyzed using the nonlinear dynamics approach. The greatest positive Lyapunov exponent (LE) and 0-1 test methods were applied to a time series s...

This book constitutes the refereed proceedings of 14th International Conference on Augmented Cognition, AC 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, in July 2020. The conference was planned to be held in Copenhagen, Denmark, but had to change to a virtual conference mode due to the COVID-19 pa...

This book constitutes the refereed proceedings of 14th International Conference on Augmented Cognition, AC 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, in July 2020. The conference was planned to be held in Copenhagen, Denmark, but had to change to a virtual conference mode due to the COVID-19 pa...

Human-AI teaming refers to systems in which humans and artificial intelligence (AI) agents collaborate to provide significant mission performance improvements over that which humans or AI can achieve alone. The goal is faster and more accurate decision-making by integrating the rapid data ingest, learning, and analyses capabilities of AI with the c...

ABSTRACT This article aims to investigate the Grand Challenges which arise in the current and emerging landscape of rapid technological evolution towards more intelligent interactive technologies, coupled with increased and widened societal needs, as well as individual and collective expectations that HCI, as a discipline, is called upon to address...

This book constitutes the refereed proceedings of the 13th International Conference on Augmented Cognition, AC 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCII 2019, in Orlando, FL, USA in July, 2019. The 1274 full papers and 209 posters presented at the HCII 2019 conferences were carefully reviewed and...

The 20th International Conference on Human-Computer Interaction, HCI International 2018, was held in Las Vegas, NV, USA, during July 15–20, 2018. The event incor- porated the 14 conferences/thematic areas listed on the following page. A total of 4,373 individuals from academia, research institutes, industry, and gov- ernmental agencies from 76 coun...

This two-volume set LNCS 10915 and 10916 constitutes the refereed proceedings of the 12h International Conference on Augmented Cognition, AC 2018, held as part of the 20th International Conference on Human-Computer Interaction, HCII 2018, in Las Vegas, NV, USA in July 2018. The 1171 papers presented at HCII 2018 conferences were carefully reviewed...

In current military operations, a team of human operators, often distributed across multiple locations, is required to manage even a single Unmanned Vehicle (UxV). For future multi-UxV control, effective dynamic task sharing strategies – the ability to quickly re-assign tasks and responsibilities between operators or between operators and autonomou...

Humans tend to represent and to understand the world in terms of stories, while computer reasoning tends to require formal, mathematical representations. This paper describes a research prototype that enables computers to parse human stories and use collections of those stories to inform causal modeling of political, military, economic, social, inf...

This volume constitutes the proceedings of the 11th International Conference on Augmented Cognition, AC 2017, held as part of the International Conference on Human-Computer Interaction, HCII 2017, which took place in Vancouver, BC, Canada, in July 2017. HCII 2017 received a total of 4340 submissions, of which 1228 papers were accepted for publicati...

This volume constitutes the proceedings of the 11th International Conference on Augmented Cognition, AC 2017, held as part of the International Conference on Human-Computer Interaction, HCII 2017, which took place in Vancouver, BC, Canada, in July 2017. HCII 2017 received a total of 4340 submissions, of which 1228 papers were accepted for publicati...

This volume constitutes the refereed proceedings of the 10th International Conference on Foundations of Augmented Cognition, AC 2016, held as part of the 18th International Conference on Human-Computer Interaction, HCII 2016, which took place in Toronto, Canada, in July 2016. HCII 2016 received a total of 4354 submissions, of which 1287 papers were...

This volume constitutes the refereed proceedings of the 10th International Conference on Foundations of Augmented Cognition, AC 2016, held as part of the 18th International Conference on Human-Computer Interaction, HCII 2016, which took place in Toronto, Canada, in July 2016. HCII 2016 received a total of 4354 submissions, of which 1287 papers were...

Nikola Tesla was one of the greatest inventors of all time. His life was full of interesting twists and turns. He is most famous for inventing the Alternating Current motor, Tesla Coil, and the Bladeless Turbine, to name a few. Other inventors inspired him, such as, Thomas Edison and George Westinghouse. His trust in the people that inspired him ev...

This book constitutes the proceedings of the 9th International Conference on the Foundations of Augmented Cognition, AC 2015, held as part of the 17th International Conference on Human-Computer Interaction, HCII 2015, which took place in Los Angeles, CA, USA, in August 2015. HCII 2015 received a total of 4843 submissions, of which 1462 papers and 2...

The past several years have seen significant advances in research and development across the human social culture behavior modeling domain. In order to document and share these advances, The MITRE Corporation has produced Sociocultural Behavior Sensemaking: State of the Art in Understanding the Operational Environment. This edited volume features c...

This book constitutes the refereed proceedings of the 5th International Conference on Augmented Cognition, AC 2013, held as part of the 15th International Conference on Human-Computer Interaction, HCII 2013, held in Las Vegas, USA in July 2013, jointly with 12 other thematically similar conferences. The total of 1666 papers and 303 posters presente...

The previous special issue featured articles on the theme of presenting the latest advances in cognitive readiness research in terms of both theory development and practical applications across domains. For researchers, this translates into addressing important research gaps and identifying areas warranting further investigation. For practitioners,...

Cognitive readiness refers to the mental preparation (including knowledge, skills, abilities, and attitudes) an individual needs to establish and sustain competent performance in complex and unpredictable operational environments (Morrison & Fletcher, 2002). Over the past decade, the scientific research community has endeavored to optimize human pe...

Given the ever-increasing complexity of the tasks performed by individuals and teams to meet organizational goals, it is imperative to ensure that operators are cognitively ready to meet the demands of the operational environment. Cognitive readiness refers to the mental preparation (including knowledge, skills, abilities, and attitudes) an individ...

To build a global " Social Radar, " an integrated set of capabilities supporting strategic and operational level situation awareness, alerts, and option awareness, there is a need for an overarching enterprise approach. Social Radar's objective is to demonstrate a useful, mission focused, end-to-end environment—a dashboard of socio-cultural indicat...

IntroductionCognitive State AssessorsHuman–System AugmentationRobust ControllersApplication DomainsConclusions AcknowledgmentsReferences

Broadly defined as the science and technology of systems responding to neural processes in the brain, neuroadaptive systems (NASs) has become a rapidly developing area of study. One of the first books available in this emerging area, Neuroadaptive Systems: Theory and Applications synthesizes knowledge about human behavior, cognition, neural process...

This volume explores decision-making styles, including cooperative, collaborative, avoidant, competitive, and dominate that are commonly modified by the culture. Culture is not a stagnant phenomenon, and many variables need to be considered to accurately evaluation cultural differences in decision-making styles. Among many cultural factors, the ind...

This paper provides a summary of the presentations presented in the Operational Neuroscience session during Augmented Cognition International 2011 at Human Computer Interaction International 2011 in Orlando, Florida, July, 2011.

Team performance represents an extension of individual performance. It is an outcome of individuals joining together for the purpose of exploiting one another’s individual skills to successfully accomplish a common goal (Fitts 1964, Dreyfus & Dreyfus 1980, Ericsson et al. 1993, Offermann & Spiros 2001, Dyer 1984). Current team performance measures...

In the current global security environment, the success of operations increasingly depends on leveraging all instruments of national and international power in a coherent fashion. Such a "comprehensive approach to operations" (CA) involves coordinated and coherent action by multiple operational entities that may include national/international gover...

Traditionally, the development of training systems follows the well-established paths defined through Instructional Systems Design (ISD) guidelines. This approach works well when developing training for domains where core knowledge content is relatively static, such as Aviation, Marksmanship, Mathematics or Language. However, with the recent shift...

Augmented cognition is a form of human-systems interaction in which a tight coupling between user and computer is achieved via physiological and neurophysiological sensing of a user's cognitive state. This interactive paradigm seeks to revolutionize the manner in which humans engage with computers by leveraging this knowledge of cognitive state to...

Command and Control (C2), especially in the uncertain environments associated with counterinsurgency, stability operations, and irregular warfare actions, require a knowledge and shared understanding of not only an elusive adversary, but the socio-cultural layer in which the adversary lives and enjoys a support structure. C2 in these situations mus...

The ability to imagine alternative futures and to make decisions based on predicted outcomes is uniquely human. Our power to build and track these mental simulations is absolute; our evolved human skill of daydreaming endless possibilities, combined with twenty-first century technological capabilities, may provide us with yet another unique surviva...

The 13th International Conference on Human–Computer Interaction, HCI Inter- tional 2009, was held in San Diego, California, USA, July 19–24, 2009, jointly with the Symposium on Human Interface (Japan) 2009, the 8th International Conference on Engineering Psychology and Cognitive Ergonomics, the 5th International Conference on Universal Access in Hu...

This chapter explores the role of virtual reality (VR) systems in the military training environment. It begins with a discussion of some of the current and future training challenges that VR can solve. These include enabling the warfighter to continue training despite the ever-increasing operational tempo and enabling the military to provide low-co...

The increasingly complex environment of the 21st century demands unprecedented knowledge, skills and abilities for people from all walks of life. One powerful solution that blends the science of learning with the technological advances of computing is Virtual Environments. In the United States alone, the Department of Defense has invested billions...

This discussion panel was organized to offer HFES members an opportunity to learn more about the burgeoning field of Augmented Cognition and to discover the multi-dimensional aspects of the discipline. The session will feature six invited panelists who were selected to represent a cross-section of the Augmented Cognition International Society commu...

The Office of Naval Research (ONR) Infantry Immersion Trainer (IIT), at Area 62 in the San Mateo area of Camp Pendleton, began training Marines using many innovative technologies in 2008. Although many of the fundamental techniques and technologies have been used elsewhere, the IIT is a a unique prototyping facility where the latest ideas can be sc...

This paper presents a historical overview of intelligent tutoring systems and describes an adaptive instructional architecture based upon current instructional and adaptive design theories. The goal of such an endeavor is to create a training system that can dynamically change training content and presentation based upon an individual's real-time m...

The effectiveness of neurophysiologically triggered adaptive systems hinges on reliable and effective signal processing and cognitive state classification. Although this presents a difficult technical challenge in any context, these concerns are particularly pronounced in a system designed for mobile contexts. This paper describes a neurophysiologi...

In augmented cognition (AugCog), mitigation strategies are used as real-time intervention techniques that are triggered by the outcome of cognitive state assessment and context to significantly improve human-systems performance. Yet, no common ground has been established regarding best practices and what aspects to consider during implementation. T...

Blend of live training and virtual technologies immerses U.S. Marines in the urban fight

This special issue includes a collection of articles on the use of optical devices to measure brain activity in humans in a noninvasive manner and is intended to provide a sample of different aspects of this area of research.

Office of Naval Research (ONR) initiatives such as Human Performance Training and Education (HPT&E) as well as Virtual Technologies and Environments (VIRTE) have primarily focused on developing the strategies and technologies for creating multimodal reality or simulation based content. Resulting state-of-the-art training and education prototype sim...

The purpose of this paper is to describe a new, user-friendly tool that will enable researchers and instructors to setup and run virtual environment scenarios that adapt to the VE user’s real-time performance and cognitive status. This tool, the Automated SAF Adaptation Tool (ASAT), will work with existing performance and cognitive state assessment...

The 1st Augmented Cognition International (ACI) conference was held in July 2005 in conjunction with the HCI International conference in Las Vegas, Nevada. A full day working group session was held during this inaugural ACI conference to facilitate the development of an Augmented Cognition R&D agenda for the near- (1-2 years), medium- (within 5 yea...

Augmented Cognition is distinct from other disciplines due to its focus on using modern neuroscientific tools to determine the ’in real time’ cognitive state of an individual and then adapting the human-system interaction to meet a user’s information processing needs based on this real-time assessment [1], [7], [14]. Augmented Cognition systems emp...

Designers of a closed loop scenario based training systems must have specifications to drive the decisions of whether or not performance feedback is appropriate in response to student behavior, the most effective content of that feedback, and the optimal time and method of delivery. In this paper, we propose that physiological measures, when interp...

To address the issue of system simplicity and subject applicability, a brain controlled HCI system derived from steady state visual evoked potential (SSVEP) based brain computer interface (BCI) is proposed in this paper. Aiming at an external input device for personal computer, key issues of hardware and software design for better performance and u...

The 12th International Conference on Human-Computer Interaction, HCI Inter- tional 2007, was held in Beijing, P.R. China, 22-27 July 2007, jointly with the Symposium on Human Interface (Japan) 2007, the 7th International Conference on Engineering Psychology and Cognitive Ergonomics, the 4th International Conference on Universal Access in Human-Comp...

Augmented Cognition is distinct from other disciplines due to its focus on using modern neuroscientific tools to determine the 'in real time' cognitive state of an individual and then adapting the human-system interaction to meet a user's information processing needs based on this real-time assessment [1], [7], [14]. Augmented Cognition systems emp...

This report (Volumes I and II) documents the successful completion of Phase II of the Defense Advanced Research Projects Agency (DARPA) Improving Warfighter Information Intake Under Stress (IWIIUS) program. Volume I discusses the Phase II integration of cognitive state gauges into operational systems that could demonstrate the viability of augmenti...

IntroductionCognitive State AssessorsHuman–System AugmentationRobust ControllersApplication DomainsConclusions AcknowledgmentsReferences

One of the biggest challenges in designing Virtual Environment (VE) training systems is identifying the fidelity requirements for the component technologies. Initial fidelity-related design decisions are often motivated by the belief that the more accurately the VE stimulates individual components of the human sensory system, the more likely the sy...

The primary goal of any training system is to enhance performance on the real world tasks it simulates (Lathan et al., 2002). The principal benefit that Virtual Environment (VE) systems have over field exercises and similar real world training is that VEs present training situations that would be too hazardous or too costly to reproduce in the real...

Military operators are often put into complex human-machine interactive environments shown to fail when stressful situations are encountered. To enhance warfighter readiness and operational capability, DARPA's efforts in Augmented Cognition are developing a new generation of technologies to enable computational systems to adapt to the human's cogni...

The Defense Advance Research Projects Agency Augmented Cognition program is developing innovative technologies that will transform human-computer interactions by making information systems adapt to the changing capabilities and limitations of the user. The first phase of the Augmented Cognition program was to empirically assess the ability of vario...

Recent advances in sensor technologies have enabled a net centric view of the battlespace, substantially increasing situational awareness for the warfighter. However, this net centric awareness has yet to be extended to the status of the warfighters themselves. Medical technologies have progressed to the degree that portable, rugged, and wireless d...

Over the past decade, Virtual Environment (VE)-based training systems have become commonplace within the military training domain. These systems offer such benefits as small footprint, rapid reconfiguration, and enhanced training delivery. In addition, they appear to offer significant relief for a market starved for low cost training systems, and h...

The DARPA Augmented Cognition funded Boeing team designed and built a fully-functioning closed loop system which uses neurophysiologically derived cognitive state data to drive aa dynamically adaptive user interface for the ultimate purpose of improving human performance by improving cognitive throughput.

Increased dismounted environments, such was those demanded by the U.S. Army's Future Force Warrior (FFW) program, will required the realization of super ordinate goals such as a ten-fold improvements in operational task performance because soldiers within fur units of action will be required to process more information afforded by C4ISR, assume mor...

Not Available

Not Available

Current U.S. Naval doctrine places increasing emphasis on providing just-in-time training. This means training the deployed sailor when they need the training, wherever they happen to be. This differs from classic training doctrine that calls for placing a completely trained expert in the field. This shift in doctrine is a direct response to reduct...

A crucial need for the modern military is the ability to rapidly assemble a set of disparate information systems into a coherently interoperating whole. This must be done without system redesign and may include interoperation with non-DoD governmental systems, with systems separately designed by coalition partners or with COTS and open-source syste...

The Augmented Cognition (AugCog) program will extend, by an order-of-magnitude or more, the information management capacity of the "human-computer" combination by developing and demonstrating enhancements to human cognitive ability in diverse and stressful operational environments. Specifically, this program will develop the technologies needed to...

Progress in computer vision and speech recognition technologies has recently enabled multimodal interfaces that use speech and gestures. These technologies o er promising alternatives to existing interfaces because they emulate the natural way in which ...

A variety of Virtual Environment (VE) systems exist today and there are as many terms to them. Synthetic environments, virtual reality (VR), and VEs, to name a few, all refer to a simulation of some operational environment. The gamut of VEs runs anywhere from desktop displays to fully immersive and interactive scenes. What these systems share is th...

Force information is often required for tele-operation system and virtual reality. Conventional force displays are active systems with actuators. This, however, is inherently active, so that it may become danger. Consequently, passive force display is ...

Naval Aviation has redoubled its long-standing efforts to eliminate mishaps, especially those linked to human error. The focus was expanded not only to cover aircrew error, but maintainer error as well. To examine maintainer error, the Naval Safety Center's Human Factors Analysis and Classification System (HFACS) was adapted to analyze reportable N...

The Maintenance Extension of the Human Factors Analysis and Classification System (HFACS-ME) was employed to categorize errors present in 470 FY 90-97 Naval Aviation Maintenance Related Mishaps (MRMs). HFACS-ME identified common error types present in MRMs: maintenance supervision, crew coordination, maintainer error, and procedural violations. The...

maximum 200 words) This study evaluates the usability of a U.S. Navy Decision Support System (DSS). The DSS was developed to enhance the performance of tactical decision-makers within a Navy Combat Information Center. The goals of this study were to test the DSS against usability criteria and objectives to track future redesign efforts and system i...

Naval Aviation is in the midst of a major transformation as it attempts to come to terms with the demands of maintaining operational readiness in the face of diminishing budgets and reduced manning. Diminishing operating and procurement budgets mean that Naval Aviation is for the most part "making do" with existing aircraft. Over the past decade, o...

Naval Aviation has redoubled its long-standing efforts to eliminate mishaps, especially those linked to human error. The focus was expanded not only to cover aircrew error, but maintainer error as well. To examine maintainer error, the Naval Safety Center's Human Factors Analysis and Classification System (HFACS) was adapted to analyze reportable N...