The Role of Risk in Aviation Decision Making: How Pilots Perceive and Manage Flight Risks

Abstract

The current direction in CRM training is threat and error management. In order to manage threats and to prevent them from leading to unsafe situations, pilots must first assess the risks associated with them. Risk assessment feeds into decision making in two ways: during assessment of the precipitating event (or threat) that requires a decision to be made, and in evaluating potential courses of action. Survey data were collected from over 100 airline pilots concerning their everyday experience with five types of risk: economic, physical, productivity, professional, and social. These risks may be pitted against each other, creating goal conflicts and decision dilemmas. Data from this study are helping us to understand what risks are salient to pilots in everyday flight and how pilots manage those risks when making decisions.

Full text

http://pro.sagepub.com/
Ergonomics Society Annual Meeting
Proceedings of the Human Factors and
http://pro.sagepub.com/content/45/2/58
The online version of this article can be found at:
DOI: 10.1177/154193120104500213 2001 45: 58Proceedings of the Human Factors and Ergonomics Society Annual Meeting
Judith Orasanu and Jeannie Davison
The Role of Risk in Aviation Decision Making: How Pilots Perceive and Manage Flight Risks
Published by:
http://www.sagepublications.com
On behalf of:
Human Factors and Ergonomics Society
can be found at:Proceedings of the Human Factors and Ergonomics Society Annual MeetingAdditional services and information for
http://pro.sagepub.com/cgi/alertsEmail Alerts:
http://pro.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://pro.sagepub.com/content/45/2/58.refs.htmlCitations:
What is This?
- Oct 1, 2001Version of Record >>
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from

PROCEEDINGSof theHUMANFACTORSANDERGONOMICSSOCIETY45thANNUALMEETING-2001
THE ROLE OF RISK IN AVIATION DECISION MAKING:
How Pilots Perceive and Manage Flight Risks
,
JudithOrasanu,NASA-Ames Research Center
Moffett Field, CA 9403 5-1000
jorasanu@mail.arc.nasa.gov
Jeannie Davison,San Jose State University/NASA-Ames Research Center
Ute Fischer, Georgia Institute of Technology
The currentdirection in CRM training is threat and error management. In
order to manage threats and to prevent them from leading to unsafe situations,
pilots must first assess the risks associated with them. Risk assessment feeds into
decision making in two ways: during assessment of the precipitating event (or
threat) that requires a decision to be made, and in evaluating potential courses of
action. Survey data were collected from over 100 airline pilots concerning their
everyday experience with five types of risk: economic, physical,productivity,
professional, and social. These risks may be pitted against each other, creating
goal conflicts and decision dilemmas. Data from this study are helping us to
understand what risks are salient to pilots in everyday flight and how pilots
manage those risks when making decisions.
BACKGROUND particular experience,goals,sense of controland
vulnerability. An experienced instrument-qualified
The current direction in CRM training for flight pilot will see a weather system quite differently than
crews is threat and error management (Gunther, 2001; will a newly certified non-instrument rated private
Helmreich, Klinecht, & Wilhelm, 2001). In order to pilot.
manage threats and to prevent them from leading to Slovic (1987) has studied factors that influence
unsafe situations, pilots must first assess the risks how people perceive risks and has found a consistent
associated with them. A traditional definition of risk is set across many cultures, despite surface variations.
"the possibility of loss," or the occurrence of an First, risk perception depends on the amount of
undesired consequence. Two factors combine to experience or familiarity people have with a situation.
determine how significant a loss is: the probability of More familiarity generally translates into a lower
the loss and the magnitude of the loss (Yates & estimation of risk level. Second is personal
Stone, 1992). For example, if you are considering an vulnerability: can this risk hurt me? Clearly, pilots
investment in the stock market and you learn that have greater vulnerability than do controllers. Third
one stock has a 5% chance of dropping while the is the level of control one exercises over a situation.
other has a 25% chance of dropping, you might be Many people are afraid to fly because they feel totally
inclined to choose the first one. In addition, the out of control. They would rather drive a car than be a
amount you already have invested when an advisor passenger. Finally, the time frame of a risk is
warns you of a 25% chance that the stock will drop important. Consequences that will occur in the
will probably affect your eagerness to sell it: If you immediate future loom larger than those more distant
have invested $25,000 you may respond differently in time.
than if you have only $5,000 invested.
Although both probability and magnitude of loss 'Why Study Risk in Aviation?
are relevant, the significance of a risk is not just a
statistical phenomenon. It depends on one's Despite its apparent significance to aviation
reference point. If you have a recent inheritance, you safety, there is surprisingly little research on the role
may be less interested in taking a risk to win a of risk perception in pilot decision making. Most of
substantial sum than if you have just lost your job. the existing work deals with general aviation pilots
Loss is inherently subjective; the meaning of the loss (O'Hare and Smitheram, 1995), either in weather-
or gain depends on one's perspective. For example, related decision situations or in setting "personal
pilots and air traffic controllers may see the same minimums" (e.g., Jensen, 1995). Our interest in
traffic situation very differently based on their studying risk perception was prompted by three
58
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from

PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING-2001
factors. First, a sorting study designed to identify want to get his passengers to their destination on
similarities in problem structuring by commercial Christmas Eve, but may face delays in getting there
airline captains and first officers found that potential due to bad weather. An airport curfew and low fuel
risk was the most salient dimension for captains but may create a dilemma: to hold and hope to get in
not for first officers, regardless of years of experience before the curfew and a fuel critical situation arises, or
(Fischer, Orasanu, & Wich, 1995). to divert and avoid the uncertainty (Fischer &
Second, examination of tactical decision errors Orasanu, 2001). Risks in this situation include
(TDE) in hull-loss accidents (National Transportation productivity (getting passengers where they want to
Safety Board, 1994) indicated that 73% of the TDE go at an important time), safety (possibility of
could be classified as "plan continuation errors" running out of fuel), and economic (hotels/meals for
(Orasanu, Martin & Davison, in press). These are passengers at the divert location and then transporting
errors that occur when pilots continue with their them to their original destination). The decision
original plan of action, even though conditions have made by the crew involves balancing these risks and
changed or an event has occurred that suggests deciding which are most important at that particular
updating their situation model and reconsidering the time. As Nygren (1995) pointed out, risk dimensions
plan. Most of these cases occurred during the are additive: as the number of risks increases, the more
approach and landing phases of flight, when workload complicated the decision problem becomes.
was high, time was limited, and there was little margin
for error. Pilots' behavior appears to follow March & STUDIES OF AVIATION RISK
Shapira's (1987) observation that "managers pursue
risky actions because they fail to perceive accurately We are presently conducting several studies to
the risks involved" (p. 333.) In dynamic situations, address two central issues: First, what risks are of
pilots may not realize that they need to update their greatest concern to pilots? Second, why do they take
model, they may underestimate the risk associated risks? The first question was addressed by a survey
with the changing conditions, or they may conducted with pilots from a major US carrier. The
overestimate their ability to cope with the situation, second issue was addressed in a think-aloud study with
A third reason for studying risk is that although a second major carrier that used evolving decision
we have observed little explicit discussion of risk dilemmas. Pilots' risk framing and information use
(either during jumpseat rides, in cockpit voice recorder helped to determine their decision strategies.
transcripts from accidents, or in crew conversations in
full-mission simulations), pilots clearly do assess risk Risk Survey Method
in flight situations. In a full-mission study designed to
determine pilots' strategies for correcting an error The survey included three open-ended questions,
committed by a confederate pilot, the levels of risk ratings of frequency of encounters with five types of
associated with the errors were varied. When risk risk, and requests for descriptions of actual experience
levels were high, both first officers and captains with the same five types of risk. The three open-
caught more errors and were more effective in what ended questions were: (1) When you think of risk in
they said to correct the errors (Orasanu, van Aken, your professional life, what concerns or issues come to
McDonnell & Fischer, 2001) than when risk was low. mind? (2) Please describe a difficult professional
In addition, Rhoda and Pawlak (1999) found that the decision and what made it difficult. (3) Have you
likelihood of pilots penetrating thunderstorms in experienced a flight situation that generated real fear
terminal areas reflected the severity of the storms, or anxiety? Please describe it. Demographic
information also was collected. Booklets were either
Many Types of Risk distributedpersonallyat the carrier's Operations
Center or were placed in pilots' mailboxes and were
A conversation with pilots about the risks they then returned by mail. To date, 101 replies have been
perceive in flight suggests that they are concerned not _eceived and 64 have been coded (from 26 captains
only about flight safety, but also about passenger and 38 first officers).
satisfaction, fuel consumption, and the legality of The rationale for the structure of the survey is as
their actions. The research literature supports this follows: for pilots to make "good" and safe decisions
diversity. Several researchers have pointed out that they must be aware of the risks they face. When one
there are several types of risk: physical harm, faces risks every day, sensitivity to risk becomes part
professional, economic, social, and psychological of one's perceptual filter. Thus, the first survey
(Jacoby & Kaplan, 1972; Nygren, 1995). Moreover, question ("When you think of risk in your professional
these risks may occur in the same situation and be life, what concerns or issues come to mind? ")sought
pitted against one another. For example, a pilot may to determine which risks were most salient to pilots.
59
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from

PROCEEDINGSof theHUMANFACTORSANDERGONOMICSSOCIETY45thANNUALMEETING-2001
Second, we wanted to know whether the salience of Physical
various risks detected in the first question reflects the Fear of making a bad decision, not managing
frequency of pilots' encounters with them. Pilots risk.Deferred maintenance, substandard de-icing
were asked to indicate on a scale from one tofive how facilities,high traffic density, runway collision,
often they encountered each of fivetypesof risk jammed radio frequencies, terrorism, turbulence,
(economic, physical harm, productivity, professional, fatigue, terrain.
and social). Third,we were interested in the specificProductivity
typesofevents thatpilots actually encountered within Delays associated with weather or ATC, rejecting
eachrisk category. Pilots were asked to describe an unsuitable aircraft.
situations they had personally experienced thatdealtProfessional
with each of the fivetypesof risk. Finally, we wanted Will I correctly handle my next check ride? Being
pilots' perceptions about the factors that make their charged with a crime, losing medical certificate or
decisions difficult, the second open-ended question, job.
The five different types of risk in the survey Social
were defined as follows: Telling a captain to go around or divert, reporting
Economic: Actions that may have negative a junior officer.Making a smooth landing.
economic consequences for your flight, such as fuel
usage or missed passenger connections. The majority of responses reflected concerns
Physical: Professional actions that may have with safety: 77% of responses fell intothe physical
negative consequences for flight and passenger and risk category. Concerns with job security ranked a
safety, distant second, with 18%of responses falling into
Productivity: Actions that may have professional risk. Thus, these twocategories
productivity consequences, such as delaying a flight accounted for 95%of the responses, with the
departure or arrival, or rejecting a placarded aircraft, remainder distributed across the other three.
Professional: Actions whose consequences could Physical risk. While all responses in this
negatively impact career goals or job security, category were concerned with the possibility of
Social: Actions that may negatively impact how physicalharm, the sources of the physical risks were
others (e.g., pilots, managers, controllers, passengers, quite diverse. Most frequent for both captains and first
flight attendants) judge your competence and skill, officers were self-referential responses that we labeled
"individual" factors. These refer tothe possibility
Results that one's own behavior may contribute toa reduction
of safety. Approximately one- third of the physical
Responses to only three items will be reportedhere: risk responses fell intothis category. Pilots
open-ended question 1, frequency of encounter (particularly captains) referred to their own ability to
ratings, and difficulty explanations, in that order, manage risks and to make good decisions. Most first
officers' "individual" responses referred to their
Q1." When you think of risk in your professional life, physical health or fatigue as safety-compromising
what concerns or issues come to mind? factors. These crew differences may reflect differing
crew responsibilities and experiences.
Pilots' responses to the first question were sorted The next two subcategories contributing to
into one of the five risk categories described above, physical risks were environmental factors and aircraft
Items were sorted by outcomes or consequences, all of equipment. Typical environmental factors were
which are losses of some type (e.g., safety, job, weather and turbulence (especially important to
money, respect). All responses fit into these five captains), traffic and midair collisions, and general
categories, though some categories were quite diverse, uncertainty about environmental events. Both
Examples of responses in each of the five risk captains and first officers were concerned with aircraft
categories are shown in Figure 1. The examples equipment failures, automation problems, and fire or
typically reflect causes of a type of loss. These are smoke. Together these two factors accounted for
specific operational issues that can have distinct another 35% of the physical risk responses.
consequences for pilots. Professional risk. The second most commonly
mentioned concern expressed by the pilots dealt with
Figure1. Examples of Responses in each Risk the risk of losing their jobs dueeither to medical
Category conditions that would affect their license or tolegal
Economic violations that would result in FAA actions. This type
Passenger missed connections due to medical of concern was expressed slightly more often by first
emergency or delayed feeder flight,officers than by captains (21% vs. 14%).
60
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from

PROCEEDINGSoftheHUMANFACTORSANDERGONOMICSSOCIETY45thANNUALMEETING-2001
getting them to their destination on time). A second
Frequencyof Encounters with Various Types of subcategory referred to conflicts in which there was no
Risks goodoption(e.g.,destinationis closeddueto weather,
Rating Scale." "How often do you feel each type of alternate weather is deteriorating, and fuel is low).
risk plays a role in decisions you are routinely called These decisions often involve risk trade-offs. Both
upon to make as a pilot? "(Rate from 1 to 5) captains and first officers reported this source of
difficulty.
Given the pattern of findings for relative
salience of risk concerns to pilots, the second question Table 2. Frequency of Mention of Factors Contributing
was whether risk salience reflects frequency of to Decision Difficulty
encounters with the various types of risks. Pilots were
asked to indicate on a scale from one to five how Contributing Captains First officers
often they needed to consider each type of risk when Factor (N= 26) (N =38)
makingflightdecisions.AsshowninTable1, Safety 10 11
frequencyof encounterclearlywas not relatedto Cognitive Effort 9 3
salience of risk concerns. No statistical differences
were found between captains and first officers. Career/Personal 3 15
Table 1.Mean Ratings of Frequency of Encounter with The second category is essentially metacognitive.
Various Types of Risk It refers to difficulty arising from heavy workload,
time pressure, or the uncertainty inherent in many
Risk Type MeanFrequency Ratingdecision situations. Captains, who are responsible for
(from 1 to 5)makingdecisions, were moresensitivetothisfactor
Productivity2.9 than first officers. Factors relating tocareer choices,
Economic 2.8 personal healthand well being were mentioned
Professional 2.6 primarily by first officers.
Physical 2.3 EVOLVING DECISION DILEMMAS
Social 2.3
Why pilots makerisky decisions, our second
Pilots reported encounteringproductivity,majorissue,wasaddressed throughthink-aloud
economicand professional risks most frequently, protocolsfordynamically evolvingdecision problems
Physical safetyrisks, which dominated pilots'(see Fischerand Orasanu, 2001, fordetails). Two
concerns, werereported to be encountered less scenarioswere developed that created dilemmasfor
frequently and did notdiffer significantly from social pilots. The first wasatake-off scenario involving
risks. Hence, pilots' concerns with various types of wind-shear reported by landing aircraft and 20 planes
risks clearly do not reflect how often they encounter in line behind the participant's aircraft. The second
those risks in actual decision making situations, involved approach and landing on Christmas Eve with
holding due to traffic, bad weather, a destination
What Makes Decisions Difficult? curfew, and low fuel. In each case,there were risks
Q2: "Please describe a very difficult decision you associated with bothcontinuingwiththe original plan
have had to make as a professional pilot and what (the riskier choice but the greaterpayoff) and delaying
made it difficult."take-off ordiverting (the less riskychoices). Pilots
read the first stage of a scenario and were asked to
In thisquestion we sought to determine whetherthinkaloud, verbalizing their concernsand
pilots' perceptionsof whatmakesdecisionsdifficult information needs, and to make a decision ateach
corresponded with predictions of decision difficulty stage. They proceeded this way through each of the
based on our aviation decision process model (Orasanu sequential steps of the scenario.
& Fischer, 1997). Pilots' responses were categorized
according to their consequences. Three major Results
categories emerged: safety, cognitive effort, and
career/personal, as shown in Table 2. [Note: Not all Analysis of 30 pilots' responses (17 captains and
pilots responded to this question.] 13first officers) indicated that in fact the problems
Within safety the most common response represented decision dilemmas. Half of the pilots
involved goal conflicts, primarily those that pitted diverted or delayed takeoff and half continued with
safety against achieving passenger satisfaction (e.g., the original plan. Individual pilots were not
61
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from

PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING-2001
consistent; most continued on one scenario and Fischer, U., & Orasanu, J. (2001, March). The role of risk
changed their plan on the other. No evidence was perception in pilot decision making. Paper presented at the 11th
International Symposium on Aviation Psychology, Columbus,
found for either risk-taking or risk-averse personal OH.
styles. That is, pilots did not consistently choose the
risk-taking or risk-averse options. Most significantly, Fischer, U., Orasanu, J., & Wich, M. (1995). Expert pilots'
how pilots framed the situations seemed to influence perception of problem situations. In Proceedings of the Eighth
which decision they made. If they created a situation International Symposium on Aviation Psychology (pp. 777-782).
model that was clearly negative (there is windshear at April 24-27, 1995, Columbus, OH.
the airport, storm is heading toward us),they took the Gunther, D. (2001, March). A new training program in threat
cautious approach. If they painted a picture that was and error management. Paper presented at the 1lth International
less negative (loss of airspeed is diminishing, my Symposium on Aviation Psychology, Columbus, OH.
takeoff path is away from the storm), they were more Helmreich, R. L., Klinect, J. R., & Wilhelm, J. A. (2001,
likely to pursue their original course of action. March). Models of threat, error, and CRM in flight operations.
Moreover, continuations were usually accompanied by Paper presented at the 11th International Symposium on Aviation
a strategy to mitigate the risk,such as reviewing Psychology, Columbus, OH.
windshear procedures prior to takeoff or declaring a
fuel emergency and requesting special handling to Jacoby, J., & Kaplan, L. B. (1972). The components of
reduce their holding time. perceived risk. In M. Ventakesan (Ed.), Proceedings of the Third
Annual Conference of the Association for consumer Research
DISCUSSION (pp. 382-393). Chicago" Association for ConsumerResearch.
Jensen, R. S. (1995). Pilot judgment and crew resource
These two studies represent an initial effort to management. Aldershot, Hants, UK: Avebury.
understand the risks that pilots face and why they Nygren, Y. E. (1995, June). Effective risk perception and the
make the decisions they do. The inherently subjective communication of risk in flight crew decision making.
nature of risk is clearly evident in the differences Presentationat American Society ofElectricalEngineers meeting
between captains and first officers in the risk atNASA Ames Research Center.
perception survey and in the situation models created
National Transportation Safety Board, (1994). A review of
by pilots in the think-aloud study. The survey flightcrew-involved, major accidents of U.S.Air Carriers, 1978-
indicated that pilots' main concern was with factors I990(NTSB/SS-94/01). Washington, DC:NTSB.
that compromise flight safety. Their recognition of
their own possible contribution to loss of safetywas O'Hare, D., & Smitheram, Y. (1995). "Pressing on" into
unexpected. This was reinforced by the captains' deteriorating conditions" An application of behavioraldecision
awareness of how cognitive load and uncertainty theory to pilot decision making. International Journal of
contribute to decision difficulty. Aviation Psychology, 5 (4),351-370.
The decision dilemmas show how decisions reflect Orasanu, J., & Fischer, U. (1997). Finding decisions in
pilots' evaluations of risks presented by the situation, natural environments: The view from the cockpit. In C. E.
The extent to which pilots considered the conditions Zsambok & G. Klein (Eds.), Naturalistic decision making.
risky influenced their willingness to continue with Mahwah, NJ: Erlbaum.
their original plan. Coupled with their risk assessment Orasanu, J., Martin, L., & Davison J. (in press). Cognitive
was the extent to which they felt they could control and contextual factors in aviation accidents. In G. Klein and E.
the risks associated with various choices. Preparations Salas (Eds.). Applications in naturalistic decision making.
for windshear and ATC special handling were Mahwah,NJ: Lawrence Erlbaum.
associated with accepting the riskier option. How risk Orasanu, J., Van Aken, C., McDonnell, L. & Fischer, U. (2001,
perception affects decision strategies by pilots with March). Communication strategies for successful monitoring and
varying levels of experience and under diverse challenging. Paper presented at the 11th International Symposium
conditions is a topic that we are investigating further, on Aviation Psychology, Columbus, OH.
ACKNOWLEDGEMENTS Rhoda,D. A., & Pawlak,M. L. (1999). An Assessment of
Thunderstorm Penetrations and Deviations by Commercial
Aircraft in the Terminal Area. Project Report #NASA/A2 to NASA
We would like to thank Yuri Tada for her assistance with Ames Research Center, Moffett Field, CA.
statistical analyses, Yuri Tada,Jon Holbrook and Amy Lynn for
their help with coding, the airline representatives who helped us Slovic, P. (1987). Perception of risk. Science, 236, 280-
find pilots, and all the pilots who so generously took time to 285.
participate in the studies and share with us their experiences.
Yates, J. F., & Stone, E. R. (1992). The risk construct. In J. F.
REFERENCES Yates (Ed.), Risk-taking Behavior. West Sussex, UK: Wiley.
62
at NASA AMES RESEARCH CENTER on July 17, 2012pro.sagepub.comDownloaded from