On Cognitive Biases in Architecture Decision Making

Abstract

The research carried out to date shows that architectural decision-making is far from being a rational process. Architects tend to adopt a satisficing approach, rather than looking for the optimal architecture, which is a result of many human and social factors. The results of a workshop, carried out with 14 software engineering practitioners show that cognitive biases are commonly present in architecture decision-making. A systematic approach to analysing the influence of biases on decision making has been introduced. Twelve cognitive biases identified during the workshop were analysed with regard to the elements of the decision-making context that affected the aspects of architectural decision making. Finally, we analyse the interactions between cognitive biases and the conditions of real-world software development.

Full text

On Biases in Architecture Decision Making
Andrzej Zalewski,1 Klara Popławska,1 Andrzej Ratkowski1
1 Warsaw University of Technology, Institute of Control and Computation Engineering,
Warsaw, Poland
2 a.zalewski@ia.pw.edu.pl
Abstract. The research carried out to date shows that architectural decision-
making is far from being a rational process. Architects tend to adopt a
satisficing approach, rather than looking for the optimal architecture, which is a
result of many human and social factors. The results of a workshop, carried out
with 14 software engineering practitioners show that cognitive biases are
commonly present in architecture decision-making. A systematic approach to
analysing the influence of biases on decision making has been introduced.
Twelve cognitive biases identified during the workshop were analysed with
regard to the elements of the decision-making context that affected the aspects
of architectural decision making. Finally, we analyse the interactions between
cognitive biases and the conditions of real-world software development.
Keywords: cognitive biases, architectural decisions, architectural decision-
making.
1 Introduction
The concept of architectural decisions enables the design rationale and architectural
knowledge to be captured, but equally important is the focus it places on the act of
deciding on software design. This has shaped anew our perception of software
development as a decision-making process, and triggered research into its nature.
The research on how architectural decisions are made revealed that, despite the in-
trinsic complexity of architecting, architects as decision makers remain normal human
beings: their judgement is more often bounded rational than fully rational, which is a
result of the inherent properties of the human mind.
As normal human beings, architects are subject to cognitive biases [4], [8-19]. This
exploratory paper, which has been motivated by the outcomes of a workshop carried
out with 14 software engineering practitioners, investigates how cognitive biases in-
fluence architectural decision-making. Our research focuses on answering the follow-
ing research questions:
 RQ.1. Are biases in architecture decision making commonly observed by software
engineering practitioners?
 RQ.2. What are the most significant cognitive biases that influence architecture
decision making?

2
 RQ.3. Which of these biases result from cognitive biases inherent to the conditions
of the human mind?
 RQ.4. Which elements of the decision-making context can bias architects' deci-
sions?
 RQ.5. Which aspects of architectural decision making are influenced by the biases
that have been identified?
 RQ.6. How do practical conditions influence the extent of the influence of the bi-
ases on architectural decision making?
The research presented in this paper shows that biases are commonly observed by
software practitioners (section 3.1). In order to capture how cognitive biases influence
architectural decision making, we propose a model comprising contextual factors that
are transformed by the biases into influence on the identified aspects of the
architectural decision-making process (section 3.2). The identified cognitive biases
are presented and discussed according to this model (section 3.2). The ways in which
the practical conditions of architecture decision making affect the ‘mechanics’ of
biases’ influence is presented in section 4. The discussion of the findings (section 5) is
presented in section 6, the summary of the paper and the research outlook in section 7.
2 Related work
The social and human factors in software engineering have been studied since the
advent of software engineering as a scientific discipline – compare reports from
NATO conferences from 1968 and 1969 [1], [2]. Probably the earliest observation of
the influence of social factors on software architecture is the now famous Conway’s
law [2].
The research on cognitive biases was pioneered by Twerski and Kahneman (e.g.
[4], [8], [12]), and the latter was awarded a Noble Prize. Their research has been later
extended by many researchers, e.g. [8-19]. Kahneman and Renshon [20] define cogni-
tive biases as “predictable errors in the ways that individuals interpret information and
make decisions”.
The dual process theory, as stated by Kahneman [4], is one widely known and
accepted in psychology and helps to explain the mechanisms of the human mind.
According to the theory, our thoughts are controlled by two parallel systems: System
1 and System 2.
System 1 controls the part of our mind that is fast, runs effortlessly, and completely
out of our control. It is very useful, letting us, for example, instantly react to danger
and save our lives in a dire situation. Sadly, its associative nature may make our line
of thought illogical, often creating associations where there are none. System 1 is
unable to process rule-based logic, and thus does not always perform well.
System 2 is the complete opposite of System 1. It is slow, requires a great amount
of conscious effort to use, and is very logical. Since the use of System 1 is something
unavoidable, System 2 serves the purpose of correcting the premature conclusions of
System 1, but only if we put effort into it. It is when this correction does not happen,
or is not strong enough, that cognitive biases occur.

3
The process of architectural decision making has been thoroughly investigated by
Zannier et al. [3] in 2007. Their research shows that architects use naturalistic
(looking for a satisfactory solution) or rational decision making (looking for an
optimal solution). The naturalistic approach is common for poorly-structured
problems, while the rational one for well-structured problems.
In 2015, Tang and van Vliet observed [6] that most architects need only a few
reasons before concluding a decision, which is a result of satisficing behaviour of
naturalistic decision making.
The research record on the role of biases in software engineering in general, and in
software architecture, is rather small. Tang and van Vliet, in 2016 [7], indicate only a
couple of papers that can be related to the role of biases in design processes. They
show some examples of anchoring, framing and confirmation biases in software
engineering, and conclude that “biases do play a role; and we probably cannot fully
prevent them from occurring; they are simply too human.”
The research presented in this paper aims to expand upon these observations, and
to systematise the way we analyse how cognitive biases influence architectural
decision making.
3 Investigating Biases in Architectural Decision Making
3.1 Workshop on Biases in Architecture Decision-Making
Fourteen people with a wide variety of professional experience (8 novices – 1-2
years of experience and 6 experts with at least 10 years of experience) were gathered
on a workshop. The purpose was to gather data about biases that may have an influ-
ence over the process of software development and sort out those that affect architec-
ture-decision making. The workshop agenda was as follows:
1. Short presentation on cognitive biases;
2. Poll of the participants – they were asked to write down any examples of biases
that they could have observed from their previous projects;
3. Survey and an open discussion on biases indicated by the participants, aimed at
improving the understanding of people’s statements;
4. Rating of the biases - each participant was asked to rate the frequency of cases
when they experienced the effect of every cognitive bias listed (on a scale from 0
to 3, where 0 means something that was never observed, and 3 means an often ex-
perienced phenomenon);
5. Identifying related cognitive biases;
6. Wrap-up and conclusion.
The results of the workshop are summarised in table 1.

4
Table 1. Biases relevant to architecture decision making indicated by the workshop participants
Reported Bias
Related cog-
nitive bias
Average Frequen-
cy Assessment
Judging the quality of a system by the form of
its presentation by marketing specialists.
Framing
effect
2.58
Estimating the time needed to complete a task
wrongly, because of the expectations placed on
the development team by the client, rather than
the true complexity of the problem.
Confirmation
bias
2.33
Excessive overvaluation of a solution that we
created ourselves.
IKEA effect
2.33
Spending long hours on meetings discussing
trivial problems (like whether we should use
spaces or tabs in our code) instead of truly
important ones.
Parkinson's
Law of trivi-
ality
2.25
Insisting on continuing using certain COTS
despite the long record of errors
Anchoring
2.25
Errors created by miscommunication between
the technical staff and a client, because of them
having a completely different background.
Curse of
knowledge
2.16
Focusing mainly on a set of standards, believ-
ing that if they are met, then the quality of the
product will be good.
Anchoring
2.16
Belief that "new is better". Quick abandonment
of old tools and technologies while they are
still properly working.
Pro-
innovation
bias
2.08
Dismissing serious issues as trivial, without
any further thought.
Parkinson's
Law of trivi-
ality
2.08
Evasion of solutions that were not used previ-
ously in our industry.
IKEA effect
2.00
Underestimating a problem by assuming that it
could be solved by "writing some code", even
with no detailed plan of the implementation.
Planning
fallacy
1.91
Wrongly assuming that a solution found on the
web is correct and appropriate for our problem,
especially if it is popular.
Bandwagon
effect
1.83

5
Reported Bias
Related cog-
nitive bias
Average Frequen-
cy Assessment
Avoidance of redoing work on a system that
was already done – even if it was done poorly.
Irrational
escalation
1.75
Being unable to admit that there is an error.
Logic similar to "it's not a bug, it's a feature".
Anchoring
1.75
Applying the design patterns that we know
everywhere - even if it's not the best solution.
Law of the
instrument
1.66
Underestimating the possible load put on a
system. Guessing without any evidence to sup-
port our claims.
Optimism
bias
1.66
Focusing only on hardware when solving opti-
misation problems.
Anchoring
1.36
3.2 Influence of Cognitive Biases on Architecture Decision-Making
Expanding on the workshop results, we discussed and analysed the identified biases in
order to identify how they influence the decision-making process. As a result we de-
veloped the model presented in fig. 1.
Fig. 1. How biases influence architecture decision making
Fig. 1 shows that cognitive biases subconsciously influence the decision-making
process by associating some contextual factors with a given architectural problem.
These factors usually have no connection, or just a limited connection, with the ana-

6
lysed issue itself. The biases include these factors into the decision-making process
and alter the critical aspects of the decision-making process. Analysing the biases on
the basis of our experience we identified sets of:
 contextual factors affecting architecture decision making, for example: form of
presentation, architect’s beliefs, “topics of the year”, time spent on a given design,
order, etc. – for a complete list see table 2, column (3).
 aspects of architecture decision-making process affected by the biases, for example
such architect’s preferences (expressed by a decision’s rationale), scope of consid-
ered architectural issues, etc. – for a complete list see table 2, column (4).
Below we describe and study in a greater detail the cognitive biases indicated in
table 1.
Framing effect
The example that achieved the highest average rating in the experiment was repre-
sentative of the framing effect. The bias itself happens when information is judged
differently on the basis of how it was presented. This effect was examined thoroughly
by Tversky and Kahneman [8]. In the course of their research, they discovered that
slight differences in the formulation of the choice problem may significantly impact
decision making.
The exact scenario that the workshop participants pointed at was that software
products are often purchased not on the basis of their quality, but of the way they are
advertised, for example: properly advertised products (like COTS) are more likely to
be chosen, even if they are less suited to the needs of the project. The framing bias
affects mainly architect’s preferences, scope of considered architectural issues and
alternatives.
Confirmation bias.
Another widely appearing case is an example of conformation bias. Confirmation
bias itself, as stated by Nickerson [9], is a natural tendency to look for evidence sup-
porting our claims, because we believe that this is an effective way of showing what
is right, if it really is right.
The subjects linked the wrong estimation of the time needed to complete a project
with their superiors’ (or clients’) expectations. Concerning architectural decision
making, it is easy to observe a tendency to look for arguments confirming our beliefs
that certain solutions are better than others. Believers of microservices would always
find evidence to confirm it is a best choice. In order to confirm their beliefs, architects
may narrow the scope of considered architectural issues, requirements and alterna-
tives. Such an architect would prefer to choose the options, which comply with his
beliefs.
IKEA effect.
Easily associated with the phenomenon of the globally-known producer of ready-
to-assemble furniture – the IKEA effect is one that makes us biased when it comes to

7
items we have created or assembled ourselves [10]. When comparing to the products
of our competitors, even if they are renowned professionals in their field, we are easy
to prey for the unstoppable gut feeling that our creation is the best. This effect may be
strengthened by the time and effort that we have put into our work, even if our hard
work proved to be almost fruitless.
The participants also pointed to the reverse form of this effect – that we tend to
avoid solutions that have not been yet used in our industry, thus boxing ourselves in a
small pool of alternative choices. It is a popular phenomenon that authors of a given
system (application etc.) are rather reluctant to replace it even if a definitely better one
is currently available. This will certainly narrow the set of considered alternatives and
obviously affect architect’s preferences.
Parkinson's Law of triviality.
According to Parkinson's Law: 'work expands to fill the time available'. This una-
voidable effect rules over most workplaces, although most managers would prefer to
overlook it. Parkinson's Law of triviality is a narrower version of Parkinson's Law
stating that we spend an enormous amount of time debating over trivial unimportant
issues [11].
Most of our subjects had the unfortunate displeasure of taking part in meetings that
seemed to be endless and led to nothing. Although wasting time does not have to lead
to wrong decisions by itself, it does shorten the time available to resolve more com-
plex issues. This may result in ‘system 1’ being used to resolve the crucial problems
instead of ‘system 2’. This may indirectly affect all the aspects of architectural deci-
sion making.
Anchoring.
The effect of anchoring is created by the way in which the human brain estimates
probabilities. Naturally, we tend to cling to the first fact that comes to our mind when
contemplating an issue. This means that the first piece of information we obtained, or
one that we have a particularly fond memory of, heavily influences decision making
[12]. When 'anchored' to an idea, it becomes hard to notice different solutions, and
even if we do, it is very unlikely that we will choose them.
Although examples of the anchoring effect were rated very differently by the par-
ticipants, it is worth noting that this was the bias for which they found the greatest
amount of examples. Anchoring seems to have an influence over almost every kind of
decision: hardware, technologies, design and even implementation. It influences
mainly architect’s preferences but also possibly the scope and perception of im-
portance of requirements.
Curse of knowledge
We may be put in a situation when we have to communicate with someone of a
completely different background, or with a different level of experience, or even
simply someone younger than us. We fall prey of the curse of knowledge, if at some
point we falsely assume that the other side possesses the same knowledge that we
have about any kind of issue. This may be more apparent in contacts between children

8
and adults, when the young ones find it hard to grasp concepts that are new to them,
but adult relations are not free from this effect [13].
The curse may cause misunderstandings at any level of human interaction, which is
especially crucial when understanding the requirements of our client and choosing
appropriate solutions for their problems. Team members with different experience
levels can also be influenced when an issue is not explained properly, they can mis-
understand the way in which their tasks should be handled. Therefore, the curse of
knowledge bias influences the scope of requirements and the architect’s perception of
their priorities, as well as scope of considered architectural issues and alternatives,
and as a result this may also alter architect’s preferences.
Pro-innovation bias
The false belief that innovation should always be adopted is what we call the pro-
innovation bias [16]. Humans naturally have a positive attitude associated with inno-
vation. However, it does not mean that innovation should always be pursued.
The pursuit of a novel solution is not always necessary; sometimes it may even be
harmful. What needs to be taken into account is the high risk of every innovative
project. If a stable and reliable system is to be created in a reasonable timeframe, usu-
ally innovation should be avoided. As one of the subjects pointed out, it is not always
the case that potentially high rewards await those that successfully bring new ideas to
life. Relating these observations into architectural decision making, we observe that
pro-innovation bias means that innovative design alternatives are considered and pre-
ferred.
Planning fallacy.
The planning fallacy is the tendency to underestimate the time required to complete
a task. Interestingly, it seems to affect the individuals who are supposed to complete
the tasks more than observers – of course, if they have information about the individ-
uals’ past performance [17].
This cognitive bias has a great influence over the planning phase of any project.
The amount of information needed to avoid it is so big that it is almost impossible to
process it, especially if a problem is complex. Although various methodologies have
their ways of soothing this problem (e.g. Planning Poker in Scrum), there are almost
none that can prevent it on the early decision-making level. Let us also observe that
the planning fallacy may affect the preferences of an architect who may choose solu-
tions which are more difficult to implement than he thought when making a decision.
Bandwagon effect.
The bandwagon effect is a universal phenomenon that appears in almost any do-
main where human beings are given a choice. People naturally want to wear, buy, do,
consume and behave like their fellows, thus becoming part of a group [19]. This re-
sults in popular choices and popular decisions becoming even more popular.
The danger this bias poses should not be downplayed – it puts our mind in a small
box, limiting our possibilities and potentially forces bad decisions on us. Especially in
cases of pressure from higher-ups to solve a problem in the way they wish us to. In

9
some cases even, due to the organisation culture in a company, it may even be impos-
sible to have any influence on this kind of decisions, which could result in choosing
solutions being very far from perfect. Therefore the bandwagon makes an architect to
prefer the same solutions that have already been widely accepted by a similar organi-
sations, by an industry branch or our community.
Irrational escalation.
Irrational escalation takes place when one continues to commit to an initial course
of action, even if it is obviously no longer the most beneficial choice [18].
As the participants noticed, this may affect performance when old technologies,
code or components that are unfit for the task, are forced on us. Often these old prod-
ucts should have been abandoned long ago due to their doubtful quality, but since at
some point they were invested in, there is a stubborn reluctance to let go of them.
Irrational escalation means that architect gets fixed on an existing solution.
Optimism bias.
We do not usually assume failure before trying something. Most healthy people’s
brains are hardwired that way [14]. When writing his example, one of our participants
told us a story where a client he worked for judged how many messages his system
would have to handle daily. Unfortunately, in reality, the estimated value turned out
to be a hundred times too low, which triggered later multiple serious issues. As this
example shows, being overly optimistic can hurt badly not only planning, but the
architected system’s quality as well. Architect’s optimism can potentially influence all
the aspects of architectural decision-making.
Law of the instrument.
Known in software architecture as the Golden Hammer anti-pattern [15] - when a
single technology or design pattern is used in every possible place. This obviously
results in the creation of numerous inefficient and mismatched solutions. The bias
experienced here may simply be a symptom of the lack of necessary skill or
knowledge that forces us to use well-known solutions. Such cases were pointed out by
the participants of the workshop. Furthermore, there is one more scenario that re-
quires further consideration – does spending money on a technology in the past force
us to use it? This seems to be the case in many big companies, where decisions to
invest in expensive technologies are often made independently of the technical con-
text of specific projects.
The above findings have been summarised in table 2. Note that in column (3), only
the most important factors influencing the decision making have been listed. Natural-
ly, there are numerous other factors that may modulate (magnify or diminish) the
influence posed by a give cognitive bias, for example the architect’s
knowledge/experience, organisation’s culture.

10
Table 2. Influence of cognitive biases on architecture decision making
Bias
(1)
Description
(2)
Main contextual factors
that influence archi-
tect’s judgment
(3)
Influenced as-
pects of architec-
ture decision
making
(4)
Framing
effect
Drawing different conclu-
sions from the same in-
formation depending on
the form of presentation
[8]
Form of presentation
Architect’s pref-
erences, scope of
considered archi-
tectural issues
and alternatives
Confir-
mation
bias
Focus on searching for
facts that confirms one's
beliefs, while ignoring
opposing information. [9]
The architect’s beliefs
All the aspects.
IKEA
effect
Overvaluing items that
were created or assembled
by us personally. [10]
Who was the author of a
given design; time spent
on a given design.
Scope of consid-
ered alternatives,
preferences.
Parkin-
son's Law
of trivi-
ality
Focusing time and effort
on trivial matters while
often omitting the truly
important ones. [11]
None.
All the aspects.
Anchor-
ing
Relying on one piece of
information more heavily
than any other, usually on
the first one that we were
exposed to. [12]
Order of obtaining in-
formation.
Scope and per-
ception of im-
portance of con-
sidered require-
ments; prefer-
ences.
Curse of
knowledg
e
When individuals, due to
having a different level of
knowledge, interpret facts
differently.[13]
The knowledge, experi-
ence and background of
the stakeholders.
All the aspects.
Pro-
innova-
tion bias
An overly optimistic ap-
proach in adopting innova-
tive solutions. [16]
The architect’s state of
mind with respect to
innovative solutions.
Preferences,
scope of consid-
ered alternatives.
Planning
fallacy
Underestimation of the
time it will take to com-
plete a task. [17]
Problem complexity
Preferences

11
Bias
(1)
Description
(2)
Main contextual factors
that influence archi-
tect’s judgment
(3)
Influenced as-
pects of architec-
ture decision
making
(4)
Band-
wagon
effect
The phenomenon of peo-
ple more likely adapting
ideas/buying products that
have already been widely
accepted. [19]
Existing widely accept-
ed solutions.
Preferences
Irrational
escalation
Continuing an ac-
tion/investment, because
of a similar prior one, even
if the previous one turned
out to be a wrong decision.
[18]
Existence of an initial
solution, course of ac-
tion contradicting the
use of an initial solution.
Preferences
Law of
the in-
strument
Using a tool/skill that you
possess everywhere, even
in contexts where it is not
appropriate. [15]
Architectural solutions
focal for an architect.
Scope of consid-
ered alternatives,
preferences.
Optimism
bias
Overestimating the proba-
bility of favourable out-
comes of our decisions.
[14]
The architect’s state of
mind.
All the aspects
4 Biases in the practical conditions of architectural decision
making
Having established that cognitive biases are a common phenomenon in architecture
decision making, we explore real-world factors that can influence their manifestation
or affect the magnitude of their influence on architecture decision making.
Biases and time.
Cognitive biases are an integral element of human nature. They have been shaped
by the evolution of the human mind, and as such are a result of the adaptation to the
conditions of the environment. Although they possibly distract architects from craft-
ing a fully rational, thoroughly deliberated design, they potentially enable the qualities
desirable by today’s hectic software industry: rapid architecting and quick response to
changes or emergencies.
As the “need for speed” concerns more and more software engineers, the role of
system 1 will certainly be increasing at the cost of diminishing the role of system 2. It
means that even more decisions will be made intuitively without a thorough delibera-

12
tion. This may substantially hinder the quality of a software architecture. At the same
time, architecting efficiently under the pressure of time is something very desirable in
the frenetic software industry, as well as in emergency cases.
It seems that there are two basic ways of addressing this challenge:
1. Applying debiasing techniques – this seems to be generally difficult, as the main
factor limiting rational judgement is the lack of time and other external pressures.
In order to ensure rational decision making, we have to give architects more time
to conclude a decision and to restrain the external pressures. This is in many cases
impossible, as we have limited or no control over the conditions that are external to
architectural decision making;
2. Accepting “the rules of the game” (biases) and trying to exploit them to our ad-
vantage – this requires the development of techniques that lead to reasonable archi-
tectures under pressure of time. Hypothesising further, they could take the form of
a specific training for architects, probably similar to those used by students prepar-
ing for programming competitions: this training supposedly makes system 1 closer
to system 2 with regard to algorithms and computer programming, as trained stu-
dents decide at a glance which algorithms should be used in order to solve their ex-
ercise.
Biases and teams.
Applying group architecture decision making techniques has the potential to limit
the influence of biases, as decisions are made by people with different mindsets. This
justifies assessing an architecture by a group of stakeholders, such as in ATAM. At
the same time, group decision making brings with it the risk posed by the ‘law of
triviality’ bias.
Biases and cultural factors.
Cultural factors may magnify or diminish the influence of cognitive biases. For ex-
ample, in many cultures it is difficult for people to admit they cannot understand
something or accomplish a certain design. This will certainly strengthen the curse of
knowledge bias.
Biases and tools and methodologies.
It is also important to recognise that cognitive biases introduce a feedback between
what we create and how it is created, i.e. what we create, what we know influences
how it is created by us. This is exactly what most of the biases do – consider, for ex-
ample, anchoring bias, irrational escalation and the law of instrument biases. There-
fore, it is worth investigating how different software development methodologies,
architecture decision-making techniques, software development tools etc. interact
with cognitive biases and vice versa.

13
5 Results
RQ.1 Are biases in architecture decision making commonly observed by software
engineering practitioners?.
It turned out that the participants commonly observe biases in deciding on software
design. Both novices (less than 2 years of experience) and experts (more than 10 years
of experience) in software engineering noticed biases. Novices indicated on average 1
bias each, experts about 4 biases each. Experts have a much broader experience than
novices, which explains the observed difference.
RQ.2. What are the most significant biases in architecture decision-making?
As a result of our research, we identified 12 cognitive biases that influence architec-
ture decision making. The list of these can be found in tables 1 and 2.
RQ.3. Which of these biases result from cognitive biases inherent to the conditions of
the human mind?
All these biases, concerning architectural decision making, indicated by the work-
shop participants and listed in table 1, can be related to well-known cognitive biases.
RQ.4. Which elements of the decision-making context can bias architects' decisions?
These identified elements of the decision-making context that bias architects’ deci-
sions are: form of presentation, the architect’s beliefs, who was the author of a given
design, the time spent on a given design, the order of obtaining information, the
knowledge, experience and background of the stakeholders, the architect’s state of
mind, the problem complexity, the existing widely-accepted solutions, the course of
action contradicting the use of an initial solution, and architectural solutions focal for
an architect.
RQ.5. Which aspects of architectural decision making are influenced by the biases
that have been identified?
The above aspects of architectural decision making are: the architect’s preferences
(finally expressed by the rationale for a decision), the scope of the considered archi-
tectural issues, alternatives and requirements, and the perception of the importance of
requirements (compare section 3.2 and table 2).
RQ.6. How do practical conditions influence the extent of the biases’ influence on
architectural decision making?
Time, teams, cultural factors as well as tools and methodologies used for software
development can affect the extent of the biases’ influence on architecture decision
making.

14
6 Discussion, limitations
The volume of research on cognitive biases in software engineering is rather small
(compare section 2). Let us observe that our research confirms the findings of the
seminal paper by Tang and van Vliet [7], namely, that anchoring, framing and con-
firmation biases are among the most often observed by software engineering practi-
tioners as influencing architectural decision making.
The contribution of this paper comprises:
 the proposition of a model of how biases influence architectural decision making,
which enables a systematic, uniform analysis of various biases;
 the identification of 12 cognitive biases that influence architectural decision mak-
ing;
 an analysis of how each bias affects decision making, by identifying the elements
of the model mentioned above (elements of the context affecting decision making
and aspects of the decision-making process influenced by each bias);
 an analysis and identification of real-world factors that can potentially influence
the extent of the influence of biases on architecture decision making.
The obvious limitation of the presented results are:
 the number of workshop participants may influence the representativeness of the
results;
 although the claims of section 4 seem to be logically sound, the analysis of real-
world factors is only exploratory, hence it requires empirical substantiation to
strengthen the claims of section 4.
7 Summary and research outlook
Cognitive biases are commonly present in architecture decision making. By asking
practitioners, we identified 12 cognitive biases that can be observed most frequently.
In order to analyse their influence on architectural decision making in a uniform way,
we have developed a model of how biases ‘work’.
The common presents of cognitive biases is both virtue and vice. On one side, they
enable rapid architecting by an intuitive resolution of the architectural issues, on the
other, they may lead to suboptimal solutions and in extreme cases to a design disaster.
We can either try to accept and exploit them, or fight them. Probably, we need a kind
of a decision-making approach that balances ‘system 1’ and ‘system 2’ decision mak-
ing.
The further research outlook includes:
 obtaining a more statistically significant confirmation of the above results by inter-
views with a larger group of practitioners or by a broader industrial survey;
 investigating the interactions that may exist between the biases;

15
 developing techniques of using the knowledge about biases and their influence on
decision-making process, in order to align the architecting process with the stake-
holders’ expectations;
 carrying out an in-depth analysis of each of the identified biases.
References
1. Naur, P., Randell, B., Software Engineering Techniques. Report on a conference spon-
sored by the Nato Science Committee, Garmisch, Germany, 7th to 11th October 1968.
2. Buxton, J. N., Randell, B., Software Engineering Techniques. Report on a conference
sponsored by the Nato Science Committee, Rome, Italy, 27th to 31st October 1969.
3. Zannier, C., Chiasson, M., Maurer, F., A model of design decision making based on empir-
ical results of interviews with software designers, Information and Software Technology,
Volume 49, Issue 6, June 2007, Pages 637-653.
4. Kahneman, D.: Thinking, fast and slow. Penguin (2011)
5. Tang, A., van Vliet, H, Design Strategy and Software Design Effectiveness. IEEE Soft-
ware, vol. 29, no. 1, pp. 51-55, Jan.-Feb. 2012.
6. Van Vliet, H., Tang, A., Software Designers Satisfice. LNCS vol. 9278, Springer, 2015.
7. Van Vliet, H., Tang, A., Decision Making in Software Architecture. Journal of Software
and Systems, 2016.
8. Tversky, A., and Kahneman, D., Rational choice and the framing of decisions, Journal of
business, 1986.
9. Nickerson, R.S., Confirmation bias: A ubiquitous phenomenon in many guises, Review of
general psychology, 1998.
10. Norton, M. I., Mochon D., Ariely, D., “The IKEA Effect: When Labor Leads to Love.”
Journal of Consumer Psychology 22 (3), pp. 453–460, July, 2012.
11. Parkinson, C. Northcote (1958). Parkinson's Law, or the Pursuit of Progress. Penguin,
2002.
12. Tversky, A., Kahneman D., Judgment under uncertainty: Heuristics and biases, Utility,
probability, and human decision making. Springer Netherlands, 1975. 141-162.
13. Birch, S. AJ, Bloom P., The curse of knowledge in reasoning about false beliefs, Psycho-
logical Science 18.5 (2007): pp. 382-386.
14. Sharot, T., Neural mechanisms mediating optimism bias, Nature 450.7166 (2007): pp. 102-
105.
15. Brown, W.H., et al. AntiPatterns: refactoring software, architectures, and projects in crisis.
John Wiley & Sons, Inc., 1998.
16. Rogers, E. M, Diffusion of innovations. Simon and Schuster, 2010.
17. Buehler, R., Griffin D., Ross M., Exploring the planning fallacy: Why people underesti-
mate their task completion times, Journal of personality and social psychology 67.3
(1994): 366.
18. Bazerman, M. H., and Neale M. A., Negotiating rationally. Simon and Schuster, 1993.
19. Leibenstein, H., Bandwagon, snob, and Veblen effects in the theory of consumers' de-
mand, The quarterly journal of economics 64.2 (1950), pp. 183-207.
20. Kahneman, D., Renshon, J., Hawkish biases. In American Foreign Policy and the Politics
of Fear: Threat Inflation Since 9/11, pp. 79-96, Routledge, 2009.