Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review

Full text

Algorithmic Decisions, Desire for Control, and the Preference for
Human Review over Algorithmic Review
Henrietta Lyons
hlyons@student.unimelb.edu.au
The University of Melbourne
Melbourne, Victoria, Australia
Tim Miller
tmiller@unimelb.edu.au
The University of Melbourne
Melbourne, Victoria, Australia
Eduardo Velloso
eduardo.velloso@unimelb.edu.au
The University of Melbourne
Melbourne, Victoria, Australia
ABSTRACT
In this paper, we explore why decision subjects generally express a
preference for human reviewers of algorithmic decisions over algo-
rithmic reviewers. We theorise that decision subjects desire control
over the decision-making process in order to increase their chance
of receiving a favourable outcome. To this end, human reviewers
will be seen as easier to inuence than algorithmic reviewers, thus
providing more control. Using an online study we nd that: (1)
people who have a greater Desire for Control over their lives ex-
hibit a stronger preference for human review; (2) interaction with
a reviewer is important because it enables inuence and ensures
understanding; and (3) the higher the impact of a decision, the
greater the incentive to inuence the outcome, and the greater the
preference for human review. Our qualitative results conrm that
outcome favourability is a driver for reviewer preference, but so is
the desire to be treated with dignity.
CCS CONCEPTS
•Human-centered computing →Empirical studies in HCI.
KEYWORDS
algorithmic decision-making; contestability; reviewability; algorith-
mic fairness, accountability, and transparency
ACM Reference Format:
Henrietta Lyons, Tim Miller, and Eduardo Velloso. 2023. Algorithmic De-
cisions, Desire for Control, and the Preference for Human Review over
Algorithmic Review. In 2023 ACM Conference on Fairness, Accountability,
and Transparency (FAccT ’23), June 12–15, 2023, Chicago, IL, USA. ACM, New
York, NY, USA, 11 pages. https://doi.org/10.1145/3593013.3594041
1 INTRODUCTION
The growing use of algorithms in high-stakes decision-making has
prompted people to highlight the importance of contestability–the
ability to challenge algorithmic decisions (e.g.[
2
,
19
,
42
,
43
,
51
]).
However, there is limited guidance on what contestability for algo-
rithmic decisions should look like in practice [
1
]. Some advocate for
building decision-making systems that are contestable by design
[
2
,
19
,
38
], while others explore post-hoc contestability, researching
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic permission
and/or a fee. Request permissions from permissions@acm.org.
FAccT ’23, June 12–15, 2023, Chicago, IL, USA
©2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-0192-4/23/06.. .$15.00
https://doi.org/10.1145/3593013.3594041
perceptions of dierent review processes for algorithmic decisions
[
35
]. These options are not mutually exclusive and could be com-
bined to ensure the accountability of decision makers while also
providing decision subjects with autonomy over decisions that
impact their lives. In this paper, we focus on post-hoc review pro-
cesses. Various elements must be considered when designing such
processes, including who can appeal a decision, how a review should
be run, and who will decide the review outcome [
34
]. We explore
preferences for who should review an algorithmic decision—a hu-
man reviewer or an algorithmic reviewer—seeking to understand
the reasons behind these preferences. This research can inform the
design of algorithmic decision-making systems and processes that
enable contestation.
Recent work considering contestability nds that people prefer
human reviewers over algorithmic reviewers for algorithmic deci-
sions [
35
]. Lyons et al.’s [
35
] qualitative analysis suggests a number
of reasons for this preference, including that: humans are perceived
as more capable of understanding objections to a decision and tak-
ing into account the nuances of people’s situations; humans are
more likely to provide a dierent decision to the original decision
than another algorithm; and, humans are seen as more malleable
in their decision-making, allowing decision subjects inuence over
the outcome. In line with procedural justice literature, Lyons et al.
[
35
] also found that people prefer review processes that provide
them with process control or voice—the ability to provide informa-
tion and input to the decision [
49
]. Process control has been found to
increase perceptions of fairness of decision-making processes [
30
].
One theory explaining this nding is that having a voice allows peo-
ple to persuade and inuence the decision-maker, giving them the
feeling of control over the outcome and providing an opportunity
to achieve a more favourable outcome [27, 30, 50].
Building on these ndings, we hypothesise that decision subjects
prefer a human reviewer over an algorithmic reviewer because they
feel more in control of the process. As such, we expect human
reviewers to be viewed as easier to inuence than algorithms and,
therefore, more likely to provide a favourable (to the decision sub-
ject) outcome. In contrast, we expect algorithmic decision-makers
to be viewed as rigid in their decision-making abilities [
33
] and,
therefore, more dicult to inuence. If our proposition is true, we
would expect to see: people with higher scores for the personality
trait Desire for Control exhibiting a stronger preference for human
review over algorithmic review; a weaker preference for human
review over algorithmic review when a decision subject’s ability
to interact with, and therefore exert inuence over, a reviewer is
limited; and, a stronger preference for human review when the
stakes of the decision are high because there is a greater incentive
for people to want to exert inuence over the outcome.
764

FAccT ’23, June 12–15, 2023, Chicago, IL, USA Lyons, et al.
In an online experiment, we presented 234 participants with
three contextually dierent algorithmic decisions. For each of the
three scenarios, participants rated the impact of the decision. They
were then provided with information about how the decision could
be reviewed: in one condition, the level of interaction with the
reviewer was expressly limited. Participants were asked whether
they would prefer an algorithm or a human to review the initial al-
gorithmic decision. Having provided their preference, participants
were asked to explain it. They then completed the Desirability of
Control Scale [
11
]. Overall, participants preferred human reviewers
compared to algorithmic reviewers. We found that the higher a per-
son’s Desire for Control, the more they preferred human reviewers.
Our qualitative analysis indicated that the ability to reason with
humans and appeal to their emotions resulted in human reviewers
being seen as easier to inuence than algorithms.
We also found that the preference for human review increased
as the perceived impact of the decision increased: having a re-
viewer that understood the consequence of the decision was seen
as important, with humans being seen as more capable of this than
algorithms. Though our quantitative results did not indicate that
the level of interaction with a reviewer impacted preferences, our
qualitative results suggested otherwise. On the one hand, interac-
tion with the reviewer was seen as an essential building block for
exerting inuence. On the other hand, this interaction is important
because it enables decision subjects to feel heard and understood.
These ndings indicate that when designing algorithmic decision-
making systems and contestation processes for algorithmic deci-
sions, decision subjects value the ability to exercise some inuence,
or control, over the process and, ultimately, the decision outcome.
Control becomes even more important as the impact of the decision
increases.
Our study contributes theoretically and practically to work ex-
ploring the concept of contestability in the context of algorithmic
decision making and algorithm aversion. First, we propose the the-
ory that people prefer human reviewers over algorithmic reviewers
due to the desire for control. Second, we provide experimental evi-
dence to support this theory. Third, our qualitative analysis provides
in-depth reasons for why participants hold certain preferences, as
called for by Mahmud et al. [
36
]. Fourth, based on these ndings,
we outline design implications for algorithmic decision-making
systems.
2 RELATED WORK
2.1 Contestability: Reviewing algorithmic
decisions
The need for contestability concerning algorithmic decision-making
has been raised by academics, industry, and governments (e.g.
[
17
,
19
,
20
,
38
,
52
]). However, what contestability means in the
context of algorithmic decisions is still being conceptualised [
1
,
34
].
There is a small but growing body of research exploring contestabil-
ity in algorithmic decision-making, with work on embedding con-
testability into decision-making systems by design (e.g. [
2
,
19
,
38
])
and work exploring post-hoc review mechanisms [
35
,
51
]. Lyons et
al. [
35
] explored participants’ preferences for appeal processes that
diered across three dimensions through a conjoint analysis: who
the reviewer was, how the review was conducted, and how long the
review would take. The authors found that participants preferred
the review process that was faster, allowed decision subjects to
provide additional information and make objections to the original
decision, and that had a human reviewer. Our paper extends this
work by focusing specically on the perceptions of the reviewer
and putting forward a theory to explain such preferences through
a lens of control.
2.2
Perceptions of algorithmic decision-making
in the context of contestability
Though our focus is on secondary decision-making—i.e. contesting
algorithmic decisions and perceptions of algorithmic reviewers and
human reviewers—we draw from, and contribute to, the rapidly
growing body of research that explores perceptions of algorithmic
decision-making in the rst instance. This literature has produced
results that support the notion of algorithmic aversion—the ten-
dency to view algorithms negatively compared to humans [
14
,
21
].
In contrast, it also supports the concept of algorithmic apprecia-
tion—where people show a preference for algorithms over humans
[
4
,
21
,
31
]. The mixed results in this body of work have been at-
tributed to dierences in the decision-making contexts, decision
tasks, perceived capabilities of the decision-maker, and participants’
characteristics.
In terms of capabilities, algorithms tend to be viewed as rigid
and inexible decision-makers that, unlike humans, cannot take
into account people’s unique characteristics [
33
]. Algorithms are
also seen as less able to make subjective [
24
] or complex decisions
[
39
] compared to humans. Humans tend to be seen as capable of
checking algorithmic systems to ensure that mistakes have not
been made, acting as a form of quality control [
2
,
9
,
35
]. However,
humans are not perfect; they can be biased [
55
] and inconsistent
decision-makers. People often see algorithms as more objective
than human decision-makers [
4
,
15
]. While algorithms have been
shown to embed systematic bias [
3
,
41
], some argue that such bias
is easier to detect and x in an algorithm than in a human [
6
,
37
].
Advantages of algorithmic review also include scalability, eciency,
and lower costs in comparison to human review [2, 25].
With regard to personal characteristics, people’s knowledge of
algorithms is positively associated with perceptions of fairness
of algorithmic decision-making [
4
]. Familiarity with algorithms
making certain types of decisions has also been found to increase a
person’s preference for algorithmic decision-makers [
22
]. People’s
lifetime of exposure to human decision-making makes them more
knowledgeable about what it entails than what algorithmic decision-
making entails. Consequently, we predict that this knowledge will
make people more condent in their ability to persuade a human
reviewer to achieve a favourable outcome. Further, we expect that
humans will be viewed as more capable of interacting with the
decision subject and grasping the nuances of a situation. In contrast,
we expect algorithms to be viewed as rigid and dicult to interact
with, making them dicult to persuade.
Hypothesis 1: Decision subjects will prefer hu-
man reviewers over algorithmic reviewers for
algorithmic decisions
765

Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review FAccT ’23, June 12–15, 2023, Chicago, IL, USA
2.3 Desire for Control and the ability to
inuence the reviewer
People nd decision-making processes fairer when they can exert
control over the outcome [
35
,
50
]. According to procedural justice
literature, process control enables participation, interaction, and an
opportunity to inuence the outcome of a decision [
27
,
50
]. The
human-computer interaction community also recognises the impor-
tance of people feeling in control when interacting with technology
[
28
]. For example, one of Shneiderman’s [
47
] ‘Eight Golden Rules
of Interface Design’ includes that users of technology feel they
are in control of an interface and that it responds appropriately to
their inputs. There are various conceptualisations of control. In this
work, we focus on the personality trait Desire for Control, which
concerns “the extent to which people want control” [
10
]. To our
knowledge, the algorithmic aversion literature has not explored
the relationship between the Desire for Control and perceptions of
contestation processes in algorithmic decision-making.
People exhibiting higher Desire for Control are more likely to
try to inuence other people when such behaviour suggests an
advantage [
10
]. People with higher Desire for Control are also
more likely to believe that they can inuence or control events
that they have no control over (i.e. events that are determined by
chance) [
10
]. In consumer research, having high Desire for Control
has been linked to consumers who are less likely to accept or adopt
new products over traditional products [
16
]. It has been suggested
that this phenomenon may be due to people not having a sense of
mastery over a new product or because a new product does not
t within a person’s existing cognitive category or schema [
16
].
While this research relates to consumer products, we suggest that
these concepts generally apply to new experiences, technologies,
and ways of doing things, such as algorithmic decision-making.
Indeed, Syahrivar et al. [
48
] found that people with a higher Desire
for Control tend to perceive automated vehicles more negatively
than those with a lower Desire for Control because drivers feel less
in control of a vehicle when it is automated. We predict that people
with higher Desire for Control will prefer human reviewers over
algorithmic reviewers because they feel they have more control
over a human reviewer.
Hypothesis 2: The higher a decision subject’s
Desire for Control, the stronger their preference
for human review over algorithmic review
2.4 Procedural justice: Fair decision-making
processes provide decision subjects with the
ability to participate and interact
Process control relates to the inuence or control over the decision-
making procedure, such as the ability to present evidence and the
time to state a case. In their seminal studies in 1975, Thibaut and
Walker [
49
] found that people perceived decision-making processes
where they had process control to be fairer than processes without
it. Process control is related to the notion of voice—the ability to
provide information and input relevant to the decision. It has been
theorised that having process control and voice increases fairness
perceptions because allowing people to present information and
have a say oers a chance to persuade and inuence the decision-
maker and potentially achieve a more favourable outcome [
27
,
30
,
50
]. Importantly, there are two dimensions to voice: not only do
people want the ability to express their views and have their say,
but they also need to feel as though what they have said has been
heard and taken into account [29].
Based on these ndings, we expect people to feel more comfort-
able presenting their case to a human rather than an algorithm. Peo-
ple know how to interact with other humans and will have knowl-
edge based on experience from many previous interactions about
how to persuade and potentially inuence a human reviewer. In
comparison, people will be less familiar with algorithmic decision-
makers, representing greater uncertainty for decision subjects. Peo-
ple tend to view algorithms as rigid and inexible decision-makers
[
33
], so they will likely question their ability to inuence an algo-
rithm. Further, we expect people to question whether algorithms
are capable of interacting in a way that ensures people’s views
are understood and taken into account, thus impacting a person’s
ability to inuence.
Based on the above, the ability to inuence relies on the ability to
interact: if the ability to interact is limited, there is less opportunity
to inuence. So, when there is less opportunity to interact, we
expect the advantage of a human reviewer to be less pronounced.
Hypothesis 3: Limiting a decision subject’s abil-
ity to interact with a reviewer will result in a
weaker preference for human review over algo-
rithmic review
2.5 Decisions with a signicant impact
The ability to contest or challenge a decision, if provided at all, is
most likely to be provided for high-stakes decisions. Indeed, guide-
lines for the ethical development and use of algorithmic systems
largely focus on algorithmic decisions that have a ‘signicant’ im-
pact on a person’s life [
13
,
20
]. Article 22(1) of the European Union’s
General Data Protection Regulation contains a similar proviso: “The
data subject shall have the right not to be subject to a decision
based solely on automated processing, including proling, which
produces legal eects concerning him or her or similarly signicantly
aects him or her” (emphasis added).
Several studies have explored how the impact of a decision inu-
ences people’s perceptions of algorithmic decision-makers. Longoni
et al. [
33
] explored people’s preference for a human compared to
an algorithmic medical provider by manipulating the impact of
the medical condition while holding the context (screening for eye
conditions) constant. The high-stakes condition was a diagnosis of
macular degeneration, while the low-stakes condition was a diag-
nosis of dry eye. While a human medical provider was preferred in
both conditions, people were less averse to the algorithmic provider
for the low-stakes decision. Exploring the use of algorithms for
26 tasks that diered over various factors, including context and
impact, Castelo et al. (2019) [
12
] found that trust in algorithms
decreased as the perceived consequence of the task increased.
We expect that the greater the consequence or detriment of
a decision, the greater the incentive to control or inuence the
outcome and, therefore, the greater the preference for a human
reviewer over an algorithmic reviewer.
766

FAccT ’23, June 12–15, 2023, Chicago, IL, USA Lyons, et al.
Hypothesis 4: The greater the impact of a deci-
sion, the stronger the decision subject’s prefer-
ence for human review
3 METHOD
Akin to Lee [
24
] and Wang et al. [
53
], we conducted an online
vignette experiment to explore reviewer preference for algorithmic
decisions.
3.1 Participants
We recruited 260 participants for our study. Participants were re-
cruited using Prolic, an online crowdsourcing platform designed
for participant recruitment by researchers [
44
]. We pre-screened
participants to ensure that our sample included only residents of the
United States who were over 18 years old. The study was expected
to take approximately 15 minutes, and participants were paid £2.47.
The Human Ethics Committee of our university provided ethics
approval.
3.2 Scenarios
We developed 18 scenarios to use in our study. Scenarios are fre-
quently used in social psychology, ethics, and HCI research to
explore people’s views on various matters [
7
,
24
]. We used a rst-
person perspective. We chose three dierent contexts for our sce-
narios that, while ctional, were inspired by recent instances of
consequential algorithmic decision-making:
(1)
The use of algorithms to detect fraud by rideshare companies
[32] (RidR scenario)
(2)
The use of algorithms in performance assessment of employ-
ees [23] (Ammonite scenario)
(3)
The use of algorithms in hiring decisions [
18
,
46
](MakeAd
scenario)
To ensure that we had decisions with various impacts, we de-
veloped three decisions for each decision context. For example, in
the RidR scenario, the impact of the decision could be a formal
warning, a nancial deduction, or suspension. The nine scenarios
are described in Appendix A.
We also varied the level of interaction a decision subject would
have with the reviewer. We included two conditions:
(1)
Limited Interaction Condition: This limited the interac-
tion between the decision subject and reviewer to email only
and, similar to [
33
], specically stated “You will not interact
directly with the reviewer”
(2)
Not Limited Interaction Condition: This condition did
not limit the level of interaction.
In both conditions, we held the performance of the reviewers
constant by specifying that the reviewers were equally good at
conducting reviews and that neither was more likely than the other
to change the decision.
These 18 scenarios were used in a vignette experiment with a
3 (scenario context: RidR/Ammonite/MakeAd)
×
3 (impact of the
decision) within-subjects design, and
×
2 between-subject factor
(Interaction Condition/Not Limited Interaction Condition).
3.3 Measures
3.3.1 Subject variables. We measured Desire for Control (M=98.3,
SD=13.3) using the Desirability of Control Scale [
11
], which con-
tains 20 items and is used to measure a person’s general desire to
control the events in their life. Items include: “When it comes to
orders, I would rather give them than receive them” and “I enjoy
having control over my own destiny”. Responses were made on a
7-point scale ranging from 1 (The statement doesn’t apply to me
at all) to 7 (The statement always applies to me). For our partic-
ipants, Cronbach’s alpha was 0.80, which suggests high internal
consistency.
In line with previous research [
12
], we used one question to mea-
sure participants’ subjective impact rating of the decisions: “How
much of an impact does this decision have on your life” on a scale
with 0 (No impact) to 100 (High impact).
3.3.2 Dependent variable. Reviewer preference was measured with
a single item, “Which reviewer would you prefer to review the
decision”? Responses were on a scale that ranged from 0 (Denitely
algorithm) to 100 (Denitely human), including a midpoint at 50
(No preference). In summary, we operationalised our hypotheses
as follows:
•
H1: Decision subjects will prefer human reviewers over al-
gorithmic reviewers for algorithmic decisions. As such, the
lower bound of the condence interval of the mean of re-
viewer preference will be greater than 50.
•
H2: The higher a decision subject’s Desire for Control, the
stronger their preference for human review over algorithmic
review. As such, a decision subject’s score on the Desirability
of Control Scale will have a signicant eect on reviewer
preference, with higher scores resulting in higher reviewer
preference scores.
•
H3: Limiting a decision subject’s ability to interact with a
reviewer will result in a weaker preference for human review
over algorithmic review. As such, the interaction condition
will have a signicant eect on reviewer preference, with a
higher mean in the Not Limited Interaction Condition
than in the Limited Interaction Condition.
•
H4: The greater the impact of a decision, the stronger the
decision subject’s preference for human review. As such, a
decision subject’s subjective impact rating will have a signi-
cant eect on reviewer preference, with higher impact ratings
resulting in higher reviewer preference scores.
3.4 Procedure
Participants were provided with a Plain Language Statement and
instructions about the study. Upon providing their informed consent
to participate in the study, participants provided their demographic
information (age, gender,and ethnicity). Participants were then told
that they would be asked questions about three dierent scenarios.
They were provided with the following denition of an algorithm:
“An algorithm is a computer program that processes data to produce
a decision. In the scenarios below, an algorithm makes a decision
without human intervention.”
Participants were randomly assigned to either the Not Limited
Interaction Condition or the Limited Interaction Condition.
767

Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review FAccT ’23, June 12–15, 2023, Chicago, IL, USA
Participants were assigned the three dierent scenarios within their
condition with each scenario having a dierent impact. The pairing
between scenarios and impact was counterbalanced across the par-
ticipant sample, and the scenarios were presented in a randomised
order to each participant.
After reading Part 1 of a scenario (see Appendix A), participants
rated how much of an impact the decision in the scenario would
have on their lives. After reading Part 2 of a scenario (see Appen-
dix A) participants were asked which reviewer they would prefer
to review the decision. They were asked to explain this answer
in a free-text box. Following the three scenarios, participants re-
sponded to items in the Desirability for Control scale. We included
an attention check and presented the Desirability for Control scale
statements in a randomised order.
3.5 Analysis
3.5.1 antitative analysis. We used a linear mixed-eects model
to test whether the level of interaction, desire for control, and stakes
of the decision impact reviewer preference. Linear mixed-eects
models estimate the impact that variables set as xed eects have
on the dependent variable while controlling for variables set as
random eects. Mixed-eects models can be used to analyse non-
independent data, such as ours, where participants have provided
multiple data points.
3.5.2 alitative analysis. After indicating on a scale which re-
viewer would prefer to review the decision, we asked participants
to explain their answers in a free-text box. As each participant con-
sidered three scenarios, we received three separate pieces of text
from each participant explaining why they preferred a particular
reviewer for that context. These responses formed the qualitative
data for the study, which we analysed using Braun and Clarke’s [
8
]
six-stage approach to reexive thematic analysis. The rst author
used NVivo 12 to inductively code the data. Based on the coding,
initial themes were generated and rened through reviewing and
writing.
4 RESULTS
4.1 Participants
We excluded data from 26 participants who gave free-text answers
that consisted of one word, did not answer the question, gave non-
sensical answers [
35
]; failed the attention check [
51
]; or provided
straightlined responses [
51
] to the Desirability of Control Scale. Of
the remaining 234 participants, 50% identied as men, 47% identi-
ed as women, and 3% identied as non-binary. Participants’ ages
ranged from 18 to 72 years, with an average of 31.9 years. The
average time to complete the study was 15.6 minutes (SD = 9.7
minutes).
4.2 Quantitative analysis
Each participant provided a reviewer preference for three scenarios,
resulting in 702 preference ratings. Of these, 83% were rated 51
to 100 (Denitely human), 6% were rated 50 (No preference), and
11% were rated from 0 (Denitely algorithm) to 49. We used the
simpleboot
R package to calculate a 20% trimmed bootstrapped
mean (M = 85.86) for reviewer preference across all scenarios and
conditions with a 95% condence interval (83.83, 87.67) (based on
2000 bootstrap replicates). Given that the condence interval does
not include 50, this indicates that Hypothesis 1 is supported: de-
cision subjects prefer human reviewers over algorithmic reviewers
for algorithmic decisions.
We built a linear mixed-eects model in RStudio, using the
lme4
package [
5
], to test whether the level of interaction, desire for
control, and stakes of the decision impact reviewer preference.
Unlike linear regression models, mixed-eects models can be used
to analyse non-independent data, such as ours, where participants
have provided multiple data points. Linear mixed-eects models
estimate the impact that variables set as xed eects have on the
dependent variable while controlling for variables set as random
eects. To control for repeated measures, we specied participants
as a random eect in our model. The dependent variable in our
model was reviewer preference.
We initially ran a null model to determine the intra-class correla-
tion (ICC) to check if multi-level modelling was the correct choice
for our analysis. The ICC measures the degree of clustering in the
data. The ICC was 0.3, which suggests that a mixed-eects model is
a good choice for analysis because our data is nested within individ-
uals. We then built a model that included the scenario (the context
and the decision), the interaction condition, and the interactions
between these variables as xed eects. We also included Desire
for Control and impact rating as xed eects. We specied partici-
pants as a random eect in our model. The dependent variable was
reviewer preference. Predictors that were not signicant (
𝑝<.
05)
were discarded. This was true for the interaction condition and
scenario as well as the interactions between these variables. The
model was recalculated, excluding these variables. The nal model
is reported in Table 1. The results indicate that participants’ impact
ratings and Desire for Control both signicantly predicted reviewer
preference. Both variables aect reviewer preference in a positive
direction: an increase in impact rating and an increase in Desire for
Control both increase preference for a human reviewer. Thus, we
have support for Hypothesis 2 and Hypothesis 4. However,
we did not have support for Hypothesis 3: expressly limiting
the interaction did not signicantly aect reviewer preference.
Table 1: Results of the nal mixed-eects model
Variable Estimate SE tp-value
(Intercept) 41.38 9.00 4.60 <0.001
Impact rating 0.25 0.05 5.56 <0.001
Desire for Control 0.17 0.09 2.01 0.045
Compared to the null model, the mixed-eects model is statisti-
cally signicant (
𝜒2(
2
)=
35
.
94
, 𝑝 <.
001). We used the
piecewiseSEM
package [
26
] to calculate goodness-of-t: the model explains 5% of
the variance in reviewer preference (Marginal
𝑅2=.
05, Conditional
𝑅2=.
31). Marginal R
2
describes the variance caused by the xed
eects, while Conditional R
2
describes the variance explained by
both the xed and the random eects [40].
Visual inspection of residual plots indicated deviations from
homoscedasticity and normality. However, mixed-eects models
768

FAccT ’23, June 12–15, 2023, Chicago, IL, USA Lyons, et al.
have been shown to be robust even when assumptions of normality
and homoscedasticity are violated [45].
4.3 Qualitative results
To add depth to the quantitative results, we analysed participants’
responses about why they chose their preferred reviewer using
Braun and Clarke’s method for thematic analysis [
8
]. All responses
were coded inductively in NVivo 12. Two key themes help to ex-
plain participants’ reviewer choice: (1) Self-interest: the desire for a
favourable outcome, and (2) Dignity: being heard and understood.
4.3.1 Self-interest: the desire for a favourable outcome. Outcome
favourability was a strong driver of reviewer preference: many
participants believed that a human reviewer would be more likely
to provide them with a dierent, more favourable decision than
an algorithmic reviewer: “I think I would have a better chance of
a human reviewer changing the original decision as opposed to an
algorithm which computerized” (P26).
For some, this belief was inuenced by the experience of re-
ceiving a negative initial decision from an algorithmic decision-
maker. There were concerns that an algorithmic reviewer would
have the same or similar programming to the initial decision-
maker and would therefore produce the same (unfavourable)
result. Indeed, a number of participants who received an initial
algorithmic decision they viewed as positive preferred an algorith-
mic reviewer: “I believe the algorithm that got me shortlisted the rst
time, will shortlist me again” (P79).
Many participants, noting the text of the scenario that stated
that neither reviewer was more likely to change the outcome, stated
that they had no preference for the reviewer because neither
reviewer provided them with a better chance at receiving a
favourable decision. Despite this, many participants preferred a
human reviewer based on their belief that they could inuence a
human reviewer and improve the decision outcome. In justifying
their reviewer choice, participants used terms such as “sway the
opinion”, “persuade”, and “convince” to change the outcome: “The
human reviewer would explain why and then you’d be able to convince
him to reverse the decision” (P205).
The ability to inuence requires an interaction with the
reviewer. Participants highlighted the need for an explanation about
the decision, space for a decision subject to be able to explain the
situation and put forward a case, and a reviewer who can listen
and understand the points raised: “I would want my reviewer to be
a human because I feel that I would be able to explain myself more
and be able to make a connection to the individual that is reading my
review. I would be able to go into depth in my reasons as to why my
decision should be reviewed, and be condent that the individual will
be able to reevaluate the algorithm’s decision” (P231).
Participants saw their ability to interact with, and therefore
inuence, an algorithm as reduced due to rigid programming and
limited ability to allow for an explanation, understand the situation
and understand the points being raised: “An algorithm does not give
you the ability to explain yourself and the events that occurred. It
just makes a decision based on the variables it is given that is already
programmed into its code. However, you can convince a person to be
on your side in this situation” (P64).
The ability to inuence also requires a decision subject to have
knowledge of the reviewer and tactics that could be used to
persuade them. One participant explained that their preference
was for an algorithm because they felt unable to eectively persuade
a human reviewer: “I prefer the objectiveness of an algorithm, I don’t
typically get along very well with people, and I nd it hard to convince
them of my position or to understand what they are thinking” (P101).
Another participant felt condent in their ability to inuence the
reviewer based on their personality: “I could probably sway a human
in my favor since I have a good personality but the algorithm is set in
stone, there’s no way around it” (P13). In terms of persuasion tactics,
many participants raised the ability to convince a human reviewer
using rational argument or ‘reason’:“I can actually reason with a
human being and see if I can persuade them to see that the algorithm
was wrong about me, and possibly get them to give me another chance.
I can’t reason with a computer algorithm. I feel I have a better shot at
getting the decision overturned if I speak with a real person” (P186).
Participants noted that they would try to appeal to a human
reviewer using an emotional plea, relying on the reviewer’s abil-
ity to feel empathy and hopefully change the decision. This was
particularly the hope in high-impact decisions:“A human be-
ing will understand the impact of someone losing their job and may
feel bad and let me keep mine with a promise to improve” (P77). In
contrast to human reviewers, algorithmic reviewers were seen as
“set in stone” (P13) and unable to empathise or be reasoned with.
Participants did not refer to any tactics they could use to convince
an algorithm but tended to conclude that computers were limited
by their programming.
When a decision has signicant consequences, participants tend
to prefer the reviewer they think will provide them with a favourable
outcome, in this case, a human. When the decision carries less
weight, participants tend to care less about who conducts the
review. Decisions deemed “more detrimental”, “so important”, or
“aects me so strongly” resulted in participants not wanting to “take
chances” with an algorithmic reviewer. A number of participants
felt as though they had a better chance at a positive outcome with
a human reviewer, whereas an algorithm represented uncertainty:
“I would have a better chance of keeping my job if I was reviewed by
a human. Because it is so important I do not want to take chances”
(P8).
4.3.2 Dignity: being heard and understood. Participants were keen
to have the reviewer understand their situation, with many
participants highlighting the need to explain the situation to the
reviewer. Many felt that it would be easier to explain to a human: “I
could more easily explain the situation to a human without feeling like
I need to use keywords a computer would recognize” (P6). Participants
also highlighted the need to be heard. It was not enough to just have
voice, there was also a need to be listened to and understood:
“It seems the more automated things get, the harder it is becoming to
depend on human interaction being understood or even listened to”
(P81).
In regard to the RidR Scenario in particular, participants believed
that a human would understand that a phone can run out of bat-
tery given that a majority of people would have experienced this
occurrence: it is a shared human experience. In contrast, most
participants did not think that an algorithm would understand this
769

Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review FAccT ’23, June 12–15, 2023, Chicago, IL, USA
particular scenario and would not register the decision subject’s
explanation as valid: “A fellow human would understand that some-
times, people forget to charge their phones and as a result, they die.
We’ve all been there. An algorithm is simply unable to understand
this” (P186).
Many participants highlighted the desire for reviewers to under-
stand how much a decision impacts their life, particularly in the
case of high-consequence decisions such as losing a job or income.
Participants wanted these high-impact decisions to be consid-
ered with due care. Humans were seen as far more capable of
understanding the detrimental consequences of decisions than algo-
rithms: “I prefer a human review this because it’s extremely important
to me, which an algorithm can’t understand but a human can” (P120).
Again, empathy was seen as valuable by allowing human reviewers
to truly understand the gravity of a decision, and as a consequence,
make a considered decision: “A human can better empathize with
how severely the formal warning can aect my income and review
with connection to my human experience” (P44). For some partici-
pants, the inability of machines to exhibit empathy and understand
the situation resulted in a lack of trust: “Being considered or rejected
for an important job is too critical to be left up to an algorithm, espe-
cially when it is for subjective reasons such as body language. I would
not trust an algorithm to have the compassion needed to understand
such a situation” (P66).
For some participants, having an algorithm determine an impor-
tant decision was an aront to their dignity:“But honestly who
wants to the decision that could impact a person’s raise or even their
job to a machine?” (P40).
5 DISCUSSION
The results of this study provide support for the theory that the
preference for human reviewers over algorithmic reviewers can be
partially explained by the fact that humans are viewed as easier to
inuence than algorithms, thus providing more control over the
decision-making process and increasing the likelihood of achieving
a favourable outcome.
5.1 The higher the Desire for Control, the
stronger the preference for human review
Our hypothesis that the higher a person’s Desire for Control, the
stronger their preference for human review was supported by the
quantitative results. We suggested that people exhibiting high De-
sire for Control would prefer a human reviewer because they would
be more familiar with how to inuence a human compared to an
algorithm and would therefore feel more comfortable in their abil-
ity to control the outcome. Our qualitative results demonstrate
that people have a good understanding of human capabilities. This
understanding provided condence that not only would a human
reviewer understand a decision subject’s point of view and situation,
but that the decision subject would have the capacity to exercise a
level of control over the review process by being able to inuence
the reviewer’s decision. Understanding how people think and what
types of appeal (e.g. rational argument, emotional plea) can inu-
ence decision-making is developed through years of interacting
with other people. In contrast, algorithmic decision-making is a
relatively new form of technology. A number of participants noted
that they did not know enough about algorithms, which impacted
their trust in an algorithmic reviewer. In line with previous research
(e.g. [
33
]), many participants’ responses suggested a narrow view
of algorithmic decision-making, with most seeing them as rigid
systems that rely on programming with limited ability to consider
relevant context and nuance.
5.2 The ability to interact with the reviewer
Our hypothesis that limiting the level of interaction with the re-
viewer would result in a weaker preference for a human reviewer
was not supported by the quantitative results. As predicted, the
mean reviewer preference for the Not Limited Interaction Con-
dition was higher than the Limited Interaction Condition
(indicating a stronger preference for human review), however, the
dierence was not signicant. We acknowledge three possible ex-
planations for this: (i) being able to interact with the reviewer is
not a factor in participants’ preference for human review; (ii) our
manipulation did not work; or (iii) given that this was a between-
subjects manipulation, our study was underpowered to detect any
signicant eect. A visual inspection of the two distributions indi-
cates that they were similar and, given that the qualitative results
strongly suggest that the ability to interact with the reviewer is an
important consideration for participants in choosing a reviewer,
we suspect that the manipulation did not work as planned.
Our manipulation was based on two expectations: (1) by omit-
ting any information about interaction, participants would assume
that they could interact with the reviewer, and they would favour a
human reviewer over an algorithmic reviewer because they know
how to interact with, and inuence, a human (Not Limited In-
teraction Condition); and (2) specifying that there would be
no direct interaction with the reviewer and limiting interaction to
writing/email would impact the desire for human review due to
there being less ability to inuence (Limited Interaction Condi-
tion). The qualitative results indicate that across these conditions,
algorithms were seen as less capable of interacting meaningfully
with decision subjects compared to humans due to a limited ability
to provide an explanation, reason with, and understand a decision
subject’s situation. The manipulation may have worked if the choice
had been between interacting with a human reviewer in person
compared to interacting with a human reviewer over email, but
the choice was between a human and an algorithm. Ultimately, we
think that perceptions around algorithms’ limited capabilities to
meaningfully interact outweighed the impact of limiting the inter-
action with the reviewer, so there was no marked change in the
preference for human review.
In contrast, our qualitative results make it clear that interaction
is a key driver of reviewer preference. We nd that people want
to interact with the reviewer for two key purposes: (1) to be able
to inuence them and achieve a more favourable outcome, (2) to
make sure they have been heard and understood.
5.3 Design implications
Based on our ndings, allowing decision subjects some control over
the review process and/or outcome is important, especially for deci-
sions that have a signicant impact. Determining how much control
to provide and how to provide it will likely be context dependent.
770

FAccT ’23, June 12–15, 2023, Chicago, IL, USA Lyons, et al.
As explored in this work, one option for providing decision subjects
with control is to allow decision subjects to interact with the re-
viewer. The level of interaction and how that interaction occurs can
vary greatly. For example, interaction could include the ability to
ask questions, provide additional information, argue your case etc.
Further, interaction can be synchronous or asynchronous. For ex-
ample, review processes for human decisions often involve lodging
written complaints and receiving written responses (asynchronous)
or arguing your case in person (synchronous). While unable to be
“in person” with an algorithm, a chatbot can provide synchronous
responses.
Designing algorithms that allow for meaningful interactions is
still a work in progress. Hirsch et al. [
19
] and Mulligan et al. [
38
]
have been exploring ways to build algorithmic decision-making
systems that allow for contestability, where users of the systems,
as well as decision subjects, can identify and correct errors and
can also lodge complaints or disagreements with the system’s out-
put and decision-making. This ability to “argue with machines”
would provide decision subjects with a better understanding of the
decision-making algorithm; the ability to interact with the system;
and a feeling of control over the decision-making process.
Providing for interaction is one thing, ensuring that people feel
that they have been provided with an adequate opportunity to
interact and that they have been heard is another matter. While
algorithms may be technically able to interact, perceptions of their
capability to understand and empathise may limit decision subjects’
satisfaction around having been heard.
While we focus on the entity conducting the review in this work,
there are other aspects of a review process that can be designed
to allow for control. For example, the ability to make objections
to the initial decision or to provide additional information gives
decision subjects some control [
35
]. Further, the provision of clear
information about how the decision-making algorithm works and
an explanation specically explaining the algorithmic decision may
also help to increase trust and provide decision subjects under-
standing of how an algorithmic decision could be changed, thus
increasing their feelings of control.
6 LIMITATIONS AND FUTURE WORK
Our study took a scenario-based approach, which allowed us to
manipulate certain variables in a controlled manner and, as a re-
sult, increase the internal validity of the work. A consequence of
this approach, however, is that it has lower external validity than
conducting a naturalistic eld study exploring these questions. In
reality, people who are in the position of the decision subject are
likely to feel the consequences of the decision more acutely. Despite
these shortcomings, the use of scenarios to understand people’s
attitudes and perceptions of various dilemmas is frequently used
[
24
], and various studies suggest that people respond similarly in
reality to how they respond to hypothetical scenarios [54].
We focused on reviewers connected with the company that made
the initial algorithmic decision. Having received a negative initial
algorithmic decision, many participants highlighted a lack of trust
in the company’s algorithms. They suggested that any reviewing
algorithm would contain similar programming to the decision-
making algorithm and, as such, would produce a similar result.
Future studies could explore the impact of being able to seek a
review of the decision from an independent reviewer. For example,
if a reviewing algorithm was developed by a research institute or
an independent government body, such as an ombudsman, people
may be more willing to select and accept an algorithmic reviewer.
Given that the focus of our research is on contesting algorithmic
decisions, our scenarios contained unfavourable decisions made by
algorithms that people would wish to contest. A consequence of
this was that some participants expected an algorithmic reviewer
to provide the same, unfavourable decision on review and so picked
a human reviewer. Future studies could include scenarios where
humans make an initial, unfavourable decision to test the impact
of this initial decision on reviewer choice.
There are particular groups of people who are often disadvan-
taged by human decision-making due to human bias. For example,
a recent study by Bigman et al. [
6
] explains how human bias in med-
ical decision-making often disadvantages Black people and people
with a low income. Bigman et al. [
6
] nd that aversion to algorithmic
decision-making decreases when biases in human decision-making
are explicitly highlighted. The authors acknowledge that systemic
bias is often embedded in algorithmic decision-making but suggest
that such bias is easier to locate and x than human bias [
6
]. We
did not target a specic sample of participants based on their de-
mographics, but note that future research on review processes for
algorithmic decision-making should explore whether people who
are often disadvantaged by human decision-making have dierent
reviewer preferences compared to those who are not.
Finally, while we have explored how a number of dierent vari-
ables impact reviewer preference, we acknowledge that reviewer
preference will be multiply determined: many variables will im-
pact a person’s preference for a particular reviewer. More work is
needed to isolate the specic impact of factors such as outcome
favourability, dignity, and control on reviewer preferences. Future
research could also explore additional variables that might impact
reviewer preference, such as the style of review, personal charac-
teristics such as ethnicity, as well as psychological measurements
such as self-ecacy.
7 CONCLUSION
Over a number of decision-making contexts, we nd that humans
are preferred to algorithms as reviewers of algorithmic decisions.
Our ndings suggest that this preference can, at least in part, be
attributed to the desire to exercise control or inuence over a re-
viewer in order to achieve a favourable outcome. The personality
trait, Desire for Control, was linked to a stronger preference for
human review, with participants exhibiting greater condence in
their ability to inuence a human reviewer compared to an algo-
rithmic reviewer. The preference for human review also increased
with the perceived impact of the decision, with participants em-
phasising the importance of receiving a favourable outcome when
the decision was more consequential and also a desire for the de-
cision to be considered with due care. These results can be used
to inform the design of algorithmic decision-making systems as
well as contestation processes for such decisions. These ndings
also contribute to the growing literature exploring perceptions of
771

Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review FAccT ’23, June 12–15, 2023, Chicago, IL, USA
algorithmic decision-making by exploring Desire for Control as a
driver of perception.
ACKNOWLEDGMENTS
We would like to thank our reviewers for their valuable feedback
on previous versions of this paper. This research was partly funded
by Australian Research Council Discovery Grant DP190103414.
Henrietta Lyons is supported by the Faculty of Engineering and
Information Technology Ingenium scholarship program. Eduardo
Velloso is the recipient of an Australian Research Council Discovery
Early Career Researcher Award (Project Number: DE180100315)
funded by the Australian Government.
REFERENCES
[1]
Kars Alfrink, Ianus Keller, Gerd Kortuem, and Neelke Doorn. 2022. Contestable
AI by Design: Towards a Framework. Minds and Machines (2022), 1–27.
[2]
Marco Almada. 2019. Human Intervention in Automated Decision-making:
Towardthe Construction of Contestable Systems. In Proce edings of the Seventeenth
International Conference on Articial Intelligence and Law (ICAIL ’19). 2–11.
[3]
Julia Angwin, Je Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias.
Retrieved 24 July 2020 from https://www.propublica.org/article/machine-bias-
risk-assessments-in-criminal-sentencing.
[4]
Theo Araujo, Natali Helberger, Sanne Kruikemeier, and Claes H De Vreese.
2020. In AI we trust? Perceptions about automated decision-making by articial
intelligence. AI & Society (2020), 1–13.
[5]
Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2014. Fitting linear
mixed-eects models using lme4. arXiv preprint arXiv:1406.5823 (2014).
[6]
Yochanan E Bigman, Kai Chi Yam, Déborah Marciano, Scott J Reynolds, and Kurt
Gray. 2021. Threat of racial and economic inequality increases preference for
algorithm decision-making. Computers in Human Behavior 122 (2021), 106859.
[7]
Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel
Shadbolt. 2018. “It’s Reducing a Human Being to a Percentage”: Perceptions of
Justice in Algorithmic Decisions. In Proceedings of the 2018 CHI Conference on
Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM,
1–14. https://doi.org/10.1145/3173574.3173951
[8]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.
Qualitative Research in Psychology 3 (2006), 77–101.
[9]
Kiel Brennan-Marquez and Stephen Henderson. 2019. Articial Intelligence
and Role-Reversible Judgment. Journal of Criminal Law and Criminology 109, 1,
Article 2 (2019), 27 pages.
[10]
Jerry M Burger. 1992. Desire for control: Personality, social and clinical perspectives.
New York: Plenum Press.
[11]
Jerry M Burger and Harris M Cooper. 1979. The desirability of control. Motivation
and emotion 3, 4 (1979), 381–393.
[12]
Noah Castelo, Maarten W Bos, and Donald R Lehmann. 2019. Task-dependent
algorithm aversion. Journal of Marketing Research 56, 5 (2019), 809–825.
[13]
D Dawson, E Schleiger, J Horton, J McLaughlin, C Robinson, G Quezada,
J Scowcroft J, and S Hajkowicz. 2019. Articial Intelligence: Australia’s Ethics
Framework. Data61 CSIRO (2019).
[14]
Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm
aversion: People erroneously avoid algorithms after seeing them err. Journal of
Experimental Psychology: General 144, 1 (2015), 114.
[15]
Jaap J Dijkstra, Wim BG Liebrand, and Ellen Timminga. 1998. Persuasiveness of
expert systems. Behaviour & Information Technology 17, 3 (1998), 155–163.
[16]
Ali Faraji-Rad, Shiri Melumad, and Gita Venkataramani Johar. 2017. Consumer
desire for control as a barrier to new product adoption. Journal of Consumer
Psychology 27, 3 (2017), 347–354.
[17]
Organisation for Economic Co-operation and Development. 2019. OECD
Principles on Articial Intelligence. Retrieved 24 February 2020 from
https://www.oecd.org/going-digital/ai/principles/.
[18]
Drew Harwell. 2019. A face-scanning algorithm increasingly de-
cides whether you deserve the job. Retrieved August 18, 2021 from
https://www.washingtonpost.com/technology/2019/10/22/ai-hiring-face-
scanning-algorithm- increasingly-decides-whether- you-deserve-job/
[19]
Tad Hirsch, Kritzia Merced, Shrikanth Narayanan, Zac E. Imel, and David C.
Atkins. 2017. Designing Contestability: Interaction Design, Machine Learning,
and Mental Health. In Proceedings of the 2017 Conference on Designing Interactive
Systems. 95–99.
[20]
Anna Jobin, Marcello Ienca, and Ey Vayena. 2019. The global landscape of AI
ethics guidelines. Nature Machine Intelligence 1 (2019), 389–399.
[21]
Ekaterina Jussupow, Izak Benbasat, and Armin Heinzl. 2020. Why are we averse
towards Algorithms? A comprehensive literature Review on Algorithm aversion..
In Twenty-Eighth European Conference on Information Systems (ECIS2020).
[22]
Max F. Kramer, Jana Schaich Borg, Vincent Conitzer, and Walter Sinnott-
Armstrong. 2018. When Do People Want AI to Make Decisions?. In Proceedings
of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (New Orleans, LA,
USA) (AIES ’18). 204–209. https://doi.org/10.1145/3278721.3278752
[23]
Colin Lecher. 2019. How Amazon automatically tracks and res ware-
house workers for ‘productivity’. Retrieved August 18, 2021 from
https://www.theverge.com/2019/4/25/18516004/amazon-warehouse-
fulllment-centers- productivity-ring-terminations
[24]
Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fair-
ness, trust, and emotion in response to algorithmic management. Big Data &
Society 5, 1 (2018), 1–16.
[25]
Min Kyung Lee, Anuraag Jain, Hea Jin Cha, Shashank Ojha, and Daniel Kusbit.
2019. Procedural Justice in Algorithmic Fairness: Leveraging Transparency and
Outcome Control for Fair Algorithmic Mediation. Proceedings ACM Hum.-Comput.
Interact. 3, CSCW, Article 182 (Nov. 2019), 26 pages. https://doi.org/10.1145/
3359284
[26]
Jonathan S Lefcheck. 2016. piecewiseSEM: Piecewise structural equation mod-
elling in r for ecology, evolution, and systematics. Methods in Ecology and
Evolution 7, 5 (2016), 573–579.
[27]
Gerald Leventhal. 1980. What Should Be Done With Equity Theory? In Social
Exchange, Willis R.H Gergen K.J., Greenberg M.S. (Ed.). Plenum Press.
[28]
Hannah Limerick, David Coyle, and James W Moore. 2014. The experience of
agency in human-computer interactions: a review. Frontiers in human neuro-
science 8 (2014), 643.
[29]
E. Allan Lind and Christiane Arndt. 2016. Perceived Fairness and Regulatory Policy:
A Behavioural Science Perspective on Government-Citizen Interactions. OECD
Regulatory Policy Working Papers 6. OECD Publishing. https://doi.org/10.1787/
1629d397-en
[30]
E Allan Lind, Ruth Kanfer, and P Christopher Earley. 1990. Voice, control, and
procedural justice: Instrumental and noninstrumental concerns in fairness judg-
ments. Journal of Personality and Social Psychology 59, 5 (1990), 952.
[31]
Jennifer M Logg, Julia A Minson, and Don A Moore. 2019. Algorithm appreciation:
People prefer algorithmic to human judgment. Organizational Behavior and
Human Decision Processes 151 (2019), 90–103.
[32]
Natasha Lomas. 2021. Dutch court rejects Uber drivers’ ‘robo-ring’ charge
but tells Ola to explain algo-deductions. Retrieved August 18, 2021
from https://techcrunch.com/2021/03/12/dutch-court- rejects-uber-drivers- robo-
ring-charge- but-tells-ola- to-explain-algo- deductions/
[33]
Chiara Longoni, Andrea Bonezzi, and Carey K Morewedge. 2019. Resistance to
medical articial intelligence. Journal of Consumer Research 46, 4 (2019), 629–650.
[34]
Henrietta Lyons, Eduardo Velloso, and Tim Miller. 2021. Conceptualising Con-
testability: Perspectives on Contesting Algorithmic Decisions. Proceedings of the
ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–25.
[35]
Henrietta Lyons, Senuri Wijenayake, Tim Miller, and Eduardo Velloso. 2022.
What’s the appeal? Perceptions of review processes for algorithmic decisions. In
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems.
1–15.
[36]
Hasan Mahmud, AKM Najmul Islam, Syed Ishtiaque Ahmed, and Kari Smolander.
2022. What inuences algorithmic decision-making? A systematic literature
review on algorithm aversion. Technological Forecasting and Social Change 175
(2022), 121390.
[37]
Sendhil Mullainathan. 2019. Biased Algorithms Are Easier to Fix Than Biased
People. Retrieved August 31, 2021 from https://www.nytimes.com/2019/12/06/
business/algorithm-bias- x.html
[38]
Deirdre K. Mulligan, Daniel Kluttz, and Nitin Kohli. 2020. Shaping Our Tools:
Contestability as a Means to Promote Responsible Algorithmic Decision Making
in the Professions. In After the Digital Tornado: Networks, Algorithms, Humanity,
Kevin Werbach (Ed.). New York: Cambridge University Press.
[39]
Rosanna Nagtegaal. 2021. The impact of using algorithms for managerial deci-
sions on public employees’ procedural justice. Government Information Quarterly
38, 1 (2021), 101536.
[40]
Shinichi Nakagawa and Holger Schielzeth. 2013. A general and simple method
for obtaining R2 from generalized linear mixed-eects models. Methods in ecology
and evolution 4, 2 (2013), 133–142.
[41]
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019.
Dissecting racial bias in an algorithm used to manage the health of populations.
Science 366, 6464 (2019), 447–453.
[42]
Australian Government Department of Industry Innovation and Science.
2019. AI Ethics Framework. Retrieved on 6 December 2019
from https://www.industry.gov.au/data-and-publications/building-australias-
articial-intelligence-capability/ai-ethics-framework/ai-ethics-principles.
[43]
Independent High-Level Expert Group on Articial Intelligence. 2019.
Ethics Guidelines for Trustworthy AI. Retrieved 24 February
2020 from https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-
trustworthy-ai.
[44]
Stefan Palan and Christian Schitter. 2018. Prolic.ac—A subject pool for online
experiments. Journal of Behavioral and Experimental Finance 17 (2018), 22–27.
772

FAccT ’23, June 12–15, 2023, Chicago, IL, USA Lyons, et al.
[45]
Holger Schielzeth, Niels J Dingemanse, Shinichi Nakagawa, David F Westneat,
Hassen Allegue, Céline Teplitsky, Denis Réale, Ned A Dochtermann, László Zsolt
Garamszegi, and Yimen G Araya-Ajoy. 2020. Robustness of linear mixed-eects
models to violations of distributional assumptions. Methods in Ecology and
Evolution 11, 9 (2020), 1141–1152.
[46]
Natalie Sheard. 2022. Employment Discrimination by Algorithm: Can Anyone
be Held Accountable? University of New South Wales Law Journal (forthcoming)
45, 2 (2022).
[47]
Ben Shneiderman, Catherine Plaisant, Maxine S Cohen, Steven Jacobs, Niklas
Elmqvist, and Nicholas Diakopoulos. 2016. Designing the user interface: strategies
for eective human-computer interaction. Pearson.
[48]
Jhanghiz Syahrivar, Tamás Gyulavári, Melinda Jászberényi, Katalin Ásványi,
László Kökény, and Chairy Chairy. 2021. Surrendering personal control to
automation: Appalling or appealing? Transportation Research Part F: Trac
Psychology and Behaviour 80 (2021), 90–103.
[49]
John Thibaut and Laurens Walker. 1975. Procedural Justice: A Psychological
Analysis. Lawrence Erlbaum Associates, Hillsdale, NJ.
[50]
John Thibaut and Laurens Walker. 1978. A theory of procedure. Calif. L. Rev. 66
(1978), 541.
[51]
Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. “At the End
of the Day Facebook Does What It Wants” How Users Experience Contesting
Algorithmic Content Moderation. Proceedings of the ACM on Human-Computer
Interaction 4, CSCW2 (2020), 1–22.
[52]
Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2018. Counterfactual
Explanations without Opening the Black Box: Automated Decisions and the
GDPR. Harvard Journal of Law and Technology 31, 2 (2018), 841–887.
[53]
Ruotong Wang, F. Maxwell Harper, and Haiyi Zhu. 2020. Factors Inuencing
Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, De-
velopment Procedures, and Individual Dierences. In Proceedings of the 2020
Conference on Human Factors in Computing Systems (Honolulu, HI, USA). 1–14.
https://doi.org/10.1145/3313831.3376813
[54]
Sarah Woods, Michael Walters, Kheng Lee Koay, and Kerstin Dautenhahn. 2006.
Comparing human robot interaction scenarios using live and video based meth-
ods: towards a novel methodological approach. In 9th IEEE International Workshop
on Advanced Motion Control, 2006. IEEE, 750–755.
[55]
John Zerilli, Alistair Knott, James Maclaurin, and Colin Gavaghan. 2019. Trans-
parency in algorithmic and human decision-making: Is there a double standard?
Philosophy & Technology 32, 4 (2019), 661–683.
A SCENARIOS
A.1 RidR
A.1.1 Part 1. Imagine that you work as a driver for a ride share
company called “RidR”. This is your only source of income. RidR
uses an algorithm to automatically penalise drivers if fraudulent
activity is detected. Fraudulent activity includes acts such as collu-
sion between drivers to trigger ‘surge pricing’. This occurs when a
group of drivers congregate in one place and turn o their phones
to decrease driver supply, which triggers an increase in the price of
rides in that area.
Last night while you were driving a customer home your phone
ran out of battery and turned o. You attached your phone to your
charger and followed the customer’s directions to their destination.
The next morning, when you check your earnings from the previous
night you see that an automated decision has been made by the
algorithm to [participants were provided with one of the following
three decision impacts:]
•
place a formal warning on your record because fraudulent
activity has been detected
•
deduct $42 from the $110 you earnt because fraudulent ac-
tivity has been detected
•
suspend your access to the RidR app based on the detection
of fraudulent activity, which means you can no longer earn
income from RidR
A.1.2 Part 2. [Participants were provided with the text of one of the
following two conditions:]
•
You want to have this decision reviewed. [Not Limited
Interaction Condition]
•
You want to have this decision reviewed. To start the review
process, you need to provide written reasons for why the
decision should be reviewed. The outcome of the review will
then be emailed to you. You will not interact directly with
the reviewer. [Limited Interaction Condition]
The reviewer could be a human from RidR’s internal reviews
team or an algorithm that has been developed by RidR specically
for reviewing decisions. Both reviewers are equally good at con-
ducting reviews and neither reviewer is more likely than the other
to change the initial decision.
A.2 Ammonite
A.2.1 Part 1. Imagine that you work at the distribution centre of
a freight company called Ammonite. Over the past four years your
work performance assessments at Ammonite have been positive.
This year, Ammonite has decided to use an algorithm to assess
its employees’ work performance. The algorithm uses numerous
performance metrics to produce its assessment, such as worker
productivity, whether quotas have been met, as well as feedback
from peers and supervisors.
The decision to use an algorithm was based on evidence from
previous years that under-performing employees received high
performance assessments because their supervisors based the as-
sessments mainly on peer review.
Based solely on the work performance assessment made by the
algorithm, a decision has been made [participants were provided
with one of the following three decision impacts:]
•
that you will not receive a “high performer” certicate from
the company this year
•that you will not be eligible for a promotion this year
•that your contract of employment will not be renewed
A.2.2 Part 2. [Participants were provided with the text of one of the
following two conditions:]
•
You want to have this decision reviewed. [Not Limited
Interaction Condition]
•
You want to have this decision reviewed. To start the review
process, you need to provide written reasons for why the
decision should be reviewed. The outcome of the review will
then be emailed to you. You will not interact directly with
the reviewer. [Limited Interaction Condition]
The reviewer could be a human from the internal reviews team or
an algorithm that has been developed by Ammonite specically for
reviewing decisions. Both reviewers are equally good at conducting
reviews and neither reviewer is more likely than the other to change
the initial decision.
A.3 MakeAd
A.3.1 Part 1. Imagine that you have applied for your dream job
as a creative director at a well known advertising agency, MakeAd.
You have ve years of experience in the advertising eld. You are
required to upload a 5 minute video introducing yourself. To decide
who will be shortlisted for a formal panel interview, MakeAd uses
773

Algorithmic Decisions, Desire for Control, and the Preference for Human Review over Algorithmic Review FAccT ’23, June 12–15, 2023, Chicago, IL, USA
an algorithm that analyses body language and speech patterns to
assess whether a person will be suitable for the job.
In the 5 minute video you introduce yourself, give a brief sum-
mary of your work experience, and highlight your strengths. The
video you record is very similar to how you would introduce your-
self at an in-person interview.
Based on the analysis of the video the algorithm has [participants
were provided with one of the following three decision impacts:]
•
shortlisted you for the role of creative director but deter-
mined that you will need to create a short presentation
showcasing your work in addition to attending the panel
interview
•
shortlisted you for a dierent role (as Junior Creative Direc-
tor) at MakeAd
•
not shortlisted you for the job and has prevented you from
applying to similar jobs at MakeAd for the next year
A.3.2 Part 2. [Participants were provided with the text of one of the
following two conditions:]
•
You want to have this decision reviewed. [Not Limited
Interaction Condition]
•
You want to have this decision reviewed. To start the review
process, you need to provide written reasons for why the
decision should be reviewed. The outcome of the review will
then be emailed to you. You will not interact directly with
the reviewer. [Limited Interaction Condition]
The reviewer could be a human from the internal reviews team
or an algorithm that has been developed by MakeAd specically for
reviewing decisions. Both reviewers are equally capable of making
the right decision and neither reviewer is more likely than the other
to change the initial decision.
774