Content uploaded by Inês Fortes
Author content
All content in this area was uploaded by Inês Fortes on Feb 19, 2021
Content may be subject to copyright.
The Impact of Autonomous Vehicles’Active
Feedback on Trust
Ana Mackay
1(&)
,Inês Fortes
1
, Catarina Santos
1
,Dário Machado
2
,
Patrícia Barbosa
1
, Vera Vilas Boas
3
,João Pedro Ferreira
3
,
Nélson Costa
1,4
, Carlos Silva
2
, and Emanuel Sousa
1,2
1
ALGORITMI Research Centre, School of Engineering,
University of Minho, Guimarães, Portugal
anamackay@gmail.com, ines.fortes@gmail.com,
pmscnb@gmail.com, cfssantos_13@hotmail.com,
2
Center for Computer Graphics, Guimarães, Portugal
drmachado@gmail.com,
{carlos.silva,Emanuel.sousa}@ccg.pt
3
Bosch Car Multimedia, Braga, Portugal
{vera.vilasboas,joao.ferreira5}@pt.bosch.com
4
Department of Production and Systems, School of Engineering,
University of Minho, Guimarães, Portugal
ncosta@dps.uminho.pt
Abstract. The successful introduction of self-driving technology may depend
on the ability of the vehicles’human-machine interface to convey trust to the
vehicle occupants. Using a driving simulator, in this experiment we aimed to
evaluate drivers’trust on an autonomous system, depending on the feedback the
vehicle provided by an assistive cluster’s interface. Forty participants were
divided into three groups regarding levels of feedback: (a) cluster without
feedback (N = 13); (b) cluster with feedback regarding the surrounding vehicles
(N = 14); (c) cluster with feedback regarding the surrounding vehicles and the
vehicle’s own decisions (N = 13). For all groups, a visual search task was
introduced as an indirect indicator of trust in the autonomous system. Results
showed an inverse relation between available feedback and correct answers. The
system was evaluated as trustable and safe by all groups. Overall, the results
may contribute to design requirements for future vehicle HMIs, as they indicate
that more information does not necessarily convey more trust.
Keywords: Autonomous driving Trust Cluster’s feedback
Visual search task
1 Introduction
The introduction of autonomous driving technology may fundamentally change the
role of the driver. Freed from driving responsibilities, he/she may be allowed to engage
in leisure or working related tasks that were previously deemed as incompatible with
driving. However, for this to happen vehicle occupants must trust that the autonomous
system is aware of surrounding events and is deciding its course of action accordingly.
©Springer Nature Switzerland AG 2020
P. M. Arezes (Ed.): AHFE 2019, AISC 969, pp. 342–352, 2020.
https://doi.org/10.1007/978-3-030-20497-6_32
anamackay@gmail.com
A seemingly obvious way of conveying such information is through the
Human-Machine Interfaces (HMIs), but deciding on exactly what type and amount of
information and vehicle performance/decision feedback should be presented is a
leading question in autonomous driving (AD) transportation research. More specifi-
cally, which information concerning the vehicle’s current state should be presented to
facilitate trust in the system?
Several studies have shown that informing users about the capabilities and limi-
tations of the autonomous system, as well as continuously communicating its current
state promotes a safer and more appropriate use of the system (Hoffman et al. 2013).
One of the first experiments that aimed to study whether communicating automation
uncertainty improves driver–automation interaction was developed by Beller et al.
(2013). They showed a face-like icon expressing uncertainty whenever the autonomous
system had ambiguous or incomplete sensory information. They concluded that,
compared with a group that had no uncertainty information, (a) the time to collision
increased in case of a system failure, (b) situation awareness increased, and (c) trust and
acceptance in the system increased.
The results from Beller et al. (2013), suggest that presenting information regarding
the uncertainty of the system may be useful. However, in their study, there were only
two levels of uncertainty under testing: uncertainty and no uncertainty. Helldin et al.
(2013), also addressed the display of uncertainty by the vehicle but investigated the
effect of communicating it as a continuous variable. The experimental group saw a
figure in the cluster indicating, on a scale from 1 to 7, how certain the car was that it
could handle the situation autonomously. When the confidence level was 2 or less,
automation could not be guaranteed. The control group had no information regarding
the current status of the car. The results showed that the drivers of the experimental
group were quicker to take-over, looked more often away from the road, and even
though the difference was small, had less trust in the system. In this case, the behavior
of the experimental group (trusting less) revealed to be more appropriate as the vehicle
sometimes asked for manual take-over. Moreover, participants in this group were more
efficient in responding to take-over requests.
Even if the results from the previous studies might indicate that presenting status
information could be useful, it is not clear whether it is enough to inform about the
status of the vehicle, or it is better to also inform why the vehicle is doing what it is
doing. Regarding this question, Koo et al. (2015), investigated whether the content of
verbal messages stating the vehicle’s autonomous actions affected the driver’s trust in
the system. For that, they conducted a driving simulator experiment where a semi-
autonomous system prevented frontal collisions by activation of an automated brake
function. They were asked to drive, and whenever an unexpected risk situation
appeared on the course, a voice warning and/or auto braking was activated. There were
four conditions concerning the message content: (1) ‘how’the car is acting (e.g., “Car
is braking”); (2) ‘why’it is acting the way it is (e.g., “Obstacle ahead”); (3) ‘how +
why’message (e.g., “Car is braking due to obstacle ahead”); and (4) no message. The
‘how’condition yielded the poorest results on driving performance, and the ‘why’was
the most preferred by the drivers. Combining the ‘how’and ‘why’messages resulted in
the safest driving performance measured by number of road edge excursions, but also
led to more negative emotional states (these were inferred by asking the participants
The Impact of Autonomous Vehicles’Active Feedback on Trust 343
anamackay@gmail.com
how much they felt “anxious”,“annoyed”, and “frustrated”while driving).
Overall, drivers who received some type of information about the driving environment
expressed higher system trust and acceptance when compared with those who received
no message.
In sum, informing users regarding the system’s status seems to be beneficial, both
in terms of trust and performance. One way to convey that information is by means of
displays. However, it is yet not clear which type of information users need and find
useful. The present study aimed to explore the role of visual feedback cues that may
affect trust in the interaction process between the user and the autonomous vehicle’s
interface. Trust was measured by objective and subjective metrics. Many physiological
methods are already used as indirect indications of a driver’s trust in the vehicle, and in
this study, heart rate was measured. In addition, a study by Llaneras and Green (2013)
found that increased trust could lead drivers to allocate less visual attention to the road
ahead. Therefore, a visual search task was introduced without previous notice, and
prompted on a display inside the vehicle away from the typical central area of the road
gaze. Task performance was analyzed as an indicator of trust in the autonomous
system. Finally, a trust questionnaire was used.
2 Method
2.1 Participants
Forty participants with a driver’s license were recruited to take part on this experiment,
ten of which were female, with a mean age of 29 years old (SD = 9.11). The mean
years of driving experience was 10 years (SD = 8.41).
2.2 Apparatus, Materials and Setup
The experiment was conducted in a fixed-base Driving Simulator Mockup (DSM),
composed of two seats, a steering wheel, two pedals, and three monitors for rear view
projection. The DSM is connected with the simulation software (SILAB v.5.0, WIVW
2018) which controls the simulation environment. The frontal visualization was dis-
played in a curved screen with 5-m width by 2-m height, mounted in a metal structure.
Along this curved surface three projectors (1920 1080 pixels each) displayed the
simulation environment in the curved surface. A Head-Up Display (HUD) was installed
and the Assistive Cluster (AC) was mounted on the right side of the steering wheel, at
approximately 75 cm from the drivers’head. A touchscreen display on the lower
dashboard was placed within reach to prompt the visual search task (Fig. 1).
Heart Rate (HR) was measured using BIOPAC’s MP160 data acquisition system
that can record data at a frequency of 2000 Hz and is coupled with the AcqKnowledge
5.0 data analysis software. Three pre-gelled electrodes were applied to the skin, on the
participant’s right clavicle (negative electrode), left clavicle (positive electrode) and left
lowest rib (ground electrode).
344 A. Mackay et al.
anamackay@gmail.com
3 Procedure
The participants were invited to an autonomous driving experience, and upon arrival,
they read and signed an informed consent form. Then, they were randomly assigned to
one of three groups: the group without feedback (No Feedback Group, N = 13), the
group with feedback regarding the surrounding vehicles (Sensors Group, N = 14), and
the group with feedback regarding the surrounding vehicles and the vehicle’s own
decisions (Decision group, N = 13). The No Feedback group had a cluster with no
information (Fig. 2a); the Sensors Group had information regarding the proximity of
surrounding vehicles: lateral and longitudinal control lines turned yellow as other
objects got closer - e.g., while in a queue or during an overtake (Fig. 2b); and the
Decision group had full feedback: the same sensor information as the Sensors group
and also arrows that informed the driver of the vehicle’s immediate future behaviour
regarding lane changes (Fig. 2c).
Fig. 1. DSM: (a) Head-up display (orange dashed outline); (b) Assistive cluster interface (blue
solid outline); (c) Touchscreen display (green dotted outline).
Fig. 2. Cluster’s level of feedback: (a) No feedback; (b) Sensors feedback; (c) Sensors and
Decision feedback.
The Impact of Autonomous Vehicles’Active Feedback on Trust 345
anamackay@gmail.com
All clusters had an illustration of the vehicle at the centre, and an automation bar
below that represented the driving mode within four possible autonomy levels. For the
purpose of this study, the AD mode –level 4 –represented by a full green automation
bar, was presented to all participants during the experiment. The HUD (see Fig. 1a) had
a speedometer at the centre and an AD icon, as shown in Fig. 3.
3.1 Route Description
For an overview of the experiment see Fig. 4. The scenario was composed of a 12-min
drive in a highway during which several events occurred such as a free traffic segment,
an overtaking and a traffic queue that required the system to brake. While on AD mode,
a visual search task was prompted twice, 3 and 6 min after the beginning of the
experiment (t1 and t2), with no previous notice. Finally, the car stopped at a gas station
for recharge and the experimenter asked the participant to fill a trust questionnaire.
4 Measures
4.1 Visual Search Task
In the search arrows task (e.g., Engström et al. 2005), the participant had to search for
an upward facing arrow in a grid of equal but differently oriented arrows by pressing a
“Yes”button (if present) or “No”(if absent). Figure 5shows an example of an arrow
grid, with a “Yes”as a correct response for this trial.
Fig. 3. Example of the centre of the Head-Up Display (HUD) with AD mode engaged.
Fig. 4. Scheme of the experiment.
346 A. Mackay et al.
anamackay@gmail.com
While the car was on AD, this task was presented twice (visual search task t1 and
t2). Each time the task was presented, it was composed of three trials/images, forcing
the participants to deviate their attention from the road or the environment. Each trial
started with a fixation cross displayed for 500 ms, followed by the arrows grid and
ended either with the driver’s response or after a 6-s presentation without response.
After the third trial, the touchscreen was turned off until the second evaluation. The
participant’s performance was measured based on the percentage of correct/incorrect
and/or missing answers. As higher levels of trust are associated with lower monitoring
frequencies (Hergeth et al. 2016), the best performance was expected from the Decision
group. As this group has been given more information regarding the vehicle’s beha-
viour, it was hypothesized a higher level of trust in the system and a better performance
on the visual search task.
4.2 Heart Rate
It was expected that the more the cluster’s feedback, the higher the trust, and conse-
quently, the lower the heart rate. It has been shown that the use of a simulated
autonomous vehicle can increase stress (Morris et al. 2017), and heart rate is a fre-
quently used physiological variable that reflects cognitive stress of subjects (e.g.,
Reimer and Mehler 2011). Therefore, it was hypothesized that the No Feedback group,
due to the lack of information which is a factor that triggers an anxiety state (Lee et al.
2016), showed the highest heart rate.
4.3 Trust Questionnaire
To evaluate trust, a custom-made questionnaire was presented at the end of the
experiment. The participant was asked to classify each sentence on a scale from 0
(totally disagree) to 5 (completely agree):
•The autonomous driving system is trustable;
•The autonomous driving system is safe;
•I understood the intentions of the autonomous driving system;
•I understood the actions of the autonomous driving system.
It was expected that, as the cluster’s feedback increased, the reported trust in the
system also increased.
Fig. 5. Example of an image for the visual search task.
The Impact of Autonomous Vehicles’Active Feedback on Trust 347
anamackay@gmail.com
5 Results and Discussion
5.1 Heart Rate
For each participant, the mean heart rate was calculated in the 5 s immediately before
the presentation of the visual search task and in the 5 s immediately after the visual
search task has ended. To test whether heart rate increased when the visual task was
introduced, the heart rate values before the first and the second visual task presentation
were averaged (M = 78.48 beats/min), and the same was done for the values after the
tasks (M = 85.96 beats/min). Reimer and Mehler (2011) found that heart rate and skin
conductance levels were lower in a driving simulator than under actual on-road driving,
but that the relative increases in these measures across cognitive tasks of increasing
difficulty were equivalent.
A mixed Analysis of Variance (ANOVA) with the amount of information (3 levels:
no feedback, sensors feedback, and decision feedback) as the between-subject factor
and the moment relative to the visual search task (2 levels: before and after) as the
within-subject factor was conducted. The increase in heart rate with the occurrence of a
visual search task was statistically significant, F(1, 36) = 19.09, p = .0001, g2
p= 0.35.
However, no significant differences were found between the three levels of information,
F(2, 36) = 0.06, p = .94, g2
p= 0.004, meaning that heart rate was similar across the
different levels of information.
5.2 Visual Search Task
Figure 6shows the percentage of missing answers to the visual search tasks according
to the amount of available information in the assistive cluster, in the first moment (t1,
left panel) and in the second moment (t2, right panel).
Both in t1 and in t2 the percentage of missing answers was lowest for the No
Feedback group, intermediate for the Sensors group and highest for the Decision group.
Fig. 6. Average percentage of missing answers to the visual search tasks according to the
amount of available information in the assistive cluster in each moment of presentation (t1, t2).
348 A. Mackay et al.
anamackay@gmail.com
A mixed ANOVA with amount of information (3 levels) and the instance of the
visual search task (2 levels: t1 and t2), in the number of missing answers to the visual
search task was conducted. Significant differences were found for the information level,
F(2, 37) = 8.18, p = .001, g2
p= 0.31, and for the instance of the visual search task,
F(1, 37) = 9.70, p < .01, g2
p= 0.21. A post-hoc analysis of the information level using
least-squares means with Bonferroni corrections, revealed that the Decision group had
significantly more missing answers than the No Feedback group, t = 4.01, p < .01, and
then the Sensors group, t = 2.47, p = .05. These results could indicate that a more
complex cluster may be more distracting. The cause for that distraction is not clear:
participants may be looking at the information cluster (because it was showing almost
continuous information) or the environment (e.g., to compare their knowledge of the
environment with the cluster’s information) instead of looking to the visual search task.
Concerning the instance of the visual search task, there were fewer missing answers in
the second presentation of the task, which was expected since the element of surprise
was removed with the first task presentation, so participants were more aware that the
visual search task could be shown at any time.
It seems that having more information on the cluster has an influence on the
missing answers to the visual task. However, when participants do answer the task,
does the percentage of correct answers differ across groups? Figure 7shows the per-
centage of correct/incorrect answers to the visual search task, calculated after excluding
the missing answers. In this analysis, 6 participants were eliminated because they either
failed to respond to all three trials in the first task (N = 2, all from Sensors group), to all
three trials in the second task (N = 1, from Decision group), or to all trials, in both tasks
(N = 3, all from Decision group).
From the results, it seems the percentage of correct and incorrect answers was very
similar across groups. A mixed ANOVA with amount of information (3 levels) and the
instance of the visual search task (2 levels: t1 and t2) was conducted regarding the
percentage of correct answers (excluding missings). No significant differences were
Fig. 7. Percentage of correct and incorrect answers to the visual search tasks according to the
amount of available information in the assistive cluster in each moment of presentation (t1, t2).
The Impact of Autonomous Vehicles’Active Feedback on Trust 349
anamackay@gmail.com
found for the information level, F(2, 31) = 0.14, p = .87, g2
p= 0.009. Conversely, a
significant effect was found for the instance of the visual search task, F(1, 31) = 6.10,
p = .02, g2
p= 0.16, with the percentage of correct answers increasing considerably in
the second appearance of the task (M
t1
= 82.4% vs. M
t2
= 93.1%). Although the
number of trials was small, this increase in performance may reflect the effect of training.
5.3 Trust Questionnaire
Figure 8shows the answers of the three groups to the four questions concerning the
trust on the system. As depicted in Fig. 8, the overall system was perceived by all
groups as trustable and safe.
Generally, participants agreed with all the questions. The lowest mean score was
3.8 and it was obtained for the Decision group in the questions regarding the auton-
omous system being trustable and safe (first two questions in Fig. 8). The intentions
and actions of the autonomous driving system (last two questions in Fig. 8) were well
understood, as the mean score rate for both items was higher than 4 for all groups.
Although Young et al. (2015) argue that subjective measures, like self-reports, are
rather complicated, Schmidt et al. (2017) report to have successfully used verbal
assessment of drivers’condition regarding perceived sleepiness and cooling sensations.
Looking at Fig. 8, there seems to be a tendency for perceiving the system as more
trustable and safe (first and second questions in Fig. 8) the less information it has. Also,
the comprehension of system’s intention and actions (third and fourth questions in
Fig. 8) seem to be rated with lower scores as the cluster’s feedback increases on
complexity, with the worst scores in the Decision group.
One-way ANOVAs were conducted to analyse differences between the three
information levels for each one of the questionnaire items. The differences were
Fig. 8. Mean scores of the Trust questionnaire. The score 0 means “totally disagree”and score 5
means “completely agree”with the sentence.
350 A. Mackay et al.
anamackay@gmail.com
non-significant for all ANOVAs, except for the item “I understood the actions of the
autonomous driving system”, F(2, 37) = 3.36, p = 0.05, g2
p= 0.15, were the result was
partially significant. Post hoc analysis indicate that the participants from the Decision
group performed worse than the No Feedback group, t = −2.321, p = 0.08.
6 Conclusion
Self-driving technology has the potential to profoundly change how we use automo-
biles. However, generalized adoption will strongly depend on the perception of trust
experienced by the users (Choi and Ji 2015). Given that the HMI is one of the main
sources of information for the vehicle occupants, it may play a key role in increasing
system transparency, a factor known to affect trust (Choi and Ji 2015), thus influencing
acceptance.
In this study, we investigated how conveying feedback regarding perception of
surrounding driving environment and from the autonomous systems decisions may
affect trust. Results of our study do not point clearly to a direct relation between the
available feedback information and the amount of trust assessed by the questionnaire.
This may be due to the particular design of the interface, which provided only limited
information regarding the location of surrounding vehicles and of the vehicle’s own
course of action. It may be that more explicit and complete information is required to
affect trust. For instance, in a recent work by Haeuslschmid et al. (2017) which also
compared different feedback visualizations a “world in miniature”concept (inspired on
the one used in Tesla vehicles) was the most effective in conveying trust and sense of
safety. Our results also showed an inverse relation between available feedback and
performance on a visual search task. That may imply that the more complex assistive
cluster created the greatest cognitive load, leading to the worst task performance.
In conclusion, more information does not necessarily lead to more trust and may in
fact negatively affect cognitive load. These results point to the need of investigating
which types of feedback are more appropriate and how the particular design choices for
the visual HMI may affect trust and influence cognitive load. Other ways of conveying
information to the user should also be studied. For instance, this experiment focused on
visual feedback, but the use of other sensory modalities, either in combination with the
visual cues or by themselves, should also be explored. Other approach to convey trust
on autonomous vehicles may be by anthropomorphizing them, by for example pro-
viding it a voice and simulating intelligent conversation (Ruijten et al. 2018).
Despite a different cluster feedback design or other modalities that could be
implemented to transmit a feeling of safety and trust to the user, this study calls for the
need to test different approaches on active feedback of autonomous driving systems.
Acknowledgments. This work has been supported by FCT –Fundação para a Ciência e Tec-
nologia the scope of the Project: UID/CEC/00319/2019 and by: European Structural and
Investment Funds in the FEDER component, through the Operational Competitiveness and
Internationalization Programme (COMPETE 2020) [Project no. 039334; Funding Reference:
POCI-01-0247-FEDER-039334].
The Impact of Autonomous Vehicles’Active Feedback on Trust 351
anamackay@gmail.com
References
Beller, J., Heesen, M., Vollrath, M.: Improving the driver-automation interaction: an approach
using automation uncertainty. Hum. Factors 55(6), 1130–1141 (2013). https://doi.org/10.
1177/0018720813482327
Choi, J.K., Ji, Y.G.: Investigating the importance of trust on adopting an autonomous vehicle. Int.
J. Hum.-Comput. Interact. 31(10), 692–702 (2015). https://doi.org/10.1080/10447318.2015.
1070549
Engström, J., Johansson, E., Östlund, J.: Effects of visual and cognitive load in real and simulated
motorway driving. Transp. Res. Part F Traffic Psychol. Behav. 8(2 SPEC. ISS.), 97–120.
https://doi.org/10.1016/j.trf.2005.04.012
Haeuslschmid, R., von Buelow, M., Pfleging, B., Butz, A.: Supporting trust in autonomous
driving. In Proceedings of the 22nd International Conference on Intelligent User Interfaces -
IUI 2017, pp. 319–329 (2017). https://doi.org/10.1145/3025171.3025198
Helldin, T., Falkman, G., Riveiro, M., Davidsson, S.: Presenting system uncertainty in
automotive UIs for supporting trust calibration in autonomous driving. In: Proceedings of
International Conference on Automotive User Interfaces Interactive Vehicles, vol. 5, pp. 210–
217 (2013). https://doi.org/10.1111/j.1600-0714.1976.tb01762.x
Hergeth, S., Lorenz, L., Vilimek, R., Krems, J.F.: Keep your scanners peeled: gaze behavior as a
measure of automation trust during highly automated driving. Hum. Factors J. Hum. Factors
Ergon. Soc. 58(3), 509–519 (2016). https://doi.org/10.1177/0018720815625744
Hoffman, R.R., Johnson, M., Bradshaw, J.M., Underbrink, A.: Trust in automation. IEEE Intell.
Syst. 28(1), 84–88 (2013). https://doi.org/10.1109/MIS.2013.24
Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., Nass, C.: Why did my car just do that?
Explaining semi-autonomous driving actions to improve driver understanding, trust, and
performance. Int. J. Interact. Des. Manuf. (IJIDeM) 9(4), 269–275 (2015). https://doi.org/10.
1007/s12008-014-0227-2
Lee, J., Kim, N., Imm, C., Kim, B., Yi, K., Kim, J.: A question of trust: an ethnographic study of
automated cars on real roads. In: Proceedings of the 8th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications –Automotive UI 2016,
pp. 201–208 (2016). https://doi.org/10.1145/3003715.3005405
Llaneras, R.E., Green, C.A.: Human factors issues associated with limited ability autonomous
driving systems : drivers’allocation of visual attention to the forward roadway, pp. 92–98
(2013)
Morris, D.M., Erno, J.M., Pilcher, J.J.: Electrodermal response and automation trust during
simulated self-driving car use. In: Proceedings of the Human Factors and Ergonomics
Society, pp. 1759–1762 (2017). https://doi.org/10.1177/1541931213601921
Reimer, B., Mehler, B.: The impact of cognitive workload on physiological arousal in young
adult drivers: a field study and simulation validation. Ergonomics 54(10), 932–942 (2011).
https://doi.org/10.1080/00140139.2011.604431
Ruijten, P.A.M., Terken, J.M.B., Chandramouli, S.N.: Enhancing Trust in Autonomous Vehicles
through Intelligent User Interfaces That Mimic Human Behavior (2018). https://doi.org/10.
3390/mti2040062
Schmidt, E., Decke, R., Rasshofer, R., Bullinger, A.C.: Psychophysiological responses to short-
term cooling during a simulated monotonous driving task. Appl. Ergon. 62,9–18 (2017).
http://doi.org/10.1016/j.apergo.2017.01.017
Young, M.S., Brookhuis, K.A., Wickens, C.D., Hancock, P.A.: State of science: mental
workload in ergonomics. Ergonomics (2015). https://doi.org/10.1080/00140139.2014.956151
352 A. Mackay et al.
anamackay@gmail.com
