Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs

Abstract

Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver. This paper applies a user-centred design approach to designing a new AV interior to better support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with 44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were identified as the principal concerns about future autonomous vehicles. Based on these findings, Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information display to facilitate users' activities and situational awareness while driving. Study 3 consists of a series of laboratory simulator evaluation studies, and this indicated that drivers in the proposed design condition had better situational awareness in an AV when dealing with take-over situations. Such findings suggest the possibility of applying rear-facing seats in autonomous vehicles to support in-vehicle non-driving activities. Some specific implications of designs to enhance a driver's situational awareness have also been suggested.

Full text

1
Accepted manuscript
Sun, X., Cao, S., & Tang, P. (accepted in 2020). Shaping driver-vehicle interaction in
autonomous vehicles: How the new in-vehicle systems match the human needs. Applied
Ergonomics
Shaping Driver-Vehicle Interaction in Autonomous Vehicles: How the
New In-vehicle Systems Match the Human Needs
Abstract
Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver.
This paper applies a user-centred design approach to designing a new AV interior to better
support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with
44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an
AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were
identified as the principal concerns about future autonomous vehicles. Based on these findings,
Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information
display to facilitate users’ activities and situational awareness while driving. Study 3 consists
of a series of laboratory simulator evaluation studies, and this indicated that drivers in the
proposed design condition had better situational awareness in an AV when dealing with take-
over situations. Such findings suggest the possibility of applying rear-facing seats in
autonomous vehicles to support in-vehicle non-driving activities. Some specific implications
of designs to enhance a driver’s situational awareness have also been suggested.
Keywords
driver-vehicle interaction; autonomous vehicles; in-vehicle system; human needs

2
Introduction
Many well-known companies, such as Jaguar Land Rover (JLR), Google, Tesla, Daimler AG,
GM, Uber, Toyota and Baidu, are now developing autonomous vehicles. In recent years, such
vehicles have become commonplace in the news headlines, and it seems they will be leading
the future development of personal transportation. The proposed benefits of such intelligent
vehicles include enhanced safety, a reduced need for infrastructural investments, improved fuel
economy, reduced congestion and, most importantly, they may make driving safer, more
relaxing and, ultimately, more enjoyable (Jorlov et al. 2017). However, such benefits can only
be achieved when a human driver feels comfortable in an autonomous vehicle, and when the
optimal level of interaction between the driver and the automation system can be developed
(Helldin et al. 2013). Researchers have suggested that comfort plays an essential role in the
acceptance of autonomous vehicles (e.g., Beggiato et al. 2019; Bellem et al. 2018), and the
factors influencing driving comfort have been investigated and discussed in previous research.
For example, noise, vibration, and harshness have been identified as the main variables
affecting driving comfort (e.g., Elbanhawi et al. 2015; Qatu, 2012); psychological determinants,
such as personality, apparent safety, trust in the system, feelings of control, the familiarity of
driving manoeuvres, and information about system states and actions (e.g., Bellem et al. 2018;
Bellem et al. 2016; Elbanhawi et al. 2015) have also been discussed. Additionally, potential
measures to relieve any causes of discomfort have been suggested. For example, Beggiato et al.
(2019) studied the correlation between HR, pupil diameter, and interblink and discomfort-
inducing situations. They advocated that, by changing driving styles and information
presentation, autonomous vehicles could provide a more personalised driving experience and
thereby relieve any such causes of discomfort. Meanwhile, there is still a paucity of
corresponding design implications based on the existing research on comfort, as well as a
profound understanding of human driver characteristics. There is a compelling need for
designers to design autonomous vehicle (AV) behaviours based on such understanding (Mok
et al. 2015).
The Society of Automotive Engineering (SAE) categorised six levels of autonomy in a vehicle,
whereby the human driver performs part or all of the dynamic driving tasks (DDT) from L0 to
L2, while the automated driving system performs the entire DDT (while engaged) from L3 to
L5 (SAE, 2018). According to their definitions, in L3 automation, when there is a failure in
DDT, the system will issue a timely request to intervene to the fallback-ready driver, whereas
in L4 and L5 automation, it is the system that is responsible for a DDT fallback. Human drivers
play an essential role in SAE L3 automation, whereby they don’t need to monitor the system,
which separates L3 from other partly automated systems (Gold et al. 2018), but they are

3
expected to operate the vehicle in conditions not supported by the system (Kyriakiddiset al.
2017). Such special circumstances make L3 automation the main challenge, and this is also the
focus of current human factors research into automated driving (Jarosch and Benlger, 2018;
Kyriakiddis et al. 2017).
The main priority to be addressed is the safety of the transition of control from the system to
the human driver, and the application of L3 automation is more likely to evolve in stages (Merat
et al. 2014). Jarosch and Benlger (2018) suggested that in L3, there are typically two types of
take-over, namely, long- and short-term take-over situations. Long-term takeover situations are
known to the system well in advance of the human driver having to intervene, whereas short-
term takeover situations are not known to the system or the human driver until they are detected
by the system, and the human has to react promptly. With the improvement of relevant
automation capabilities in L3 automation (Endsley, 2017) and following legislation in favour
of the operation of ‘autonomous cars’ (Meratet al. 2014), it is anticipated that the evolving L3
automation will become more capable of long-term take-over situations where the frequency
of take-overs would be much lower and the time allowance would be much longer. There has
been an increasing amount of research investigating short-term take-over behaviour (Jarosch
and Benlger, 2018). The current research, therefore, focuses on the evlolving L3 automation in
which the system performs the entire dynamic driving tasks (while engaged). When there are
driving conditions beyond its capability, the system is expected to allow the driver to have
sufficient transition time to resume control comfortably. In this level of automated vehicles, the
fundamental role of the driver changes significantly, and such changes impose new needs,
requirements and challenges for designers when designing safe, comfortable and optimal AV
human interaction systems (Strömberg et al., 2018).
First, driving in autonomous vehicles is clearly different from driving traditional manual
vehicles. A traditional vehicle interaction design is based on the requirement that the driver
should be able to reach the pedals, the steering wheel and the gear shifter in any sitting position
(Yang & Klinkner, 2018). This both restricts the possibilities for interior design (e.g. vehicle
seats and displays) to support optimal user experience and limits the driver’s behaviour (e.g.
movements, postures). In autonomous vehicles, there is no longer the need for drivers to drive
at all times, nor are they required to sit behind the steering wheel at all times. These new
situations open up the possibilities for new spatial orientations and to provide free time for the
driver to perform non-driving activities (Pettersson & Karlsson, 2015). However, driving
posture may no longer be optimal if the majority of time spent in a vehicle does not involve the
need to drive (Yang & Klinkner, 2018). There is an urgent need for designers to observe both
driving and non-driving user behaviours and to understand how the elements of vehicle

4
interiors can support user activities, and facilitate the transition between autonomous driving
and passing control back to the driver.
Second, when non-driving activities become drivers’ primary tasks, and the driving activities
(e.g. road monitoring) become secondary tasks, a differential effect on situational awareness
can occur (Chandrasekaran et al. 2019). Situational awareness is defined as “knowing what’s
going on so you can figure out what to do” (Adam, 1993, p.319). Endsley’s model (1995) is
often cited to explain the concept of situational awareness. In the model, situational awareness
is defined as the ability to perceive the related elements of the environment, to comprehend the
given situation, and to anticipate the future status. It is relevant to the concept of mode
awareness which has been studied mainly in the field of advanced automated systems (e.g.
Miller et al. 2002; Vakil et al. 1995). Mode awareness is the ability to anticipate the behaviour
of automated systems (Hew et al. 2013), while situational awareness focuses on awareness of
the surroundings, as mentioned by Endsley (2016). The concept of situational awareness is one
of the focuses of this study because it is a crucial component of safe driving (Chandrasekaran
et al. 2019). The driver needs to rapidly acquire information about the surroundings which is
relevant to the driving tasks of navigation and hazard-avoidance. Large et al. (2017) identified
that when drivers are engaged in non-driving activities, situational awareness decreases with
automated driving. However, if they are instructed to detect objects in the environment, this
effect is reversed. Such a situation highlights the challenge of keeping the driver attentive when
automation is not fully implemented, and when they will be required to resume driving under
specific circumstances. It is becoming increasingly important to understand how humans
interact with autonomous vehicles and how the operational information systems are best
delivered to ensure a high level of take-over performance.
In this paper, we have reported three empirical studies to understand driver-vehicle interaction
in autonomous vehicles. All the studies took place in the city of Ningbo (China), using local
citizens as participants. We first describe a study designed to investigate the variety of potential
activities and requirements of users in an autonomous vehicle. Then, we propose a possible
new seating position for autonomous vehicles with the support of an in-vehicle information
display to assist drivers in both their driving and non-driving activities, while raising their
situational awareness and achieving recovery of their driving posture in the take-over procedure.
Finally, we implemented our design in a driving simulator before evaluating the system with
human drivers. In any take-over situations, significant changes are expected to take place in the
in-vehicle user behaviour and any corresponding driver-vehicle interactions, and this is a focus
of this paper. The contribution of this paper lies in the addressing of current gaps and challenges
in driver-autonomous vehicle interaction, by adopting a user-centred approach to understand

5
user needs and behaviours in autonomous vehicles and to design in-vehicle elements to support
any observed interactions and user needs. All user-entred design (UCD) activities undertaken
in this research have been approved by the Ethics Committee of the Faculty of Engineering at
the University of Nottingham.
Literature Review
Since, in the 1980s, UCD has been widely used in industrial and interaction design (e.g. Costa,
Holder, & MacKinnon, 2017). UCD begins with an understanding of users and then places user
characteristics, goals, needs, requirements and preferences at the centre of each stage of the
design process (Costa et al. 2017; Robins et al. 2010). However, the application of UCD in the
research of future technology is limited (Jorlov et al. 2017; Odom et al., 2012; Pettersson &
Karlsson, 2015). One of the possible reasons could be that, when applying UCD in a context of
the use of future technology such as autonomous vehicles, problems are faced because it must
support “a shift in focus from the present into the future” (Pettersson & Karlsson, 2015). The
technology that enables autonomous vehicles is at an early stage of development, which results
in a huge leap from the current stage of relevant research to the understanding of actual users
and their needs and requirements, as well as the expected design state (Strömberg et al., 2018).
In spite of the difficulties of user research on autonomous vehicles, there are a few studies on
AV research from the perspective of end-users. One area which has been explored is the variety
of potential in-vehicle activities. In their longitudinal simulator study which lasted for five days
for each participant, Large et al. (2017) found that the participants tended to bring routine
objects such as paper documents and computing devices for their non-driving activities. The
most common activities included reading, social networking activities using a mobile device,
web-browsing, and watching videos on a laptop or iPad. By using the methods of traditional
participatory design and “setting the stage”, Pettersson and Karlsson (2015) identified a series
of possible activities including relaxation, working, sleeping, reading, socialising, eating,
drinking, and video entertainment, among others. Krome et al. (2015) also described how
playing games and communication activities could be popular in the daily use of autonomous
vehicles. This empirical evidence from users indicates high expectations for autonomous
vehicles to provide the opportunity for other activities for which they do not currently have the
time in manually controlled vehicles(Jorlov et al., 2017). Such non-driving related activities are
open to all kinds of possibilities, which also makes any corresponding driver-vehicle interaction
subject to high levels of diversity.
Another focused question of UCD research on autonomous vehicles is the possibility of how
being engaged in non-driving tasks would potentially lead to a decline in the situational

6
awareness of a driver (Birrell & Young, 2011; Merat & Jamson, 2009). An optimal automation
system is expected to maintain the situational awareness of the driver, while at the same time
helping them to monitor any situational developments when they are engaged in non-driving
tasks (Merat et al. 2012;). In such a case, the information offered by the vehicle, as well as the
way the information is offered, will play a vital role (Reimer, Mehler, & Coughlin, 2016). There
are studies which have specifically investigated the interaction between the driver and the
information offered by the vehicle, and any corresponding influence on the driving performance.
Mok et al. (2015) conducted a simulator study to gain a specific understanding of the types of
messages to offer, and behaviours to be performed by the vehicle, to achieve efficient
communication and interaction. Some dominant themes were identified. For example, drivers
in autonomous vehicles were found to have a preference for shared controls, and if a request
could not, or should not, be performed by the vehicle, it is important to declare the
corresponding reasons. Beller et al. (2013) conducted a simulator study to ascertain the effect
of the presentation of automation uncertainty on a driver’s understanding of the system. It was
found that the presentation of uncertainty improved situational awareness and enhanced
knowledge of fallibility. Kaber et al. (2012) assessed the effects of visual, cognitive, and
simultaneous visual and cognitive distractions on operational (braking, accelerating) and
tactical (manoeuvring) control of vehicles. They found that the workload increased following
all three types of distractions, among which the simultaneous distraction task resulted in the
highest number of steering errors. There are also studies which have investigated the effect of
different forms of information presentation (e.g. visual, auditory, tactile) on driver performance,
some of which found that tactile information prompted more rapid responses (Abbink et al.
2012).
Research on take-over behaviour during automation failure, or when a situation is beyond the
functional boundaries of automation, has now become a topic undergoing intense study. The
nature of autonomous driving presents conflicting goals, such that when the system is activated,
a complete transfer of the driving task to the system will comfort the driver, whereas when there
is a take-over request, a safe and smooth process is also essential (Radlmayr et al., 2014). The
process of transferring control is new to traditional driving techniques (Large et al., 2017; Zeeb
et al., 2015). It involves actions such as regaining eye fixation on the road and returning the
driver’s body to a driving position (Gold, Damböck, Lorenz, & Bengler, 2013; Zeeb, Buchner,
& Schrauf, 2016). Such a process will take place under the influence of many factors, including
the time available for the driver to regain control (Gold et al., 2013; Zeeb et al., 2016), the
cognitive load (Zeeb et al., 2016), the traffic situation, and any non-driving related tasks
(Radlmayr et al., 2014). Understanding the interaction mechanism between the driver and the

7
vehicle from a UCD perspective is critical for ensuring the smooth and safe transfer of control
(Large et al., 2017; Zeeb et al., 2015).
Different UCD approaches have been applied and discussed in relevant studies. User enactment
has been used as a rapid means of envisioning new ideas with future technology and exploring
future use in an evaluation-like context (e.g. Odom et al., 2012; Strömberg et al., 2018). It is a
method in which, by using drama or enactment, designers may develop both the physical form
and the social context of future designs, with users enacting loosely scripted scenarios dealing
with different situations using both familiar and novel technical interventions (Odom et al.,
2012). Karlsson & Pettersson (2015) used existing car models, car concepts, and images of
Copenhagen to elicit participants’ perceptions of the future design of autonomous vehicles and
any relevant changes which may be brought about. Similarly, Jorlov et al. (2017) used the
technique of “setting the stage” to investigate seating positions and activities in autonomous
vehicles. They built a stage consisting of four chairs and a simple sketch of a car, which
simulated an abstract design for a future autonomous vehicle, and this was used to facilitate the
participants in enacting imagined scenarios. Simulator study is another possible method of user
research into AV use. Driving simulation, in a sense, is a more strictly defined stage which
offers the participants more predefined characteristics of a vehicle for the future, and such an
attribute means that driving simulators are used frequently to test specific design features and
also to test the relevant user behaviour under specific conditions. For example, in their driving
simulator study, Beattie et al. (2014) compared five separate auditory feedback methods during
different driving scenarios. In a driving simulator, Hergeth et al. (2015) investigated the
relationship between gaze behaviour and automation trust. In this research of future
autonomous vehicle designs, the methods should employ a UCD approach to facilitate the users
by creating a space to imagine and test possible future experiences.
Study 1: Requirement analysis
Research Method
A scenario-based context interview study was conducted with 44 drivers, using an AV
simulator to understand their needs and requirements in terms of in-vehicle activities and their
relevance to interior design. The methods of simulator study (Pettersson & Karlsson, 2015) and
user enactment (Odom et al., 2012) were integrated to set the context for future autonomous
vehicles, as well as to make the experience more concrete for the participants and help them to
clarify their needs and requirements for future design (Strömberg et al., 2018). User enactment
is a method whereby, using drama or enactment, designers may conduct both the physical form
and the social context of future designs, with users enacting loosely scripted scenarios of

8
dealing with different situations of both familiar and novel technical interventions (Odom et al.,
2012). For example, it has been used as a rapid means of envisioning new ideas with future
technology and for exploring future use in an evaluation-like context (e.g., Odom et al., 2012;
Strömberg et al., 2018).
Participants. We intended to access a sample as representative as possible of the population of
licenced drivers in the locality in which the experiment took place. However, as there is a lack
of official information on the population distribution of licensed drivers in China (Salvendy,
2014), a random sample selection was used in this study. A total of 44 participants were
recruited. The drivers were screened for their experience of driving, with the criterion that they
could demonstrate at least one year’s driving experience. In addition, the driving frequency
criterion was at least three times per week. The proportion by age group was 20.4 % young
drivers (less than 30 years old), 68.2% middle-aged drivers (between 30-50 years old) and 11.3 %
senior drivers (> 50 years old). Their average driving experience was 7.5 years (SD = 7.8 years).
The participants were from different professional backgrounds, including education, finance,
marketing, and freelancing, among others, while no professional drivers were involved. Only
three participants reported previous experience with autonomous vehicles, of which two had
had a trial run in a Tesla Model X, and the other had some experience with the adaptive cruise
control system and the autonomous parking system of a Volvo vehicle. Additional demographic
information is shown in Table 1.
Table 1 Participants’ demographics
Age
Gender
< 30
30-50
>50
Female
Male
Driver
9
30
5
15
29
Apparatus. The driving scenarios were designed using the driving simulation software
OpenDS
1
to cover a broad range of scenarios and contexts. This included a drive to work, and
a leisurely drive, with both scenarios set in the contexts of both urban and highway
environments. A stop at the end of the highway scenario was designed to simulate SAE L3 take-
over situations.
1
https://opends.dfki.de/

9
Figure 1. The urban scene (left) and the highway scene(right)
The interviews were conducted at an Innovative Design laboratory. The laboratory contained a
driving simulator, four projectors and projection screens and a data collection platform to
capture audio and video data from both outside and inside the simulator. During the experiment,
the simulated driving trip was run on laboratory computers and projected onto four roof-
mounted projection screens (1920 x 1080 pixels each) so that the driver had a 360° field of
view (see Figure 2). A compact car (Chery QQ) was placed in the laboratory, with the interior
removed to create a simple and open design space. Such a design was expected to spark the
imaginations of the participants in the driving scenarios, while also stimulating their ideal
driving preferences.
Figure 2. A fixed-based driving simulator with a 360° visual field of view
Procedure. On arrival, participants were briefed about the study and were screened by
questionnaire and a trail drive in the simulator to ensure that they were at low risk of motion
sickness while experiencing the driving simulation. The participants were advised that the
simulator represents an autonomous vehicle that enables the driver to cede full control of all
DDT under certain traffic or environmental conditions. They were also advised that when the
driving conditions exceed the boundaries of the automation capabilities, the system will alert
the driver in advance, with sufficient time to complete the transition procedure. After the
introduction, they were invited to sit in the simulator. The driving scenario began with the urban

10
street scene. The participants were given five minutes to sit by themselves in the simulator to
adapt to the environment, after which the researcher would join them to ask questions about
what activities could take place in each relevant scenario, and any corresponding needs and
requirements. Similar questions were repeated until the relevant answers were exhausted. The
researcher would then change the simulation to the highway scenario, following which the
previous procedures were repeated. The participants would then be left by themselves in the
simulator for ten minutes to experience the scenario. They were instructed to perform any
activity they wished. Their activities during this period were video recorded for post hoc
analysis. The study lasted for a total of approximately 40 to 60 minutes.
Data collection and analysis. Two types of data were collected, the first of which came from
the interview transcripts, which were transcribed by the researchers. A content analysis was
applied to the transcripts, which were divided into two categories, namely activities and
functions. The second type of data came from the two 10-minute periods of video recording
when the participants were sitting in the driving simulator. The activities shown in the video
were also transcribed and categorized according to the specific activities adopted.
Results and Discussion.
All activities mentioned by the participants during the interviews were categorized into five
main types, namely, driving-related activity, rest, social activity, work and study and daily
routine (see Figure 3). A majority of the participants were found to be concerned about the
driving performance under Level 3 automation, and they exhibited strong needs to monitor the
driving tasks to enhance their situational awareness. This finding is in accordance with studies
by Mok et al. (2015) and Habibovic et al. (2016), who identified the need for drivers of an
autonomous vehicle to monitor driving performance. This monitoring mainly focused on two
types of controls, namely the operational control of the vehicle (braking, accelerating), and the
tactical control, meaning the manoeuvring of the vehicle in response to the road conditions
(Kaber et al., 2012). For the operational control, participants were concerned about the speed
and wanted to have real-time feedback. For the tactical control, participants expressed the need
to receive information about the traffic situation, which would include the surrounding traffic
and the road signs.

11
Figure 3. Reported activities in an autonomous vehicle
In addition to their safety concerns, drivers were also concerned about their comfort during the
journey. Participants were found to expect autonomous vehicles to release time for conducting
different activities. Entertainment and relaxation were two major types of activity mentioned
by the participants. For example, participants wanted to maintain the function of playing music
in a conventional vehicle but also functions for relaxation related activities such as
sleeping/resting and eating/drinking. Besides these, other types of activity expected to be
undertaken were related to work or social communication, where the most common activities
included working/studying and social networking using a mobile device.
The activities observed when the participants were left by themselves in the simulator for 10
minutes showed that more than half of the participants were resting (i.e. sitting in their seat and
watching the projected scene ahead). A small number of participants looked around to see the
scenes projected to the sides and the rear. The other activities included drinking/eating, writing
something. A majority of participants commented that “such a period was too boring [and]
should be filled with some activities”. The observed activities revealed that it is highly likely
that being released from the task of driving will cause drivers to feel bored when sitting in a
self-driving vehicle, which leads to the high potential of new interior designs that could engage
the driver through high cognitive input.

12
Figure 4. Activities observed in the simulator - The other activities include drinking/eating,
writing somethings.
This requirement study identified a number of common activities people would expect to be
able to undertake in an AV. In general, as a consequence of being released from providing
permanent manual control, there is an unlimited variety of activities that could be undertaken
within a vehicle. The participants tended to request an in-vehicle space which was adapted for
comfort and non-driving orientated activities.
Study 2: Participatory Design
The above requirement study 1 showed that more than half of the participants referred to a
design requirement featuring a more relaxed and comfortable environment. A typical finding
was the request for a more adjustable seat design to support common activities that were
preferred in all level of automation, such as monitoring the driving, sleeping/resting,
eating/drinking, working/studying, listening to music, etc. This was mentioned by two-thirds
of the participants, and it echoes the results from Pettersson and Karlsson (2015) and Jorlov et
al. (2017). It offers valuable references for understanding driver-vehicle interaction in
autonomous vehicles, in addition to suggestions for AV design. Thus, participatory design
workshops were conducted by the six design team members and twelve drivers to design a new
type of seat, in terms of both the functionality and ergonomics which could be adjusted for use
in non-driving orientated activities.
Research method
Participatory workshops are a design method based on Participatory Design which stimulates
the formation of ideas and concepts and allows designers to investigate specific situations and

13
environments by adopting the principles of co-design and genuine user participation
(Droumeva and Wakkary, 2006).
Participants. The participants in this research consisted of a group of professional designers,
HCI researchers and a group of drivers. The design team included six experts in interaction
design, industrial design and HCI. Similar to the first study, a random sample selection was
used to recruit the drivers’ team, which consisted of 12 licenced drivers with at least three years
of driving experience. The proportion of the age groups was 33.3 % young drivers, 58.3%
middle-aged drivers and 8.3 % older drivers. Their average driving experience was 9.2 years
(SD = 11).
Table 2 Participants’ demographics
Age
Gender
< 30
30-50
>50
Female
Male
Designer
3
3
0
4
2
Driver
4
7
1
6
6
Procedure. Participants initially read the information sheet containing details about the
automation level introduced in study 1, the steps involved in the study, and then signed the
consent form. In the beginning, the researcher invited all members to take their turn to sit in the
simulator where the driving scenario used in the first study was projected onto the four roof-
mounted projection screens. Then, participants were divided into three groups, with six
participants in each group. The researcher ensured that at least two designers were included in
each group. Next, the researcher presented the findings of the previous study (Figures 3 &4) to
all members. After this, each group was invited to design a new vehicle seat to support the way
people would like to use an autonomous vehicle. Finally, all members gathered to review and
discuss the concepts generated. The total duration of the study was 3 hours.
Data collection and analysis. A thematic analysis technique for qualitative data was employed
in this study. The participants’ comments were included to exemplify some of the points.
Results and discussion
Design concepts. Several concepts were generated by the design team to define the seat layout,
including 1) the front seats face backwards, which will give the drivers/passengers a social
context in which to better communicate, and will provide more space to perform non-driving
relevant activities; 2) the front seats can be folded when not in use, which will maximize the
interior space to enhance comfort levels; 3) the front seats should be rotatable, thereby
providing flexibility for the front seat to vary between facing forward and rearward, which will

14
provide the drivers/passengers with the possibility of changing between individual and social
formations.
Front seats face backwards
Rotatable front seats
Folded front seats (a)
Folded front seats (b)
Figure 5. Design concepts for AV seating
Review of design concepts. A majority of the participant commented that the concept of
rotatable front seats was most preferred, as follows:
'This (rotatable front seats) is the best idea. I can choose to rotate the seat and talk to
other passengers face to face.'
'I like this idea the most because I can control when sitting facing forwards and when to
sit facing backwards.
'This (concept) allows flexibility. I can decide on the different seating positions based on
my activities in the car. Sitting facing backwards allows me to recline the seat without
affecting others.'
Having the front seats facing backwards was favoured for its function of supporting social
communication. However, some users also expressed concerns:
'The front seats facing backwards is good for me for engaging in social communication.
However, it limits the potential use of the seats in the front row and also limits any
potential customisation'.

15
Enabling the front seats to be folded was appreciated as this would provide extra space and
would enhance the level of comfort. However, it would limit the number of passengers in the
vehicle and a clear front view of the road, as explained:
‘This idea can provide more interior space during travel, although it limits the number
of passengers.’
‘I cannot get the best view of the road when sitting in the back seat’.
The concept of rotatable front seats was selected for development within the participatory
design workshop, and it was also mentioned in a few of the previous studies (Pettersson and
Karlsson, 2015; Cuddihy and Rao, 2016). Such a design is expected to better support
conversation and social interactions between drivers and passengers (Jorlöv et al., 2017). Sitting
in rear-facing seats is also expected to be safer during frontal collisions, as demonstrated in a
computer simulation study by Jin et al. (2018). Wu et al. (2020) also highlighted how, in a SAE
L3 highly autonomous vehicle, it is possible that an imminent stop could be detected well in
advance prior to the initial contact. The availability of this additional time could be used
strategically to actively position the driver into a safer position for a stop. If a take-over request
warning can be issued in advance, taking into consideration the seat turning time, is it still
feasible for a driver take-over process in rotatable seats? How is it possible to facilitate proper
situational awareness to support the take-over procedure? The workshop also raised these
questions, which were brainstormed and discussed among the designers and drivers, focusing
on how to enhance situational awareness in the selected concept. This included: 1) a foldable
in-vehicle roof-mounted display that delivers real-time information when sitting in a rear-facing
position; and 2) the content of the information presented to drivers. The discussion highlighted
the fact that too much information would overwhelm the driver, and a poorly designed method
of presenting information could add to the cognitive load and/or decrease a driver’s sense of
responsibility for the take-over procedure.
Figure 6. The foldable in-vehicle roof-mounted display

16
The design of a multimodal interface that distributes information across various sensory
channels, including visual and auditory, was proposed by both the designers and the potential
users. This design approach is based on the Multiple Resource Theory (Wickens, 2008), which
postulates that modalities represent separate attention resources, and thus resource competition
is reduced when tasks are presented via different modalities.
Continuous feedback regarding the
performance of the automated vehicle
Warning of urgency
Figure 7. The interface of the interactive in-vehicle display
Based on previous requirement studies, two types of information were identified during the
workshop. The first is to provide drivers with continuous feedback regarding the performance
of the automated vehicle, including information about real-time driving and the traffic situation
(i.e. road conditions, surrounding traffic and traffic signs). The second is warning of urgency,
which includes an appropriate warning about a take-over procedure. A combination of the use
of audio and visual information was designed to provide warnings for a take-over procedure to
improve the recognition of the urgency of any such warnings, while visual information was
designed to present continuous feedback of the driving and its situational development.
The location of the foldable in-vehicle roof-mounted display was decided to be as close as
practicable to the driver’s normal sightline when sitting in a rear-facing position (Burns &
Ekfjorden, 2000). The display is compatible with the feature of rotatable front seats.
Final concept. A final design concept was developed, consisting of rotatable front seats and an
in-vehicle interactive display, merging different user needs and requirements and refining them
based on the input from the participatory design workshops. We implemented our final concept
in a driving simulator which had been used for previous requirement studies.

17
Figure 8. Final design concept implemented in the driving simulator
The simulator consists of the same compact car body as in Study 1, equipped with a throttle
pedal and a brake, a steering wheel, four vehicle seats and one touch-screen display, sized 228
× 700 pixels (approximately 70 × 210mm). The front seats can swivel through 180 degrees to
allow both rear- and forward-facing travel. Both seats can also be adjusted to a maximum of
approximately 514mm in forward/rearward movement. A foldable in-vehicle interactive roof-
mounted display was installed to present the information within a view angle of 45 degrees and
a horizontal distance of 450mm in front of the users in a rearward-facing position.
Study 3: User evaluation
Even though the final design concept was developed together with the continuous involvement
of users in Study 2 to ensure the design could meet their needs, preferences and expectations,
there is still a need to investigate the effectiveness of how the final concept could ensure safe
driving, and in particular, how it could facilitate the transition between autonomous driving and
handing back control to the driver. The final evaluation aimed to address this question by
exploring and comparing the take-over performance of the drivers, as well as their situational
awareness under a variety of design conditions, including a traditional forward seating position
and our design of a rearward seating position, both with and without the support of an
information display.
Research method
Participants. A random sample selection recruited a total of 36 participants, all of which were
screened for their experience of driving, with the criterion that they could demonstrate at least
one year’s experience. Meanwhile, their driving frequency criterion was set at a minimum of
three times per week. The proportion of the age groups was 36.1% young drivers, 58.3%
middle-aged drivers and 5% senior drivers. Their average driving experience was 6.2 years (SD
= 4.2 years).
Table 2 Participants’ demographics

18
Age
Gender
< 30
30-50
>50
Female
Male
Driver
13
21
2
13
23
Apparatus. The user evaluation was conducted in the driving simulator which was built based
on Studies 1 and 2. The driving simulation trip incorporated both the highway and the urban
environments. The speed limit varied over the trip from 60 miles per hour in urban areas to 120
miles per hour on the highway.
Experiment design. A within-subject design was used. The independent variable was the in-
vehicle seating and display design with three conditions, including a forward-seating position,
a rearward-seating position without an information display, and a rearward-seating position
with an information display. The dependent variables included the objective take-over
performance (in terms of response time and the number of crashes) and subjective ratings of
situational awareness under a variety of design conditions.
Each participant performed three take-over procedures in the simulated AV. A collision
situation (an obstacle in the road) was built and programmed for all design conditions in the
simulator to examine the take-over procedure, during which the drivers were requested to turn
from sitting facing rearward to facing forward to take control of the vehicle, and requiring
emergency braking or changing lanes. An audio and visual warning was designed to alert the
driver of an impending collision situation seven seconds in advance (Wan & Wu, 2018). For
the audio warning, a voice with a standard accent with no particular mood was employed. The
visual warning would only appear under the condition of a rearward seating position with the
support of an information display (see Figure 7).
The information display was turned on to present the video feed of real-time traffic captured in
front of the vehicle. This was presented to the participants in real-time under the condition of
rearward seating with an information display, thereby allowing the participants to monitor the
real-time traffic situation around them. A semi-transparent level was set on the video presenting
information about the real-time driving and traffic situation (i.e. road conditions, surrounding
traffic and road signs). In the collision situation, a visual warning was displayed on the screen
nine seconds in advance.
The first study showed that a wide range of non-driving activities, including monitoring driving,
listening to music, watching a video, playing a game, reading, typing and sleeping, could impact
on take-over performance. Among these activities, many occupy both visual and motor
resources. Standard tests have been established to represent typical non-driving tasks. A
surrogate reference task (SuRT) can be used to occupy visual and motor resources, representing

19
visual-motor tasks such as using mobile phones or a vehicle’s centre console. A SuRT requires
participants to visually search for a target circle (slightly larger) among other smaller circles
and press keys to select and identify the target (International Organization for Standardization,
2012). For this experiment, we designed a SuRT task (Radlmayr et al. 2014) during the
experiments to simulate the distraction.
Procedure. Participants were first briefed in writing about the purpose of the experiment and
the automation. They were informed that the automated simulator did not have to be monitored
and they did not have to drive most of the time, although there were situations which the
automation could not handle so they would be requested to re-engage with driving the vehicle
in advance. They were also advised about how to conduct the SuRT task and how to use the
information display. The participants were then given a test drive to familiarise themselves with
the simulation and automation, including a take-over exercise in which they had to turn from
sitting facing rearward to facing forward. The experiment began when participants understood
the brief and were comfortable with the simulator. The experiment included three take-over
situations in three design conditions, as explained in the experiment design section, above. The
situational awareness rating technique (SART) (Selcon et al., 1992) was completed at the end
of each take-over situation to assess the situational awareness of the participants.
Dependent measure. We measured the time and quality aspects of the take-over performance
and the cognitive aspects of situational awareness under the various design conditions. With
regard to the time aspects, the response time was evaluated. We defined the response time as
the time between the warning and the moment at which the brake was applied by 10% or the
steering wheel was turned by 2 degrees, so the response time indicates how long it took the
driver to return to a driving position. The quality of the take-over was measured by the number
of crashes, which represents how successfully the driver managed to avoid collision situations.
The crashes recorded included the collisions with the obstacle in the road and/or with the curb
during the take-over procedures. Participants’ situational awareness was evaluated immediately
following the task performance, using a subjective rating of the 3D SART on three dimensions
that included attentional demands (D), attentional supply (S), and understanding (U). The
ratings for each of the three dimensions were combined into a single SART value according to
the formula (Selcon et al., 1992): Situational Awareness = U − (D − S).
Data collection and analysis. The response time was collected in the OpenDS simulator, while
the number of crashes was collected by the researchers. Three take-over situations were tested
for each participant, resulting in a total of 108 examples. An ANOVA was performed with
repeated measures. When Mauchly’s test for sphericity showed significance, values were
corrected, and all post hoc pairwise comparisons were Bonferroni corrected.

20
Results and Discussion
Take-over performance. With regard to the response time, participants in a forward-facing
sitting position took 3.65 seconds (SD 10.2) on average to take control of the vehicle, while
those in the rearward-facing position needed 4.53 seconds (SD 6.08), and those in the rearward-
facing seating with the support of a display screen required 4.33 seconds (SD 6.73). The seating
position showed a significant main effect on take-over performance in terms of response time
F (1, 8.29), p=.011 under the three conditions. The post hoc tests indicated that there was a
significant difference in response time between the rearward-facing seating position supported
with a display screen and the forward-facing seating (p=.0005), and between the rearward-
facing seating without a display screen and the forward-facing seating (p=.003), but there were
no significant differences between rearward-facing seating with a display and that without a
display (p >.05).
Figure 9. Response times under different experiment conditions
This observed significantly slower responses of participants in the two rearward-facing
positions might mainly contribute to the duration of the physical turning action from the
rearward-facing position to the forward-facing sitting position. To further explore the cognitive
effects on take-over performance under the three conditions, the physical turning action of 2
seconds, which was calculated in the simulator with 20 participants, was reduced in the two
rearward-facing position conditions. After processing the data, participants in a forward-facing
position required 3.65 seconds (SD 10.2) on average to take control of the vehicle, while those

21
in the rearward-facing position required 2.53 seconds (SD 6.08), and those in the rearward-
facing seating with the support of a display screen took 2.33 seconds (SD 6.73).
A significant difference was identified in terms of response time among the three conditions F
(2, 20.3), p=.0005. The post hoc tests indicated that there was a significant difference in
response time between the rearward-facing seating position supported with a display screen and
the forward-facing seating (p=.003), and between the rearward-facing seating without a display
screen and the forward-facing seating (p=.0005), but there were no significant differences
between rearward-facing seating with a display and that without a display (p >.05).
Figure10. Response times under different experiment conditions (after excluding the physical
action of turning)
With regard to the quality of the take-over performance, two crashes occurred under the
condition of forward-facing seating, four under the condition of rearward-facing seating
without a display, and one when participants were sat facing rearwards with a display. The
general linear mixed model for analysing the binomial variable (crash or no crash) from each
trial showed no significant difference identified in the number of crashes across the three
conditions F (2, .915), p= .404.

22
Figure 11. Number of crashes under the different experiment conditions
Situational awareness. The average situational awareness for participants sitting facing
forward was 51.5 (SD = 1.68), while the participants sat facing rearwards showed an average
situational awareness of 50 (SD = 1.81) and 55.4 (SD = 1.3) for participants sat facing rearwards
with a display. The analysis of the participants’ responses in respect of their situational
awareness showed that, on average, participants had a higher level of situational awareness
when sat facing rearwards with the support of a display, compared to that of the other two
conditions.
The seating position also demonstrated a significant effect on the cognitive aspect of situational
awareness F (4.99), p=.014 across the three experimental conditions. Situational awareness
under the rearward-facing seating condition with a display was significantly higher than that of
the forward-facing seating (p=.025) and the rearward-facing seating without a display (p= .013).
There was no significant difference in SA between the conditions of forward-facing and
rearward-facing seating without a display (p >.05).

23
Figure 12. Situational awareness under the different experiment conditions
Discussion
UCD approach
From a UCD approach, there is only a limited amount of research into gaining an understanding
of how users should be supported in an autonomous vehicle. Although current UCD studies
have highlighted that the interaction between driver and vehicle in autonomous driving is
subject to tremendous changes, when compared to manual driving (Jorlov et al. 2017), there
has been a lack of effort in developing a comprehensive understanding of driver-vehicle
interaction in autonomous vehicles from the perspective of end-users, and in offering guidance
for AV design. This research has employed a UCD approach to investigate how to design in-
vehicle elements of autonomous vehicles to support user requirements while also ensuring the
necessary level of safety. Representative end-users were involved in the requirement studies all
the way through the design workshops and evaluation experiments. Different scenarios and
simulator studies were applied to bridge the gap between the present and the future development
of personal transportation and to immerse the participants and designers in concrete driving
scenarios, while at the same time leaving sufficient space for the imaginary exploration of the
possibilities of an AV design (Brandt & Grunnet, 2000). The simulator provided the users with
a space in which to imagine and test possible future experiences. Together with the virtual
driving scene projected as a 360° visual field of view, it successfully guided the participants
into each driving scenario, while also leaving sufficient space for any new ideas to be inspired
and explored. The participatory design workshops invited designers and participants to immerse
themselves in the various scenarios which helped them explore any potential problems in the
imagined interactions, while also going deeper to generate more fruitful discussion concerning
such interactions, both “in terms of discussion and in design ideas” (Strömberg et al., 2018. p.
19). This process revealed useful information that has inspired our design.
New vehicle interior design
Findings from our requirement studies and participatory design showed how user needs and
requirements in an AV are different from those in traditional manual driving vehicles. In
accordance with Jorlov et al.’s (2017) claim, participants tended to expect autonomous vehicles
to release time for activities that, owing to a lack of time, would not otherwise be undertaken.
Thus, drivers would spend their time on non-driving related activities, such as eating, relaxing,
etc. Seating comfort was one of the major concerns for drivers. Kamp (2011) observed different
passengers’ postures and activities while travelling on trains in a semi-public situation, while
Yang et al. (2018) confirmed that there are going to be different non-driving postures in

24
autonomous vehicles, in relation to various non-driving related activities. Jorlov et al. (2017)
specifically focused on the comfort of the seats as one of the main concerns of new interior
designs, which may be overcome by making use of reclinable seats. We have proposed three
different seating positions, of which the most preferred position was that where the front seats
would be rotatable so that those in the front could turn around to interact with those in the rear
of the vehicle, and to release more space to support non-driving activities. These findings echo
the results from both Pettersson and Karlsson (2015) and Jorlov et al. (2017), who proposed
that more adjustable seats would be a significant trend in future vehicles.
Nevertheless, safety remains a priority for our design of a SAE Level 3 autonomous vehicle.
The possibility of being engaged in non-driving tasks would inevitably lead to a decline in the
situational awareness of drivers during autonomous driving (Birrell & Young, 2011; Merat &
Jamson, 2009). A key aspect of the new in-vehicle interaction is to maintain the situational
awareness of the driver while at the same time helping the driver to monitor situational
developments when engaging in non-driving tasks. This is to keep the drivers engaged in-the-
loop in case of critical situations arising (Hew et al. 2013; Reimer et al. 2016). We have
proposed an in-vehicle information display to provide drivers with continuous feedback
regarding the performance of the automated vehicle. A combination of visual and audio
communication to indicate the status of the vehicle was proposed to present information, as this
was expected to help in distributing cognitive loads as well as maintaining situational awareness.
Pettersson and Karlsson (2015) further highlighted that the use of in-vehicle screens might
provide a sense of hi-tech novelty and also enable non-driving activities, such as enjoying media
and performing work. Jorlov et al. (2017) also found that participants tended to relate screens
to the feeling that they were experiencing something luxurious, which revealed how they tended
to envision autonomous driving “as more than just a means of transportation” (p. 18).
The impact of the new design
Taking into consideration those situations when automation is not applicable for covering a
driving task in the SEA Level 3 of automation, the process of transferring control from the
vehicle back to the driver is new to traditional driving techniques (Large et al., 2017; Zeeb et
al., 2015). It is important to understand how the take-over process will work with any new
vehicle seating layout to guarantee safety (Kamp et al. 2011). The effectiveness of how the new
interior design can facilitate the transition between autonomous driving and handing back
control to the driver was also evaluated with the participants.
The findings indicate that drivers under the rearward-facing conditions had significantly slower
response times in take-over situations, which was not surprising. The slower response times
observed in participants in the two rearward-facing positions were mainly due to the extra time

25
spent physically turning the seat from rearward-facing to forward-facing. However, it is
interesting to note that the delay in the take-over reaction times (approximately 0.8 seconds)
was shorter than the time required to rotate the seat, which was approximately two seconds on
average. This indicates that in a take-over situation when drivers were sat facing rearwards,
after turning the seat to face forwards, they took actions more quickly than they would when
already in the forward-facing condition. They did not fully use the additional two seconds to
compensate for the time needed to rotate the seat. A possible explanation is that the drivers
perceived a higher level of time pressure when in the rearward-facing conditions because they
were aware that turning the seat would delay them from taking actions. When people experience
higher levels of stress, they consider fewer factors and make decisions involving less conscious
reasoning, which could lead to more rapid responses (Weltman et al., 1971). When people act
more quickly, it may be at the cost of reduced accuracy (Petermann et al. 2013), however, this
was not observed in this study. There were no significant differences in respect of the number
of crashes across the three conditions. One observation was that when providing a visual and
audio presentation of real-time driving information over the information display featured a
characteristic of the autonomous driving that was transparent to drivers, the participants
allocated more resources to the monitoring tasks, which may have resulted in enhanced take-
over performance.
Our findings also show that drivers sat facing rearwards with the support of a display, had the
highest level of situational awareness, when compared to the other two conditions. We found
that the take-over task included the key aspects of rebuilding situational awareness, scanning
the environment and planning and executing a dynamic manoeuvre, including longitudinal and
lateral control. The complexity of such a process makes it radically different from the
established mental models of current primary controls and manoeuvres in a traditional vehicle.
While Winter et al. (2014) pointed out that drivers of automated vehicles are likely to engage
in tasks unrelated to driving, and would exhibit a deteriorated level of situational awareness,
according to our observations, the information display in the facing rearward condition had
intuitive appeal to participants. This effect could enable participants to monitor the driving with
less cognitive burden, and would support drivers’ rapid reallocation of their situational
awareness and thereby facilitate their take-over behaviours. This explanation is also supported
by previous researchers who have designed information displays to enhance situational
awareness to help users communicate and respond to security alerts both efficiently and
effectively (Chen et al. 2016).
Design implications

26
One promising means of enhancing the take-over performance and enhancing situational
awareness was found through the design of a multimodal information display that distributes
information across various sensory channels, including visual and auditory. The design of in-
vehicle displays has already been explored in several studies. Seppelt and Lee’s (2007)
highlighted how representation of continuous information would enhance driving performance.
Costa et al. (2017) found that the combination of a graphical representation of traffic conditions
with a haptic and/or sound information could result in better performance with regard to
intervention times and situational awareness. Politis et al. (2013) evaluated a set of multimodal
abstract warnings across three levels of urgency. They found that an increased number of
modalities increased ratings of urgency and annoyance. Similarly, in their studies of multimodal
warnings signifying the handover of control in autonomous driving, Politis et al. (2017) found
multimodal cues to be more urgent and more effective. However, Lu et al. (2013) found urgent
interruption signals are better presented via the auditory channel only, while Stanney and Hale
(2014) recommended the use of haptic signals together with audio signals for warnings when
response times are critical. More research is required to support the design of effective, user-
centred multimodal interfaces for autonomous in-vehicle displays.
Conclusions, limitations and future work
Compared with existing works in the domain of automated driving vehicles, little research has
been done in respect of designing a vehicle interior based on an understanding of human drivers.
This paper has reported a UCD approach to designing a new interior in AVs, which was
informed by, and tested against, studies of human behaviour in the driving context. Our results
suggest the design of rearward-facing rotatable seats supported by a multimodal information
display is a promising approach for facilitating in-vehicle non-driving activities and assisting
drivers’ situational awareness to prepare for take-over situations in the SEA Level 3 long-term
take-over autonomous environment.
There are some limitations worthy of further discussion. First, the effects on safety of the
proposed design were investigated in terms of response times and the overall number of crashes
during the take-over situations. We were not able to analysis other dimensions, such as time to
collision, steering performance and braking performance, to measure the precision and safety
margins with which drivers took over control. Future work could validate such potential aspects
of future autonomous vehicle design by establishing metrics to enable a closer examination of
the impact on safety. Second, the new rotatable seat installed in the simulator was able to swivel
through 180 degrees. However, it did not have the features of adjustable supports (i.e. the ability
to change height and angles). As safety and comfort remain priorities for auto manufacturers,
future work will develop the adjustable features of front rotatable seats to improve the

27
experience of passengers within autonomous vehicles. Finally, we recruited more middle-aged
drivers than other age-group drivers, and the average driving experience of participants was
relatively low. The proportion of the representatives of each age group in the sample, and their
respective driving experience, reflected the corresponding proportions in the local context. A
recent population-based controlled study in a capital city in China indicated a proportion of
18.5 % young drivers, 61% middle-aged drivers and 20% older drivers (Shen et al. 2018). With
regard to their driving experience, Li and Henk (2014) pointed out that a large proportion of
the drivers in China have relatively little experience, and the number of drivers with less than
three years’ experience represented approximately one third of the entire driver population.
There were also more male than female drivers involved in the first and third studies. This
gender distribution was also found to be similar to that in the empirical studies in China (Li et
al. 1998; Shen et al. 2018). While the sample demographics seem reasonable, there is a lack of
official information on the overall driver population distribution in this context to confirm the
appropriateness of the sample selection (Stevenson, 2011).
References
Adam, E. V. (1993). Fighter cockpits of the future. In Proceedings of the 12th IEEE/AIAA Digital
Avionics Systems Conference, Fort Worth, TX, 318–323
Abbink, D. A., Mulder, M., & Boer, E. R. (2012). Haptic shared control: Smoothly shifting control
authority? Cognition, Technology and Work, 14(1), 19–28
Beattie, D., Baillie, L., Halvey, M., & Mccall, R. (2014). What’s around the corner? Enhancing Driver
Awareness in Autonomous Vehicles via In-Vehicle Spatial Auditory Displays. In Proceedings of the 8th
Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational - NordiCHI ’14, 189–198,
Helsinki, Finland.
Beggiato, M., Hartwich, F., & Krems, J. (2019). Physiological correlates of discomfort in automated
driving. Transportation Research Part F: Traffic Psychology and Behaviour, 66, 445-458
Bellem, H., Thiel, B., Schrauf, M., & Krems, J. F. (2018). Comfort in automated driving: An analysis of
preferences for different automated driving styles and their dependence on personality traits.
Transportation Research Part F: Traffic Psychology and Behaviour, 55, 90-100
Bellem, H., Schönenberg, T., Krems, J. F., & Schrauf, M. (2016). Objective metrics of comfort:
Developing a driving style for highly automated vehicles. Transportation Research Part F: Traffic
Psychology and Behaviour, 41, 45–54
Beller, J., Heesen, M., & Vollrath, M. (2013). Improving the driver-automation interaction: An approach
using automation uncertainty. Human Factors, 55(6), 1130–1141
Birrell, S. A., & Young, M. S. (2011). The impact of smart driving aids on driving performance and
driver distraction. Transportation Research Part F: Traffic Psychology and Behaviour, 14(6), 484–493
Brandt, E., & Grunnet, C. (2000). Evoking the future: Drama and props in user centered design.
Proceedings of Participatory Design Conference PDC 2000, 11–20
Burns, P.C. A. & Ekfjorden, H.A. (2000) Placing Visual Displays in Vehicles: Where should they go?
International Conference on Traffic and Transport Psychology At: Bern, Switzerland

28
Chandrasekaran, L., Crookes, A., & Lansdown, T. C. (2019). Driver situation awareness – Investigating
the effect of passenger experience. Transportation Research Part F: Psychology and Behaviour, 61, 152–
162
Chen Y.V., Qian, Z.C. and Lei, W.T. (2016) Designing a Situational Awareness Information Display:
Adopting an Affordance-Based Framework to Amplify User Experience in Environmental Interaction
Design, Informatics, 3, 6
Costa, N. A., Holder, E., & MacKinnon, S. N. (2017). Implementing human centred design in the context
of a graphical user interface redesign for ship manoeuvring. International Journal of Human Computer
Studies
Droumeva, M. and Wakkary, R. (2006)The role of participatory workshops in investigating narrative
and sound ecologies in the design of an ambient intelligence audio display. London
Elbanhawi, M., Simic, M., & Jazar, R. (2015). In the Passenger Seat: Investigating Ride Comfort
Measures in Autonomous Cars. IEEE Intelligent Transportation Systems Magazine, 7(3), 4–17
Endsley, M.R. (1995) Measurement of situation awareness in dynamic systems. Hum. Factors 37, 65–84
Endsley, M.R. (2016) Designing for Situation Awareness: An Approach to User-centered Design, CRC
press, Boca Raton, FL
Endsley, M. R. (2017). Autonomous Driving Systems: A Preliminary Naturalistic Study of the Tesla
Model S. Journal of Cognitive Engineering and Decision Making, 11(3), 225–238.
doi:10.1177/1555343417695197
Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). “Take Over!” How Long Does It Take To Get
the Driver Back Into the Loop? In The Human Factors and Ergonomics Society, 57th Annual Meeting ,
1938–1943
Gold C., Naujoks F., Radlmayr J., Bellem H., Jarosch O. (2018) Testing Scenarios for Human Factors
Research in Level 3 Automated Vehicles. In: Stanton N. (eds) Advances in Human Aspects of
Transportation. AHFE 2017. Advances in Intelligent Systems and Computing, vol. 597. Springer.
Habibovic, A., Andersson, J., Nilsson, M., Lundgren, V. M., & Nilsson, J. (2016). Evaluating
Interactions with Non - existing Automated Vehicles : Three Wizard of Oz Approaches. In 2016 IEEE
Intelligent Vehicles Symposium (IV), 32–37
Helldin, T.,Falkman, G., Riveiro,M., Davidsson,S. (2013) Presenting system uncertainty in automotive
UIs for supporting trust calibration in autonomous driving. Proceedings of the 5th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‚13),
Eindhoven, The Netherlands.
Hergeth, S., Lorenz, L., Vilimek, R., & Krems, J. F. (2015). Keep Your Scanners Peeled: Gaze Behavior
as a Measure of Automation Trust during Highly Automated Driving. Human Factors, 58(3), 509–519.
Hew, P. (2013) Reconciling situation awareness in humans versus machines via mode awareness and
schemata, Theoretical Issues in Ergonomics Science, 14:4, 330-351
Jarosch, O., & Bengler, K. (2018). Rating of Take-Over Performance in Conditionally Automated
Driving Using an Expert-Rating System. Advances in Human Aspects of Transportation, 283–294.
Jorlov, S., Bohman, K., & Larsson, A. (2017). Seating Positions and Activities in Highly Automated
Cars–A Qualitative Study of Future Automated Driving Scenarios. IRCOBI Conference 2017, 749–750.
Kaber, D. B., Liang, Y., Zhang, Y., Rogers, M. L., & Gangakhedkar, S. (2012). Driver performance
effects of simultaneous visual and cognitive distraction and adaptation behavior. Transportation Research
Part F: Traffic Psychology and Behaviour, 15n(5), 491–501
Kamp,M.F., Kilincsoy, & Vink (2011) A light weight car seat shaped by human body contour,
International Journal of Human Factors Modelling and Simulation. Vol. 2(4), 2011

29
Krome, S., Goddard, W., Greuter, S., Walz, S. P., & Gerlicher, A. (2015). A context-based design process
for future use cases of autonomous driving. In Proceedings of the 7th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI ’15 ,265–272,
Nottingham, UK.
Kyriakidis,M.,de Winter, J. C. F., Stanton, N., Bellet,T., van Arem, B. Brookhuis,K. M. Martens, H.,
Bengler,K., Andersson, J., Merat, N. Reed, N.Flament,M. Hagenzieker M. & Happee R.(2017): A
human factors perspective on automated driving, Theoretical Issues in Ergonomics Science, DOI:
10.1080/1463922X.2017.1293187
Large, D. R., Banks, V., Burnett, G., & Margaritis, N. (2017). Putting the Joy in Driving: Investigating
the Use of a Joystick as an Alternative to Traditional Controls within Future Autonomous Vehicles.
Automotiveui 2017: Proceedings of the 9Th International Conference on Automotive User Interfaces
and Interactive Vehicular Applications, 31–39.
Li, J.W., Wu,S.M., Wang, M.Q, Yang, X.L., Chen,B. (1998) Health condition of licensed drivers in
Guiyang, China. Journal of Occupational Health and Damge 1998,Vol . 13 No. 4 (in Chinese)
Li, J. and Henk, J.Z. (2014) Road Traffic in China, Procedia - Social and Behavioral Sciences, Vol. 111:
107 – 116
Llaneras R., Salinger J., Green C.(2013). Human factors issues associated with limited ability
autonomous driving systems: Drivers’ allocation of visual attention to the forward roadway. Proceedings
of the 7th International Driving Symposium on Human Factors in Driver Assessment, Training and
Vehicle Design, 92–98
Lu, S. A., Wickens, C. D., Prinet, J. C., Hutchins, S. D., Sarter, N., & Sebok, A. (2013). Supporting
interruption management and multimodal interface design: Three meta-analyses of task performance as
a function of interrupting task modality. Human Factors, 55(4), 697–724
Merat, N., Jamson, A. H., Lai, F. C. H., Daly, M., & Carsten, O. M. J. (2014). Transition to manual:
Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part
F: Traffic Psychology and Behaviour, 27, 274–282
Miller S P , Barber S , Carlson T M , Lempia, D., Tribble, A. (2002) A methodology for improving
mode awareness in flight guidance design, Digital Avionics Systems IEEE Conference.
Merat, N., & Jamson, a. H. (2009). How do drivers behave in a highly automated car? In
PROCEEDINGS of the Fifth International Driving Symposium on Human Factors in Driver Assessment,
Training and Vehicle Design HOW,514–521, Big Sky, Montana. Iowa City, IA
Merat, N., Jamson, A. H., Lai, F. C. H., & Carsten, O. (2012). Highly automated driving, secondary task
performance, and driver state. Human Factors, 54(5), 762–771
Miller S P , Barber S , Carlson T M , Lempia, D., Tribble, A. (2002) A methodology for improving
mode awareness in flight guidance design, Digital Avionics Systems IEEE Conference.
Mok, B. K., Sirkin, D., Sibi, S., David Bryan Miller, & Ju, W. (2015). Understanding Driver-Automated
Vehicle Interactions Through Wizard of Oz Design Improvisation. In the Eighth International Driving
Symposium on Human Factors in Driver Assessment, Training and Vehicle Design , 380–386, Salt Lake
City, Utah: Public Policy Center, University of Iowa.
Odom, W., Zimmerman, J., Davidoff, S., Forlizzi, J., Dey, A. K., & Lee, M. K. (2012). A Fieldwork of
the Future with User Enactments. In DIS 2012. In the Wild, 338–347, Newcastle, UK.
Pettersson, I., & Karlsson, I. C. M. (2015). Setting the stage for autonomous cars: a pilot study of future
autonomous driving experiences. IET Intelligent Transport Systems, 9(7), 694–701
Politis, I., Brewster, S., and Pollick, F. (2013) Evaluating Multimodal Driver Displays of Varying
Urgency. Automotive UI 2013, ACM Press, 92 – 99

30
Politis, I., Brewster, S., and Pollick, F. (2017) Using Multimodal Displays to Signify Critical Handovers
of Control to Distracted Autonomous Car Drivers, International Journal of Mobile Human Computer
Interaction 9(3)
Qatu, M. S. (2012). Recent research on vehicle noise and vibration. International Journal of Vehicle
Noise and Vibration, 8(4), 289
Radlmayr, J., Gold, C., Lorenz, L., Farid, M., & Bengler, K. (2014). How traffic situations and non-
driving related tasks affect the take-over quality in highly automated driving. Proceedings of the Human
Factors and Ergonomics Society, 2014-Janua(1988), 2063–2067
Reimer, B., Mehler, B., & Coughlin, J. F. (2016). Reductions in self-reported stress and anticipatory
heart rate with the use of a semi-automated parallel parking system, Applied Ergonomics, 52, 120–127
Robins, B., Ferrari, E., Dautenhahn, K., Kronreif, G., Prazak-Aram, B., Gelderblom, G. J., … Marti, P.
(2010). Human-centred design methods: Developing scenarios for robot assisted play informed by user
panels and field trials. International Journal of Human Computer Studies, 68(12), 873–898
SAE (2018) Taxonomy and definitions for terms related to driving automation systems for on-road motor
vehicles (Surface Vehicle Recommended Practice: Superseding J3016 Jun 2018), SAE International,
September 2018
Salvendy, G. (2014) Handbook of Human Factors and Ergonomics, Wiley.
Samuel, S. & Fisher, D.L. 2015, Evaluation of the Minimum Forward Roadway Glance Duration,
Transportation Research Record: Journal of the Transportation Research Board, no.
2518, 9-17.
Selcon, S., Taylor, R., & Shadrake, R. (1992). Multi-Modal Cockpit Warnings: Pictures. Words. or Both?
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 36, 57–61.
Seppelt, B. D., & Lee, J. D. (2007). Making adaptive cruise control (ACC) limits visible. International
Journal of Human Computer Studies, 65(3), 192–205
Shen, B.Y., Qu,W., Yan,G, Sun, X.H., Zhang, K. (2018) The relationship between personalities and self-
report positive driving behavior in a Chinese sample,PLoS One. 2018; 13(1)
Stanney, K. M., and Hale, K. S. (2014). Handbook of Virtual Environments: Design, Implementation,
and Applications. Boca Raton, FL: CRC Press.
Strömberg, H., Pettersson, I., Andersson, J., Rydström, A., Dey, D., Klingegård, M., & Forlizzi, J. (2018).
Designing for social experiences with and within autonomous vehicles – exploring methodological
directions. Design Science, 4(13), 1–29.
Vakil,S.S., Hansam, R.J., Midkiff, A.H., Vaneck, T. (1995) Mode Awareness in Advanced Autoflight
Systems, IFAC Proceedings Vol.28 (15), 371-376
Wickens, C., Small, R., Andre, T., Bagnall, T., & Brenaman C. (2008). Multisensory Enhancement of
Command Displays for Unusual Attitude Recovery. The International Journal of Aviation Psychology,
18, 255-267
Winter, J.C.F., Happee, R., Martens, M.H., and Stanton, N.A.Effects of adaptive cruise control and
highly automated driving on workload and situation awareness: A review of the empirical evidence.
Transportation Research Part F: Traffic Psychology and Behaviour 27, (2014), 196–217
Yang, Y. C.& Klinkner, J.N.(2018) How Will the Driver Sit in an Automated Vehicle? – The Qualitative
and Quantitative Descriptions of Non-Driving Postures (NDPs) When Non-Driving-Related-Tasks
(NDRTs) Are Conducted, Proceedings of the 20th Congress of the International Ergonomics Association
(IEA 2018), 409-420
Zeeb, K., Buchner, A., & Schrauf, M. (2016). What determines the take-over time? An integrated model
approach of driver take-over after automated driving. Accident Analysis and Prevention, 78, 212–221