ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip for Non-Visual Navigation

Abstract

Non-visual feedback (e.g., auditory and haptic) has been used as directional cues for the blind and visually impaired (BVI) users. This paper presents the design and the evaluation of ThermalCane, a white-cane grip instrumented with multiple flexible thermal modules, to offer thermotactile directional cues for BVI users. We also conducted two thermotactile experiments on users’ perception of ThermalCane. Our first experiment with twelve BVI users reports on the selection of the thermal-module configuration, considering the BVI users’ perceptive accuracy and preference. We then evaluated the effectiveness of the four-module ThermalCane in walking with 6 BVI users, in comparison with vibrotactile cues. The results show that the thermal feedback yielded significantly higher accuracy than the vibrotactile feedback. The results also suggested the feasibility of using thermal directional cues around the cane grip for BVI users’ navigation.

Full text

ThermalCane: Exploring Thermotactile
Directional Cues on Cane-Grip for Non-Visual Navigation
Arshad Nasser Kai-Ning Keng Kening Zhu*
arshad.nasser@my.cityu.edu.hk kaining1101@gmail.com keninzhu@cityu.edu.hk
School of Creative School of Creative School of Creative
Media, City University of Hong Kong Media, City University of Hong Kong Media, City University of Hong Kong
Hong Kong, China Hong Kong, China Hong Kong, China
ABSTRACT
Non-visual feedback (e.g., auditory and haptic) has been used as direc-
tional cues for the blind and visually impaired (BVI) users. This paper
presents the design and the evaluation of ThermalCane, a white-cane
grip instrumented with multiple flexible thermal modules, to offer
thermotactile directional cues for BVI users. We also conducted two
thermotactile experiments on users’ perception of ThermalCane. Our
first experiment with twelve BVI users reports on the selection of the
thermal-module configuration, considering the BVI users’ perceptive
accuracy and preference. We then evaluated the effectiveness of the
four-module ThermalCane in walking with 6 BVI users, in compari-
son with vibrotactile cues. The results show that the thermal feedback
yielded significantly higher accuracy than the vibrotactile feedback.
The results also suggested the feasibility of using thermal directional
cues around the cane grip for BVI users’ navigation.
CCS CONCEPTS
• Human-centered computing → Haptic devices
;
Accessibility
systems and tools.
KEYWORDS
Thermotactile, thermal haptic, tactile white cane, directional cue,
navigation, visually impaired
ACM Reference Format:
Arshad Nasser, Kai-Ning Keng, and Kening Zhu. 2020. ThermalCane: Ex-
ploring Thermotactile Directional Cues on Cane-Grip for Non-Visual Naviga-
tion. In The 22nd International ACM SIGACCESS Conference on Computers
and Accessibility (ASSETS ’20), October 26–28, 2020, Virtual Event, Greece.
ACM,New York, NY, USA,12 pages. https://doi.org/10.1145/3373625.3417004
1 INTRODUCTION
According to 2018 data of the World Health Organization, there are
around 440 million visually impaired (blind or low-vision) people in
the world [
5
]. It is common for BVI people using white canes while
navigating, mostly for recognizing the obstacles. However, white
*Corresponding author.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
© 2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-7103-2/20/10. . .$15.00
https://doi.org/10.1145/3373625.3417004
Figure 1: Concept of ThermalCane grip with thermotactile
directional cues (red: hot, blue: cold).
canes may have a limited field of “view” due to their fixed length. In
addition, it is challenging for the passive/non-interactive cane itself to
notify directional cues for BVI users’ navigation. Besides the white
canes, BVI users often utilize various digital technologies, such as
GPS, to support daily space navigation and target searching. GPS
devices usually offer the auditory representation of the information
and could potentially help the BVI persons for spatial navigation.
However, the auditory information could be interfered in a noisy
environment [
15
], and it may interrupt users’ on-going verbal conver-
sation, music, or radio program. Researchers started investigating the
feasibility of adopting haptic feedback, mainly vibrotactile feedback,
as the navigational cues for sighted and BVI people. The vibrotactile
feedback has been applied for different purposes, including navi-
gation [
32
,
50
,
57
] and notifications/warnings [
47
,
48
]. Also, the
vibrotactile feedback has been tested individually [
32
,
57
] and in com-
bination/comparison with other modalities [
47
,
48
] for notification on
the move. However, sometimes it could be difficult to detect the exact
vibration location [
32
] in the context of multi-point spatial vibrotactile
feedback, as the natural turbulence or movements during walking or
driving may affect the perception of vibration [
13
,
24
,
40
,
45
]. Besides
the vibrotactile feedback, there is an increasing amount of research
interest in the recent years in the application of thermal feedback
for human-computer interaction (HCI). The thermal feedback may
not have these disadvantages of vibration, and the characteristics of
single-spot and multi-spots thermal feedback have been investigated
for mobile devices [
20
,
21
,
59
,
61
] and smart wearable accessories
[
44
,
45
,
63
] for general purposes, with a reliable recognition accuracy.
In addition, the thermal feedback can be integrated on the steering
wheel for notifying lane changes and directions in driving simulation

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
[
12
,
13
]. While thermal feedback has shown great potential in facil-
itating information representation, it is still unclear how it could be
perceived by BVI users, and how it can support BVI users’ navigation.
Research showed that the palm area is one body part with high ther-
mal sensitivity [
7
,
21
], and it is commonly used by BVI peoplein many
daily activities of information comprehension [
42
]. In this paper, we
present the design of ThermalCane (Fig. 1), integrating the spatial ther-
mal feedback into the white-cane grip to support BVI users’ spatial
navigation. BVI users can receive the feedback of temperature change
on their hands while holding the ThermalCane, and further induce the
navigation direction. We first designed three prototypes of Thermal-
Cane using flexible Peltier modules, and each prototype contained a
unique layout of the modules (i.e., 3, 4, and 5 Peltier modules). Our
first experiment involving 12 BVI users showed that the users can rec-
ognize the spatial thermal stimuli with an average accuracy of 84.7%
and an average response time of 2.71s. The prototypes with three and
four Peltier modules were perceived significantly more accurately
than the prototype with five modules. Furthermore, the BVI users com-
mented that the cold stimuli was easier and more pleasant to be recog-
nized as a feedback cuewhen compared to the hot stimuli.Considering
the participants’ preference on the four-modules grip configuration for
directional notification, we designed ThermalCane, and investigateed
its effectiveness of offering the directional cues for BVI users. The
walking experiments involved 6 BVI users, and showed that the ther-
mal feedback of ThermalCane outperformed the vibrotactile feedback
with a significantly higher recognition accuracy. This further showed
that the spatial thermal stimuli around the white-cane grip could be
an effective directional cues for BVI users’ walking activities.
The contributions of this paper are three folds:
•
We present an optimal layout for thermal feedback on the
white-cane grip based on a detailed evaluation involving BVI
users on different configurations of thermal modules and ther-
mal signals;
•
Based on the selected four-module configuration, we designed
a set of directional cues provided by the flexible Peltier mod-
ules around the ThermalCane grip;
•
We validated the effectiveness of ThermalCane directional
cues with BVI users in the walking context in comparison with
the vibrotactile cues.
2 RELATED WORKS
The research of ThermalCane was highly inspired by existing research
on thermotactile feedback in HCI and haptic navigation aids for both
sighted and BVI users.
2.1 Thermal Feedback in HCI
As one early study on thermal feedback, Jones and Berris [
29
] sug-
gested a list of design recommendations for the thermal display based
on psychological evidence. Some comprehensive research on thermal
feedback in HCI has provided important insights such as: 1) hand is
a body part with high thermal sensitivity [
21
]; 2) the perception of
thermal feedback could be strongly affected by clothes [
21
] and the
environment [
20
]; 3) a set of thermal icons with an overall recognition
accuracy of 83% can be designed using the rate and the direction of
temperature change [
59
]. Following Wilson et al.’s insights, Tewell
et al. showed that thermal feedback could enhance the emotional per-
ception of text-based information [
55
] and could be used to support
on-screen navigation [
56
] for sighted users. Singhal and Jones [
52
]
evaluated thermal pattern recognition on the hand and arm with single
thermoelectric module, and proposed the model-based approach for
designing thermal icons. More recently, researchers started investi-
gating the spatial thermal feedback in wearable accessories, such as
finger ring [
63
], bracelet [
45
], earhook [
44
], etc. In this paper, we
expand the research of thermal feedback for HCI to the accessibility
technology for BVI users, with the ThermalCane prototype.
2.2 Haptic Navigation Aids
The visual navigation information could be sometimes distracting
for sighted users and not accessible for BVI users, and the auditory
notification can be affected by the ambient noise [
15
]. Therefore,
researchers investigated the feasibility of haptic information repre-
sentation for navigation.
2.2.1 Vibrotactile Navigation Aids. As a type of on-body haptic
feedback, vibrotactile feedback has been used for a wide range of nav-
igation research. Zelek et al. [
62
] developed a wearable glove device
with piezo-electric vibrotactile units to represent the stereo-image
information for BVI users’ real-world environment understanding.
Bouzit et al. [
6
] developed the tactile handle, a handheld device with
a 4
×
4 array of vibration motors around the handle, to help BVI users
to navigate in an unfamiliar space. Johnson and Higgins’s work [
28
]
facilitated BVI users’ obstacle avoidance with a vibrotactile vest.
Ertan et al. [
16
], who employed a 4x4 grid of vibrotactile actuators on
the back of a vest to indicate turn information. Similarly, Scheggi et
al. [
51
] used wrist-worn devices with two haptic actuators that were
controlled by a remote observer to help blind users avoid obstacles
while walking. Mann et al. [
39
] employed a haptic helmet combined
with a depth camera to help blind users avoid collisions, although no
user evaluation was reported. Cosgun et al. [
11
] evaluated a haptic
belt with eight motors to provide directional guidance in discrete
45
◦
increments. They showed that users could recognize which of
the eight motors was vibrating with 55-97% accuracy, depending on
the motor. Similarly, Dura-Gil et al. [
14
] compared the multi-spot
vibrotactile directional cues in the form factor of a waist belt, and
found that BVI users preferred the directional cues presented on the
front side of the waist, and the instructional vibrations (e.g., stop) on
the back. Cassinelli et al. [
8
] developed Haptic Radar, a vibrotactile
headband for BVI users, to augment the users’ spatial awareness in
everyday life. Adopting the similar hardware design, Chen et al. [
9
]
investigated the effectiveness of visual, auditory, and vibrotactile di-
rectional cues in VR, suggesting that vibrotactile could be suitable for
multi-task situation in VR. Flores et al. [
17
] defined a set of around-
waist vibrotactile icons for guiding the BVI users’ navigation. Hong
et al [
26
] investigated the directional vibrotactile feedback around the
wrist to support BVI users navigating on the 2D touch screen. The
vibrotactile feedback was also widely applied in the navigation white
cane for BVI users. Balakrishnan et al. [
1
] developed a cane-mounted
accessory for obstacle detection and warning. Maidenbaum et al. [
38
]
developed EyeCane with single vibration motor to notify the obstacle,
and validated their design with both blindfolded sighted users and BVI
users. More recently, Meshram et al. [
41
] developed NaviCane, which

ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
contains multiple ultrasonic sensors and vibration motors, to represent
the distances of the obstacles at different height through vibration.
Among these research, the vibration intensity of single vibrator or
all of the multiple vibrators was often used for representing the dis-
tance of the obstacle, while the direction of the obstacle was presented
by the on-hand/on-body vibration location. Though these studies
showed the effectiveness of the vibrotactile devices in different form
factors, researchers argued that the vibration feedback could be weak-
ened due to the turbulence generated by the moving process, such as
walking [40, 45] and driving [13, 24].
2.2.2 Thermotactile Navigation Aids. Compared to the vibrotactile
feedback, there is relatively less research focusing on thermotactile
feedback as directional cues. As a conceptual application, Wettach
et al. [
58
] suggested that thermal feedback could be activated on a
hand held device for the pedestrians. The users got a warm stimuli
on the device as they moved close to the set destination. Tewell et
al. [
56
] evaluated this concept, by employing the warm stimuli for
guiding a 2D maze game on the touch screen. Di Campli San Vito
et al. [
13
] investigated thermal navigation feedback on the steering
wheel and compared it with cutaneous push and audio feedback dur-
ing a simulated driving task. It was observed that the performance
increased with the thermal haptic feedback. Lecuyer et al. [
36
] devel-
oped a device to indicate the sun direction to BVI users. Balata et al.
[
3
] developed a handheld haptic device that presented the heat-map
navigation information through thermotactile feedback for BVI users.
Quido [
2
] used both thermal and vibrotactile feedback to guide partic-
ipants in a 2D maze. Nakashige et al. [
43
] studied a spatial navigation
task where users searched for an object on a computer display with a
temperature-augmented mouse. Peiris et al. [
46
] demonstrated the the
use of thermal devices embedded in the head-mounted display to offer
directional cues in VR. They further developed ThermalBracelet [
45
],
evaluating the recognition of multi-spot thermal stimuli around the
wrist while walking. Recently, Zhu et al. [
63
] investigated the spatial
thermotactile feedback in the form factor of finger ring, and proposed
the application of direction cues for biking. More recently, Kim et al.
[
33
] developed flexible peltier-based thermal modules, and proposed
to attach the flexible module on the grip of a white cane for obstacle
notification for blind people. Although their technical experimen
proved the temperature-changing performance of the flexible pelit
module, the proposed application for blind people has not yet been
properly evaluated from a user’s point of view. There is a need for
more in-depth investigation on how BVI users may perceive this type
of tactile feedback, especially how to design assistive devices with
thermal feedback.
Taking BVI users’ feedback and preference into account, we de-
signed ThermalCane with four flexible peltier modules around the
grip. Our user studies showed that the thermal directional cues could
reliably be perceived by BVI users while walking, and it outperformed
the vibrotactile feedback in terms of accuracy. Thus, thermal feedback
can potentially serve as a new type of navigational aid for BVI users.
3 THERMALCANE DESIGN
Considering the common posture of BVI people grasping their white
canes and the hardware configuration of thermal actuators, we de-
signed ThermalCane, a white-cane grip with multi-spot spatial ther-
motactile feedback.
Figure 2: Three different grasping postures of a whitecane.
ts
er
Figure 3: Three different configurations of thermal cane grips
used in Study 1. The Peltier modules are placed in equal slots
around the grip.
3.1 Grasping Posture of White Cane
White canes are widely used by BVI people these for obstacle detec-
tion in walking. There are mainly two types of white canes: rigid and
foldable, either of which contains a grip at one end for firm grasping.
As shown in Fig. 9, there are three common postures for grasping
white canes [
27
]: the index-finger grasp, the pencil grasp, and the
handshake grasp. While it is stated that BVI users may select a par-
ticipant method of cane grasping depending on the situation [
27
], we
chose the hand-shake gripping posture to design and validate Ther-
malCane, as the related literature shows that it is the most preferred
posture by BVIs [
35
]. In addition, a survey from the local blind asso-
ciation also indicates that the hand-shake gripping is the most adopted
posture by the BVIs in the authors’ region. More specifically, we
arranged the Peltier modules around the grip in the layouts of three,
four, and five slots, as shown in Fig. 3. To ensure the closed contact
between the modules and the palm during the handshake grasp, we
placed one Peltier module on the top surface of the grip, and the rest
of the modules are placed in equal gaps around the grip.

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
Figure 4: a) Flexible Peltier module. (b) Flexible Peltier affixed
to a four-module grip. Figure 5: Experiment setup in Study 1.
3.2 Flexible Thermal Modules
We used thermoelectric modules (Peltier modules) as actuators for
thermal cues. Specifically, the current design of ThermalCane was
equipped with multiple flexible Peltier modules
1
(Fig. 4a) around
the grip(Fig. 4b). The flexible Peltier module is in the dimension of
23
×
79
×
2 mm, and consists of a matrix of micro Peltier elements
without the ceramic insulation. We selected these flexible modules
because of two reasons: 1) They are flexible and lightweight, and can
fit the surface of the cane grip which is not possible with the solid
Peltier modules; 2) the flexible Peltier module offers higher thermal
efficiency, in comparison with the solid counterparts, due to the ab-
sence of ceramic insulation and 96 micro Peltier elements in a larger
active area of 17×75 mm.
3.3 System Description
Fig. 3 depicts the 3D-printed models of the thermal cane grips used
in our research. The grips were in the shape of cylinder tube with
the inner diameter of 1.5cm and the outer diameter of 3.6cm, for the
ease of installation on the existing white cane. The grips were 3D
printed with PLA (Polylactic Acid). Considering the dimension of
the flexible Peliter module and the average hand size of human [
34
],
we designed and fabricated the grips with three, four, and five slots for
the attachment of the flexible Peltier modules, to ensure the modules
closely in touch with the whole palm. Once the Peliter modules were
attached on the grip, the grip was further tightened with two rubber
bands, ensuring the modules firmly attached.
All the Peltier modules are driven using an H-bridge driver mod-
ule (Model No.: L298N) and Arduino Uno microcontroller, with an
external 9V battery. Each Peltier module draws a maximum of 1A at
9V during the stimulation. The system was controlled by the Arduino
Uno connected to a laptop through USB, to ensure the fine control
of the temperature through Pulse Width Modulation (PWM). Follow-
ing the recent related research on wearable thermal devices [
45
,
63
],
we adopted the temperature-changing rate of
±3◦C
/
s
, and activated
the thermal stimuli for 1.5s (on for 1.5s and then switched off), for
a comfortable yet perceivable temperature feedback. While recent
related research [
45
] activated the stimuli for 1 second, it is unclear
if this short duration is appropriate for BVI users. Thus, we extended
the duration of the thermal stimuli in ThermalCane as 1.5 seconds, to
ensure the detection by BVI users and reduce their mental load.
1http://tegway.co/tegway/
To optimize and validate the design of ThermalCane, we conducted
two user studies. Study 1 was conducted to understand the overall
response of the BVI for the thermal haptic feedback, and to obtain the
peltier-module configuration that can be reliably perceived by the BVI
users. We then investigated the feasibility of using ThermalCane in the
outdoor environment by conducting a directional walking experiment
with BVI usersin Study 2.
4 STUDY 1: ON-PALM THERMAL
PERCEPTION OF GRIP HOLDING
Before designing the thermal cues for direction/navigation, there is
a need to understand how BVI users perceive the on-palm multi-spot
thermal feedback. To our best knowledge, there was no existing study
on this issue. To this end, we conducted Study 1, as the support of Ther-
malCane design. The main goal of this study was to understand BVI
users’ thermal perception of on-palm multi-spot thermal feedback
and optimal arrangement of thermal haptic feedback on the palm.
4.1 Participants
Twelve visually impaired participants (11 male and 1 female) aging
from 25 to 35 years old (Mean = 31.5, SD = 4.42) were recruited
through a local non-profit blind association. 6 out of 12 participants
were congenitally blind, and 2 of them suffered form retinal pigmen-
tosa. 10 of them have previous experience with existing vibrotactile
white-cane products for notifying obstacles, and the other two par-
ticipants have experience on vibrotactile feedback of mobile phones.
7 out of 12 the participants were left handed, and they all held the
cane grip in their dominant hands. The average palm temperature was
33.2◦C and the average room temperature was 30.3◦C.
4.2 Apparatus
We used three 3D-printed cane grips for this study. The thermal
system on the cane grip was connected to a MacBook Pro laptop
through a USB cable. Besides controlling the thermal stimuli and
the experiment flow, the laptop also displayed the Processing-based
graphical user interface (GUI), as shown in Fig. 5, for registering the
participants’ responses.

ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
4.3 Study Design
We designed a within-subject study with the configuration (i.e. the
number) of the Peltier modules and the direction of temperature
change as the independent variables. The dependent variables in-
cluded the accuracy, the response time, and the perceived task load of
stimuli detection. In each configuration, each of the Peltier modules
around the grip was activated using hot (
3◦Cs
) and cold (
−3◦Cs
) stim-
uli. The order of the three configurations were counter-balanced using
Latin Square method for each participant, and the positions and the di-
rection (hot/cold) of the stimuli were randomly presented within each
configuration. Each stimulus were repeated twice, which amounted
to (3+4+5) positions
×
2 directions of change
×
2 repetitions = 48
trials for each participant.
4.4 Procedure and Task
The experiment contained one participant and one experimenter, and
was divided into three sessions (one for each configuration). Each
session consisted of one training block and one testing block. After
the initial introduction of the experiment’s logistics and filling the pre-
questionnaire, the participant was asked to hold the ThermalCane grip
on his/her dominant hand in a proper orientation. The thermal stimuli
were then activated in a clock-wise order with the top position as
the start. Each stimulus lasted for 1.5s. Meanwhile, the experimenter
verbally explained the nature of each stimulation to familiarize the
participant with the stimuli. During the explanation, the experimenter
numbered the position of the stimulus following the scheme shown
in the bottom part of Fig. 3. The participant could choose to repeat
the current stimulus for training or move to the next one by verbally
reporting to the experimenter.
After training, the participant started the test block, where the stim-
uli were presented in a randomized order. The selection interface was
displayed after each stimulation.
As it is difficult for the BVI participant to operate the GUI by
him/herself, he/she was instructed to rest the other hand on the space
bar of the laptop. The participant was also instructed to press the space
bar as soon as they felt and confirmed the stimulus. The participant
then spoke out the temperature-changing direction (hot or cold) and
the numbered position of the stimulus, and the experimenter registered
the corresponding stimuli on the selection interface. The timestamp
of the participant pressing the space bar was used to calculate the
reaction time. There was a 15s break between two consecutive stimuli.
Between two configuration sessions, a temperature-resetting period
of 5 minutes were give to the participant. After each the configuration
session, the experimenter read out the NASA-TLX questionnaire
items [
22
] to the participants and recorded the responses in a Google
form. A short semi-structured interview was conducted in the end of
the experiment to collect the participant’s subjective comments on
his/her experience of the ThermalCane grip. The overall experiment
duration per participant was approximately one hour.
4.5 Results
4.5.1 Accuracy. Fig. 6(LEFT) shows the overall accuracy of stimuli
identification in different conditions of module configuration and
temperature. Fig. 6(RIGHT) shows the accuracy of individual stimuli
identification, and Fig. 7 shows the confusion tables in different mod-
ule configurations. We performed a multi-factorial repeated measures
ANOVA on the accuracy of recognizing the on-hand thermal stimuli.
The results revealed the significant effect of the temperature-changing
direction (F(1,11) = 18.54,
p
< 0.005,
η2
= 0.628). Post-hoc Boferroni
p
test showed that the cold stimuli yielded significantly higher accu-
racy than the hot ones (
p
< 0.005). The Peltier-module configuration
also showed a significant effect on the accuracy (F(2,22) = 8.26,
p
< 0.005,
η2
= 0.429). The post-hoc Boferroni test revealed that for
p
the cold stimuli, the configuration with five Peltier modules yielded
significantly more errors than the three-module configuration (
p
<
0.005) and the four-module configuration (
p
< 0.05), while there was
no significant difference between the accuracy of the three-module
configuration and the four-module configuration. For the hot stimuli,
the three-module configuration was significantly more accurate than
the one with five modules (
p
< 0.01), while there was no significant
difference between three-module vs four module and four module vs
five module.
4.5.2 Response Time. We define the response time as the time for
a user to respond once the stimulus starts. Similar to the accuracy of
recognizing the thermal stimuli, the multi-factorial repeated measures
ANOVA revealed the significant effect of the temperature-changing
direction on the participants’ response time to the stimuli (F(1,11) =
20.04,
p
< 0.005,
ηp
2
= 0.646). Post-hoc Boferroni test showed that the
hot stimuli yielded significantly longer response time than the cold
stimuli did (
p
< 0.005). There was also a significant effect from the
type of the thermal-module configuration (F(2,22) = 8.73,
p
< 0.05,
η2
p
= 0.442). For both hot and cold stimuli, the five-module configuration
resulted in significantly longer response time than the one with three
modules (
p
< 0.05) and four modules (
p
< 0.05), while there was no
significant difference between the response time to the three-modules
and the four-modules configurations. Fig. 8(LEFT) illustrates the
comparison of response time across different temperature-change
direction and configurations.
4.5.3 Perceived Task Load. The Friedman tests on the participants’
responses on NASA-TLX questionnaire revealed the significant effect
of the thermal-module configuration on the participants’ ratings on
the mental demand (
χ2
(2) = 18.05, p
<
0.0005), the physical demand
(
χ2
(2) = 10.06, p
<
0.005), the effort (
χ2
(2) = 6.95, p
<
0.05), and
the total score of the NASA-TLX questionnaire (
χ2
(2) = 10.51, p
<
0.005,. Pairwise Wilcoxon Signed Ranks Test showed that there
was no significant difference between the three-modules and the four-
modules configurations in all aspect of the NASA-TLX questionnaire.
On the other hand, the five-modules configuration was rated signif-
icantly higher than the three- and the four-modules configuration in
terms of mental demand (5 vs 3: Z = 2.85, p
<
0.005; 5 vs 4: Z = 2.95,
p
<
0.005), effort (5 vs 3: Z = 2.26, p
<
0.05; 5 vs 4: Z = 2.53, p
<
0.05), and the total score (5 vs 3: Z = 2.75, p
<
0.005; 5 vs 4: Z = 2.36,
p
<
0.05). For the physical demand, the five-modules configuration
received significantly higher ratings than the three-modules one (Z
= 2.67, p
<
0.05). Fig. 8(RIGHT) depicts the results of the subjective
task-load ratings.
4.6 Discussion of Study 1
Generally speaking, the participants performed better (i.e. higher
accuracy and shorter response time) with the cold stimuli than the
hot ones in all the configurations. The confusion matrices in the Fig.

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
Figure 6: (LEFT): Overall accuracy of stimuli identification in different module configuration in Study 1 with BVI participants.
(RIGHT): Accuracy of individual stimuli identification: a) three-module configuration, b) four-module configuration, c) five-module
configuration (red - outter circle: hot, blue - inner circle: cold
Cold Hot
1 2 3 1 2 3
1
100%
4.17%
83.33%
4.17%
2
100%
4.17%
100%
3
95.83%
12.50%
95.83%
Cold Hot
1 2 3 4 1 2 3 4
1
95.83%
8.33%
95.83%
20.83%
4.17%
2
4.17%
83.33%
4.17%
75.00%
12.50%
3
4.17%
95.83%
4.17%
4.17%
75.00%
4.17%
4
4.17%
4.17%
95.83%
4.17%
95.83%
Cold Hot
123451 2 3 4 5
1
91.67%
4.17%
95.83%
4.17%
2
87.50%
4.17%
8.33%
70.83%
16.67%
4.17%
3
4.17%
8.33%
91.67%
16.67%
16.67%
75.00%
20.83%
4
4.17%
70.83%
4.17%
8.33%
66.67%
8.33%
5
4.17%
4.17%
12.50%
91.67%
8.33%
8.33%
91.67%
Three-module Configuration Four-module Configuration
Five-module Configuration
Figure 7: Confusion tables for Study 1 with BVI participants.
Figure 8: (LEFT):Response Time in Study 1 with BVI participants.(RIGHT):NASA-TLX scores for Study 1 with BVI participants.
7 show that the majority of the errors in the cold stimuli happened
one point away from the correct location. For the hot stimuli, the
errors were more widely distributed. This could be because the skin
contains more cold receptors than hot receptors suggesting people are
more sensitive to cold stimuli than hot stimuli [
30
]. Considering the
subjective feedback from the users, all the participants stated that it
was a new experience for them to try the thermal feedback on the cane
grip. They found cold stimuli were more comfortable, and preferred
cold stimuli over the hot ones. This could be due to the hot stimuli
being closer to the pain threshold.One participant said that “Cold
sensation is better. Hot is too difficult to absorb, especially when the
hand sweats.” Four participants reported the hot sensation to be very
“feeble”. Another participants explicitly stated that the two modules
at the bottom (#3 and #4) were difficult to distinguish.
The results of the recognition accuracy decreasing along with the
increment of the module number is aligned with the existing research
showing that the spatial acuity reduces with the reduction on the dis-
tance between two thermal stimuli [
53
]. While the three-modules grip
yielded the best performance, we decided to choose the four-module
configuration over the three-modules one for further study. This was
primarily because the four-modules configuration could offer higher
expressiveness for communication with more Peltier modules. In ad-
dition, there was no sigifnicant difference between the three-modules
and the four-modules configuration, in terms of the accuracy, the
response time, and the rated workload. During the post-experiment
interview, the participants were also asked to rank their preference on
the configurations for guiding navigation. 11 participants preferred
the configuration with four modules. Furthermore, they tended to
assign the cold stimuli, instead of the hot ones, for the directional cues
due to the ease to perceive, and map the directions of forward, turn left,
and turn right, to top, left, and right stimuli respectively. Even though
the U-turn cue is not widely used by the BVI users, all the participants
agreed to the idea of using the bottom cold stimuli for the U-turn.
We further performed a multi-factorial repeated measures ANOVA
on the participants’ data of the four-modules configuration, taking
the temperature-change direction and the positions of the four stimuli
as the independent variables. The results revealed that there was no
significant effect of the stimuli position on the participants’ perfor-
mance (i.e., the accuracy and the response time). On the other hand,
the temperature-change direction placed a significant effect on the

ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
accuracy (F(1,11) = 5.21,
p
< 0.05,
η2
= 0.321) and the response time
p
(F(1,11) = 18.92,
p
< 0.005,
ηp
2
= 0.632). The post-hoc Bonferroni
test showed that the cold stimuli yielded significantly higher accuracy
(
p
< 0.05) and shorter response time (
p
< 0.005) than the hot ones.
These results implied that one may want to consider the cold stimuli
more than the hot ones for designing reliable thermal icons, which is
consistent with the previous studies [45, 63].
5 THERMAL DIRECTIONAL
CUES WITH THERMALCANE
The first study suggested the four-modules configuration yielded
significantly higher recognition accuracy than the five-modules con-
figuration, and it can offer more design options than the three-modules
configuration. Therefore, we chose this configuration for the Ther-
malCane grip, and designed the directional cues accordingly.
The common instructions in most existing navigation devices (e.g.,
GPS) include “Go forward”, “Stop”, “Turn left”, “Turn right”, and
“U-turn” [
49
]. These cardinal directions are also important and widely
used in the mobility and the orientation training for BVI persons [
18
].
Noted that the turn instructions here indicate taking the turn with-
out pausing the pace. Existing research on thermal feedback showed
that users tended to associate the hot sensation with uncomfortable
and dangerous information, and the cold sensation was rated to be
more comfortable and safe [
60
,
63
]. In addition, Study 1 shows no
significant difference between the response time toward the top-hot
(2.63s) and the top-cold stimuli (2.54s). There is no significant dif-
ference between the top-hot and cold stimuli in terms of accuracy
(95.% and 95%) Therefore, we decided to map the hot feedback at
the top module to the instruction of “Stop”. In addition, Zhu et al.’s
research [
63
] on the user-defined application of multi-spot thermal
feedback also showed that users tended to map the four spots of cold
feedback with their respective directions. That is, the cold feedback
on the top is mapped to “Go forward”, the left for “Turn left”, the right
for “Turn right”, and the bottom for “U-turn” or turning backward
180 degree. Additionally, the first study showed that the perception
of the cold stimuli were more significant in comparison to the hot
ones. To ensure the accuracy of perceiving navigation instructions
and reduce the risks in navigation, we adopted the mapping shown
in Fig. 9 for our further study.
6 STUDY 2: WALKING EXPERIMENTS
FOR THERMALCANE EVALUATION
Following the evaluation methods adopted by the other related work
on haptic feedback for accessibility [
26
,
38
], we conducted the second
user study with BVI users in a semi-realistic walking context, to eval-
uate the effectiveness of on-grip thermal directional cues for actual
walking users without any visual feedback. Existing research [
4
] on vi-
brotactile feedback for outdoor navigation showed that users can accu-
rately recognize the single-spot vibration across fingers and palm with
the average accuracy above 95%. Therefore,we were also interested in
the comparison between the thermal and vibrotactile feedback which
has been widely used as a type of haptic notification for BVI users.
Figure 9: Thermal and vibrational directional cues (red: hot,
blue: cold, zig-zag lines: vibration).
Figure 10: System flowchart
6.1 Participants
We first recruited 6 sighted participants (3 female and 3 male, aver-
agely aging 23 years old) for a pilot test, to validate the the feasibility
and the safety of using ThermalCane while walking. All these 6
sighted participants were blindfolded during the pilot test. We didn’t
observe any safety issue during the pilot validation.
Therefore, we carried on the same protocol with 6 BVI participants
(1 female and 5 male, averagely aging 28 years old). Among the BVI
participants, four were congenitally blind, and two has 25% vision.
All of the BVI participants have previous experience with existing
vibrotactile white-cane products for notifying obstacles. All the BVI
and sighted participants had experience on vibrotactile feedback of
mobile phones. All the participants were right-handed, and they all
held the cane grip in their dominant hands.
The average skin temperature of the BVI participants were 32.3
◦
C
(SD = 1.8). All the experiments were completed within 7 days in April.
The outdoor temperature was between 23
◦
C and 32
◦
C (Mean = 27.2,
SD = 2.1).
6.2 Apparatus
For the pilot validation of blindfolded sighted participants, we at-
tached the four-Peltier grip to a wooden stick. The length of the
wooden stick is 128 cm, and the average height of the BS participants
is 166.67cm (SD = 6.23). In addition, the wooden stick weighs 0.258
kg, which is similar to the white canes popular in the market. We
didn’t receive any report of difficulty from the BS participants while

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
Figure 11: (a) Cane grip with vibration modules, (b) User setup in Study 2
using the wooden stick with thermal/vibrotactile feedback in walking.
For the BVI participants, we allowed them to use their own white
canes, and mounted the haptic grips on the handles of the canes.
Fig. 10 shows the flow of the prototype system used in the study.
Besides using the same temperature control mechanism as the one
in Study 1, we developed an Android application to record the par-
ticipant’s real-time GPS data while walking. The application was set
up in an Google Nexus 6P mobile phone which was strapped to the
shoulder pad of the backpack using a velcro band, as shown in Fig.
11b. The backpack contained the micro-controllers, the motor-driver
modules, the battery pack of 9V 2.5A, and a Microsoft Surface lap-
top which ran the Processing-based experiment software for stimuli
control and data recording.
For the vibrotactile counterparts, we developed a cane grip embed-
ded with four vibration actuators (3V70mA and 12000RPM) with
the similar placement as the Peltier modules. As shown in Fig. 11a,
the vibrotactile grip was designed and fabricated in a modular way,
allowing the participant to slide and adjust the position of each actua-
tor to ensure its full contact with the participant’s palm. Furthermore,
the vibration motor was attached to the grip through a 3mm layer of
foam, to avoid the vibration propagation. Vibrotactile stimuli was
controlled by the same system used for the thermal stimuli. Based
on previous work on vibrotactile feedback on the wrist [
45
], we ac-
tivated the vibration at 170Hz for 1.5 seconds. As shown in Fig.
??
,
the vibrotactile direction cues of “Go”, “Stop”, “Turn left”, and “Turn
right” were directly mapped to the respective positions of the actu-
ators. Similar to the previous research on around-waist vibrotactile
directional cues [
17
], we activated all the vibration actuators at the
same time, to indicate “Stop”.
6.3 Study Design
We designed a within-subjects evaluation with the modality and the
type of directional cues as the independent variables. The dependent
variables were the accuracy and the reaction time of the direction
taken with respect to the corresponding stimulus, and the perceived
task load of the overall walking task. The thermal and the vibrotactile
directional cues were presented to the participant in the Latin-Square
counter-balanced order. All the directional cues within the same
modality were repeated twice and presented in a randomized order.
There was a 15-second gap between two consecutive cues. While
perceiving the turning cues during walking, the participant was in-
structed to perform the turn without pausing his/her movement. While
perceiving the turning cues in a stand-still status, the participant was
asked to perform the turn and start walking at the same time. In ad-
dition, we avoided the situation of two consecutive “Stop” cues in the
order randomization, to ensure the fluent walking process.
6.4 Procedure
We conducted this study in an open field with the size of around two
football fields. The experiment contained one participant and one ex-
perimenter, and was divided into two sessions (one for each modality).
Each session consisted of one training block and one testing block.
Upon arriving at the field, the participant filled the pre-questionnaire
to collect their personal information, and the experimenter introduced
the procedure of the experiment. For the vibrotactile cues, the ex-
perimenter also helped the participant to adjust the placement of the
vibrotactile actuators to achieve the full contact between the actuators
and the participant’s palm while grasping the grip. For the sighted
participants, the experimenter then blindfolded them with a normal
sleep eye mask, and conducted a 10-minute training on how to use the
white cane for walking. Then they started the training block. During
the training block, the participant received the demonstration of the
directional cues with the corresponding modality in the clock-wise
order, with “Go” as the start and “Stop” as the end. Meanwhile, the ex-
perimenter explained the navigation instructions mapped to the cues.
Once the participant reported that he/she was familiar with the cues,
the experimenter helped the participant wear the backpack, attached
the phone to the shoulder strap, and started the GPS recording mobile
app. While receiving the “Go” instruction on the palm, the participant
started walking, and was asked to perform the navigation according
to his/her perception of the directional cues on the palm. After com-
pleting each modality session, the participant was asked to complete
the NASA-TLX questionnaire. In the end, we interviewed the partic-
ipants for his/her subjective comments. The overall experiment lasted
about 45 minutes per participant.
6.5 Analysis & Results
In this session, we mainly focus on the analysis of the data of the BVI
participants. Averagely, the participants finished the walking session
within 5 minutes (Thermal: Mean = 289.67s, SD = 16.75; Vibrotactile:

ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
Thermal Cues Vibrotactile Cues
Go Stop
Turn Left
Turn Right
UTurn Go Stop
Turn Left
Turn Right
UTurn
Go 100% 55% 30% 5% 5% 5%
Stop 100% 10% 85% 2.78% 5.56% 5.56%
Turn Left
80.0% 20.0% 6.67% 66.67% 6.67% 20%
Turn Right
93.33% 6.67% 6.67% 13.33% 66.67% 13.33%
UTurn 100% 6.67% 13.33% 6.67% 73.33%
Figure 12: Confusion table of navigation instructions in Study 2 for BVI participants: row is the stimuli and the column is the
participants’ responses.
Mean = 291.0s, SD = 18.33). The overall average walking distance
was 140.87 meters (Thermal: Mean = 141.49 metres, SD = 36.33;
Vibrotactile: Mean = 140.26 metres, SD = 36.27). We further statis-
tically analyzed the accuracy, the response time, and the participants’
subjective task-load rating. Contrary to Study 1 where we compared
accuracy using ANOVA, we will rely on non-parametric statistical
tests due to the different homogeneity of variances between levels of
factors in this experiment.
6.5.1 Accuracy & Response Time. The Wilcoxon Signed Ranks Test
showed that there was a significant effect of the feedback modality (Z
= 2.21,
p
< 0.05). There was no significant difference on the accuracy
cross different navigation instructions. In terms of the response time,
the Wilcoxon Signed Ranks Test showed no significant effect of the
feedback modality or the type of the navigation instruction, though the
thermal cues yielded a slightly but not significantly longer response
time (2.96s) than the vibrotactile cues(2.87s). Fig. 12 shows the con-
fusion table of the navigation instructions for the BVI participants.
Fig. 13 shows the response time of the BVI participants.
6.5.2 Subjective Task-load Rating and Comments. Friedman tests
on the participants’ responses to the NASA-TLX questionnaire ( Fig.
14) showed no significant difference between the thermal and the vi-
brational cues, though the thermal cues was rated with slightly lower
task load than the vibration cues.
All the BVI participants expressed their preference on the thermal
cues over the vibrotactile ones. BVI User#1 said “I need to really
focus my attention on the the grip to constantly check for the vibration
and as a result I am losing the focus on my surroundings and surface
that I am walking on.” BVI User#2 commented, “I don’t have to
put much of effort in understanding the (thermal) sensation (cues)
as it is very intuitive. I can focus on my surrounding noises and still
get to know when to stop with the hot sensation (cue)." About the
vibrotactile cues, he commented, “Tapping on the ground creates a
sort of vibration, and sometimes we drag the stick on the ground to
understand the surface, so I feel that I can easily miss a vibration
during that process.” BVI User#3 praised the design of hot feedback
for stop/emergency. On the other hand, she mentioned that she felt
the whole grip vibrating when one single vibration motor was on.
BVI User#5 mentioned that he usually loosen the grip while tapping
the cane on the ground, but loosening affected his perception of vi-
bration. On the feeling of skin contact, BVI User#5 mentioned that
the texture of the peliter modules on the four sides feel nice with a
grippy feeling. Both BVI User#4 and BVI User#5 stated that they
have been using vibration of indicating obstacles for long, so they
prefer keeping this feature for vibrotactile cues, and using the thermal
Figure 13: Response time in Study 2 for BVI users.
Figure 14: NASA-TLX scores for Study 2.
cues for navigation. All the participants suggested the combination of
thermal and vibrotactile cues in the cane, “A combination of thermal
and vibration would be perfect that it can solve both navigational
and obstacle issues with a single device.” “Make a combination of
thermal and vibration and that would be the best. The cues/feedback
of the thermal grip is easy to understand and well thought of. It can
pretty much address necessary navigation commands.”
One of the participants also reported about the sweating of the
palm area due to the prolonged grip holding, and this might affect his
performance. The other participant commented, “The temperature
stimulation is very soft for me, so it is difficult to detect them”. This
implies that he may have a different thermal threshold compared to the
rest of the participants, further suggesting the support of temperature
customization in the future design.
7 GENERAL DISCUSSION OF STUDY 1 & 2
Our studies with BVI participants indicated that the thermal feedback
in ThermalCane could offer more reliable directional notice than the
vibrotactile feedback while a user is walking. Although the global

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
health problem places difficulties in participant recruitment for our
user studies, the current results with six BVI participants show the
effectiveness of ThermalCane and encourage more investigations
with a larger number of subjects. In this section, we discuss more
general observation and possible future work in our studies.
7.1 Response Time
The overall average response time of the BVI participants in Study
2 was 2.91s (SD = 0.34). With the 1.5s stimulus in our studies, the
overall average response time in Study 2 means the BVI participants
responded to a stimulus 1.4s after it stopped. Considering the general
walking speed for BVI persons ( 0.41m/s without any assistance)
[
10
,
19
], the system with the feature of GPS positioning could activate
the thermal directional cue around 0.5 meter ahead.
7.2 Gender and Handedness
Previous research show that the thermal perception and threshold
could be affected by body parts or handedness [
7
], genders [
31
], and
ages [
23
]. In Study 1, the left-handed users achieved 87.2% accu-
racy and 2.24s response time, and the right handed with 91.66% and
3.38s respectively. In Study 2, the females achieved 90.48% accuracy
with a response time of 2.32s, and the males with 92.78% and 2.21s
respectively. Also, the existing works show that females may have
higher thermal sensitivity than males [
31
], and there is no significant
difference in terms of thermal sensitivity for left-handed and right-
handed persons [
7
]. Our studies show slightly different results: Study
1 shows the right-handed persons can perceive the thermal feedback
more accurately but more slowly than the left-handed; Study 2 shows
the males performed slightly better than the females. While we didn’t
focus on the gender/handedness difference in the current stage, the
presented studies may have a gender/handedness bias. There is a need
for in-depth experiments on the perceptive difference of ThermalCane
among different genders, handedness, and ages.
7.3 Potential
Applications Besides Walking Navigation
While navigation was the most popular application proposed by the
BVI participants during the Study 1, they also suggested potential
add-on features and other possible applications of ThermalCane. For
instance, one male participants with retinal pigmentosa stated, “It
would be better if there could be a replay feature with double-tapping
a button.” One participant who lost 90% of his sight mentioned that it
is better to put the thermotactile directional cues on the cane than on
the wrist since the wrist could be in various orientation depending on
the body posture. One participant who is currently using a vibrotactile
smart-cane product commented, “It could be really helpful for the
navigation using SmartCane [with thermal feedback] if I can make my
custom directional cues.” Two participants mentioned the similar ther-
mal stimuli could also integrated in the bicycle handles or the steering
wheel, to benefit the sighted people, and this comment echoed with Di
Campli San Vito et al.’s research on haptic steering wheel for driving
[
13
]. It also points to a potential future direction on supporting BVI
users’ driving with multimodal haptic feedback. Another two partic-
ipants suggested that this type of around-grip thermal feedback could
also be used for phone-call notification. One future direction could be
Figure 15: Response time in Study 2 for blindfolded participants.
investigating BVI users’ perception on the animated spatial thermal
patterns around the grip and exploring their potential applications.
7.4 User
Performance with ThermalCane in Walking
Though we didn’t focus on comparing the performance of the sighted
and the BVI participants, we observed a difference between the per-
formance of the blindfolded sighted (BS) participants in the pilot
validation and the BVI participants in the actual study. The recorded
data showed that there was nosignificant statisticaldifference between
the BVI participants’ and the blindfolded-sighted users’ perception
accuracy towards the thermal directional cues, though the BVI par-
ticipants performed more accurately than the BS users (94.77% vs
86.67%). We also found that the BS users perceived the vibrotactile
cues slightly more accurately than the BVIs did (71.11% vs 65.33%).
This could be explained by our observation during the pilot test that
although being instructed and trained to use the cane as the BVIs do,
some sighted participants could not follow the way how BVIs use the
cane in the actual testing process. For instance, we observed three BS
participants who didn’t tap the cane on the ground while they were
walking. This reduced the in-cane turbulence which may weaken the
vibrotactile cues as reported by the BVI participants.
In terms of response time, we found that the BS participants re-
acted faster than the BVI participants towards the thermal cues (2.21s
vs 2.96s). Furthermore, we observed a larger within-group variation
in terms of the response time for the BS participants than the BVI
participants. Fig. 15 depicts the response time of the BS participants
in the pilot test.
These observation echoed with existing comparison research where
the BVI orients/navigates more accurately (but slowly) than the BS
[
37
], and shows more consistency in the haptic rotation task[
25
]. In
addition, the difference of response time between the two users group
in our study could be due to the different environmental temperature
during the pilot test and the actual study. As the environmental tem-
perature may affect one’s thermal perception [
20
], we found that the
average outdoor temperature for the BVI participants was 25.5
◦C
,
and 29.7
◦C
for the sighted participants. Therefore, this may be a
confounding factor that caused the different response time between
these two groups. This deduction could be partially backed up with
the results in Study 1 where the BVI participants could react to the
cold stimuli in the 4-module configuration within 2s averagely, which
is close to the sighted participants’ response time in Study 2, in an av-
erage room temperature of 30.3
◦C
. The different performance across

ThermalCane: Exploring Thermotactile Directional Cues on Cane-Grip ASSETS ’20, October 26–28, 2020, Virtual Event, Greece
different users further places the needs for the feature allowing users
customizing the thermal level by themselves.
7.5 Limitations and Future Work
We identified a few limitations in the current prototype of Thermal-
Cane. Firstly, two BVI participants in Study 1 commented that the
use of ThermalCane may be affected in the situation of the user wear-
ing gloves. While it could be true that it is difficult to perceive the
around-grip thermal feedback with gloves, it suggests a potential need
of thermotactile smart gloves with flexible Peltier modules which
requires further research in the future.
Secondly, one participant commented that users may perceive the
thermotactile cues differently while walking in different weather. For
example, the hot stimuli may be perceived more clearly in a cold
day. In addition, previous research showed that thermal perception
could vary among different ethnic groups [
54
]. In the future, we plan
to conduct more outdoor experiments of ThermalCane in different
ambient conditions.
Thirdly, in the current stage we mainly focus on the hand-shake
gripping posture, due to its popularity. However, it is possible that
BVI users adopt other postures of gripping their canes, and differ-
ent gripping postures will lead to different hand-contact areas which
could affect users’ perception on haptic feedback. Therefore, there is a
need on investigating the optimal haptic-module layouts for different
hand-gripping postures in the future work.
Last but not the least, our walking experiments compared the user
performance with thermotactile and vibrotactile directional cues. We
chose the vibrotactile feedback due to its wide adoption in the assistive
devices for BVI users, such as notifying the obstacle through the in-
cane vibration. On the other hand, there are other types of force-based
haptic feedback, such as poking, pushing, and stretching. However,
implementing these force-based feedback may involve complicated
and bulky electronic and mechanical structures which may be difficult
to be integrated into the form factor of a thin grip. Therefore, we will
extend our research to further explore the optimal design for different
types of force-based directional cues in the white cane, and evaluate
their effectiveness.
8 CONCLUSION
In this paper, we present ThermalCane, a white-cane grip instru-
mented with multiple flexible thermal modules, to offer thermotactile
directional cues for BVI users’ spatial navigation. Our first user study,
involving 12 BVI participants, suggested that the participant can re-
liably identify the spatial thermal stimuli with four flexible Peltier
modules around the grip. We further designed a set of directional cues
with the configuration of four Peltier modules in ThermalCane. Our
second user study with 6 BVI users showed that the thermotactile
directional cues offered by ThermalCane significantly outperformed
the vibrotactile cues in the walking situation. Taking ThermalCane as
an important step, we aim to investigate future design of multimodal
assistive devices for the BVIs.
ACKNOWLEDGMENTS
This research was supported by the Young Scientists Scheme of the
National Natural Science Foundationof China(Project No. 61907037),
the Strategic Research Grants (Project No. 7005172 & 7005361), the
Applied Research Grant (Project No. 9667189), and the Centre for
Applied Computing and Interactive Media (ACIM) of School of Cre-
ative Media, City University of Hong Kong. We thank the participants
who took part in the user evaluation.
REFERENCES
[1]
M Balakrishnan, Kolin Paul, Ankush Garg, Rohan Paul, Dheeraj Mehra, Vaibhav
Singh, PVM Rao, Vishwas Goel, Debraj Chatterjee, and Dipendra Manocha. 2009.
Cane mounted knee-above obstacle detection and warning system for the visually
impaired. In ASME 2007 International Design Engineering Technical Conferences
and Computers and Information in Engineering Conference. American Society
of Mechanical Engineers Digital Collection, 143–151.
[2]
Jan Balata, Katerina Prazakova, Anna Kutikova, Miroslav Macik, and Zdenek
Mikovec. 2013. Quido: arcade game with thermo-haptic feedback.". In IEEE 4rd
international conference on cognitive infocommunications (CogInfoCom 2013).
[3]
Balata, Jan and Macik, Miroslav and Mikovec, Zdenek. 2014. Heat Map
Thermal Display for Visually Impaired. https://www.researchgate.net/publication/
264509264_Heat_Map_Thermal_Display_for_Visually_Impaired. CHI 2014 -
Assistive Augmentation Workshop. Online; accessed 17-09-2019.
[4]
Dominik Bial, Dagmar Kern, Florian Alt, and Albrecht Schmidt. 2011. Enhancing
Outdoor Navigation Systems through Vibrotactile Feedback. In CHI’11 Extended
Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI
EA’11). Association for Computing Machinery, New York, NY, USA, 1273–1278.
https://doi.org/10.1145/1979742.1979760
[5]
Rupert RA Bourne, Seth R Flaxman, Tasanee Braithwaite, Maria V Cicinelli, Aditi
Das, Jost B Jonas, Jill Keeffe, John H Kempen, Janet Leasher, Hans Limburg,
et al
.
2017. Magnitude, temporal trends, and projections of the global prevalence
of blindness and distance and near vision impairment: a systematic review and
meta-analysis. The Lancet Global Health 5, 9 (2017), e888–e897.
[6]
M Bouzit, A Chaibi, KJ De Laurentis, and C Mavroidis. 2004. Tactile feedback
navigation handle for the visually impaired. In ASME 2004 International Me-
chanical Engineering Congress and Exposition. American Society of Mechanical
Engineers Digital Collection, 1171–1177.
[7]
JOSEPH C. STEVENS KENNETH K. CHOO. 1998. Temperature sensitivity of
the body surface over the life span. Somatosensory & motor research 15, 1 (1998),
13–28.
[8]
Alvaro Cassinelli, Carson Reynolds, and Masatoshi Ishikawa. 2006. Augmenting
spatial awareness with haptic radar. In 2006 10th IEEE International Symposium
on Wearable Computers. IEEE, 61–64.
[9]
Taizhou Chen, Yi-Shiun Wu, and Kening Zhu. 2018. Investigating different
modalities of directional cues for multi-task visual-searching scenario in virtual
reality. In Proceedings of the 24th ACM Symposium on Virtual Reality Software
and Technology. ACM, 41.
[10]
DD Clark-Carter, AD Heyes, and CI Howarth. 1986. The efficiency and walking
speed of visually impaired people. Ergonomics 29, 6 (1986), 779–789.
[11]
Akansel Cosgun, E Akin Sisbot, and Henrik I Christensen. 2014. Guidance
for human navigation using a vibro-tactile belt interface and robot-like motion
planning. In 2014 IEEE International Conference on Robotics and Automation
(ICRA). IEEE, 6350–6355.
[12]
Patrizia Di Campli San Vito, Stephen Brewster, Frank Pollick, Stuart White,
Lee Skrypchuk, and Alexander Mouzakitis. 2018. Investigation of Thermal
Stimuli for Lane Changes. In Proceedings of the 10th International Confer-
ence on Automotive User Interfaces and Interactive Vehicular Applications
(Toronto, ON, Canada) (AutomotiveUI ’18). ACM, New York, NY, USA, 43–52.
https://doi.org/10.1145/3239060.3239062
[13]
Patrizia Di Campli San Vito, Gözel Shakeri, Stephen Brewster, Frank Pollick,
Edward Brown, Lee Skrypchuk, and Alexandros Mouzakitis. 2019. Haptic
Navigation Cues on the Steering Wheel. In Proceedings of the 2019 CHI
Conference on Human Factors in Computing Systems. ACM, 210.
[14]
Juan V Durá-Gil, Bruno Bazuelo-Ruiz, David Moro-Pérez, and Fernando
Mollà-Domenech. 2017. Analysis of different vibration patterns to guide blind
people. PeerJ 5 (2017), e3082.
[15]
Enes Selman Ege, Furkan Cetin, and Cagatay Basdogan. 2011. Vibrotactile
feedback in steering wheel reduces navigation errors during GPS-guided car
driving. In 2011 IEEE World Haptics Conference. IEEE, 345–348.
[16]
S. Ertan, C. Lee, A. Willets, H. Tan, and A. Pentland. 1998. A Wearable Haptic
Navigation Guidance System. In Proceedings of the 2Nd IEEE International
Symposium on Wearable Computers (ISWC ’98). IEEE Computer Society,
Washington, DC, USA, 164–. http://dl.acm.org/citation.cfm?id=857199.858004
[17]
German Flores, Sri Kurniawan, Roberto Manduchi, Eric Martinson, Lourdes M
Morales, and Emrah Akin Sisbot. 2015. Vibrotactile guidance for wayfinding of
blind walkers. IEEE transactions on haptics 8, 3 (2015), 306–317.
[18]
Mira Goldschmidt. 2018. Orientation and Mobility Training to People with Visual
Impairments. In Mobility of Visually Impaired People. Springer, 237–261.

ASSETS ’20, October 26–28, 2020, Virtual Event, Greece Nasser et al.
[19]
Luis A Guerrero, Francisco Vasquez, and Sergio F Ochoa. 2012. An indoor
navigation system for the visually impaired. Sensors 12, 6 (2012), 8236–8258.
[20]
Martin Halvey, Graham Wilson, Stephen Brewster, and Stephen Hughes. 2012.
Baby it’s cold outside: the influence of ambient temperature and humidity on
thermal feedback. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems. ACM, 715–724.
[21]
Martin Halvey, Graham Wilson, Yolanda Vazquez-Alvarez, Stephen A Brewster,
and Stephen A Hughes. 2011. The effect of clothing on thermal feedback
perception. In Proceedings of the 13th international conference on multimodal
interfaces. ACM, 217–220.
[22]
Sandra G Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In
Proceedings of the human factors and ergonomics society annual meeting, Vol. 50.
Sage publications Sage CA: Los Angeles, CA, 904–908.
[23]
MJ Hilz, B Stemper, FB Axelrod, EH Kolodny, and B Neundörfer. 1999. Quan-
titative thermal perception testing in adults. Journal of clinical neurophysiology
16, 5 (1999), 462.
[24]
Eve Hoggan, Andrew Crossan, Stephen A Brewster, and Topi Kaaresoja. 2009.
Audio or tactile feedback: which modality when?. In Proceedings of the SIGCHI
conference on human factors in computing systems. 2253–2256.
[25]
Mark Hollins. 1986. Haptic mental rotation: More consistent in blind subjects?
Journal of Visual Impairment & Blindness (1986).
[26]
Jonggi Hong, Alisha Pradhan, Jon E Froehlich, and Leah Findlater. 2017.
Evaluating Wrist-Based Haptic Feedback for Non-Visual Target Finding and
Path Tracing on a 2D Surface. In Proceedings of the 19th International ACM
SIGACCESS Conference on Computers and Accessibility. ACM, 210–219.
[27]
William Henry Jacobson. 1993. The art and science of teaching orientation and
mobility to persons with visual impairments. American Foundation for the Blind.
[28]
Lise A Johnson and Charles M Higgins. 2006. A navigation aid for the blind using
tactile-visual sensory substitution. In 2006 International Conference of the IEEE
Engineering in Medicine and Biology Society. IEEE, 6289–6292.
[29]
Lynette A Jones and Michal Berris. 2002. The psychophysics of temperature
perception and thermal-interface design. In Proceedings 10th symposium on haptic
interfaces for virtual environment and teleoperator systems. HAPTICS 2002. IEEE,
137–142.
[30]
Lynette A Jones and Hsin-Ni Ho. 2008. Warm or cool, large or small? The
challenge of thermal displays. IEEE Transactions on Haptics 1, 1 (2008), 53–70.
[31]
Sami Karjalainen. 2012. Thermal comfort and gender: a literature review. Indoor
air 22, 2 (2012), 96–109.
[32]
Dagmar Kern, Paul Marshall, Eva Hornecker, Yvonne Rogers, and Albrecht
Schmidt. 2009. Enhancing navigation information with tactile output embedded
into the steering wheel. In International Conference on Pervasive Computing.
Springer, 42–58.
[33]
Seongho Kim, Taeyeon Kim, Choong Sun Kim, Hyeongdo Choi, Yong Jun Kim,
Gyu Soup Lee, Ockkyun Oh, and Byung Jin Cho. 2020. Two-Dimensional Thermal
Haptic Module Based on a Flexible Thermoelectric Device. Soft Robotics (2020).
[34]
Yong-Ku Kong and Brian D Lowe. 2005. Optimal cylindrical handle diameter
for grip force tasks. International Journal of Industrial Ergonomics 35, 6 (2005),
495–507.
[35]
Steven J LaGrow and Marvin J Weessies. 1994. Orientation and mobility:
Techniques for independence. Dunmore Press Palmerston North, New Zealand.
[36]
Anatole Lécuyer, Pascal Mobuchon, Christine Mégard, Jérôme Perret, Claude
Andriot, and J-P Colinot. 2003. HOMERE: a multimodal system for visually
impaired people to explore virtual environments. In IEEE Virtual Reality, 2003.
Proceedings. IEEE, 251–258.
[37]
Jack M Loomis, Roberta L Klatzky, Reginald G Golledge, Joseph G Cicinelli,
James W Pellegrino, and Phyllis A Fry. 1993. Nonvisual navigation by blind
and sighted: assessment of path integration ability. Journal of Experimental
Psychology: General 122, 1 (1993), 73.
[38]
Shachar Maidenbaum, Shlomi Hanassy, Sami Abboud, Galit Buchs, Daniel-Robert
Chebat, Shelly Levy-Tzedek, and Amir Amedi. 2014. The “EyeCane”, a
new electronic travel aid for the blind: Technology, behavior & swift learning.
Restorative neurology and neuroscience 32, 6 (2014), 813–824.
[39]
Steve Mann, Ryan E. Janzen, and Tom Hobson. 2011. Multisensor Broadband High
Dynamic Range Sensing: For a Highly Expressive Step-based Musical Instrument.
In Proceedings of the Fifth International Conference on Tangible, Embedded, and
Embodied Interaction (Funchal, Portugal) (TEI ’11). ACM, New York, NY, USA,
21–24. https://doi.org/10.1145/1935701.1935706
[40]
Anita Meier, Denys JC Matthies, Bodo Urban, and Reto Wettach. 2015. Exploring
vibrotactile feedback on the body and foot for the purpose of pedestrian navigation.
In Proceedings of the 2nd international Workshop on Sensor-based Activity
Recognition and Interaction. 1–11.
[41]
Vidula V Meshram, Kailas Patil, Vishal A Meshram, and Felix Che Shu. 2019. An
Astute Assistive Device for Mobility and Object Recognition for Visually Impaired
People. IEEE Transactions on Human-Machine Systems (2019).
[42]
Valerie S Morash, Allison E Connell Pensky, Steven TW Tseng, and Joshua A
Miele. 2014. Effects of using multiple hands and fingers on haptic performance
in individuals who are blind. Perception 43, 6 (2014), 569–588.
[43]
Mutsuhiro Nakashige, Minoru Kobayashi, Yuriko Suzuki, Hidekazu Tamaki,
and Suguru Higashino. 2009. Hiya-Atsu media: augmenting digital media with
temperature. In CHI’09 Extended Abstracts on Human Factors in Computing
Systems. ACM, 3181–3186.
[44]
Arshad Nasser, Kening Zhu, and Sarah Wiseman. 2019. Thermo-haptic Earable
Display for the People with Hearing and Visual Impairment. In Proceedings of
ACM ASSETS’19. ACM.
[45]
Roshan Lalitha Peiris, Yuan-Ling Feng, Liwei Chan, and Kouta Minamizawa.
2019. ThermalBracelet: Exploring Thermal Haptic Feedback Around the Wrist. In
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.
ACM, 170.
[46]
Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Mi-
namizawa. 2017. Thermovr: Exploring integrated thermal haptic feedback with
head mounted displays. In Proceedings of the 2017 CHI Conference on Human
Factors in Computing Systems. ACM, 5452–5456.
[47]
Ioannis Politis, Stephen Brewster, and Frank Pollick. 2013. Evaluating multimodal
driver displays of varying urgency. In Proceedings of the 5th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications.
ACM, 92–99.
[48]
Ioannis Politis, Stephen A Brewster, and Frank Pollick. 2014. Evaluating
multimodal driver displays under varying situational urgency. In Proceedings of the
SIGCHI conference on Human Factors in Computing Systems. ACM, 4067–4076.
[49]
Karl Rehrl, Elisabeth Häusler, and Sven Leitinger. 2010. Comparing the
effectiveness of GPS-enhanced voice guidance for pedestrians with metric-and
landmark-based instruction sets. In International Conference on Geographic
Information Science. Springer, 189–203.
[50]
Yael Salzer, Tal Oron-Gilad, and Adi Ronen. 2010. Vibrotactor-belt on the
thigh–directions in the vertical plane. In International Conference on Human
Haptic Sensing and Touch Enabled Computer Applications. Springer, 359–364.
[51]
Stefano Scheggi, Francesco Chinello, and Domenico Prattichizzo. 2012. Vi-
brotactile haptic feedback for human-robot interaction in leader-follower tasks.
In Proceedings of the 5th International Conference on PErvasive Technologies
Related to Assistive Environments. ACM, 51.
[52]
Anshul Singhal and Lynette A Jones. 2018. Creating Thermal Icons: A Model-
Based Approach. ACM Transactions on Applied Perception (TAP) 15, 2 (2018), 14.
[53]
Joseph C Stevens. 1989. Temperature and the two-point threshold. Somatosensory
& motor research 6, 3 (1989), 275–284.
[54]
Nigel AS Taylor. 2011. Human heat adaptation. Comprehensive Physiology 4,
1 (2011), 325–365.
[55]
Jordan Tewell, Jon Bird, and George R Buchanan. 2017. The heat is on: a
temperature display for conveying affective feedback. In Proceedings of the 2017
CHI Conference on Human Factors in Computing Systems. ACM, 1756–1767.
[56]
Jordan Tewell, Jon Bird, and George R Buchanan. 2017. Heat-nav: Using
temperature changes as navigation cues. In Proceedings of the 2017 CHI
Conference on Human Factors in Computing Systems. ACM, 1131–1135.
[57]
Jan BF Van Erp and Hendrik AHC Van Veen. 2004. Vibrotactile in-vehicle
navigation system. Transportation Research Part F: Traffic Psychology and
Behaviour 7, 4-5 (2004), 247–256.
[58]
Reto Wettach, Christian Behrens, Adam Danielsson, and Thomas Ness. 2007. A
thermal information display for mobile applications. In Proceedings of the 9th
international conference on Human computer interaction with mobile devices and
services. ACM, 182–185.
[59]
Graham Wilson, Stephen Brewster, Martin Halvey, and Stephen Hughes. 2012.
Thermal icons: evaluating structured thermal feedback for mobile interaction. In
Proceedings of the 14th international conference on Human-computer interaction
with mobile devices and services. ACM, 309–312.
[60]
Graham Wilson, Gavin Davidson, and Stephen A Brewster. 2015. In the heat of
the moment: subjective interpretations of thermal feedback during interaction. In
Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing
Systems. ACM, 2063–2072.
[61]
Graham Wilson, Martin Halvey, Stephen A Brewster, and Stephen A Hughes.
2011. Some like it hot: thermal feedback for mobile devices. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems. ACM, 2555–2564.
[62]
John Zelek, Richard Audette, Jocelyn Balthazaar, and Craig Dunk. 1999. A
stereo-vision system for the visually impaired. University of Guelph 1999 (1999).
[63]
Kening Zhu, Simon Perrault, Taizhou Chen, Shaoyu Cai, and Roshan Lalintha
Peiris. 2019. A sense of ice and fire: Exploring thermal feedback with multiple
thermoelectric-cooling elements on a smart ring. International Journal of
Human-Computer Studies 130 (2019), 234–247.