Art Touch with CREATE haptic interface

Abstract

A novel approach to the synthesis and the interac- tion of (with) virtual Environment is presented. In the following paper, the description of a novel multi-point VE-haptic system is given. The system has been devel- oped with the intent of interacting with an interactive VE, based on the constructivist approach. In what fol- low the main feature of the VE and of the interface are given. A special focus has been given to the design phi- losophy:several performances optimizations in terms of user interaction and object manipulation, devices joined workspace, isotropy, arms interferences, con- tinuous force, VE and inter-arm positional coherence have been performed during the design.

Full text

Art Touch with CREATE haptic interface
A. Dettori, C.A. Avizzano C. Loscos
S. Marcheschi, M. Angerilli A. Guerraz
M. Bergamasco
PERCRO Lab. Departement of Computer Science
Scuola Superiore S.Anna University College of London
Viale Rinaldo Piaggio, 34 Gower Street, WC IE 6BT
56025 Pontedera (PI), ITALY LONDON, UK
dettori@sssup.it C.loscos@cs.ucl.ac.uk
Abstract
A novel approach to the synthesis and the interac-
tion of (with) virtual Environment is presented.
In the following paper, the description of a novel
multi-point VE-haptic system is given.
The system has been developed with the intent of in-
teracting with an interactive VE, based on the con-
structivist approach.
In what follow the main feature of the VE and of the
interface are given. A special focus has been given
to the design philosophy: several performances opti-
mizations in terms of user interaction and object ma-
nipulation, devices joined workspace, isotropy, arms
interferences, continuous force, VE and inter-arm po-
sitional coherence have been performed during the de-
sign.
1 Introduction
Virtual reality (VR) and mixed reality (MR) tech-
nologies are now becoming sufficiently developed so
that simulations of cultural or architectural sites, or
virtual environments (VEs) for design are successful in
making users feel truly immersed in the environment,
especially when using head-mounted, workbench, cu-
bic or curved screen type displays [1]. Coupled with
interactive technologies that allow visitors of the vir-
tual sites to make their own choices or perform ac-
tions that trigger responses from the VE, these vir-
tual experiences become significantly richer and more
interesting. Up to now, several successful examples
of virtual worlds have been developed worldwide for
research and design, training, manufacturing, and en-
tertainment. Nonetheless, such environments suffer
either from a lack of realism or a low degree of in-
teractivity, due to technological and methodological
constraints.
Haptic interfaces have became the primary means
of interaction within the virtual environment. Several
Figure 1: Sketched concept.
experiments during the past 15 years have shown how
these interfaces can strongly improve the user per-
formances during the interaction with virtual objects
[11][14][17].
The capability of reflecting digitally programmed
forces, representing the contact interaction with the
object, physical properties, and/or several digital
force effect, highly increments the user performances
and his capabilities to understand what is happening
within the virtual environment [5][6].
The main limitation to the VE interaction based on
haptic interfaces is mainly related to the mechanical
limits of the interface in itself [16]. While the graphi-
cal feedback is now able to represent high quality and
large-scale immersive environments, the haptic inter-
faces (wearable or with grounded base) still behave
some limitation in the force/position workspace [18].
Therefore in order to provide a good quality to the
force feedback, for these immersive environments, the
design of novel interfaces is required.

2 The CREATE project
The global scope of CREATE1(Constructivist
Mixed Reality for Design, Education, and Cultural
Heritage) is to develop a mixed reality framework that
will enable highly interactive real-time construction
and manipulation of realistic, virtual worlds based on
real data sources. This framework will be tested and
applied to cultural heritage content in an educational
context, as well as to the design and review of archi-
tectural/urban planning settings.
The CREATE project:
•Develops design methodologies to determine
user requirements, based on a human-centred,
“constructivist” approach to working and
learning, with special attention paid to evalu-
ation of the resulting mixed reality experience.
•Adapts, develops, and combines novel content
creation, display and audio technologies based
on the requirements thus defined, to enable real-
ism with interactivity, specifically for immer-
sive VR platforms (single/multiple-screen stereo-
scopic displays).
•Constructs prototypes for two specific ap-
plications, cultural heritage and architec-
ture/urban planning, that incorporate more
natural and usable interface approaches and per-
mit assessment of both the methodology and the
technology employed.
This project is user-centered and therefore the
haptic-interface development should answer to the
user requirements as defined to correspond to the
working and learning constructivist approach. The
interface needs to be intuitive and to correspond to
natural interaction. The users of the interface will
range from experts in the domain being learnt (not in
virtual reality nor haptic) to visitors of the museum2
including children. It is important therefore that the
support structure is scalable and the complete sys-
tem safe and easy to use. We believe that to provide
the user with more natural means to interact with the
environment to perform actions such as grabbing liter-
ally virtual objects will allow them to get immediately
into the process of learning without having to learn
how to use the system and the interaction modules.
Moreover the use of this haptic interface within im-
mersive virtual environments, combining 3D-display
1CREATE is a 3-year RTD project funded by Information
Society Technologies (IST) Programme of the European Union
(EU). The official contract with the EU was signed under the
contract number IST-2001-34231. The project started in March
2002.
2In the case of the CREATE project, the museum is in the
Foundation of Hellenic World, partner of the European project.
and 3D-sound simulation should allow the user to fo-
cus immediately on the tasks assigned.
At the current state of the project, the haptic inter-
face has been fully designed inspired by a previous
haptic interface developed for the GRAB3project. In
the following section will be presented the descrip-
tions of the mechanical subsystem and of the control
subsystem that compose the HI. In the next steps of
the CREATE EU project, the system will be built,
assembled and then evaluated.
3 Haptic rendering
The discovery or exploration of an artwork involves
multiple sensorial channels, such as the visual, audi-
tory, haptic, smell, and taste ones. The majority of to-
day’s virtual reality simulations use the visual modali-
ties as 3-D stereo displays, and auditory modalities as
interactive or 3-D sound. Virtual reality technologies
are now becoming sufficiently developed so that sim-
ulations of cultural or architectural sites, or virtual
environments for design, especially using immersive
displays, are successful in making users feel immersed
in the environment. Haptic technology also allows for
more direct manipulation, which could specifically be
relevant to 3D shape conceptualization [7][8].
Haptic feedback groups the modalities of force feed-
back, tactile feedback, and the proprioceptive feed-
back [1]. Force feedback integrated in a virtual real-
ity simulation provides data on a virtual object hard-
ness, weight, and inertia. Tactile feedback is used
to give the user a feel of the virtual object surface
contact geometry, smoothness, slippage, and temper-
ature. Finally, proprioceptive feedback is the sensing
of the user’s body position, or posture. The haptic
rendering consists in touching object, and to feel its
mechanical characteristics, sensitivity specific to the
bones, muscles, tendons and joints which give infor-
mation about its static, balance and the displacement
of the body in space [3].
Figure 2: Desktop touch.
3Computer GRAphics access for Blind people through a hap-
tic virtual environment, IST-2000-26151

Force feedback interfaces can be viewed as com-
puter “extensions” that apply physical forces, and
torques on the user. The interfaces that are most used
today are those that are desktop, are easy to install,
clean and safe to the user. For example, the SensAble
PHANToM haptic robot is a device that has 6 degrees
of freedom and renders a three-dimensional force in-
formation, [2]. It can track the position and orienta-
tion of the tool within a workspace of 16 cm wide, 13
cm high and 13 cm deep. With such haptic device
the haptic rendering is done on virtual touchable ob-
ject, see Fig. 2, and it provides data on virtual object
hardness, weight, and inertia. Each input device has
its own strengths and weaknesses, just as each applica-
tion has its own unique demands. With the wide range
of input devices available, one of the problems that
confronts the designer is to obtain a match between
application and input technology. Part of the prob-
lem has to do with recognizing the relevant dimen-
sions along which the application’s demands should be
characterized. The other is knowing how each tech-
nology being considered performs along those dimen-
sions. With this haptic device, the aim is to get a
stable interaction with virtual environment in a large
workspace and by two contact points. The Haptic
Workspace clue and the Force intensity clue enhance
the haptic rendering. By using the GRAB haptic de-
vice for art and cultural heritage discovery in virtual
environment, the haptic rendering is more effective.
In the context of CREATE, the haptic device pro-
vides two large workspaces, that may be combined for
actions using the two hands in cooperation, and also
two contact points c1 and c2, see Fig. 3. It allows
the design of objects (scaling, modeling) or moving
objects by the way of the two hand contact points, or
by the grip movement. With such a device, the user
can feel more effectively the weight, the global shape
and contour following, and it also provides data on
virtual object hardness, weight, and inertia.
Figure 3: CREATE touch.
It appears that the CREATE haptic interface
brings more realistic direct manipulation of haptic ob-
ject for being touchable and movable. This particular
feature will benefit in simulating art, cultural or ar-
chitectural sites.
4 Mechanical subsystem
The typical workplace of the CREATE Haptic In-
terface will be within a cubic immersive display, as
sketched in the original concept drawing and in the
preliminary design configuration (see Fig. 1 and 4),
even if it can be used also in a desktop like environ-
ments. The workspace is located in front of the user
that can interacts with the virtual scenarios trough
two contact points (the 2 fingertips of the thumb and
the index of the same hand or the indexes of both
hands).
Figure 4: CAD Model of the layout of the CREATE
HI.
The mechanical sub-system is composed by two
identical robotic devices, each having a serial kine-
matics with a total of 6 Degree of Freedoms (DOFs).
For the implementation of the first 3 DOFs, 2 orthog-
onal rotational pairs followed by a prismatic pair have
been selected, while for the last 3 DOFs 3 intersecting
rotational pairs have been used to realize a spherical
joint (see Fig. 5 ). The first 3 DOFs are actuated
and sensorized to be able to replicate an independent
force vector with an arbitrary orientation on the fin-
gertip and track the position of the fingertip within a
large 3D workspace. The remaining 3 DOFs are pas-
sive and not sensorized because only the evaluation of
the absolute position of the fingertip is required and
no moments have to be exerted.
This solution allows a very high degree of isotropy
of the device w.r.t. other kinematics solutions. A high
degree of isotropy is important in order to achieve a
uniform use of the actuators in the workspace of the
device and to have a uniform reflected inertia.
In order to improve the transparency of use of the

Figure 5: Kinematics scheme.
device and the force feedback accuracy, the following
technical requirements have to be adopted:
•Low mass of the moving part that means a low
perceived inertia;
•High stiffness of the structure;
•Low friction;
•High bandwidth force feedback.
The design guidelines to satisfy such requirements
are:
•Localization of the motors on the fixed parts;
•Selection of motors with high torque to mass ratio
and high torque to rotor inertia ratio;
•Use of tendon transmissions;
•Use of low reduction ratio geared reducer;
•Use of light materials for the construction of the
moving links;
•Low or zero backlash implementation of the
joints;
•High resolution sensors.
In our case, all the above guidelines have been im-
plemented, in particular:
•The first 2 actuators have been mounted integral
with the fixed link (base) developing a differen-
tial cable mechanism capable to increase isotropic
performances, while the third actuator has been
mounted integral with the second moving link
(link2) having its center of gravity very close to
the intersection of the yaw and pitch axes of the
mechanism that is fixed w.r.t. ground;
•As actuators, brushed DC servomotors have been
selected;
•Metallic in tension tendons routed on idle pulleys
have been used as means of transmission of forces
form the actuators to the joints;
•No geared reducer have been used;
•All the structural parts have been realized in alu-
minium and carbon fiber.
The modulus of the exertable forces can be mod-
ified within the following ranges (for each contact
point):
•Peak Force Range: 0 <|F|<FP =40Nin the
case of forces to be exerted for a limited period
of time (≤1 min);
•Continuous Force Range: 0 <|F|<FC=4N
in the case of forces to be exerted for a long of
time (≥1 min). This limitation is due to the
heat dissipation of the electric motors (actuators
of the HI).
The allowable workspace is a parallelepiped, hav-
ing the base at a variable distance ZWS from the up-
per surface of the floor. The dimensions of the paral-
lelepiped are:
•HWS = 600 mm;
•DWS = 600 mm;
•WWS = 700 mm.
The stiffness of the structure in the worst case is
greater than 5 N/mm.
Figure 6: CAD Model of the robotic device.
5 Control Features
The main control architecture of the haptic inter-
face is represented in the figure 7.
Low level control software has been implemented in
order to provide whole Virtual Reality system the fol-
lowing main functionalities:

•manage the communication with Host VE com-
puter;
•verify, change and store the value for system tun-
able parameters;
•provide an elementary safety sound feedback
which improves the SW development;
•Model dynamic and kinematic of both haptic
arm;
•Compensate for non linear effects such as the
gravity acceleration on the display and the fric-
tion;
•Serve the host VE as a position controller (during
the wearing phases) and as force display (when
used as haptic display);
•Monitor simulation parameters in order to pre-
vent that SW and simple HW damages can hurt
the user;
•Generate the correct HI control motors signal
for moving two arms and for activating the force
feedback functionality.
Figure 7: Control Architecture.
All feature below have also to be synchronized and
monitored in order to allows whole system to correctly
execute all tasks requested by the host VE.
Moreover, the control system has to present the fol-
lowing general characteristics:
•the force feedback have to be realistic for allowing
an easy recognition of little shape particulars;
•ensure high safety level in all control phases;
•implement the software interface with host VE
computer based on easy communication protocol
and able to set some system parameters.
•Implement auto calibration features with the
arms’ relative position and the VE objects.
6 Calibration Features
Multi contact point virtual environments as well
as haptic and virtual immersive environment share
a common problem of coherence. Whenever the
interaction between the user and the system makes
use of several afferent channels (see Bergamasco [16]),
it is required that all the channels have time, force
and spatial synchronization.
Research experiences in this fields demonstrates that
the lack of synchronization in one or more of the
afferent channels can create in the user perception
from a lack of sense of presence up to a sense of
sickness during the virtual interoperation [6].
The proposed system has to synchronize three differ-
ent interaction feedback: two haptic information and
the relative graphical representation [13].
The inter-calibration among these components will
be achieved with a set of semi-automatic procedure
described hereafter:
Haptic graphic calibration procedures
The arm to graphic calibration is achieved by a set of
“point and click” procedures. The user is required to
place the tip of the haptic device into a set of point
which have been graphically produced.
Even is a minimum requirement of three point is
required for having a realistic match, the procedure is
iterated on a larger number of points in order to have
a better match which keeps into account statistical
error reduction and spatial distorsion of the graphical
feedback.
Inter Haptic calibration procedures
Two arms are placed and fixed, a simple mechanical
device links them reducing the relative mobility as
shown in following figure 8. This arrangement leaves
only 3 translation degrees of freedom to system and
allows the to keep the centre of Cardano’s joint in
fixed position respect to third body of both arms.
During the calibration procedure, the right arm is
moved by a position control while the left one is leaved
free to be conducted and in the same moment the con-
trol software provide to compute coordinates of spher-
ical joint expressed in local reference system of two
arms.
Two local frames are associated to right (Σ0)andleft
(Σ1) arm and an independent frame (ΣW) is set.
In order to calibrate the system a given motion is
produced on the master arm. The relative positions

Figure 8: Sketch of the intercalibration tool.
vectors and are calculate for each arm by applying
the direct cinematic equations for the mechanical de-
vice. A regressive and statistical algorithms described
in [19], finally allows the computation of the relative
position.
Acknowledgments
A novel VE system has been described. The sys-
tem is based on the constructivist approach and allow
user to higly interact with the VE objects. The whole
CREATE consortium is acknowledged for their con-
tribution to the definition of the project specification.
The EU community is acknowledged for the financial
support given to this research.
References
[1] Burdea, G., “Force and Touch Feedback for Vir-
tual Reality”, John Wiley & Sons, New York,
USA, 1996.
[2] T Massie, K. Salisbury, “The PHANToM Haptic
Interface: A Device for Probing Virtual Objects”,
ASME Winter Annual Meeting, DSC-Vol. 55-1,
pp. 295-300, 1994.
[3] A-M Wing, P. Haggard, J-R Flanagan, “Hand and
Brain - The neurophysiology and Psychology of
Hand Movements”, Academic Press, 1996.
[4] R.D. Howe, S.J. Lederman, “Haptic Interfaces for
Virtual Environment and Teleoperator Systems”,
DSC Vol. 58
[5] S.A. Wall, W.S. Harwin, “Quantification of the
Effects of Haptic Feedback During a Motor Skill
Task in a simulated environment”, 2nd PHAN-
ToM Users Research Symposium, 2000
[6] F. Infed, D.A. Lawrende et Al., “Combined Vi-
sual/haptic rendering modes for scientific visual-
ization”, DSC Vol 67
[7] R.E. Ellis N. Sarkar, M.A. Jenkins, “Numerical
Methods for the Haptic presentation of Contact:
Theory, simulations and experiments”, DSC Vol.
58
[8] H.B. Morgenbesser M.A. Srinivasan, “Force shad-
ing for haptic shape perception”, DSC Vol. 58
[9] K. Reinig, R. Tracy, et al., “Some calibration in-
formation for a PHANToM 1.5 A.”, 2nd Phantom
Users Group Workshop, 1997
[10] T. Gutierrez, J.I. Barbero, et al., “Assembly sim-
ulation through haptic virtual prototypes”, 3rd
Phantom Users Group Workshop, 1998
[11] A.E. Kirkpatrick, S.A. Douglas, “Evaluating
Haptic Interfaces in Terms of Interaction Tech-
niques”, 4th Phantom Users Group Workshop,
1999
[12] R.D. Howe, T. Debus, P. Dupont, “Twice the
Fun: Two Phantoms as a High-Performance Tele-
manipulation System”, 4th Phantom Users Group
Workshop, 1999
[13] M.C. Lin, A. Gregory et al., “Contact Determina-
tion for Real-Time Haptic Interaction in 3D Mod-
eling, Editing and Painting”, 4th Phantom Users
Group Workshop, 1999
[14] P. Buttolo, D. Kung, B. Hannaford, “Manipu-
lation in real, virtual and remote environments”,
IEEE International Conference on System, Man
and Cybernetics, 1995
[15] M. Bergamasco, “Force replication to the human
operator: the development of arm and hand exos
as haptic”, 7th Int. Symposium on Robotics Re-
search, 1995
[16] V. Hayward, R. Astley, “Performance measures
for haptic interfaces”, 7th Int. Symposium on
Robotics Research, 1995
[17] J.M. Hollerbach, H. Maekawa, “Haptic display
for object grasping and manipulating in virtual
environments”, ICRA 1998
[18] C.A. Avizzano, M. Bergamasco, “Issues in Full
Body Haptic Interfaces”, VR 2002
[19] S. Marcheschi, C.A. Avizzano, “GRAB D2, Hap-
tic Controller” User Manual, PERCRO Internal
Report, 2002