No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Effect of Force Feedback on Remote Palpation.
Abstract
Combining teletaction systems with telemanipulation systems promises to enhance task performance when interacting with remote environments. However, the force scaling inherent in the telemanipulation system affects the ability of the user to control the exploration force. The quality of the tactile signal is therefore impacted, affecting performance in tasks that benefit from spatially distributed force information. We compare performance localizing an embedded lump in a compliant environment using a telemanipulated teletaction system versus a directly manipulated teletaction system. Lump localization accuracy was found to be the same; however, time required to localize the lump was up to 150% longer for the telemanipulation trials. Based upon our results, we conclude that the ability to maintain an appropriate force in the remote environment is necessary to take full advantage of the spatially distributed force information from the tactile sensor.
... However, it has been shown that when experiments are performed through a teleoperated robotic system, the results are affected by the technical limitations of the slave robot. In the experiment performed by Feller et al. (2004), the subjects palpated a phantom tissue using direct manipulation and a teleoperated robot. Their results showed that performance using the robotic system was highly dependent on the distal exploration force. ...
... There is no obvious way to effectively measure user performance in a palpation task. When the effect of force feedback is studied, the force applied by the user during the palpation is a good mean of determining the possible effects on tissue Trejos et al., 2009;Feller et al., 2004;Talasaz et al., 2010). Conventional performance metrics such as accuracy and completion time might not be sufficient to quantitatively compare the effectiveness of various types of feedback in a complex task. ...
... tissue properties, finger orientation). Furthermore, as it was discussed in Section 3. 3.2, several studies have shown that in experiments performed through a teleoperated robotic system, the results are affected by the limitations of the slave robot (Feller et al., 2004;Talasaz et al., 2010;Trejos et al., 2009). The present study aimed to determine which channels of information are more convenient to improve the subject's performance. ...
Over the past few decades, surgical procedures have made enormous progress, shifting from traditional open procedures to less and less invasive approaches, with the promise of smaller incisions, less complications, better cosmetic results and shorter recovery times. With these developments came a reduced dexterity and more complex control through the fulcrum effect and modified eye-hand coordination. These complications were greatly mitigated by the recent introduction of surgical robots, allowing the surgeon to sit in a comfortable posture, and restoring natural visuomotor coordination. To date, the issue of the lack of haptic feedback, which initially allowed the surgeon to intervene without vision, identify pathological tissue, feel arteries, etc., has not been resolved, despite the fact that it is crucial in certain interventions. Besides, the added value of haptic feedback is subject to controversy. Surgeons experienced in robotic surgery have adapted to the lack of haptic feedback and learned how to rely only on vision and proprioceptive cues as compensation. Nevertheless, previous studies have shown that substituting haptic information through various sensory channels can increase surgeons' performance. As a complete restoration of the sense of touch is extremely challenging in minimally invasive surgery, the advantages of haptic feedback have to be demonstrated and quantified to justify the additional cost and complexity. This is the aim of the present work. We hypothesized that tactile feedback as well as force is crucial in surgery and we investigated the haptic information involved in several representative surgical tasks. Dedicated hardware and a virtual reality environment to simulate suturing and palpation tasks were developed to address these questions. Ergonomic design guidelines were established based on an in-depth literature review and surgeons' comments gathered in a survey. These guidelines were then used to design and benchmark an ergonomic haptic handle featuring active grasping feedback and additional safety features. The results of the first two studies suggest that the benefits of haptic information highly depend on the surgical task in question. During a suturing task, force feedback significantly increased users' accuracy whereas torque feedback did not result in any significant improvement. In a palpation task, a higher recognition rate was achieved with tactile feedback than with visual sensory substitution. The performance of the haptic device integrating the ergonomic handle was assessed and compared with the standard omega.7 haptic device. Results showed that the index of performance of the original device was not degraded with the additional hardware. The index of performance was slightly increased for a manipulation task involving orientations. The ergonomic assessment of the handle showed a slight decrease of the tension in the adductor pollicis and flexor digitorium muscles, and therefore a potential decrease of fatigue. Although just a subset of surgical tasks could be investigated, the results indicate that haptic feedback and sensory substitution would greatly benefit teleoperated robotic surgery. Providing the surgeon with tactile information can potentially restore the feeling of "contact with the patient" and other surgeries that are highly reliant on the sense of touch could become possible again. This thesis presents a novel multidisciplinary approach to systematically analyzing surgical tasks in order to improve safety in two dimensions. Firstly, in the area of patient safety by providing the surgeon with haptic information to increase performance and reduce the risk of errors and secondly, in the area of surgeon safety by providing a more ergonomic master console. We expect this work to be a first step in establishing a general design method for more ergonomic surgeon consoles and trust that it will inspire research to investigate the sensory mechanisms underlying highly dexterous tasks such as those performed in surgery.
... The second experiment investigated subjects' ability to locate objects in a compliant medium with reduced tactile sensation. This experiment was conducted to examine the necessity for 261 tactile sensors (such as those from [5][6][7]) in the gripper design. A haptic-sensing gripper without tactile sensors is certainly feasible, in terms of kinesthetic sensing, and would be easier to integrate into an MIS scenario. ...
... Such force sensing is commonly performed on some area of tool shaft [8][9] and / or the gripping jaws [10]. Alternatively, [11] and [7] both performed remote tactile object exploration using force-sensing industrial scale robot manipulators to position a tactile array about a compliant object. ...
A natural extension of current robotic minimally invasive surgery (MIS) is the addition of tactile and kinesthetic feedback, which would give an operating surgeon valuable haptic infonnation
about tissue under observation. Identification of haptic properties requires active exploration [1][2]. It is believed that such exploration may be facilitated by a tool capable of replicating the major exploratory procedures (EPs; haptic information gleaning maneuvers) of a surgeon. Such a tool must also fit within the
fabrication and other constraints of similar MIS tools.
In this paper the requirements of such a gripper are highlighted via experiments with surgeons. The results have been used to develop a 2-finger, 3dof (degree of freedom) gripper based on an inverted closed chain serial manipulator. A primary advantage of this system is its dexterity and compactness, compared to other
manipulators. The gripper is shown to be capable of replicating the most frequently observed two-finger exploratory procedures. It is argued that a single tactile sensor is sufficiently useful for a two fingered system and a scaled prototype of the system has been designed to accommodate such a sensor. This system has been designed with later laparoscopic grade manufacture in mind.
... One study investigated the optimal properties of the rubber layer covering the pins that acts as a spatial low pass filter [12]. Another considered force control algorithms for teleoperation when shape displays are used in conjunction with force feedback devices [13]. We have also developed signal processing algorithms to improve the realism of sensations despite continuous contact between the user's finger and the flat display surface [14], and to render correct skin deformations for simulated contact with soft tissue in virtual environments [15] Another central question is the required resolution in pin height for these devices. ...
... One study investigated the optimal properties of the rubber layer covering the pins that acts as a spatial low pass filter [12]. Another considered force control algorithms for teleoperation when shape displays are used in conjunction with force feedback devices [13]. We have also developed signal processing algorithms to improve the realism of sensations despite continuous contact between the user's finger and the flat display surface [14], and to render correct skin deformations for simulated contact with soft tissue in virtual environments [15]. ...
Tactile displays are used to convey small-scale pressure and shape information to the finger tip. Our device designs include a 1x10 pin display using shape-memory alloy actuators and a 6x6 pin display using inexpensive commercially available servomotor-gearhead packages. Our human factors studies suggest that bandwidths of about 30 Hz are required for free exploratory motions with pin-based displays, and that perception and task performance can be assisted through signal processing algorithms that compensate for the artificial interactions between the display and finger tip. One promising application area is surgery, where relaying tactile information to the surgeon's finger tip from within the patient's body may enhance minimally invasive procedures.
... The simplest solutions showcase vibrating stimulation devices, placed on top of the user's fingers [29] or over the control interface [30]. The most complex interfaces showcase the entire surface's morphology changes, acting as physical twins of the touched surface in the remote environment [31,32,33]. Those devices are able to reproduce any shape touched by the follower and convey both kinesthetic and tactile haptic information, but they are solely feedback interfaces, and cannot be used for the trajectory control of the follower. ...
Teleoperated robots have been widely accepted in several fields of medical practice, enhancing human abilities and allowing remote operation. However, such technology has not been able yet to permeate areas such as primary care and physical examination. Such applications strongly rely on the quality of the interaction between doctor and patient, and on its multimodal nature. In order to achieve remote physical examination is thus mandatory to have a good doctor-robot interface, but what does good mean? Ultimately, the goal is for the user to achieve task embodiment, making the remote task feel like the in-person one. Several research groups have proposed a wide variety of interfaces, showcasing largely different methods of control and feedback, because of the absence of design guidelines. In this work, we argue that the ideal interface for a remote task should resemble as close as possible the experience provided by the in-person equivalent, keeping in consideration the nature of the target users. To support our claims, we analyze many remote interfaces and compare them with the respective in-person task. This analysis is not limited to the medical sector, with examples such as remote abdominal surgery, but it expands to all forms of teleoperation, up to nuclear waste handling and avionics.
... One simple format is a stimulation device that can be placed directly on top of the fingertip or wrapped around the control joystick 27 and delivers a local vibration or compression 28 . In contrast, the most elaborate devices morph 2D surfaces to deliver tactile feedback by changing the surface's geometry [29][30][31] . Alternatively, tactile information can be visualized by directly showing the tactile data as a color-coded matrix on a computer screen [32][33][34] or creating a virtual simulation of the palpated object [35][36][37] . ...
Realtime visual feedback from consequences of actions is useful for future safety-critical human–robot interaction applications such as remote physical examination of patients. Given multiple formats to present visual feedback, using face as feedback for mediating human–robot interaction in remote examination remains understudied. Here we describe a face mediated human–robot interaction approach for remote palpation. It builds upon a robodoctor–robopatient platform where user can palpate on the robopatient to remotely control the robodoctor to diagnose a patient. A tactile sensor array mounted on the end effector of the robodoctor measures the haptic response of the patient under diagnosis and transfers it to the robopatient to render pain facial expressions in response to palpation forces. We compare this approach against a direct presentation of tactile sensor data in a visual tactile map. As feedback, the former has the advantage of recruiting advanced human capabilities to decode expressions on a human face whereas the later has the advantage of being able to present details such as intensity and spatial information of palpation. In a user study, we compare these two approaches in a teleoperated palpation task to find the hard nodule embedded in the remote abdominal phantom. We show that the face mediated human–robot interaction approach leads to statistically significant improvements in localizing the hard nodule without compromising the nodule position estimation time. We highlight the inherent power of facial expressions as communicative signals to enhance the utility and effectiveness of human–robot interaction in remote medical examinations.
... Tactile feedback not only provides an intuitive understanding of tissue properties, but it is also independent of the visual channel; thus, it rarely impedes the surgeon's operation. Numerous actuation methods have been employed to generate the tactile feedback for tissue palpation, such as tactile displays using shape memory alloy wires [19], multiple servomotors [20], and pneumatic systems [21] to drive pin-array elements to represent the spatially distributed reaction force. Bianchi et al. [22] and Rizzo et al. [23] utilized pneumatic air-jet and magnetorheological-fluid (MRF) devices, respectively, to display lumps (or tumors) of different sizes. ...
Tumor location, depth, and size are essential information for tumor resection surgery. In open surgery, surgeons can obtain information by palpating the tissue with their fingers. In minimally invasive surgery, where the natural sense of touch is restricted, surgeons can rely on haptic information provided by haptic devices to determine the tumor characteristics. Tactile feedback is a promising representation modality for providing haptic information intuitively to surgeons during tissue palpation. In this paper, we propose a palpation strategy using tactile feedback to determine the tumor depth and size. For the palpation strategy, the tumor depth was determined by detecting the presence of the tumor at a given indentation depth of the sensor. Tumor size may be obtained by localizing the tumor edges. Fundamental experiments were conducted to investigate the use of contact force components in determining tumor features using the proposed strategy. The results indicated that the normal force is more useful in estimating the indentation depth, and the shear force is highly effective in detecting tumor regions and edges. Users’ ability to characterize the tumor using tactile feedback from our developed tactile display was demonstrated through tissue palpation tasks. Participants who received both normal force and shear force feedback could identify the depth and size of the embedded tumor with 66 % and 65 % accuracy, respectively. These results suggest that tactile displays that provide normal and shear force feedback can be successfully used for tumor characterization.
... Feller et al. concluded that it was necessary to maintain an appropriate force throughout a telemanipulated procedure to take advantage of the spatially distributed force information from the tactile sensor. 37 This would be a focus of the next step in the development of the telemanipulated fracture reduction framework. Another limitation is the use of medical students as examiners owing to their inexperience in performing fracture reduction procedures. ...
Objectives:
Intramedullary nailing is the treatment of choice for femoral shaft fractures. However, there are several problems associated with the technique, e.g. high radiation exposure and rotational malalignment. Experimental robotic assistance has been introduced to improve the quality of the reduction and to reduce the incidence of rotational malalignment. In the current study, we compare two devices for control of the fracture fragments during telemanipulated reduction.
Methods:
Ten male and ten female subjects were asked to participate as examiners in this experiment. A computer program was developed to render and manipulate CT-based renderings of femur fracture bone fragments. The user could manipulate the fragments using either a simple joystick device or a haptic manipulator. Each examiner performed telemanipulated reduction of 10 virtual fracture models of varying difficulty with each device (five in a 'training phase' and five in a 'testing phase'). Mixed models were used to test whether using the haptic device improved alignment accuracy and improved reduction times compared to using a joystick.
Results:
Reduction accuracy was not significantly different between devices in either the training phase or the testing phase (P > 0.05). Reduction time was significantly higher for the Phantom device than for the Joystick in the training phase (P < 0.0001), but it was no different in the testing phase (P = 0.865). High spatial ability with electronics had a significant effect on the alignment of fracture reduction and time to reduction.
Conclusions:
The Joystick and the Phantom devices resulted in similarly accurate reductions, with the Joystick having an easier learning curve. The Phantom device offered no advantage over the Joystick for fracture telemanipulation. Considering the high cost of the Phantom device and the lack of a demonstrable advantage over the Joystick, its use is not justified for implementation in a fracture telemanipulation workflow. The Joystick remains as a low-cost and effective device for developing 3D fracture telemanipulation techniques.
... aunque son muchas las ventajas que la MIS tiene sobre la cirugía convencional, esta presenta algunas dificultades para los cirujanos, quienes se ven obligados a interactuar en el campo quirúrgico con reducidos grados de libertad y limitado sentido del tacto [16]; y a su vez, disponiendo sólo de realimentación visual en dos dimensiones, lo cual implica una coordinación manual-visual diferente a la habitual del cirujano [17]. con el objetivo de facilitar el trabajo del cirujano, disminuyendo problemas como el temblor debido a la fatiga muscular y mejorando la precisión en los procedimientos, se recurre a la MIRS [18], en la que se implementa una plataforma de robótica compuesta por dos elementos definidos como maestro y esclavo (Fig. 2). ...
Minimally Invasive Surgery and the adaptation of robotics to these procedures represent many advantages for the patient, the surgeon, and the health program. However, commercial devices used nowadays lack haptic feedback. This fact makes the tissue identification more difficult and increments the injuries risk during the surgical procedure. The development of systems with this kind of feedback has become a topic of interest throughout the world. The present article contains a revision of the state of the art about commercial and experimental systems developed in this area. Models for the force and torque propagation, used in Minimally Invasive Surgery, are also presented.
... This system is the only automated system apart from the work presented herein, to use tactile sensing for diagnostic purposes. To detect the presence of a 19 mm acrylic ball embedded in rubber, a master-slave robotic system equipped with tactile sensing capabilities was evaluated by [48]. When compared to direct manipulation of the tactile sensor, the results showed that the performance of the system was greatly dependent on how well the exploration force could be controlled by the user. ...
The challenges imposed by Minimally Invasive Surgery (MIS) have been the subject of significant research in the last decade. In the case of cancer surgery, a significant limitation is the inability to effectively palpate the target tissue to localize tumor nodules for treatment or removal. Current clinical technologies are still limited and tumor localization efforts often result in the need to increase the size of the incision to allow finger access for direct palpation. New methods of MIS tumor localization under investigation involve restoring the sense of touch, or haptic feedback. The two most commonly investigated modes of haptic perception include kinesthetic and tactile sensing, each with its own advantages and disadvantages. Work in this area includes the development of customized instruments with embedded sensors that aim to solve the problem of limited haptic feedback in MIS. This chapter provides a review of the work to date in the use of kinesthetic and tactile sensing information in MIS for tissue palpation, with the goal of highlighting the benefits and limitations of each mode when used to locate hidden tumors during MIS.
... aunque son muchas las ventajas que la MIS tiene sobre la cirugía convencional, esta presenta algunas dificultades para los cirujanos, quienes se ven obligados a interactuar en el campo quirúrgico con reducidos grados de libertad y limitado sentido del tacto [16]; y a su vez, disponiendo sólo de realimentación visual en dos dimensiones, lo cual implica una coordinación manual-visual diferente a la habitual del cirujano [17]. con el objetivo de facilitar el trabajo del cirujano, disminuyendo problemas como el temblor debido a la fatiga muscular y mejorando la precisión en los procedimientos, se recurre a la MIRS [18], en la que se implementa una plataforma de robótica compuesta por dos elementos definidos como maestro y esclavo (Fig. 2). ...
La cirugía mínimamente invasiva y la incorporación de la robótica en este tipo de procedimientos representa grandes ventajas para el paciente, el cirujano y los sistemas de salud. sin embargo, los dispositivos comerciales disponibles en la actualidad no cuentan con realimentación de fuerza y tacto, que faciliten al cirujano la identificación de los tejidos y consecuentemente, la reducción de errores en los procedimientos quirúrgicos; por lo cual, el desarrollo de sistemas que cuenten con este tipo de realimentación se convierte en un tema de interés a nivel mundial. el presente artículo contiene una revisión del estado de la técnica con respecto a los sistemas comerciales y experimentales desarrollados en esta área. también, se presentan algunos sensores y modelos matemáticos utilizados para calcular las fuerzas y torques en cirugía mínimamente invasiva.
... While not suitable for MIS procedures, this system stands out as the only previous automated system to use tactile sensing for diagnostic purposes. The research presented by Feller et al. (2004) evaluated the effect of using a master-slave robotic system equipped with tactile sensing capabilities to detect the presence of a 19 mm acrylic ball embedded in rubber. The results of using the robotic system were compared with the direct manipulation of the tactile sensor. ...
The 10 mm incisions used in minimally invasive cancer surgery prevent the direct palpation of internal organs, making intraoperative tumor localization difficult. A tactile sensing instrument (TSI), which uses a commercially available sensor to measure distributed pressure profiles along the contacting surface, has been developed to facilitate remote tissue palpation. The objective of this research is to assess the feasibility of using the TSI under robotic control to reliably locate underlying tumors while reducing collateral tissue trauma. The performance of humans and a robot using the TSI to locate tumor phantoms embedded into ex vivo bovine livers is compared. An augmented hybrid impedance control scheme has been implemented on a Mitsubishi PA10-7C to perform the force/position control used in the trials. The results show that using the TSI under robotic control realizes an average 35% decrease in the maximum forces applied and a 50% increase in tumor detection accuracy when compared to manual manipulation of the same instrument. This demonstrates that the detection of tumors using tactile sensing is highly dependent on how consistently the forces on the tactile sensing area are applied, and that robotic assistance can be of great benefit when trying to localize tumors in minimally invasive surgery.
... While not suitable for MIS procedures, this system stands out as the only automated system to use tactile sensing for diagnostic purposes. Research presented in [17] evaluates the effect of using a master-slave robotic system equipped with tactile sensing capabilities for tumor detection. Feedback to the user is provided via a tactile display. ...
The 10 mm incisions used in minimally invasive cancer surgery prevent direct manual palpation of internal organs, making intraoperative tumor localization difficult. A tactile sensing instrument (TSI), that uses a commercially available sensor to measure distributed pressure profiles along the contacting surface, has been developed to facilitate remote tissue palpation. The objective of this research was to assess the feasibility of using the TSI under robotic control to reliably locate underlying tumors. The performance of human and robot manipulation of the TSI to locate tumor phantoms embedded into ex vivo bovine livers was compared. An Augmented Hybrid Impedance Control scheme was implemented on a Mitsubishi PA10-7C robot to perform force/position control during the trials. The results showed that using the TSI under robotic control realized an average 35% decrease in the maximum forces applied, and more than a 50% increase in tumor detection accuracy when compared to manual manipulation of the same instrument. This demonstrates that tumor detection using tactile sensing is highly dependent on the consistent application of forces on the tactile sensing area and that robotic assistance can be of great benefit when trying to localize tumors during minimally invasive surgery.
... Large-area sensors are under development and some have been commercialized [16][17][18][19][20][21][22], and many humanoid robots have been developed which incorporate tactile skin sensors [23][24][25][26]. How to best transform the sensor signals into appropriate drive signals for the display will require analysis of the sensor mechanical properties and signal encoding, as well as better understanding of the perceptual and motor control issues [29]. ...
This work presents a new tactile display device for relaying contact information to locations along the human arm. The system is intended to facilitate teleoperation of whole-arm manipulation tasks, such remotely controlling a humanoid robot to grasp and lift large objects. The system consists of a set of five tactors, each a foam-covered paddle 42 x 48 mm. These tactors are brought into contact with the skin of the user's arm by small DC motors under computer control. The tactors are mounted on frames which are readily mounted to the upper and lower arm using Velcro straps. User tests demonstrate that the system can effectively convey contact information to the user. The benefits of this haptic display modality are evaluated in comparison with degraded visual information to estimate the resolution limits of the system.
... Kajimoto et al. [16] modeled a nerve axon model based on the properties of human skin, and proposed an electrocutaneous display using anodic and cathodic current stimulation; however, these tactile display devices sometimes involve user discomfort and even pain. Feller et al. [17] investigated the effect of force feedback on telemanipulated palpation system. The PHANToM TM haptic device and their own tactile display were used together to explore the environment; however, since the tactile display was too big and heavy to be integrated into the haptic device, the force and the tactile information were conveyed to each hand, respectively. ...
Although people usually contact a surface with some area rather than a point, most haptic devices allow a user to interact with a virtual object at one point at a time and likewise most haptic rendering algorithms deal with such situations only. In a palpation procedure, medical doctors push and rub the organ's surface, and are provided the sensation of distributed pressure and contact force (reflecting force) for discerning doubtable areas of the organ. In this paper, we suggest real-time area-based haptic rendering to describe distributed pressure and contact force simultaneously, and present a haptic interface system to generate surface properties in accordance with the haptic rendering algorithm. We represent the haptic model using the shape-retaining chain link (S-chain) framework for a fast and stable computation of the contact force and distributed pressure from a volumetric virtual object. In addition, we developed a compact pin-array-type tactile display unit and attached it to the PHANToM haptic device to complement each other. For the evaluation, experiments were conducted with non-homogenous volumetric cubic objects consisting of approximately 500 000 volume elements. The experimental results show that compared to the point contact, the area contact provides the user with more precise perception of the shape and softness of the object's composition, and that our proposed system satisfies the real-time and realism constraints to be useful for a virtual reality application.
... Recent work also provides some other examples of tactile displays attached to larger kinesthetic haptic devices. Feller et al. [2004] attached a pin array a teleoperated robot for palpation experiments. The effect of a rolling ball on the fingertip was studied in Provancher et al. [2003]. ...
A novel two-degree-of-freedom tactile display reproduces the sensations of sliding contact and incipient slip through the rotation of a ball positioned under the user's fingertip. A pair of motor-driven wheels actuates the ball via contact friction. Mechanical performance requirements are used to define the dimensions and construction method of the device. Kinematic analysis shows that the drive wheel angles and their contact locations with the ball must be carefully selected in order to accurately control the axis of rotation and speed of the ball. However, psychophysical experiments indicate that some kinematic error is tolerable; errors of up to 20° in slip angle and 30% of a nominal velocity may be applied without detection from an average user. The lightweight, modular tactile display was attached to a multi-degree-of-freedom kinesthetic interface and used to display virtual environments with slip. Experimental results demonstrate that users complete a virtual paper manipulation task with lower applied forces using combined slip and force feedback in comparison with conventional force feedback alone.
Teleoperated medical technologies are a fundamental part of the healthcare system. From telemedicine to remote surgery, they allow remote diagnosis and treatment. However, the absence of any interface able to effectively reproduce the sense of touch and interaction with the patient prevents the implementation of teleoperated systems for primary care examinations, such as palpation. In this paper, we propose the first reported case of a soft robotic bilateral physical twin for remote palpation. By creating an entirely soft interface that can be used both to control the robot and receive feedback, the proposed device allows the user to achieve remote palpation by simply palpating the soft physical twin. This is achieved through a compact design showcasing 9 pneumatic chambers and exploiting multi-silicone casting to minimize cross-noise and allow teleoperation. A comparative study has been run against a traditional setup, and both the control and feedback of the physical twin are carefully analyzed. Despite distributed tactile feedback not achieving the same performance as the visual map, the soft control and visual feedback combination showcases a 5.1% higher accuracy. Moreover, the bilateral soft physical twin results always in a less invasive procedure, with 41% lower mechanical work exchanged with the remote phantom.
Cutaneous haptic feedback has recently received great attention from researchers in the robotic teleoperation field, as it has been proven to convey rich information to the human operator while guaranteeing the safety and stability of the control loop. In fact, delivering ungrounded cutaneous cues keeps the teleoperation system stable even in the presence of time-varying destabilizing factors such as hard contacts or communication delays. This aspect is particularly relevant for all the applications and scenarios where the safety of the system is of paramount importance, as in medical robotics. This article presents an overview on cutaneous haptic interaction followed by a review of the literature on cutaneous/tactile feedback systems for robotic teleoperation, categorizing the considered systems according to the type of cutaneous stimuli they can provide to the human operator. The paper ends with a discussion on the role of cutaneous haptics in robotics and the perspectives of the field.
Medical robotics has been widely accepted in several fields in the healthcare system. However, despite several attempts and a well-understood potential, teleoperation is still far from permeating primary care and tasks such as remote palpation, despite significant advancements in both haptic and tactile feedback. With most of the current focus on the feedback channel, the control interface has largely been overlooked. However, recent studies have proposed control interfaces so as to resemble as much as possible in-person palpation. Although hypothesized, they have not been systematically proven to be more intuitive or natural than traditional ones. In this paper, we implement a hand-guided control interface and investigate its performance, comparing it to a keyboard and a joystick-based control interface. We extract both objective and subjective measurements. We show how the hand-guided control, despite being more similar to the in-person task, achieves lower speed and worse precision, especially for point-wise target reaching. Moreover, it is perceived as less intuitive by the users, conversely to what was previously theorized in literature.KeywordsTeleoperationHand trackingRemote palpation
We aim to achieve intraoperative localization of an early-stage gastric tumor that cannot be visually detected during laparoscopic surgery. In this study, we developed and evaluated a pneumatic tactile ring, which is a clinically applicable tactile device to provide instantaneous feedback from a tactile sensor directly manipulated by a surgeon. It was designed to be worn on the finger of the manipulating hand and to present pressure to the finger pad. It is lightweight, cost-effective, disposable, and sterilizable. We also developed a compact pneumatic drive unit to control the pressure and investigated its fundamental performance. The bandwidth of the pressure control was at least 1.3 Hz with a controllable range of up to 79.7 kPa. Moreover, a psychophysical experiment was performed to obtain the Weber ratio of the pressure and evaluate the effectiveness of the instantaneous tactile feedback of the sensor output through the tactile ring. The Weber ratio was 0.40 at the reference pressure of 22.7 kPa. The provided tactile feedback significantly reduced the absolute localization error and increased participants' confidence in their answers. It was shown that the tactile feedback through the ring is effective in laparoscopic tumor localization.
Cutaneous haptic feedback is considered an elegant solution to enhance the performance of robotic teleoperation systems while guaranteeing their safety. Delivering ungrounded cutaneous cues to the human operator conveys in fact rich information and does not affect the stability of the control loop. For this reason, this first part of the book presents a novel feedback approach to robotic teleoperation, that substitutes full haptic feedback with cutaneous stimuli only, with the objective of achieving high levels of performance while guaranteeing the stability of the considered systems. This technique is named sensory subtraction, in contrast to sensory substitution, as it subtracts the destabilizing kinesthetic part from the full haptic interaction to leave only cutaneous cues. Similarly to sensory substitution, this kind of feedback is expected to make the teleoperation system intrinsically stable, since the cutaneous force applied does not affect the position of the master device, thus opening the feedback loop. This chapter introduces the concept of sensory subtraction, and it briefly reviews the literature on ungrounded cutaneous devices and how they have been employed in robotic teleoperation.
In robot assisted laparoscopic surgery using a commercialized robot, such as the da Vinci, surgeons have encountered problems, such as organ impairment, because they depend only on visual information. To solve these problems, a haptic system is required. Force feedback and a bilateral control system are essential for realizing a haptic teleoperation system. The use of a force/torque sensor for force feedback is difficult in a surgical robot instrument, but previous studies confirmed that the reaction force loaded on a surgical robot instrument can be estimated using a sliding perturbation observer (SPO)-based reaction force estimation method. This paper suggests a bilateral controller applying SPO based force estimation method to realize the possibility of a haptic function in a laparoscopic surgery robot system. In the suggested bilateral controller, the master control uses impedance control and the slave control uses a sliding mode control (SMC). A torque and force sensorless teleoperation system can be implemented using the suggested bilateral control structure through an experimental evaluation. © 2015, Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers and Springer-Verlag Berlin Heidelberg.
In der konventionellen robotergestützten, minimal invasiven Chirurgie (MIC) besteht eine vollständige mechanische Entkopplung zwischen Chirurg und Patient. Die Entwicklung kinästhetischer Rückkopplungssysteme ist fortgeschritten, die Rückkopplung taktiler Eindrücke ist jedoch nach wie vor problematisch. Es wurden viele Versuche unternommen, vollständige teletaktile Eindrücke zu vermitteln, die jedoch aus unterschiedlichen Gründen scheiterten. Als hauptsächlicher Grund hierfür ist anzusehen, dass die Ausgabeschnittstelle üblicherweise ein eigenständiges Gerät ist, was die gleichzeitige Wahrnehmung von Eindrücken und Steuerung des Instruments erschwert. Die menschliche Wahrnehmung taktiler Eindrücke beruht weitgehend auf dem Betasten des Objekts mit Bewegen des "Sensors", was mit zwei verschiedenen Geräten schwer nachzubilden ist. Außerdem war die Interpretation des rückgekoppelten Signals nicht eindeutig und intuitiv genug.
Eine der größten Schwierigkeiten in der (robotergestützten) minimal invasiven Chirurgie ist die große, durch die fehlende taktile Rückkopplung begründete Gefahr einer unbeabsichtigten Arterienverletzung mit der Folge schwer kontrollierbarer Blutungen. In der offenen Chirurgie kann Gewebe betastet werden, und ein Pulsieren deutet auf eine im Gewebe liegende Arterie hin. Eine Substitution des Tastsinns wäre aber auch aus anderen Gründen wünschenswert. So können in der offenen Herzchirurgie die präoperativ geplanten Anastomosestellen durch Betasten aufgefunden werden. Stehen jedoch nur optische Untersuchungsmethoden zur Verfügung, kann ein sehr zeitaufwendiges Freipräparieren der Arterien notwendig werden, um die präoperativ geplanten Anastomosestellen aufzufinden.
Die in dieser Arbeit vorgeschlagene Lösung bietet keine umfassende taktile Rückkopplung, sondern konzentriert sich auf das ultraschallgestützte, quasi taktile Lokalisieren besonderer Strukturen unter verdeckendem Gewebe. Mit den erfassten Daten erfolgt eine Modalitätssubstitution, dem Nutzer werden intuitive haptische bzw. Mehrkanalinformationen zurückgegeben. Um die Doppler-Frequenzverschiebung des in den betreffenden Arterien fließenden Blutes zu erkennen, wird ein Ultraschalltransducer verwendet, der in ein minimal invasives Instrument integriert ist. Die gemessenen Signale werden analysiert und an ein haptisches Eingabegerät weitergeleitet, mit dem die Erkennung und die Eigenschaften der verdeckten Gefäße intuitiv erfassbar dargestellt werden. Erste Versuche zeigten, dass ein leichtes Zucken des Eingabegerätes in Verbindung mit dem typischen Doppler-Geräusch das direkte Betasten sehr zuverlässig und intuitiv ersetzen. Weiterführende Untersuchungen, die die Verlässlichkeit bestätigen und zu einem tieferen Verständnis dieser Ergebnisse führen sollen, stehen noch aus.
Es ist sehr fraglich, ob eine vollständige Rückkopplung aller möglichen taktilen Eindrücke in der robotergestützten MIC erstrebenswert ist der medizinische Nutzen scheint die Anstrengungen und Kosten nicht zu rechtfertigen. Deshalb verspricht, wie in dieser Arbeit, der Ersatz nur von Teilen der Taktilität, der mit der menschlichen Wahrnehmung vergleichbar ist, eine bessere Lösung zu sein. Im Gegensatz zu in der Literatur beschriebenen Realisierungen, die sich im praktischen Einsatz nicht durchgesetzt haben, hat das hier beschriebene System erste Tests bestanden und seine überlegene Leistungsfähigkeit bewiesen. Ein Patent hierzu wurde bereits erteilt.
Robotic surgery provides many benefits such as reduced invasiveness and increased dexterity. This comes at the cost of no direct contact between surgeon and patient. This physical separation prevents surgeons from performing direct haptic exploration of tissues and organs, imposing exclusive reliance on visual cues. Current technology is not yet able to both measure and reproduce a realistic and complete sense of touch (interaction force, temperature, roughness, etc.). In this paper, we put forward a concept based on multimodal feedback consisting of the integration of different kinds of visual and tactile cues with force feedback that can potentially improve both the surgeon's performance and the patient's safety. We present a cost-effective tactile display simulating a pulsating artery that has been integrated into a haptic workstation to combine both tactile and force-feedback information. Furthermore, we investigate the effect of different feedback types, including tactile and/or visual cues, on the performance of subjects carrying out two typical palpation tasks: (1) exploring a tissue to find a hidden artery and (2) identifying the orientation of a hidden artery. The results show that adding tactile feedback significantly reduces task completion time. Moreover, for high difficulty levels, subjects perform better with the feedback condition combining tactile and visual cues. As a matter of fact, the majority of the subjects in the study preferred this combined feedback because redundant feedback reassures subjects in their actions. Based on this work, we can infer that multimodal haptic feedback improves subjects' performance and confidence during exploratory procedures.
This paper presents a new tactile-force integrated method to localize tumors minimally invasively using robotic assistance. This method relies on using a capacitive sensor at the tip of a Tactile Sensing Instrument (TSI) which can be inserted into a patient's body in a minimally invasive manner. In this work, the operator palpates tissue containing tumors in a minimally invasive surgical (MIS) training box, representing the patient's body, through a master-slave teleoperation system which consists of a 7 degrees-of-freedom (DOF) haptic interface, used as the master, and a Mitsubishi PA10-7C robot as the slave. Using the proposed method, the operator would be able to palpate the tissue consistently, observe the pressure distribution over the tissue by a color contour map on a screen and feel the tumor on his/her fingers through a grasping mechanism of the haptic interface as a result of higher stiffness of the tumor. The tissue used for the experiments was ex vivo bovine lung and seven participants were asked to locate artificial tumors embedded in the lungs. The results show an accuracy of 93% in tumor localization using the proposed method while the average force applied to the tissue was 3.42N and the force never exceeded 6N.
This paper is aimed at exploring the effect of force feedback on the performance of a knot-tightening task in robotics-assisted minimally invasive surgery (RAMIS). In this work, we evaluate performance during the knot-tightening task in three scenarios: without force feedback, with visual force feedback and with direct force reflection on the subject's hand. Different performance measures have been implemented: quality of the knot, amount and consistency of the tightening force applied on the suture, user's control of the instrument, tissue damage, and task completion time. Seven subjects participated in this study and were asked to tighten the second throw of surgical knots using a dual arm teleoperation system that is capable of force reflection in 7 Degrees of Freedom (DOFs), 6-DOF rigid body motion plus the gripper. The results show that visual force feedback allows superior performance in the quality of the suture knots with high consistency in the tightening force, while direct force feedback can significantly improve the user's control of the instrument.
Tactile sensing and force reflection have been the subject of considerable research for tumor localization in soft-tissue palpation. The work presented in this paper investigates the relevance of force feedback (presented visually as well as directly) during tactile sensing (presented visually only) for tumor localization using an experimental setup close to one that could be applied for real robotics-assisted minimally invasive surgery. The setup is a teleoperated (master-slave) system facilitated with a state-of-the-art minimally invasive probe with a rigidly mounted tactile sensor at the tip and an externally mounted force sensor at the base of the probe. The objective is to capture the tactile information and measure the interaction forces between the probe and tissue during palpation and to explore how they can be integrated to improve the performance of tumor localization. To quantitatively explore the effect of force feedback on tactile sensing tumor localization, several experiments were conducted by human subjects to locate artificial tumors embedded in the ex vivo bovine livers. The results show that using tactile sensing in a force-controlled environment can realize, on average, 57 percent decrease in the maximum force and 55 percent decrease in the average force applied to tissue while increasing the tumor detection accuracy by up to 50 percent compared to the case of using tactile feedback alone. The results also show that while visual presentation of force feedback gives straightforward quantitative measures, improved performance of tactile sensing tumor localization is achieved at the expense of longer times for the user. Also, the quickness and intuitive data mapping of direct force feedback makes it more appealing to experienced users.
The ability to localise harder areas in soft tissues is often desired during robot-assisted surgical operations. A deformation-based tactile feedback system was tested for the detection of objects within soft tissues, after being chosen over common pressure-based designs. This system uses a biologically inspired sensor that offers a new finger-like approach to tactile sensing. A tactile shape display developed from previous successful designs was used to output the sensed tactile information. Using the tactile feedback system on a mechanical teleoperated device, test subjects palpated a number of artificial tissue models to locate objects of varying stiffness. The addition of the tactile feedback system improved the detection of the objects from 64% to 98%, reduced the localisation error from 18 to 11 mm, and also decreased the time the users spent palpating the tissue from 55 to 37 s. This demonstrates that a deformation-based tactile feedback system can be used to successfully locate hard embedded objects within soft tissue, with a significant improvement over force and visual feedback alone. During testing, it was found that the users were able to more accurately locate the softest embedded objects compared to stiffer ones. Reasons for this observation are discussed.
The reduced access conditions of minimally invasive surgery and therapy (MIST) impair or completely eliminate the feel of tool—tissue interaction forces. Many researchers have been working actively on the development of force sensors and sensing techniques to address this problem. The goal of this survey article is to summarize the state of the art in force sensing techniques for medical interventions in order to identify existing limitations and future directions. A literature search was performed from January to July 2009 using a combination of keywords relevant to the area, including force, sensor, sensing, haptics, and minimally invasive surgery.
The literature search resulted in 126 articles with valuable content. This article presents a summary of the force sensing technologies, design specifications for force sensors in clinical applications, force sensors and sensing instruments that have been developed for MIST, and the experiments performed to determine the need for force information. Open areas of research include force sensor design, development of alternative methods of sensing, assessment of the impact of force information on performance, determination of the benefits of haptic information, and evaluation of the human factors involved in the processing and use of force information.
A new device providing cutaneous feedback is
presented. Two motors are used to flex two mobile platforms,
applying a normal force to the user’s thumb and index finger
pads. The cutaneous device substitutes the typical kinesthetic
and cutaneous feedback, usually provided by grounded haptic
interfaces, with the cutaneous component only. The main
advantage of this approach is that it does not suffer from typical
stability issues and it can be considered intrinsically safe when
used in a teleoperation system.
The proposed technique can be casted in a sensory sub-
stitution framework but there are relevant differences which
are worth underlining. Its main advantage, with respect to
classic sensory substitution techniques which employ visual
and/or auditory feedback, is that the substitution occurs at
the cutaneous level and the feedback is applied directly on the
finger pads, i.e. exactly where the force feedback is expected
by the operator. A teleoperated needle insertion application is
considered, in order to evaluate the effectiveness of the device.
This chapter presents an overview of the teleoperation of robotics systems, starting with ahistorical background, and including
the description of an up-to-date specific teleoperation scheme as a representative example to illustrate the typical components
and functional modules of these systems. Some specific topics in the field are particularly discussed, for instance, control
algorithms, communications channels, the use of graphical simulation and task planning, the usefulness of virtual and augmented
reality, and the problem of dexterous grasping. The second part of the chapter includes a description of the most typical
application fields, such as industry and construction, mining, underwater, space, surgery, assistance, humanitarian demining,
and education, where some of the pioneering, significant, and latest contributions are briefly presented. Finally, some conclusions
and the trends in the field close the chapter.
The topics of this chapter are closely related to the contents of other chapters such as those on Communication in Automation, Including Networking and Wireless (Chap.13), Virtual
Reality and Automation (Chap.15), and Collaborative Human–Automation Decision Making (Chap.26).
In der konventionellen robotergestützten, minimal invasiven Chirurgie (MIC) besteht eine vollständige mechanische Entkopplung zwischen Chirurg und Patient. Die Entwicklung kinästhetischer Rückkopplungssysteme ist fortgeschritten, die Rückkopplung taktiler Eindrücke ist jedoch nach wie vor problematisch. Es wurden viele Versuche unternommen, vollständige teletaktile Eindrücke zu vermitteln, die jedoch aus unterschiedlichen Gründen scheiterten. Als hauptsächlicher Grund hierfür ist anzusehen, dass die Ausgabeschnittstelle üblicherweise ein eigenständiges Gerät ist, was die gleichzeitige Wahrnehmung von Eindrücken und Steuerung des Instruments erschwert. Die menschliche Wahrnehmung taktiler Eindrücke beruht weitgehend auf dem Betasten des Objekts mit Bewegen des "Sensors", was mit zwei verschiedenen Geräten schwer nachzubilden ist. Außerdem war die Interpretation des rückgekoppelten Signals nicht eindeutig und intuitiv genug.
Eine der größten Schwierigkeiten in der (robotergestützten) minimal invasiven Chirurgie ist die große, durch die fehlende taktile Rückkopplung begründete Gefahr einer unbeabsichtigten Arterienverletzung mit der Folge schwer kontrollierbarer Blutungen. In der offenen Chirurgie kann Gewebe betastet werden, und ein Pulsieren deutet auf eine im Gewebe liegende Arterie hin. Eine Substitution des Tastsinns wäre aber auch aus anderen Gründen wünschenswert. So können in der offenen Herzchirurgie die präoperativ geplanten Anastomosestellen durch Betasten aufgefunden werden. Stehen jedoch nur optische Untersuchungsmethoden zur Verfügung, kann ein sehr zeitaufwendiges Freipräparieren der Arterien notwendig werden, um die präoperativ geplanten Anastomosestellen aufzufinden.
Die in dieser Arbeit vorgeschlagene Lösung bietet keine umfassende taktile Rückkopplung, sondern konzentriert sich auf das ultraschallgestützte, quasi taktile Lokalisieren besonderer Strukturen unter verdeckendem Gewebe. Mit den erfassten Daten erfolgt eine Modalitätssubstitution, dem Nutzer werden intuitive haptische bzw. Mehrkanalinformationen zurückgegeben. Um die Doppler-Frequenzverschiebung des in den betreffenden Arterien fließenden Blutes zu erkennen, wird ein Ultraschalltransducer verwendet, der in ein minimal invasives Instrument integriert ist. Die gemessenen Signale werden analysiert und an ein haptisches Eingabegerät weitergeleitet, mit dem die Erkennung und die Eigenschaften der verdeckten Gefäße intuitiv erfassbar dargestellt werden. Erste Versuche zeigten, dass ein leichtes Zucken des Eingabegerätes in Verbindung mit dem typischen Doppler-Geräusch das direkte Betasten sehr zuverlässig und intuitiv ersetzen. Weiterführende Untersuchungen, die die Verlässlichkeit bestätigen und zu einem tieferen Verständnis dieser Ergebnisse führen sollen, stehen noch aus.
Es ist sehr fraglich, ob eine vollständige Rückkopplung aller möglichen taktilen Eindrücke in der robotergestützten MIC erstrebenswert ist – der medizinische Nutzen scheint die Anstrengungen und Kosten nicht zu rechtfertigen. Deshalb verspricht, wie in dieser Arbeit, der Ersatz nur von Teilen der Taktilität, der mit der menschlichen Wahrnehmung vergleichbar ist, eine bessere Lösung zu sein. Im Gegensatz zu in der Literatur beschriebenen Realisierungen, die sich im praktischen Einsatz nicht durchgesetzt haben, hat das hier beschriebene System erste Tests bestanden und seine überlegene Leistungsfähigkeit bewiesen. Ein Patent hierzu wurde bereits erteilt.
Teletaction is the sensing of a remote object by transmitting tactile information from a remote tactile sensor to an operator’s skin through tactile interface devices. A tactile interface is used to reproduce the information such as force (static and dynamic), texture, roughness, temperature, and shape. This paper describes the design, modeling, simulation, fabrication and testing of an experimental teletaction system to detect and reproduce the human pulse. It consists of three major components: pulse sensing system, pulse teleperception system and the data processing system. Two 25 μm thick polyvinylidene fluoride (PVDF) film sensors had been fabricated and calibrated for pulse detection and force measurement. A computer controlled force feedback system was developed for achieving pulse teleperception. The simulation and experimental results demonstrated the close matching between the sensed and perceived pulse. A psychophysics test showed that the fingertip feeling at the pulse teleperception end closely matched the feeling of touching a human pulse directly.
Minimally invasive surgery is a technique that provides numerous benefits to the patient, but presents challenges to the surgeon in that dexterity, hand-eye coordination and haptic perception are compromised. Robot-assisted minimally invasive approaches have addressed the problems of dexterity and coordination; however, the lack of kinesthetic and tactile feedback remains a significant drawback. Despite many advances in this area, little is currently known about what level of feedback performance is adequate to allow the surgeon to palpate tissue to detect an underlying tumour. This paper describes experiments that were conducted on ex-vivo porcine lung, using artificial tumours, to elucidate one measure of sensor performance required to detect the presence of a tumour. The results indicate that a force- sensitive probe with a sensing range of 0 to 10 N and a resolution of 0.01 N would allow a tumour to be localized via palpation using kinesthetic feedback.
Two, world-renown researchers in the science of touch (Klatzky and Lederman) have shown that there are a set of reproducible and subconscious maneuvers that humans use to explore objects. Force measuring sensors may be used to electronically identify and quantify these maneuvers. Two sensored silicone breast models were configured to represent two different clinical presentations. One-hundred clinicians attending a local breast cancer meeting performed clinical breast examinations on the models, and their performance was captured using sensor-based data acquisition technology. We have found that Klatzy and Lederman's previously defined touch maneuvers are used during the clinical breast examination and can be identified and quantified for the first time using sensor technology.
Este proyecto está contenido dentro de uno de mayor envergadura que es el diseño de el robot humanoide RH-2 en el Departamento de Ingeniería de Sistemas y Automática de la Universidad Carlos III de Madrid. El proyecto que se expone a continuación tiene como objetivo el desarrollo de un sensor exteroceptivo para la ayuda al equilibrio de un robot humanoide. Los humanos podemos mantener el equilibrio gracias a tres sentidos, estos son la vista, “el oído” y el tacto. El tacto del pie con el suelo es muy importante a la hora de mantener el equilibrio, por ejemplo con una sola pierna. Así, se pretende crear un sensor bioinspirado con dos funciones. La medida de las fuerzas de reacción del suelo contra la suela del pie del robot humanoide RH-2 (CARHU) y la absorción de pequeñas irregularidades del terreno como lo haría una zapatilla de un ser humano o el propio pie humano. Esta medida se basará en crear una matriz de sensores en el que cada una de las celdas sensoras proporcionará un vector de desplazamiento o de fuerza así se conseguirá una especie de tacto. Ingeniería Técnica en Electrónica
Part I of this tutorial described two broad paradigms of interventional assistance: surgical computer-aided design (CAD)/computer-aided manufacturing (CAM) and surgical assistance. Part II focused on the underlying concepts of surgical CAD/CAM, with a particular emphasis on percutaneous procedures. This final installment of our three-part tutorial series discusses surgical assistance. In this section, we introduce the basic concepts of a surgical workstation and briefly review several core robotic technologies used in surgical workstations.
Detection of abnormalities in subcutaneous tissue is an important issue in medical diagnosis. Surgeons palpate this type of tissue to perceive pathologies through haptic sensation. This paper presents the framework of a real-time haptic-palpation system with inclusion detection to manipulate biological soft tissues and perceive their structures and material properties non-invasively. A user with a haptic device guides a robotic manipulator to perform palpation tasks and can detect the location of inclusions inside a soft medium (silicone-molded tissue phantom). For the objective measurement of the depth of the inclusion, a finite element (FE) model was developed to predict the reaction forces for various inclusion depths. The methods presented in this paper can be applied to the early detection of prostate cancer or breast cancer in telediagnosis.
Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon and minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks.
Haptic feedback systems for RMIS are still under development and evaluation. Most provide only force feedback, with limited fidelity. The major challenge at this time is sensing forces applied to the patient. A few tactile feedback systems for RMIS have been created, but their practicality for clinical implementation needs to be shown. It is particularly difficult to sense and display spatially distributed tactile information. The cost-benefit ratio for haptic feedback in RMIS has not been established.
The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems. Surgeons, engineers, and neuroscientists should work together to develop effective solutions for haptic feedback in RMIS.
Stability is a key issue in the implementation of a bilateral telemanipulation system since both the non-negligible time delay in the communication between master and slave and the interaction with unknown environments can destabilize the whole system. In Chap. 4 it has been shown that passivity theory and port-Hamiltonian systems can be fruitfully used to build an intrinsically passive telemanipulation scheme which, therefore, has a stable behavior both in case of free motion and in case of contact with any passive, possibly unknown, environment. Scattering theory has been used to build a communication channel which is lossless independently of any constant transmission delay and the problem of wave reflection arising when coupling master and slave side through scattering based communication channels has been solved. The scheme has been extended in order to take into account the sampled data nature of controllers in a passive way. Moreover discrete scattering has been defined and packet-switching transmission lines have been considered. A communication strategy that allows to preserve passivity even in case of loss of packets and of variable transmission delay has been proposed. Finally, a passivity preserving algorithm that allows to rebuild lost packets by interpolation has been proposed.
This article reports a variety of sensory and perceptual consequences of eliminating, via a rigid fingertip sheath, the spatially distributed fingertip force information that is normally available during tactile and haptic sensing. Sensory measures included tactile spatial acuity, tactile force, and vibrotactile thresholds. Suprathreshold tasks included perception of roughness, perception of 2-D edge orientation, and detection of a simulated 3-D mass in simulated tissue via fingertip palpation. Of these performance measures, only vibrotactile thresholds and texture perception failed to show substantial impairment. The results are discussed in terms of their implications for the future design of haptic interfaces for teleoperator and virtual environment systems.
Small objects were lifted from a table, held in the air, and replaced using the precision grip between the index finger and thumb. The adaptation of motor commands to variations in the object's weight and sensori-motor mechanisms responsible for optimum performance of the transition between the various phases of the task were examined. The lifting movement involved mainly a flexion of the elbow joint. The grip force, the load force (vertical lifting force) and the vertical position were measured. Electromyographic activity (e.m.g.) was recorded from four antagonist pairs of hand/arm muscles primarily influencing the grip force or the load force. In the lifting series with constant weight, the force development was adequately programmed for the current weight during the loading phase (i.e. the phase of parallel increase in the load and grip forces during isometric conditions before the lift-off). The grip and load force rate trajectories were mainly single-peaked, bell-shaped and roughly proportional to the final force. In the lifting series with unexpected weight changes between lifts, it was established that these force rate profiles were programmed on the basis of the previous weight. Consequently, with lifts programmed for a lighter weight the object did not move at the end of the continuous force increase. Then the forces increased in a discontinous fashion until the force of gravity was overcome. With lifts programmed for a heavier weight, the high load and grip force rates at the moment the load force overcame the force of gravity caused a pronounced positional overshoot and a high grip force peak, respectively. In these conditions the erroneous programmed commands were automatically terminated by somatosensory signals elicited by the start of the movement. A similar triggering by somatosensory information applied to the release of programmed motor commands accounting for the unloading phase (i.e. the parallel decrease in the grip and load forces after the object contacted the table following its replacement). These commands were always adequately programmed for the weight.
In the field of motor control, two hypotheses have been controversial: whether the brain acquires internal models that generate accurate motor commands, or whether the brain avoids this by using the viscoelasticity of musculoskeletal system. Recent observations on relatively low stiffness during trained movements support the existence of internal models. However, no study has revealed the decrease in viscoelasticity associated with learning that would imply improvement of internal models as well as synergy between the two hypothetical mechanisms. Previously observed decreases in electromyogram (EMG) might have other explanations, such as trajectory modifications that reduce joint torques. To circumvent such complications, we required strict trajectory control and examined only successful trials having identical trajectory and torque profiles. Subjects were asked to perform a hand movement in unison with a target moving along a specified and unusual trajectory, with shoulder and elbow in the horizontal plane at the shoulder level. To evaluate joint viscoelasticity during the learning of this movement, we proposed an index of muscle co-contraction around the joint (IMCJ). The IMCJ was defined as the summation of the absolute values of antagonistic muscle torques around the joint and computed from the linear relation between surface EMG and joint torque. The IMCJ during isometric contraction, as well as during movements, was confirmed to correlate well with joint stiffness estimated using the conventional method, i.e., applying mechanical perturbations. Accordingly, the IMCJ during the learning of the movement was computed for each joint of each trial using estimated EMG-torque relationship. At the same time, the performance error for each trial was specified as the root mean square of the distance between the target and hand at each time step over the entire trajectory. The time-series data of IMCJ and performance error were decomposed into long-term components that showed decreases in IMCJ in accordance with learning with little change in the trajectory and short-term interactions between the IMCJ and performance error. A cross-correlation analysis and impulse responses both suggested that higher IMCJs follow poor performances, and lower IMCJs follow good performances within a few successive trials. Our results support the hypothesis that viscoelasticity contributes more when internal models are inaccurate, while internal models contribute more after the completion of learning. It is demonstrated that the CNS regulates viscoelasticity on a short- and long-term basis depending on performance error and finally acquires smooth and accurate movements while maintaining stability during the entire learning process.
VR and telepresence applications have placed increasing demands on
the need for effective user interfaces. To date most of the interfaces
have emphasised the use of visual and audio effects but tactile feedback
has been identified as a leading feature for future systems where there
will be an increased desire to truly interact with the virtual/remote
world rather than being observational. The paper focuses on the
cutaneous aspects of tactile feedback describing the design and
construction of pneumatically powered tactile and shear feedback
modules. It is shown that by incorporating a range of novel features
into this design it is possible to stimulate all the mechano-receptive
nerves (SAI, SAII, RAI, and RAII) with localised signals from DC to 400
Hz. All this is shown in a fully integrated, ultra-light and comfortable
package. The design control and performance results are all presented
In this paper, we examine the role of two force scaling laws in the performance, by human operators, of scaled teleoperated pick and place tasks. The experiments used a hybrid hardware-software telemanipulation system with force feedback. Human subjects were provided with visual and varying types of force feedback to perform the desired task. The force feedback depended on the scaling law used. Our results show that impedance scaling improved the performance with respect to constant (i.e., geometric only) scaling or no force feedback.
Tactile displays are used to convey small-scale force and shape information to the fingertip. We describe a 6 x 6 tactile shape display design that is low in cost and easily constructed. It uses commercially available RC servomotors to actuate an array of mechanical pins. The pins deflect a maximum of 2 mm, with a resolution of 0.1 mm. The pin center spacing is 2 mm and the pin diameter is 1 mm. For the maximum deflection of 2 mm, the display can represent frequencies up to 7.5 Hz; smaller deflections lead to achievable frequencies up to 25 Hz because the servos are slew rate limited. This design is well suited to tactile display research, as it offers reasonable performance in a robust and inexpensive package.
CLINICAL SCENARIO
The patient in your examining room is new to the practice. He is 52 years old, emigrated from Southeast Asia about 10 years ago, and has no specific complaints except fatigue. On examination you find little of note except that his liver edge is firm, easily felt, and extends about 6 cm below the costal margin across much of the right upper quadrant. The span, by light percussion, is 17 or 18cm. Should you be concerned? What does the research literature tell us about the meaning of these findings?
Relaying spatially distributed forces promises to enhance the performance of telemanipulation systems. However, the proper way to render tactile information from the sensor to the display is not clear for current displays, where the user applies a constant contact force. We present a simple approach to rendering tactile information to improve performance in a lump localization task in a compliant environment. The algorithms presented subtract uninformative background information from the tactile signal. We observed that subtracting a fixed background pressure frame reduced lump localization error by up to 20% while decreasing the time required to find the lump by up to 44%. Subtracting a background frame that depends on applied force further reduced lump localization error by another 17%.
A power and impedance scaling bilateral manipulator (such as an
extender or a macro-micro bilateral manipulator) can greatly enhance the
manual capabilities of a human operator, but it can also compromise the
inherent stability of the operator. A condition for the robust stability
of an operator/bilateral manipulator environment system is derived using
the structured singular value. The application of this condition is
illustrated with several examples of power and impedance scaling via a
two-channel bilateral manipulator
This paper presents a tactile sensor that provides transient event
information at the finger-object interface. The multi-element stress
rate sensor consists of piezoelectric polymer strips moulded into the
surface of the rubber “skin” covering the robot finger tip.
These piezoelectric elements provide localized information important to
manipulation control. We provide experimental results confirming the
sensor's ability to detect three parameters. Contact events have been
detected for signaling transitions between position and force control.
Detecting local skin curvature provided information regarding contact
shape and area. Finally, incipient slip, which is generated by small
micro slips prior to gross slip, has been detected. By sensing all of
these parameters, the utility of the multi-element stress rate sensor
seems promising
Tactile displays are used to convey small-scale force and shape information to the tip of the finger. In this paper, we present a 6×6 tactile shape display that uses commercial radio-controlled (RC) servomotors to actuate an array of mechanical pins. The display has a maximum pin deflection of 2 mm along with a resolution of 4 bits. Pin spacing is 2 mm with a pin diameter of 1 mm. The display can accurately represent frequencies up to 25 Hz for small amplitudes and the slew rate is limited at 38 mm/sec for larger amplitudes
Abstract Acommon tactile display design uses an array of mechanical pins covered ,by a ,rubber layer which acts as a, spatial low pass filter. To characterize the perceptual relationship between this rubber ,layer and shape rendering, we conducted psychophysical experiments to examine,the perception of a vertical line stimulus felt using rubber ,covers of varying thickness and,stiffness. ,We found ,no significant ,change ,in perception for rubber ,thicknesses ranging from 1.5-3.0 mm and for stiffnesses ranging from 45-200 kN/m,.
ABSTRACT This paper reviews the current state of the art and outlook in robotic tactile sensing for real-time control of dextrous manipulation. We begin with an overview of human touch sensing capabilities and draw lessons for robotic manipulation. Next, tactile sensor devices are described, including tactile array sensors, force-torque sensors, and dynamic tactile sensors. The information provided by these devices can be used in manipulation in many ways, such as finding contact locations and object shape, measuring contact forces, and determining contact conditions . Finally, recent progress in experimental use of tactile sensing in manipulation is discussed, and future directions for research in sensing and control are considered. 1
This paper has three main themes: design of a cylindrical tactile sensor, mechanical analysis using planar linear elas ticity models, and signal analysis for determining basic infor mation about the applied contact. An analysis of appropriate sensor depth has been used to allow accurate localization of contacts and reduced spatial aliasing. A comparison has been made between the cylindrical sensor and a simple two- dimensional stress-strain model. The effects of skin thickness and sensor depth on sensitivity are analyzed. Preliminary methods for determining contact location, total force, and tangential force with only normal deflection sensors have been implemented for a line contact.
This paper presents a critical study of the mechanical and electrical properties of the PHANToM haptic interface and improvements to overcome its limitations for applications requiring high-performance control. Target applications share the common requirements of low-noise/granularity/latency measurements, an accurate system model, high bandwidth, the need for an open architecture, and the ability to operate for long periods without interruption while exerting significant forces. To satisfy these requirements, the kinematics, dynamics, high-frequency dynamic response, and velocity estimation of the PHANToM system are studied. Furthermore, this paper presents the details of how the unknown subsystems of the stock PHANToM can be replaced with known, high-performance systems and how additional measurement electronics can be interfaced to compensate for some of the PHANToM's shortcomings. With these modifications, it is possible to increase the maximum achievable virtual wall stiffness by 35%, active viscous damping by 120%, and teleoperation loop gain by 50% over the original system. With the modified system, it is also possible to maintain higher forces for longer periods without causing motor overheating.
Thesis (Ph. D.)--Division of Engineering and Applied Sciences, Harvard University, 1999. Includes bibliographical references (leaves 142-147).
Literature documenting the breadth and quality of prevalent clinical breast examination (CBE) practice is briefly reviewed. A more thorough procedure, emphasizing training in tactile discrimination, is described and illustrated.
Presents an integrated tactile feedback system for a multifingered
robot hand to enable a human operator to feel contacts/interactions
between the robot finger and the environment remotely. The system
presented consists of a finger-shaped tactile sensor measuring contact
areas on the fingertip and a tactile display rendering the contact
information to the human operator. The tactile sensor, designed on the
total internal reflection principle, can capture high resolution and
high quality tactile images on the fingertip. The tactile display with
24 pins spaced at 2.5 mm uses DC solenoids structured in multi-layers to
render the contacts between the fingertip and the environment. We have
integrated the tactile sensor and the tactile display into a
five-fingered robot hand system and verified the performance of the
integrated system by experiments
A teletaction system uses a tactile display to present the user
with information about texture, local shape, and/or local compliance.
Current tactile displays are flat and rigid, and require precise
machining and assembly of many parts. This paper describes the
fabrication and performance of a one-piece pneumatically-actuated
tactile display molded from silicone rubber. Tactor spacing is 2.5 mm
with 1 mm diameter tactor elements. Tactile display compliance ensures
contact between the finger and tactile display at all times. Unlike
previous pneumatic tactile displays, there is no chamber leakage and no
seal friction. A psychophysics experiment showed that a synthetic
grating on the tactile display was perceived as well as a
low-pass-filtered real contact
Teletaction is the transmission of cutaneous information from a
remote tactile sensor to an operator's skin, typically the finger tips.
Ideally, one would like a realistic sensation of directly touching an
object with one's own finger and sense properties such as local shape,
hardness, or texture. Teletaction or tactile feedback is one component
of haptic feedback, the other component being force or kinesthetic
feedback. This paper considers design issues for teletaction systems,
particularly sampling density, aliasing, and the limitations of using an
array of 1-DOF actuators to approximate a continuous stress distribution
on the human finger
This paper discusses the design and testing of an eight-by-eight
tactile capacitive array sensor for detection of sub-millimeter features
and objects, where the entire sensor array is smaller than normal human
spatial resolution of 1 mm. Each square taction is less than 100 μm
on a side, with similar spacing between elements. The structural
material was doped polysilicon with an air gap dielectric of 0.5 μm.
A thin (50-80 μm) protective layer of silicone rubber was adhered to
the polysilicon surface of the sensor to provide interpolation of normal
loads between elements. The sensors were tested and possessed good
spatial uniformity, the capability of detecting millinewton forces, and
good interpolation between elements. The sensors had severe hysteresis
problems, but no detectable proximity effects
One of a surgeon's most important tools is a highly developed
sense of touch. Surgeons rely on sensations from the finger tips to
guide manipulation and to perceive a wide variety of anatomical
structures and pathologies. Unfortunately, new surgical techniques
separate the surgeon's hands from the surgical site. These techniques
include minimally invasive procedures such as laparoscopy and
thoracoscopy, and new techniques involving robotic manipulators. In
these situations the surgeon's perception is limited to visual feedback
from a video camera, or gross motion and force feedback through the
handles of long instruments. The authors are working to develop new
technology to rectify this sensory deficit by relaying tactile
information from the surgical site to the surgeon. They have developed a
variety of tactile sensors that can be mounted in a probe or surgical
instrument. The tactile information provided by these sensors may then
be conveyed through the tactile display devices the authors have
developed to recreate the tactile stimulus directly on the surgeon's
finger tip. By using these remote palpation devices, the surgeon may
regain some of the perceptual and manipulative skills present in
conventional open-incision surgery. Among the tactile feedback
parameters the authors are investigating are force reflection,
vibration, and small-scale shape