Content uploaded by Ana Luisa Trejos
Author content
All content in this area was uploaded by Ana Luisa Trejos
Content may be subject to copyright.
A preview of the PDF is not available
Experimental evaluation of robot-assisted tactile sensing for minimally invasive surgery
Abstract and Figures
The 10 mm incisions used in minimally invasive cancer surgery prevent direct manual palpation of internal organs, making intraoperative tumor localization difficult. A tactile sensing instrument (TSI), that uses a commercially available sensor to measure distributed pressure profiles along the contacting surface, has been developed to facilitate remote tissue palpation. The objective of this research was to assess the feasibility of using the TSI under robotic control to reliably locate underlying tumors. The performance of human and robot manipulation of the TSI to locate tumor phantoms embedded into ex vivo bovine livers was compared. An Augmented Hybrid Impedance Control scheme was implemented on a Mitsubishi PA10-7C robot to perform force/position control during the trials. The results showed that using the TSI under robotic control realized an average 35% decrease in the maximum forces applied, and more than a 50% increase in tumor detection accuracy when compared to manual manipulation of the same instrument. This demonstrates that tumor detection using tactile sensing is highly dependent on the consistent application of forces on the tactile sensing area and that robotic assistance can be of great benefit when trying to localize tumors during minimally invasive surgery.
Figures - uploaded by Ana Luisa Trejos
Author content
All figure content in this area was uploaded by Ana Luisa Trejos
Content may be subject to copyright.
... There are several types of tissue palpation devices with different structural configurations and sensing mechanisms. Among these, force [7][8][9] and pressure [10][11][12][13] sensors Table 1. Comparative of the recent high-performance sensors with different working mechanisms. ...
Palpation is a simple but effective method to distinguish tumors from healthy tissues. The development of miniaturized tactile sensors embedded on endoscopic or robotic devices is key to achieving precise palpation diagnosis and subsequent timely treatment. This paper reports on the fabrication and characterization of a novel tactile sensor with mechanical flexibility and optical transparency that can be easily mounted on soft surgical endoscopes and robotics. By utilizing the pneumatic sensing mechanism, the sensor offers a high sensitivity of 1.25 mbar and negligible hysteresis, enabling the detection of phantom tissues with different stiffnesses ranging from 0 to 2.5 MPa. Our configuration, combining pneumatic sensing and hydraulic actuating, also eliminates electrical wiring from the functional elements located at the robot end-effector, thereby enhancing the system safety. The optical transparency path in the sensors together with its mechanical sensing capability open interesting possibilities in the early detection of solid tumor as well as in the development of all-in-one soft surgical robots that can perform visual/mechanical feedback and optical therapy.
... It should be pointed out that the applied force during palpation should be thresholded to a certain limit, as if too large a force is applied to the soft tissue, safety problems may arise. The average maximum force applied by surgeons when manually palpating soft tissue was proved to be around 4.4N from the in vivo experiments in [20]. Furthermore, in [21] it is also mentioned that autonomous robot palpation can reduce the maximum applied force on the soft tissue by more than 35% compared to manual palpation with similar tools. ...
Force sensing in robotic-assisted minimally invasive surgery (RMIS) is crucial for performing dedicated surgical procedures, such as bilateral teleoperation and palpation. Due to the bio-compatibility and sterilization requirements, a specially designed surgical tool/shaft is normally attached to the sensor while contacting the organ targets. Through this design, the measured force from the sensor usually contains uncertainties, such as noise, inertial force etc., and thus cannot reflect the actual interaction force with the tissue environment. Motivated to provide the authentic contact force between a robotic tool and soft tissue, we proposed a data-driven force compensation scheme without intricate modeling to reduce the effects of force measurement uncertainties. In this paper, a neural-network-based approach is utilized to automatically model the inertial force subject to noise during the robotic palpation procedure, then the exact contact force can be obtained through the force compensation method which cancels the noise and inertial force. Following this approach, the genuine interaction force during the palpation task can be achieved furthermore to improve the appraisal of the tumor surrounded by the soft tissue. Experiments are conducted with robotic-assisted palpation tasks on a silicone-based soft tissue phantom and the results verify the effectiveness of the suggested method.
... The benefit of such an approach in comparison to direct force feedback is that it can be applied in combination with complex surgical manipulation when it is undesirable to disturb the performed manipulation. Visualization methods of stiffness distribution the TSS (tactile sensing system) [6][7][8] and the TIS (tactile imaging system) [9], were developed. However, the visual representation shows only the pressure distribution information within the sensing area. ...
Sense of touch is critical for surgeons to perform tissue palpation, and there are several tactile sensing that can be used to translate this information (Chapter 5). To overcome the loss of touch, which occurs during roboticassisted surgical procedures, methods capable of providing partial haptic feedback and mimicking the physical interaction that takes place between surgical tools and human tissue during surgery have been proposed. This chapter introduces haptic feedback modalities for robot-assisted minimally invasive surgical platforms, such as STIFF-FLOP.
... Tumors are usually much stiffer than healthy soft tissue [2][3][4]. This distinction in stiffness makes it possible to use electronic tactile sensors to non-invasively detect and assess subsurface lumps [5][6][7][8][9][10][11]. ...
Objective:
The mechanical imaging of lumps in tissues via surface measurements can permit the noninvasive detection of disease-related differences in body tissues. We present and evaluate sensing techniques for the mechanical imaging of soft tissues, using a highly compliant electronic sensing array.
Methods:
We developed a mechanical imaging system for capturing tissue properties during automatic- or human-guided palpation. It combines extremely compliant capacitive tactile sensors based on soft polymers and microfluidic electrodes with custom electronic data acquisition hardware, and new algorithms for enhanced tactile imaging by reference to nominal tissue responses.
Results:
We demonstrate that the system is able to image simulated tumors (lumps), yielding accurate estimates of cross-sectional area independent of embedding depth. In addition, as a proof of concept, we show that similar tactile images can be obtained when the sensor is worn on a palpating finger.
Conclusion:
Soft capacitive sensors can accurately image lumps in soft tissue provided that care is taken to control and compensate for electrical and mechanical background signals.
Significance:
The results underline the utility of soft electronic sensors for applications in medical imaging or clinical practices of palpation.
... In the force control setting, the robot was designed to approach the tissue (in the z direction) under force control until the ATI force/torque sensor (F/T Sensor B) located on the robot end-effector registered a pre-determined threshold force. A threshold force of 4 N was determined through preliminary experiments using the PA10-7C robot, as well as from other experiments that found that the average maximum force applied by surgeons when manually palpating tissue was 4.4 N [56,57]. Upon reaching the threshold force of 4 N, the client computer recorded the robot end-effector coordinates (also corresponding to the tissue surface coordinates) and the readings from the TSI, indicating the force profile of the contact made by the robot onto the tissue. ...
The challenges imposed by Minimally Invasive Surgery (MIS) have been the subject of significant research in the last decade. In the case of cancer surgery, a significant limitation is the inability to effectively palpate the target tissue to localize tumor nodules for treatment or removal. Current clinical technologies are still limited and tumor localization efforts often result in the need to increase the size of the incision to allow finger access for direct palpation. New methods of MIS tumor localization under investigation involve restoring the sense of touch, or haptic feedback. The two most commonly investigated modes of haptic perception include kinesthetic and tactile sensing, each with its own advantages and disadvantages. Work in this area includes the development of customized instruments with embedded sensors that aim to solve the problem of limited haptic feedback in MIS. This chapter provides a review of the work to date in the use of kinesthetic and tactile sensing information in MIS for tissue palpation, with the goal of highlighting the benefits and limitations of each mode when used to locate hidden tumors during MIS.
A doctor palpating with his or her fingertips can easily determine whether a patient's tissue contains a solid tumor, regardless of extraneous factors such as the patient's pose. We want to create automatic palpation tools that achieve this same level of robustness, but real tactile sensor signals are sensitive to small physical variations and tend to drift over time. To investigate ways of achieving tactile perceptual invariance, we collected two nearly identical data sets from a SynTouch BioTac palpating at 10,260 points distributed across an artificial tissue sample made from silicone rubber with rigid embedded lumps. Each palpation interaction was distilled down to 20 numbers (19 electrode impedances and one DC pressure). Fitting a multivariate Gaussian distribution to examples recorded far from the lumps achieved near-perfect accuracy, precision, and recall in recognizing other background examples recorded the same day. These models performed miserably, though, when used to classify the other data set. Inspired by robust perceptual methods from computer vision, we transformed the 20 tactile sensor array readings into 190 binary pairwise comparisons and used the log likelihood of observing a given binary pattern to determine whether an example was far from the embedded lumps. This novel approach achieved accuracy, precision, and recall of about 80% on reserved testing data from the same day and yielded roughly similar levels of accuracy and recall with excellent precision on the data set recorded on the other day. Pairwise comparisons between tactile sensor array readings may hold promise for achieving robust automatic tactile perception.
Hepatic tumors appear as stiff inclusions within the surrounding soft, healthy tissue. In open surgery they are searched for by manual palpation with the gloved fingertip. However, to exploit the benefits of MIS it is mandatory to implement a substitution for the human sense of touch. Therefore, a tactile instrument has been developed with the aim of enlarging the sensing area at the tool tip once it enters the abdominal cavity through the trocar. The provision of a large sensitive surface enables the detection of nearly all sizes of tumors and decreases the time needed for the performance of this task. A prototype was manufactured by laser sintering in PA serving as a carrier for an existing flexible silicone sensor. Automated as well as manual subject palpation tests have shown that a prototypical instrument with a laterally opening lid would be a suitable device for tumor detection in laparoscopic liver surgery.
This paper presents a novel portable passive robotic platform for three-dimensional scanning (3DS) of soft tissue, capable to evaluate mechanical properties and geometry in ex vivo condition. The platform comprises six degrees of freedom (DOF) passive robotic arm (Phantom Omni), a data acquisition system and a set of stiffness probes for force and stiffness measurement. The performance of the developed platform was validated by sliding indentation and uniaxial tissue indentation measurements on silicone phantoms, porcine organs and human prostates. The results show that the platform can perform effective measurements of soft tissue mechanical properties and help surgeons to identify embedded tumours.
The 10 mm incisions used in minimally invasive cancer surgery prevent the direct palpation of internal organs, making intraoperative tumor localization difficult. A tactile sensing instrument (TSI), which uses a commercially available sensor to measure distributed pressure profiles along the contacting surface, has been developed to facilitate remote tissue palpation. The objective of this research is to assess the feasibility of using the TSI under robotic control to reliably locate underlying tumors while reducing collateral tissue trauma. The performance of humans and a robot using the TSI to locate tumor phantoms embedded into ex vivo bovine livers is compared. An augmented hybrid impedance control scheme has been implemented on a Mitsubishi PA10-7C to perform the force/position control used in the trials. The results show that using the TSI under robotic control realizes an average 35% decrease in the maximum forces applied and a 50% increase in tumor detection accuracy when compared to manual manipulation of the same instrument. This demonstrates that the detection of tumors using tactile sensing is highly dependent on how consistently the forces on the tactile sensing area are applied, and that robotic assistance can be of great benefit when trying to localize tumors in minimally invasive surgery.
This paper describes a system for finding hidden arteries in remote and inaccessible locations. A tactile array sensor is located in the end of a long, 10 mm diameter probe. In a surgical application, the surgeon presses the sensor against the tissue of interest, and electronics read out the pressure distribution across the contact area at a high rate. A computer captures this information and processes the signal to find the periodic pressure variations due to the pulsatile arterial blood flow. The results are then displayed on a video monitor for the surgeon's use. This device is the first part of a complete remote palpation system that will convey tactile information to the surgeon's finger tips using shape, pressure, and vibration displays.
Early detection and removal of small pulmonary nodules significantly improves long term survival rates for patients with lung cancer. To aid in the localization of these tumors during video assisted thoracoscopic surgery (VATS), a tactile imaging system (TIS) is presented. The system consists of a capacitive array sensor mounted on a minimally invasive surgical probe that is integrated with the thoracoscopic imaging. A vision-based algorithm localizes the probe in the live video and overlays a registered pseudo-color map of the measured pressure distribution on the streaming images. The surgeon can locate the hard nodules by scanning the tactile sensor head across the surface of the lung and observing the spatial variation in contact pressures caused by the elasticity differences in the underlying tissue. A validation experiment was conducted to compare the system to a current localization technique using a rigid rod. Results indicate that subjects could localize stiff lumps in lung phantoms more quickly and accurately using the TIS.
This paper is concerned with robust position and contact force control for 7-DOF redundant robot arms. An outer-inner loop controller, called the augmented hybrid impedance control scheme is developed. A 6-DOF force/torque sensor is used to measure the interaction forces. These are fed back to the outer-loop controller that implements either a force or an impedance controller in each of the 6 DOF of the tool frame. The force controller is provided with a force set point, and desired inertia and damping are introduced in the force control loop to improve transient performance. The inner loop consists of a Cartesian-level potential difference controller, a redundancy resolution scheme at the acceleration level, and a joint-space inverse dynamics controller. Experimental results for two 7-DOF robot arms (redundant, dextrous, isotropically enhanced, seven-turning pair robot (REDIESTRO) and Mitsubishi PA10-7C) are given to illustrate the performance of the force control strategy. A successful application of the proposed scheme to a surface cleaning task is described using the REDIESTRO, while position and force tracking experiments are described for the Mitsubishi PA10-7C robot.
Tactile imaging is a newly developed mechanical sensing technology for documenting the properties of hard lumps contained in soft tissue. An examiner strokes a scan head across tissue that contains a mass and images of the distributed contact pressure between the head and the tissue are recorded. We have developed models that predict these pressure distributions from geometric and material properties. We then use inversion algorithms developed from these models to extract lump size and shape. In a limited clinical trial on 24 surgical patients, lump size was estimated with less than 17% mean absolute error when compared with ex-vivo size measurements. This is more than twice as accurate as either clinical breast examination or ultrasound examination of the same lumps. This result demonstrates that tactile imaging has the potential to improve the accuracy of clinical breast examination.
Locating arteries hidden beneath superficial tissue can be a difficult task in minimally invasive surgery. This paper reports the development of a system that finds the paths of arteries using tactile sensing. The surgeon begins by using the surgical robot to place the tactile sensor instrument on a known artery location. Signal processing algorithms locate the artery from its pulsatile pressure variation. An adaptive extrapolation algorithm then generates predicted locations for the artery based on previous measurements. After moving to the predicted location, if the artery is not located then a backtracking mechanism moves the sensor towards previously detected locations. Tests with model arteries show good tracking ability for circular arcs with curvatures as small as 80 mm, although problems with compliance in the system result in occasional loss of the artery path. Preliminary tests demonstrate the ability to transcutaneously track the radial artery in the human wrist.
Combining teletaction systems with telemanipulation systems promises to enhance task performance when interacting with remote environments. However, the force scaling inherent in the telemanipulation system affects the ability of the user to control the exploration force. The quality of the tactile signal is therefore impacted, affecting performance in tasks that benefit from spatially distributed force information. We compare performance localizing an embedded lump in a compliant environment using a telemanipulated teletaction system versus a directly manipulated teletaction system. Lump localization accuracy was found to be the same; however, time required to localize the lump was up to 150% longer for the telemanipulation trials. Based upon our results, we conclude that the ability to maintain an appropriate force in the remote environment is necessary to take full advantage of the spatially distributed force information from the tactile sensor.
For fine surgical operation by slave robot, master robot needs the information of the contact force loaded on the tip of the surgical instrument. One of major problems in sensing the force is frictional disturbance between the trocar and the instrument. One of authors proposed a new method named overcoat method where the whole instrument is supported by force sensors. This paper tried to construct a force measuring system with the basic type of the overcoat method. It discussed about the compensations of gravity and acceleration force of the instrument in relations to the pose and motion of the instrument. The tried system measured the tip-loaded force in sensitivity about 0.05 N.