Figure 1 - uploaded by Sandro Queirós
Content may be subject to copyright.
Source publication
Laparoscopy is a surgical procedure on which operations in the abdomen are performed through small incisions using several specialized instruments. The laparoscopic surgery success greatly depends on surgeon skills and training. To achieve these technical high-standards, different apprenticeship methods have been developed, many based on in vivo tr...
Contexts in source publication
Context 1
... laparoscopic procedures, surgical instruments are controlled by a surgeon, whose movements are transmitted through the incision point, via a trocar (instrument shaft), to the tip of the instrument. As a result of this motion constraint, instead of six DOFs, the laparoscopic surgical tool only presents four fundamental DOFs: translation around the instrument axis (insertion/withdrawal – 1 st DOF); rotation around instrument axis (roll – 2 nd DOF); left/right and forward/backward rotation around incision point (yaw and pitch - 3 rd and 4 th DOFs, respectively). Additionally, to track the forceps movements, one has to consider the rotation around its axis (5 th DOF), as well as its controlled behavior (open/close – 6 th DOF) (Fig. 1). According to the work of Chmarra et al. [18], the development of a device which can detect and measure the MIS instruments motion with reliable results for a realistic simulator has to fulfill the following requirements set: 1. Ability to detect the four instrument fundamental DOFs – translation and rotation around instrument axis, as well as yaw and pitch around the incision point; 2. Allow the use of real surgery instruments in a box- trainer, with or without VR environment combination; 3. Present the appropriate accuracy and sensibility standards, as close as possible to reality; 4. Low-cost and easy to produce, in order to make it affordable for every medical facility and private use; 5. PC “Plug and Play” feature – ready to use; 6. Small size, for an easier carrying and mounting. In order to meet the first requirement, the motion tracking system was conceived using a set of sensors appropriately combined (Fig. 2). Since it is fundamental to detect the instrument spatial orientation, associated with 2 nd , 3 rd and 4 th DOFs, a six DOFs Inertial Measurement Unit (IMU) was associated with a magnetometer (HMC5843). The IMU integrates (a) a tri-axis accelerometer (ADXL335), which allows the acceleration measurement in any direction in space, and (b) a tri-axis gyroscope (LPR530AL for pitch and roll, and LY530ALH for yaw), which consolidates and corrects the accelerometer data, while providing additional information regarding the three spatial axes. A magnetometer was also used in order to correct the gyroscope and accelerometer data, serving primarily as a reference for yaw measurement. A simplified filtering algorithm inspired by Kalman filter [21] was implemented to combine the accelerometer and gyroscope outputs and obtain accurate information about spatial orientation. Since there is a linear relationship between the handle opening and the forceps at the tip of the instrument, a flex sensor (FSL0093103ST) was used to assess the instrument’s handle opening. The flex sensor allows the determination of the forceps opening angle, through the conversion of a variable resistance to a voltage output (Fig. 3). This allows the measurement of the 6 th DOF. Even though a solution for capturing the 1 st DOF movement is crucial for a complete prototype, one has deliberately decided not to address this movement in the current prototype, and therefore the developed digital game does not depend on it to achieve its training results. The 5 th DOF of the instrument is actually rarely used in practice, because during a surgery, both surgeon hands are used to manipulate different instruments, thus making it very difficult to perform such movement. Given the reduced importance of this DOF in training, one preferred to simplify the current prototype and not to consider this aspect. For the interpretation of the proposed set of sensors, the second requirement (real surgery instruments use) is fulfilled, being an advantage when compared to other existing simulators, which only permit altered or unrealistic MIS instrument devices for simulation. The low-cost requirement is also met as a set of inexpensive and very common sensors is employed. In addition, the sensors can be removed from the laparoscopic instrument, using a Velcro system. This feature allows the instrument sterilization, a very important requirement, given that one is resorting to real instruments used in the operating room and, in most cases, used by several surgeons. Second, this also allows the prototype to be sold separately, without need to include the actual surgical instruments, thus lowering the price associated with a possible business solution. The small-sized sensors satisfy the sixth requirement. In the final envisioned commercial product, a box-trainer will be developed to simulate the patient’s body and to mimic the incision point with several perforated ball joints, providing a completed training set (Fig. 4). The conversion and transmission of the data obtained by the sensors to the computer is essential to integrate the system in a virtual environment. To accomplish this, the ATmega164 microcontroller was used as a physical interface. Since the magnetometer communicates through I 2 C (requiring two pins) and a set of seven other analogical inputs must be read at each update, one from the flex sensor and three from each of the remaining sensors (accelerometer and gyroscope), nine microcontroller pins are used for data transmission. The used microcontroller has a 10-bit ADC integrated module which converts each analogical voltage input into an output value in the range of 0 to 1023. Therefore, a conversion of the analogical-to-digital readings to physical units (i x ) (g-units for accelerometer and degrees/second for gyroscope) is achieved through the following formula: i x = a 1023 x ×V ss dd – zl (1) Inspired by the Kalman filter, a simplified algorithm, combining data from accelerometer and gyroscope was used to achieve accurate ...
Context 2
... -1 st DOF); rotation around instrument axis (roll -2 nd DOF); left/right and forward/backward rotation around incision point (yaw and pitch -3 rd and 4 th DOFs, respectively). Additionally, to track the forceps movements, one has to consider the rotation around its axis (5 th DOF), as well as its controlled behavior (open/close -6 th DOF) (Fig. ...
Similar publications
Surgical residents of endo-neurosurgery are required to develop several surgical skills such as eye-hand coordination, ability to use both hand coordination, and depth perception. During the process of gaining these skills, there are several effective factors on the individual performance such as gender, nature of the scenarios and hand condition t...
This paper presents an application of eye tracking approach to cognitive task analyses of the trainees in a simulation environment. Eye-tracking technology has been shown environments. In that sense in the earlier studies it is stated that, novice surgeons often hand-eye coordination problems, limited work space and the lack of tactile sensation. T...
During laparoscopy, the monitor is usually placed near the operating table, at eye level, which significantly affects hand-eye coordination. First, it is impossible for the surgeon to simultaneously observe the operative field and hand movement. Second, the axis of view of the endoscope rarely matches the natural axis of the surgeon's sight: it res...
Laparoscopic surgery (LS) has revolutionized traditional surgical techniques introducing minimally invasive procedures for diagnosis and local therapies. LSs have undeniable advantages, such as small patient incisions, reduced postoperative pain and faster recovery. On the other hand, restricted vision of the anatomical target, difficult handling o...
Citations
... Laparoscopic Surgery is a type of surgery where contrary to the conventional open surgery the surgeon makes tiny incisions that are only big enough to insert special surgical instruments and a laparoscope within a trocar [1]. These types of surgeries have grown rapidly and become more preferred by patients because of their numerous advantages [2]. ...
Because of the increasing use of laparoscopic surgeries, robotic technologies have been developed to overcome the challenges these surgeries impose on surgeons. This paper presents an overview of the current state of surgical robots used in laparoscopic surgeries. Four main categories were discussed: handheld laparoscopic devices, laparoscope positioning robots, master–slave teleoperated systems with dedicated consoles, and robotic training systems. A generalized control block diagram is developed to demonstrate the general control scheme for each category of surgical robots. In order to review these robotic technologies, related published works were investigated and discussed. Detailed discussions and comparison tables are presented to compare their effectiveness in laparoscopic surgeries. Each of these technologies has proved to be beneficial in laparoscopic surgeries.
... However, these sensors are not able to measure rotation, and are only able to quantify linear acceleration as the posture changes. Gyroscopes solve this issue by measuring the rotation, and the magnetometers compensate for the drift in the gyroscope measurements [24][29] [30]. However, the presence of ferromagnetic materials in the proximity of the magnetometers may cause disturbances in the magnetic field [31][32] [33]. ...
... The use of the three types of sensors, commonly known as IMU, has made the biometric data acquisition process easier, faster and more accurate and robust. These sensors require calibration, due to errors in the signal acquisition from the gyroscope and disturbances in the magnetic field measured by the magnetometer [30][31] [32][33] [34]. Usually, the data from the different sensors is fused using different algorithms [35] [36]. ...
... Among a number of developed applications, a major category of serious games aims at healthcare and wellbeing. For instance, Queiros et al.[6]present a serious game to train newly qualified surgeons. The hardware system of this game involves a variety of sensors, supporting trainees with instructions and assessment of their performance in laparoscopic surgery. ...
Currently, serious games are increasingly used for education and training purposes. Although the primary purpose of serious games is education, entertainment plays a crucial role in effective engagement of serious games. A variety of methods are proposed for serious game development. However, designing serious games to train users of complex systems is a non-trivial task. Consequently, enormous effort is required to integrate pedagogical content and game elements. To alleviate this problem, this paper presents a novel framework to convert the system design of software engineering methodologies into the design of serious games. Based on the descriptive documentation of system design, the proposed framework generates corresponding design diagrams to provide guidance for serious game developers. A converter for automatic diagram conversion is also implemented. Moreover, a case study, together with a survey, is conducted to confirm the effectiveness of the proposed framework.
... Games are already being used as a teaching strategy in medical education, mainly to review and reinforce lecture material for undergraduate medical students [9][10][11][12][13]. A new generation of students and professionals, familiar since childhood with the latest technologies in communication and information, appreciate the value of technology-related interactive experiences as a source of learning and personal development and there are some data indicating that medical students hold favourable views about the use of video games and related new media technology in medical education [14]. ...
We are creating a serious game in Oncobiology that will allow the player to support clinical decisions with biological data, with the aim of promoting hypothesis-driven clinical research. Serious games are becoming an important subject of research as they are a tool with increasing potential to the acquisition of knowledge and training of skills. Physicians in general and oncologists in particular are in constant need for continuous learning. In cancer management, biology plays a crucial role, as new advancements from basic science are constantly being translated into clinical practice. This paper documents the development of the prototype of the Oncobiology Game.
This paper presents the modeling and control process of recently developed manipulator with 3 limbs having prismatic-universal-universal (3-PUU) joints. It has 3 degrees of freedom (3-DOF), consisting of 2 rotational DOFs and 1 translational DOF (2R1T). To avoid the computational complexity of solving the manipulator’s kinematics in real-time application, two artificial neural networks (ANNs) are trained to estimate the forward and inverse kinematics solutions. Training and testing results show that the developed ANNs have great prediction capabilities. The manipulator’s dynamic model is deduced using MATLAB Simscape environment. Control schemes are investigated starting with motor control, first using PID controller, and then adding feed-forward control which greatly improved the motors’ response. Closed-loop trajectory control based on Cartesian space feedback of the manipulators’ position and orientation and inverse kinematics ANN is then studied. The closed-loop control scheme can enhance the system’s performance, eliminating the error resulting from any change in the manipulator’s actual model due to manufacturing or assembly defects. Simulation results of a defected manipulator model show that the closed-loop control scheme improved the manipulator’s trajectory tracking capability, reducing the z-axis position error by 89.23% and the orientation error by 86.76% and 82.83% about x-axis and y-axis directions respectively.
Computer simulation of surgical environments is always oversimplified in terms of physical behavior due to the complexity of the tissues and interactions involved, which cannot be fully simulated in real time. To better manage this trade-off between efficiency and effectiveness, we present a hybrid and adaptive environment that combines a set of methods to achieve higher accuracy and performance. Our approach merges physics-based deformation methods (Finite Elements and Mass Spring) with a non-physical method (Green Coordinates) to approximate more coarsely the behavior when the focus of the interaction is away, and more precisely during direct interaction. We experimentally demonstrate that the computational complexity of the simulation with our method does not increase with the number of objects being simulated. With our approach, a virtual surgery environment with many dynamic organs can be computed at interactive rates for the first time.
In the past few years, laparoscopic surgery has been widely performed in abdomen or pelvis surgery. It is a minimally invasive surgery (MIS) technique to conduct the operation only through small incisions. The advantages of MIS include smaller incisions, less bleeding and shorter recovery time. However, surgeons can only perform laparoscopic surgery through the video from a laparoscope and surgical instruments. This may increase the complication and difficulties while fulfilling MIS. Therefore, before conducting the real operation, surgeons are required to receive multiple related trainings to improve their surgical skills, especially in stabilization and hand-eye coordination. These skills can be estimated by analyzing the paths of the surgical instruments while simulating the operation processes. Besides, the locating information of surgical instruments can be utilized for simulating virtual reality training applications. Herein, the objective of this paper is to develop an image-based instruments locator. The system comprises a stereo camera, two surgical instruments with passive markers, and a FPGA development board. The instant location of the markers can be determined based on proposed image processing framework. The errors of the provided locating method can reach to 0.21±0.17 cm in X axis, 0.10±0.10 cm in Y axis, and 0.29±0.22 cm in Z axis.
Laparoscopic surgery (LS) has revolutionized traditional surgical techniques introducing minimally invasive procedures for diagnosis and local therapies. LSs have undeniable advantages, such as small patient incisions, reduced postoperative pain and faster recovery. On the other hand, restricted vision of the anatomical target, difficult handling of the surgical instruments, restricted mobility inside the human body, need of dexterity to hand-eye coordination and inadequate and non-ergonomic surgical instruments may restrict LS only to more specialized surgeons. To overcome the referred limitations, this work presents a new robotic surgical handheld system - the EndoRobot. The EndoRobot was designed to be used in clinical practice or even as a surgical simulator. It integrates an electromechanical system with 3 degrees of freedom. Each degree can be manipulated independently and combined with different levels of sensitivity allowing fast and slow movements. As other features, the EndoRobot has battery power or external power supply, enables the use of bipolar radiofrequency to prevent bleeding while cutting and allows plug-and-play of the laparoscopic forceps for rapid exchange. As a surgical simulator, the system was also instrumented to measure and transmit, in real time, its position and orientation for a training software able to monitor and assist the trainee's surgical movements.
