Fig 2 - uploaded by Bin Zheng
Content may be subject to copyright.
Source publication
Augmented Reality has been utilized for surgical training. During the implementation, displaying instructional information at the right moment is critical for skill acquisition. We built a new surgical training platform combining augmented reality system (HoloLens, Microsoft) with an eye-tracker (Pupil labs, Germany). Our goal is to detect the mome...
Context in source publication
Similar publications
Background
The difficulty of laparoscopic procedures and the specific psychomotor skills required support the need for a training system for intensive and repetitive practice to acquire the specific skills. The present VR training systems have some limitations with respect to the soft tissue models in the training system. This is associated with th...
Simulation training is well integrated into laparoscopic surgical training.
However, data on an objective evaluation of a structured and stepwise approach to simulation training remain scarce.
In cataract surgery, the operation is performed with the help of a microscope. Since the microscope enables watching real-time surgery by up to two people only, a major part of surgical training is conducted using the recorded videos. To optimize the training procedure with the video content, the surgeons require an automatic relevance detection ap...
Simulation-based training has been widely used in medical education. More specifically, various systems for minimally invasive surgery training have been proposed in the past two decades. The aim of this article is to review and summarize the existing simulation-based training systems for laparoscopic surgery in terms of their technical realization...
Background
It is currently not known how many trainees leave vascular surgery, and their reasons for doing so are unclear. This paper is the first to publish the number of UK trainees leaving the training programme and interrogates their reasons for doing so.
Methods
An email survey was distributed to current and recent Training Programme Director...
Citations
... In medicine, eye tracking and MR frameworks are used to train medical students and professionals on surgical skills [Lu et al. 2020]. In science, developing such frameworks enables the evaluation of novice and expert problem-solving skills during experiments and training novices based on the results [Sonntag and Bodensiek 2021]. ...
... Visual tracking is one of the most popular topics in the field of computer vision, with widespread applications in behavior analysis [1,2], human-computer interaction [3,4], video surveillance [5,6], autonomous driving [7,8], augmented reality [9,10], and more. The core task of visual tracking is to accurately determine the position and state of the target object in each frame of a video. ...
Visual tracking fundamentally involves regressing the state of the target in each frame of a video. Despite significant progress, existing regression-based trackers still tend to experience failures and inaccuracies. To enhance the precision of target estimation, this paper proposes a tracking technique based on robust regression. Firstly, we introduce a novel robust linear regression estimator, which achieves favorable performance when the error vector follows i.i.d Gaussian-Laplacian distribution. Secondly, we design an iterative process to quickly solve the problem of outliers. In fact, the coefficients are obtained by Iterative Gradient Descent and Threshold Selection algorithm (IGDTS). In addition, we expend IGDTS to a generative tracker, and apply IGDTS-distance to measure the deviation between the sample and the model. Finally, we propose an update scheme to capture the appearance changes of the tracked object and ensure that the model is updated correctly. Experimental results on several challenging image sequences show that the proposed tracker outperformance existing trackers.
... Coupling AR-based visualizations of a patient's anatomy with additional contextual information about the surgical task has the potential to reduce the surgeon's cognitive workload and improve the outcomes of AR-supported surgeries [61]. Prior studies show that such contextual guidance can be implemented in surgical training by displaying instructional information for skill acquisition through tracking the user's eye gaze to detect the surgical performance [33], predicting currently performed surgical task to visualize information corresponding to the task through a neural network [10], or providing post-trial feedback to the users including scores on users' performance and areas that need more practice [66]. In intra-operative contextual guidance, different types of domain-specific contextual guidance have been demonstrated for AR-supported endodontic [51], dental implant [26], and orthopedic [57] surgeries. ...
External ventricular drain (EVD) is a common, yet challenging neurosurgical procedure of placing a catheter into the brain ventricular system that requires prolonged training for surgeons to improve the catheter placement accuracy. In this paper, we introduce NeuroLens, an Augmented Reality (AR) system that provides neurosurgeons with guidance that aides them in completing an EVD catheter placement. NeuroLens builds on prior work in AR-assisted EVD to present a registered hologram of a patient's ventricles to the surgeons, and uniquely incorporates guidance on the EVD catheter's trajectory, angle of insertion, and distance to the target. The guidance is enabled by tracking the EVD catheter. We evaluate NeuroLens via a study with 33 medical students and 9 neurosurgeons, in which we analyzed participants' EVD catheter insertion accuracy and completion time, eye gaze patterns, and qualitative responses. Our study, in which NeuroLens was used to aid students and surgeons in inserting an EVD catheter into a realistic phantom model of a human head, demonstrated the potential of NeuroLens as a tool that will aid and educate novice neurosurgeons. On average, the use of NeuroLens improved the EVD placement accuracy of the year 1 students by 39.4%, of the year 2-4 students by 45.7%, and of the neurosurgeons by 16.7%. Furthermore, students who focused more on NeuroLens-provided contextual guidance achieved better results, and novice surgeons improved more than the expert surgeons with NeuroLens's assistance.
... Eye tracking is the process of determining and measuring a user's line of sight to the object they are observing. It is an active area of research and has many applications in multiple areas including clinical applications [1], surgical training [2], visual systems such as shopping research [3], navigation [4], psychology [5], emotion recognition [6], museum visits [7], and more. ...
In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system’s mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user’s point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches.
... Furthermore, eye trackers can enhance surgical navigation by providing real-time feedback on the location of surgical instruments relative to ocular structures. In training and education, eye trackers are employed to evaluate and refine the performance of ophthalmology residents and medical students during surgical simulations or live surgeries, assisting them in developing superior situational awareness and procedural skills [37][38][39]. Applications of eye trackers contribute to improved surgical outcomes and patient care in ophthalmology. ...
Purpose
Stereopsis is the ability to perceive depth using the slightly different views from two eyes. This study aims to conduct innovative stereopsis tests using the objective data outputted by eye tracking technology.
Methods
A laptop and an eye tracker were used to establish the test system. Anaglyphic glasses were employed to execute the stereopsis assessment. The test symbol employed was devised to emulate the quantitative measurement component of the Random Dot 3 Stereo Acuity Test. Sub-pixel technology was used to increase the disparity accuracy of test pages. The tested disparities were: 160″, 100″, 63″, 50″, 40″, 32″, 25″, 20″, 16″, and 12.5″. The test was conducted at a distance of 0.65m. Conventional and eye tracking stereopsis assessments were conducted on 120 subjects. Wilcoxon signed-rank test was used to test the difference, while the Bland-Altman method was used to test the consistency between the two methods.
Results
The Wilcoxon signed-rank test showed no significant difference between conventional and eye tracking thresholds of stereopsis ( Z = −1.497, P = 0.134). There was a high level of agreement between the two methods using Bland- Altman statistical analysis (The 95 per cent limits of agreement were −0.40 to 0.47 log arcsec).
Conclusions
Stereoacuity can be evaluated utilizing an innovative stereopsis measurement system grounded in eye tracking technology.
... It has also been reported that even after wearing the HMD, differences in the sense of physical distance to objects within the field of view can be perceived [19]. Furthermore, fatigue has been observed when using HoloLens as an HMD due to its 500 g weight, making it less suitable for long-term training [20]. ...
Extended Reality (XR) applications are considered useful for skill acquisition in dental education. In this study, we examined the functionality and usefulness of an application called “SR View for Endo” that measures root canal length using a Spatial Reality Display (SRD) capable of naked-eye stereoscopic viewing. Three-dimensional computer graphics (3DCG) data of dental models were obtained and output to both the SRD and conventional 2D display devices. Forty dentists working at the Kanagawa Dental University Hospital measured root canal length using both types of devices and provided feedback through a questionnaire. Statistical analysis using one-way analysis of variance evaluated the measurement values and time, while multivariate analysis assessed the relationship between questionnaire responses and measurement time. There was no significant difference in the measurement values between the 2D device and SRD, but there was a significant difference in measurement time. Furthermore, a negative correlation was observed between the frequency of device usage and the extended measurement time of the 2D device. Measurements using the SRD demonstrated higher accuracy and shorter measurement times compared to the 2D device, increasing expectations for clinical practice in dental education and clinical education for clinical applications. However, a certain percentage of participants experienced symptoms resembling motion sickness associated with virtual reality (VR).
... There has been an exponential technological advancement, especially in the last five years, with the introduction of low-cost VR headsets such as Vive and Quest, AR-capable smartphones and tablets, and Mixed Reality headsets such as Hololens and Magic Leap. Such advancements have enabled designers to create VR, MR, and AR-based training simulators for a wide range of applications ranging from less invasive laparoscopic surgery to complex procedures such as brain and cardiac surgery [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15]. ...
In this paper, the effect of tactile affordance during the design of Extended Reality (XR) based environments is presented. Tactile affordance is one of the Human eXtended Reality Interaction (HXRI) criteria which help lay the foundation for human-centric XR-based training environments. XR-based training environments developed for two surgical procedures have been used to study the role of tactile affordance. The first XR environment is developed for the Condylar plating surgical procedure which is performed to treat the fractures of the femur bone and the second XR environment is developed to train users in endotracheal intubation. Three studies have been conducted to understand the influence of different interaction methods to elevate tactile affordance in XR-based environments. The studies and the results of the studies have been exhaustively discussed in this paper.KeywordsAffordanceTactile AffordanceHuman eXtended Reality InteractionHXRIVirtual RealityExtended Reality (XR)
... The use of VR generates accessible metadata, e.g. head movement data required for head-based rendering can be easily recorded. In addition to the rotational and translational metadata, that can be accessed through the build-in sensors of the HMD and its motion-tracked controllers, eye tracking in VR is applied in many areas of cognitive research: For instance focusing on spatial orientation (König et al., 2021), medical training (Lu et al., 2020) or marketing (Khatri et al., 2020). Combining spatial and temporal eye data allows for the allocation of eye movement events such as fixations (i.e. ...
Combining eye tracking and virtual reality (VR) is a promising approach to tackle various applied research questions. As this approach is relatively new, routines are not established yet and the first steps can be full of potential pitfalls. The present paper gives a practice example to lower the boundaries for getting started. More specifically, I focus on an affordable add-on technology, the Pupil Labs eye tracking add-on for the HTC Vive. As add-on technology with all relevant source code available on GitHub, a high degree of freedom in preprocessing, visualizing, and analyzing eye tracking data in VR can be achieved. At the same time, some extra preparatory steps for the setup of hardware and software are necessary. Therefore, specifics of eye tracking in VR from unboxing, software integration, and procedures to analyzing the data and maintaining the hardware will be addressed. The Pupil Labs eye tracking add-on for the HTC Vive represents a highly transparent approach to existing alternatives. Characteristics of eye tracking in VR in contrast to other head-mounded and remote eye trackers applied in the physical world will be discussed. In conclusion, the paper contributes to the idea of open science in two ways: First, by making the necessary routines transparent and therefore reproducible. Second, by stressing the benefits of using open source software.
... Augmented reality is an enhanced version of reality created by using technology to overlay digital information (texts, graphic images, or 3D contents) into the user's direct vision of the real world [12,[38][39][40][41]. AR allows a person to see the real world, but it is overlaid with this layer of digital content in real time [25,33,42,43]. ...
Unlabelled:
Extended reality (XR) has emerged as an innovative simulation-based learning modality. An integrative review was undertaken to explore the nature of evidence, usage, and effectiveness of XR modalities in medical education. One hundred and thirty-three (N = 133) studies and articles were reviewed. XR technologies are commonly reported in surgical and anatomical education, and the evidence suggests XR may be as effective as traditional medical education teaching methods and, potentially, a more cost-effective means of curriculum delivery. Further research to compare different variations of XR technologies and best applications in medical education and training are required to advance the field.
Supplementary information:
The online version contains supplementary material available at 10.1007/s40670-022-01698-4.
... Pre (2) Qian et al. [167] and Galati et al. [62] Intra (5) Andress et al. [6], Deib et al. [43], Fotouhi et al. [57], Liu et al. [121], and Al Janabi et al. [3] Video (3) Intra (3) Qian et al. [164], Al Janabi et al. [3], and Qian et al. [168] Other ( [194], Van Gestel et al. [206,207], and Velazco-Garcia et al. [210] Planning (29) Pre (29) Carbone et al. [29], Condino et al. [37], Kobayashi [194], Suzuki et al. [200], Van Gestel et al. [206], and Wesselius et al. [215] Vital Signs (2) Pre (1) Qian et al. [167] Intra (1) Sirilak et al. [190] Records (2) Pre (1) Perkins et al. [159] Intra (1) Deib et al. [43] 3D non-medical [171], and Velazco-Garcia et al. [210] Intra (1) Angelopoulos et al. [7] 2D nonmedical (11) Video (7) Pre (3) Sun et al. [199], Lu et al. [127], and Schoeb et al. [187] Intra (4) Sirilak et al. [190], Mitsuno et al. [139], Glick et al. [71], and Cofano et al. [36] Docs (4) Pre (4) Hanna et al. [83], Sirilak et al. [190], Galati et al. [62], and Rositi et al. [176] other nonmedical ...
The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality display, is the main player in the recent boost in medical augmented reality research. In medical settings, the HoloLens enables the physician to obtain immediate insight into patient information, directly overlaid with their view of the clinical scenario, the medical student to gain a better understanding of complex anatomies or procedures, and even the patient to execute therapeutic tasks with improved, immersive guidance. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021, were attention is shifting towards it's successor, the HoloLens 2. We identified 171 relevant publications through a systematic search of the PubMed and Scopus databases. We analyze these publications in regard to their intended use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation. We find that, although the feasibility of using the HoloLens in various medical scenarios has been shown, increased efforts in the areas of precision, reliability, usability, workflow and perception are necessary to establish AR in clinical practice.
