Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Hardware components of the TrueDepth camera system, which is integrated in the upper part of the smartphone. A Dot Projector throws over 30,000 infrared dots onto the face of the user, and the dots and an infrared image are captured via an infrared camera to create a depth map of the face (modified from [20]).
Source publication
Background
For the classification of facial paresis, various systems of description and evaluation in the form of clinician-graded or software-based scoring systems are available. They serve the purpose of scientific and clinical assessment of the spontaneous course of the disease or monitoring therapeutic interventions. Nevertheless, none have bee...
Context in source publication
Context 1
... exact registration of facial data works via the projection of 30,000 infrared points, which are recorded at 60 times per second via an infrared camera and processed by a neural engine of the smartphone chip. An infrared flood illuminator adds infrared light to enable measurements also in the dark (Figure 1). An accurate 3D depth map of the face is calculated from the measurement data, which is converted together with the infrared image to a mathematical representation of the user's facial features. ...
Similar publications
Temporal bone rhabdomyosarcoma is an aggressive entity that simulates chronic otitis ear infection. It is the most common soft tissue sarcoma amongst pediatric patients. Herein, we would like to report a case of temporal bone rhabdomyosarcoma involving a 2-year-old boy who presented with a one-month history of otorrhea with facial asymmetry. Early...
Citations
... Erfahrungen mit einem App-Prototyp bei der Beurteilung gesunder Teilnehmer unter Verwendung einer TrueDepth-Kamera (Smartphone-Kamera, welche Tiefeninformationen liefert) zeigen ein erhebliches Potenzial für die App-gestützte, automatisierte Einstufung von Gesichtsbewegungsstörungen. Prospektive klinische Studien zur Evaluation und zur Korrelation mit gängigen Gesichtsscores stehen aus [29]. ...
Zusammenfassung
Auch in der Medizin gewinnt die Digitalisierung immer schneller an Bedeutung. Die COVID-19-Pandemie beschleunigte diesen Prozess zusätzlich, und die Politik versucht, Rahmenbedingungen für einen erfolgreichen Wissenstransfer und eine bessere digitale medizinische Versorgung zu schaffen. Im vorliegenden Artikel wird die Rolle der Telemedizin bei der Behandlung von Patientinnen und Patienten mit Fazialisparese erörtert. Eine Fazialisparese hat vielfältige Auswirkungen, von Einschränkungen der Gesichtsbeweglichkeit bis zu psychologischen Folgeerkrankungen. Während viele der akuten idiopathischen Fazialisparesen sich nach einigen Wochen bessern, entwickeln etwa ein Drittel der Betroffenen Synkinesien, unwillkürliche Mitbewegungen, welche lebenslange funktionelle und psychologische Folgen haben. Die Therapie umfasst verschiedene Modalitäten, über Medikamente und Chirurgie bis zum Bewegungstraining. Bei regionaler Unterversorgung, aber auch in der Versorgung chronischer Fazialisparesen, bietet die Telemedizin innovative Lösungsansätze. Der Artikel definiert den Begriff „Telemedizin“ im aktuellen Kontext und zeigt verschiedene Anwendungsarten auf. Eine detaillierte Analyse der Anwendungsszenarien von Telemedizin bei Patientinnen und Patienten mit Fazialisparese zeigt, dass trotz geringer Evidenz viele potenziell nützliche Konzepte existieren.
... Previous studies on the automatic evaluation of facial palsy using video or image analysis can be divided into those that focus on central facial palsy (CFP) 9,10 and those that do not distinguish between CFP and peripheral facial palsy (PFP) [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25] . For CFP, palsy presents on the lower face, whereas for PFP, palsy presents on the entire face. ...
... However, such systems were not specially developed for emergency medicine. Various methods for automatically evaluating peripheral or overall facial palsy have been proposed [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25] . These methods can be divided into those that use facial landmarks and those that do not. ...
A stroke is a medical emergency and thus requires immediate treatment. Paramedics should accurately assess suspected stroke patients and promptly transport them to a hospital with stroke care facilities; however, current assessment procedures rely on subjective visual assessment. We aim to develop an automatic evaluation system for central facial palsy (CFP) that uses RGB cameras installed in an ambulance. This paper presents two evaluation indices, namely the symmetry of mouth movement and the difference in mouth shape, respectively, extracted from video frames. These evaluation indices allow us to quantitatively evaluate the degree of facial palsy. A classification model based on these indices can discriminate patients with CFP. The results of experiments using our dataset show that the values of the two evaluation indices are significantly different between healthy subjects and CFP patients. Furthermore, our classification model achieved an area under the curve of 0.847. This study demonstrates that the proposed automatic evaluation system has great potential for quantitatively assessing CFP patients based on two evaluation indices.
... There are some promising tools as Emotrics or Auto-eFACE, based on automatic assessment of facial function, which may eliminate observer bias. Until a universally accepted objective tool is developed, we still need to rely on subjective scales that allow us to monitor changes in facial function, and therefore, it is important to validate these scales to obtain reliable results [33][34][35][36]. ...
Purpose
The clinician-graded electronic facial paralysis assessment (eFACE) is a relatively new digital tool for assessing facial palsy. The present study aimed to determine the validity and reliability of the Spanish version of the eFACE.
Methods
Forward–backward translation from the original English version was performed. Videos and photographs from 65 adult patients with unilateral facial paralysis (any severity, time course, and etiology) were evaluated twice by five otolaryngologists with varying levels of experience in facial palsy evaluation. Internal consistency was measured using Cronbach’s α and the intra- and inter-rater reliability were measured using intraclass correlation coefficient. Concurrent validity was established by calculating Spearman’s rho correlation (ρ) between the eFACE and the House–Brackmann scale (H–B) and Pearson’s correlation (r) between the eFACE and the Sunnybrook Facial Grading System (SFGS).
Results
The Spanish version of the eFACE showed good internal consistency (Cronbach’s α > 0.8). The intra-rater reliability was nearly perfect for the total score (intraclass correlation coefficient: 0.95–0.99), static score (0.92–0.96), and dynamic score (0.96–0.99) and important-to-excellent for synkinesis score (0.79–0.96). The inter-rater reliability was excellent for the total score (0.85–0.93), static score (0.80–0.90), and dynamic score (0.90–0.95) and moderate-to-important for the synkinesis score (0.55–0.78). The eFACE had a very strong correlation with the H–B (ρ = – 0.88 and – 0.85 for each evaluation, p < 0.001) and the SFGS (r = 0.92 and 0.91 each evaluation, p < 0.001).
Conclusion
The Spanish version of the eFACE is a reliable and valid instrument for assessment of facial function in the diagnosis and treatment of patients with facial paralysis.
... In addition, depth data can also provide enhanced information and replace reference values. A recent study by Taeger et al. has shown the potential of using facial recognition technologies such as Augmented Reality Kit (ARKit) and the TrueDepth sensor to assess facial motion disorders [13]. ARKit (https://developer.apple.com/augmentedreality/arkit/, ...
There is a growing consensus in the global health community that the use of communication technologies will be an essential factor in ensuring universal health coverage of the world’s population. New technologies can only be used profitably if their accuracy is sufficient. Therefore, we explore the feasibility of using Apple’s ARKit technology to accurately measure the distance from the user’s eye to their smartphone screen. We developed an iOS application for measuring eyes-to-phone distances in various angles, using the built-in front-facing-camera and TrueDepth sensor. The actual position of the phone is precisely controlled and recorded, by fixing the head position and placing the phone in a robotic arm. Our results indicate that ARKit is capable of producing accurate measurements, with overall errors ranging between 0.88% and 9.07% from the actual distance, across various head positions. The accuracy of ARKit may be impacted by several factors such as head size, position, device model, and temperature. Our findings suggest that ARKit is a useful tool in the development of applications aimed at preventing eye damage caused by smartphone use.
... A larger sample size and the inclusion of patients of diverse ethnic backgrounds would be highly recommended. The application of automated landmarking [53][54][55] , and artificial intelligence to develop an app-based grading of facial muscle movements that could be validated against clinical scores is highly desirable 56) . The development of a more comprehensive grading system of facial muscle movements that provides clinicians with detailed descriptors to quantify muscular dysfunction, including the directionality of asymmetry, should be considered in future studies. ...
Objectives: Despite the limited validity of the Sunnybrook grading index, it is routinely used for the clinical evaluation of facial palsy. This study aimed to assess the dynamic asymmetry in unilateral facial palsy and mathematically validate a modified version of the Sunnybrook facial grading system.
Methods: The Sunnybrook facial grading system was modified to provide more descriptions of the measured parameters of the distorted facial expression in unilateral facial paralysis. This correlation study was conducted on 16 patients with unilateral facial palsy and a matched control group. Three-dimensional video recordings of six facial expressions - rest, maximum smile, cheek puff, lip purse, eyebrow raising, and eye closure - were used for each case in the analysis. Advanced geometric morphometrics were applied to quantify facial asymmetry and morphology throughout the course of each expression. Seven professional assessors graded facial asymmetry for the 16 cases, twice, using the modified Sunnybrook index. Cross-correlations between the objective mathematical measurements and the subjective clinical grades were calculated.
Results: The inter- and intra-observer reproducibility of the modified Sunnybrook index was high (r = −0.8). Significant positive correlations were detected between the clinical grading of facial palsy and the mathematical measurements at rest, maximum smile, lip purse, and raising of eyebrows. The correlations between the modified Sunnybrook index and mathematical measurements were poor for cheek puff and forceful eye closure.
Conclusions: The modified Sunnybrook grading index proved reproducible and mathematically valid for the grading of unilateral facial paralysis in most facial expressions, except for cheek puff and forceful eye closure.
... 3D face scanning has wide applications in medicine and dentistry, and also in capturing facial emotions. 11 Mobile applications provide better accuracy in scanning results. Numerous 3D scanning applications are available online, for example, on the Apple store. ...
Unlabelled:
Cleft lip, a major craniofacial abnormality, is highly prevalent among people with low socioeconomic status in Indonesia. Direct two-dimensional measurement of the affected region is the gold standard for surgical preparation; however, its compliance and usability are limited in pediatric patients. Modern smartphones, including iPhones, are equipped with high-resolution cameras, which can record images and videos of a face. Here, we investigated whether a three-dimensional (3D) smartphone scanner can be used for the facial measurements of patients with unilateral cleft lip.
Methods:
Twelve facial measurements were acquired after cleft lip surgery in three female and seven male patients (aged 11-29 months) with unilateral cleft lip using direct anthropometry and a 3D smartphone scanner. The accuracy and precision of the 3D smartphone scanner were assessed through comparative analyses (t test and Bland-Altman plot).
Results:
The anthropometric data obtained using the 3D smartphone scanner matched the direct measurement data. The linear measurements did not differ significantly between two-dimensional and 3D modalities (P > 0.05). The intraobserver reliabilities of the two-dimensional smartphone scanner of the first and second observers were high (intraclass correlation coefficient 0.876-0.993 and Cronbach alpha 0.920-0.998) and moderate to high (intraclass correlation coefficient 0.839-0.996 and Cronbach alpha 0.940-0.996), respectively. Inter-observer data showed an intraclass correlation coefficient of 0.876-0.981 and a Cronbach α of 0.960-0.997.
Conclusion:
The 3D smartphone scanner is effective, efficient, economical, quick, and feasible for facial measurements of patients with unilateral cleft lip and is a viable alternative to direct two-dimensional measurements.
... Although these evaluation systems provided more objective and accurate results, most of them remained in the preclinical stage [33] which lacked clinical verification and they were unable to provide finer grading scores or demonstrate quantified scale-based facial feature alterations. Another main problem was many studies constructed new evaluation indicators [34], [35], [36], which were not easily to combine with clinical practice. Moreover, few studies [5], [18] have presented tentative evidence for the follow-up evaluation based on regional facial features to indicate the specific therapeutic alterations via video evaluation systems. ...
... Xia et al. [50] suggested it was necessary to train the facial landmark detection with palsyfaces, which had significant differences from those trained with normal-faces. Comparing with similar studies used open access repositories of healthy people [24], [35], [45], [51], the AFES used real patients to train the algorithm which meant a better performance in landmark localization [43], [52]. Additionally, the simple and smooth evaluation process of AFES improve the patients' experience and cooperation, which might be beneficial to assess the real situation of the patients. ...
Facial palsy would lead to a series of physical and mental problems, as facial function plays an important role in various aspects of daily life. However, the current strategies for evaluating facial function relied heavily on raters and the results varied from the experience of raters. Thus, an objective and accurate facial evaluation system is always claimed. In this study, a customized automatical facial evaluation system (AFES) was proposed, which might have the potential to be employed as an adjunctive and efficient assessing method in clinic. In order to investigate the feasibility of AFES, ninety-two participants with facial palsy were recruited and received scale-based subjective manual evaluation (including mHBGS and mSFGS) and objective automatical evaluation of AFES (including aHBGS, aSFGS and indicators of facial regional features) at enrollment and after two weeks. The correlations between the results of the two methods were analyzed and the participants were stratified according to the severity of facial function for further analyses. Strong positive correlations between manual and automatical HBGS and SFGS were observed and higher correlations were reported in the participants with normal-mild and moderate facial palsy. Significant improvements in clinical scales and indicator of eye synkinesis were found in forty-two participants in two weeks. Furthermore, some of the indicators were correlated with scale scores (I4, I7) and one of them presented a significant change between the baseline evaluation and follow-up evaluation (I7). According to the results, AFES could be considered as a viable method to perform objective and reliable evaluation for patients with facial palsy and provide clarified results for prognosis.
... Consequently, a mobile-based application of our approach can be realized without complex and expensive hardware, in contrast to standard clinical intervention systems [58]. Each measurement tool required, such as the camera and processor, is already included in a conventional mobile phone, so the possibility of visual self-diagnosis of laypersons and clinicians with minimal experience in FP treatment/diagnosis comes into clinical reach [59]. Moreover, the omnipresence of various mobile devices, as well as our highly accurate algorithm free of development costs, make the automated grading system a promising low-budget and easy-to-use application. ...
Background:
Reliable, time- and cost-effective, and clinician-friendly diagnostic tools are cornerstones in facial palsy (FP) patient management. Different automated FP grading systems have been developed but revealed persisting downsides such as insufficient accuracy and cost-intensive hardware. We aimed to overcome these barriers and programmed an automated grading system for FP patients utilizing the House and Brackmann scale (HBS).
Methods:
Image datasets of 86 patients seen at the Department of Plastic, Hand, and Reconstructive Surgery at the University Hospital Regensburg, Germany, between June 2017 and May 2021, were used to train the neural network and evaluate its accuracy. Nine facial poses per patient were analyzed by the algorithm.
Results:
The algorithm showed an accuracy of 100%. Oversampling did not result in altered outcomes, while the direct form displayed superior accuracy levels when compared to the modular classification form (n = 86; 100% vs. 99%). The Early Fusion technique was linked to improved accuracy outcomes in comparison to the Late Fusion and sequential method (n = 86; 100% vs. 96% vs. 97%).
Conclusions:
Our automated FP grading system combines high-level accuracy with cost- and time-effectiveness. Our algorithm may accelerate the grading process in FP patients and facilitate the FP surgeon's workflow.
... Für das eFACE-Tool (siehe oben) gibt es auch erste Vorschläge für eine vollautomatische Auswertung [21]. Erste Ansätze für eine schnelle, automatisierte Smartphone-basierte Bewertung, die dann auch eine Bewertung aus der Entfernung ohne direkten Patienten-Therapeuten-Kontakt erlaubt, wurden publiziert; eine Validierung der klinischen Routine in größerem Maßstab fehlt jedoch noch [22]. ...
The purpose of this review is to report the knowledge for otolaryngologists on standard of care, latest advances, interesting new findings and controversies about the treatment of Bellʼs palsy. This review is focusing on the acute phase of the disease. The chronic phase, with incomplete, incorrect or no recovery of the palsy, is described briefly. Treatment with prednisolone alone within 72 hours after onset still is the cornerstone of the treatment. The role of antivirals still is unclear. Since 2009 no new and breakthrough clinical trials with influence on the treatment standards have been performed. A study to clarify the role of prednisolone treatment in children is ongoing. Patient-related outcome measures like the Facial Clinimetric Evaluation Scale and the Facial Disability Index are important tools to assess the subjective severity of the disease and psychosocial impact of Bellʼs palsy next to the motor deficits. Simplified subjective electronic grading systems like the eFACE and first automated image analysis systems have been introduced. Studies clarifying the role of antivirals for severe cases are urgently needed as well as studies on the role of salvage second line therapy after insufficient response to initial corticosteroid treatment. An international consensus on the outcome measures in diagnostics and follow-up is also needed.
... As the present therapy is dependent on the surveillance of a trained therapy, it will also be of interest to develop a remote rehabilitation concept. Therefore, we will need remote EMG devices and especially a simplification of the camera technology using conventional smart phones or, for instance, special remote activity eye wear (39,40). ...
Background: There is no current standard for facial synkinesis rehabilitation programs. The benefit and stability of effect of an intensified 10-day facial training combining electromyography and visual biofeedback training was evaluated.
Methods: Fifty-four patients (77.8% female; median age: 49.5 years) with post-paralytic facial synkinesis (median time to onset of paralysis: 31.1 months) were included in retrospective longitudinal study between January 2013 and June 2016. Facial function was assesses at baseline (T0), first days of training (T1), last day of training (T2), and follow-up visit (T3) at a median time of 6 months later using the House-Brackmann (HB) facial nerve grading system, Stennert index (SI), Facial Nerve Grading System 2.0 (FNGS 2.0), and Sunnybrook Facial Grading System (SFGS). Pairwise comparisons between the time points with post-hoc Bonferroni correction were performed.
Results: No significant changes of the gradings and subscores were seen between T0 and T1 (all p > 0.01). The 10-day combined and intensified feedback training between T1 and T2 improved facial symmetry and decreased synkinetic activity. Facial grading with the FNGS 2.0 or the SFGS were most suited to depict the training effect. FNGS 2.0, regional score, FNGS 2.0, synkinesis score, and FNGS 2.0 total score improved significantly (all p ≤ 0.0001). Both, the FNGS 2.0 and the SFGS showed the strongest improvement in the nasolabial fold/zygomatic and the oral region. Neither the age of the patient ( r = 0.168; p = 0.224), the gender ( r = 0.126; p = 0.363) nor the length of the interval between onset of the palsy and training start ( r = 0.011; p = 0.886) correlated with the changes of the SFGS between T1 and T2. The results remained stable between T2 and T3 without any further significant change.
Conclusion: Intensified daily combined electromyography and visual biofeedback training over 10 days was effective in patients with facial synkinesis and benefits were stable 6 months after therapy.
