FIGURE 2 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Some existing methods of tongue alignment. The images are respectively quoted from [1](CTES), [5](CATDS), [6](Region Partition), [10](Gabor), and [11](KOPS).
Source publication
Tongue image analysis has been an active study in medical imaging. Existing tongue image processing approaches deal with the issue of image alignment in oversimplified ways. These approaches mainly extract patches or simple regions on pre-defined positions, which are severely sensitive to tongue deformations. In this paper, we present a conformal m...
Contexts in source publication
Context 1
... works is to extract several predefined key points on the tongue region, or to partition the tongue region into several sub-regions. Some works [1] [3] [6] divide the tongue surface into several regions with straight boundaries. Other works [4] [10] [11] extract from the tongue surface several blocks. The gist of these methods is visualized in Fig. 2, where the figures are directly quoted from the involved articles. We argue that these approaches suffer from a severe difficulty. Actually they treat the tongue as a rigid body and implicitly demand that the tongue body must be vertically oriented, as is demonstrated in Fig. 2. The assumption is artificial and unnecessary to the ...
Context 2
... surface several blocks. The gist of these methods is visualized in Fig. 2, where the figures are directly quoted from the involved articles. We argue that these approaches suffer from a severe difficulty. Actually they treat the tongue as a rigid body and implicitly demand that the tongue body must be vertically oriented, as is demonstrated in Fig. 2. The assumption is artificial and unnecessary to the deformability of a tongue in ...
Similar publications
Text detection and localization from text embedded natural images is still considered as a challenging problem in complex imagery environment. Foreground object segmentation followed by classification is a popular approach for this task. Component level object classification in clutter environment is therefore an important sub-problem. Appropriate...
Citations
... Zhou et al. 63 used a context-aware dilated residual block to ensure the efficiency of information without increasing extra parameters and computation; Peng et al. 57 used dilated convolution for dense feature extraction and field-of-view enlargement; and Tang et al. 12 proposed a hybrid cascade dilated convolution to extract multiscale features. The division of the tongue is also important for assessing health status, and Wu et al. 69 reported regional alignment of tongue images to improve the accuracy of disease diagnosis. Using medical ultrasound image to visualize and characterize human tongue shape and motion in real-time would do a great favor to study healthy or impaired speech production; thus, some tongue contour segmentation methods [70][71][72] of ultrasound image in real time have been proposed by the team of Hamed Mozaffari; furthermore, they 73 have used the augmented reality of ultrasound data from the extracted tongue movement to provide a real-time visual feedback to improve the training trend of language learners. ...
The rapid development of artificial intelligence technology has gradually extended from the general field to all walks of life, and intelligent tongue diagnosis is the product of a miraculous connection between this new discipline and traditional disciplines. We reviewed the deep learning methods and machine learning applied in tongue image analysis that have been studied in the last 5 years, focusing on tongue image calibration, detection, segmentation, and classification of diseases, syndromes, and symptoms/signs. Introducing technical evolutions or emerging technologies were applied in tongue image analysis; as we have noticed, attention mechanism, multiscale features, and prior knowledge were successfully applied in it, and we emphasized the value of combining deep learning with traditional methods. We also pointed out two major problems concerned with data set construction and the low reliability of performance evaluation that exist in this field based on the basic essence of tongue diagnosis in traditional Chinese medicine. Finally, a perspective on the future of intelligent tongue diagnosis was presented; we believe that the self-supervised method, multimodal information fusion, and the study of tongue pathology will have great research significance.
... After bimodal images are captured, image localization is performed and aligned bimodal ROI images are extracted. Generally, current palm alignment methods are based on keypoint matching [21], template matching [17], and shape remapping [22]. For bimodal images, the challenge is that the same point has different textures in the two images because they are generated by different optical spectrums. ...
Recently, contactless bimodal palmprint recognition technology has attracted increased attention due to the COVID-19 pandemic. Many dual-camera-based sensors have been proposed to capture palm vein and palmprint images synchronously. However, translations between captured palmprint and palm vein images differ depending on the distance between the hand and the sensors. To address this issue, we designed a low-cost method to align the bimodal palm regions for current dual-camera systems. In this study, we first implemented a contactless palm image acquisition device with a dual-camera module and a single-point time of flight (TOF) ranging sensor. Using this device, we collected a dataset named DCPD under different distances and light source intensities from 271 different palms. Then, a bimodal palm image alignment method is proposed based on the imaging and ranging models. After the system model is calibrated, the translation between the visible light and infrared light palm regions can be estimated quickly based on the palm distance. Finally, we designed a convolutional neural network (CNN) to effectively extract the fine- and coarse-grained palm features. Compared to widely used existing methods, the proposed networks achieved the lowest equal error rate (EER) on the Tongji, IITD, and DCPD datasets, and the average time cost of the system to perform one-time identification is approximately 0.15 s. The experimental results indicate that the proposed methods achieved high efficiency and comparable accuracy. In addition, the system’s EER and rank-1 on the DCPD dataset were 0.304% and 98.66%, respectively.
... The most representative work is achieved by Wu et al. [43], who presented a conformal mapping method for tongue image alignment. This method has a strong ability to resist tongue deformation. ...
Objective:
Artificial intelligence-powered screening systems of coronavirus disease 2019 (COVID-19) are urgently demanding since the ongoing outbreak of SARS-CoV-2 worldwide. Chest CT or X-ray is not sufficient to support the large-scale screening of COVID-19 because mildly-infected patients do not have imaging features on these images. Therefore, it is imperative to exploit supplementary medical imaging strategies. Traditional Chinese medicine has played an essential role in the fight against COVID-19.
Methods:
In this paper, we conduct two kinds of verification experiments based on a newly-collected multi-modality dataset, which consists of three types of modalities: tongue images, chest CT scans, and X-ray images. First, we study a binary classification experiment on tongue images to verify the discriminative ability between COVID-19 and non-COVID-19. Second, we design extensive multimodality experiments to validate whether introducing tongue image can improve the screening accuracy of COVID-19 based on chest CT or X-ray images.
Results:
Tongue image screening of COVID-19 showed that the accuracy (ACC), sensitivity (SEN), specificity (SPEC), and Matthew correlation coefficient (MCC) of the improved AlexNet and Googlenet both reached 98.39%, 98.97%, 96.67%, and 99.11%. The fusion of chest CT and tongue images used a tandem multimodal classifier fusion strategy to achieve optimal classification, and the results and screening accuracy of COVID-19 reached 98.98%, resulting in a significant improvement of 4.75% the highest accuracy in 375 years compared with the single-modality model. The fusion of chest x-rays and tongue images also had good classification accuracy.
Conclusions:
Both experimental results demonstrate that tongue image not only has an excellent discriminative ability for screening COVID-19 but also can improve the screening accuracy based on chest CT or X-rays. To the best of our knowledge, it is the first work that verifies the effectiveness of tongue image on screening COVID-19. This paper provides a new perspective and a novel solution that contributes to large-scale screening toward fast stopping the pandemic of COVID-19.
... One is that the tongue shape is highly unpredictable; the other is the consistency between the tongues of different people [106]. Wu et al. [107] present a conformal mapping method for tongue image alignment challenges. This method first establishes the mapping on the boundary by Fourier descriptors. ...
Traditional Chinese Medicine (TCM), as an effective alternative medicine, utilizes tongue diagnosis as a major method to assess the patient’s health status by examining the tongue’s color, shape, and texture. Tongue images can also give the pre-disease indications without any significant disease symptoms, which provides a basis for preventive medicine and lifestyle adjustment. However, traditional tongue diagnosis has limitations, as the process may be subjective and inconsistent. Hence, computer-aided tongue diagnoses have a great potential to provide more consistent and objective health assessments. This paper reviewed the current trends in TCM tongue diagnosis, including tongue image acquisition hardware, tongue segmentation, feature extraction, color correction, tongue classification, and tongue diagnosis system. We also present a case of TCM constitution classification based on tongue images.
... Their work focused on optimal tuning of the SVM algorithm to achieve high predictive accuracy. In addition, Wu et al. [22] have proposed a tongue image alignment technique through conformal mapping for the detection of different types of diseases including diabetes. Safia et al. [23] have proposed an intelligent diabetes detection system based on tongue images dataset created by high resolution scanning electron microscope camera. ...
Recent advances in machine learning techniques have significantly influenced the development of early diabetes detection systems. In this paper, different machine learning (ML) algorithms are employed for diabetes detection based on hand movement. The spatial and temporal information for horizontal and vertical hand movement deviations for both diabetic and non-diabetic subjects are obtained using a data acquisition system. A dataset is constructed based on the obtained hand movement information along with other related attributes such as age, gender, weight and vision health. Binary classification of diabetic and non-diabetic subjects is achieved using several machine learning algorithms. These algorithms include Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminate Analysis (LDA), Classification and Regression Tree (CART), k-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Multi-layer Perceptron (MLP). The considered ML algorithms are evaluated and ranked according to a set of key performance metrics that include accuracy, precision, recall and F-1 score. Algorithm ranking results demonstrate that LR, CART, KNN and MLP algorithms have achieved higher ranking positions as compared to other algorithms. The best ranked ML algorithms are then validated and thoroughly investigated to prove their ability to discriminate between diabetic and non-diabetic subjects. Validation results show that the best-ranked ML algorithms have achieved high classification accuracy, precision, recall and F-1 scores. Therefore, the considered machine learning algorithms can serve as efficient tools for diabetes detection based on the constructed dataset.
... Tongue image alignment is a fundamental issue of tongue diagnosis, which is the mapping of points or subregions among different tongue images. Wu et al. [6] presented a conformal mapping method for tongue image alignment. And Dai et al. [7] proposed a method called the conceptual alignment deep autoencoder to analyze tongue images that represent different body constitution types. ...
Tongue coating can provide valuable diagnostic information to reveal the disorder of the internal body. However, tongue coating classification has long been a challenging task in Traditional Chinese Medicine (TCM) due to the fact that tongue coatings are polymorphous, different tongue coatings have different colors, shapes, textures and locations. Most existing analyses utilize handcrafted features extracted from a fixed location, which may lead to inconsistant performance when the size or location of the tongue coating region varies. To solve this problem, this paper proposes a novel paradigm by employing artificial intelligence to feature extraction and classification of tongue coating. It begins with exploiting prior knowledge of rotten-greasy tongue coating to obtain suspected tongue coating patches. Based on the resulting patches, tongue coating features extracted by Convolutional Neural Network (CNN) are used instead of handcrafted features. Moreover, a multiple-instance Support Vector Machine (MI-SVM) which can circumvent the uncertain location problem is applied to tongue coating classification. Experimental results demonstrate that the proposed method outperforms state-of-the-art tongue coating classification methods.
... Tongue plays an important role in many applications, such as tongue drive system [18] and aphasia therapy [19]. As it reflects the health status [7,[20][21][22][23], it is widely used in TCM for the diagnosis [24][25][26] and treatment [9], where many tongue image analysis methods are proposed [27], such as conformal mapping method for tongue image alignment [28], color calibration algorithms [29], automatic tongue segmentation [30,31], and geometry features extraction with color correction [22]. ...
Traditional Chinese Medicine (TCM) considers that the personal constitution determines the occurrence trend and therapeutic effects of certain diseases, which can be recognized by machine learning through tongue images. However, current machine learning methods are confronted with two challenges. First, there are not some larger tongue image databases available. Second, they do not use the domain knowledge of TCM, so that the imbalance of constitution categories can not be solved. Therefore, this paper proposes a new constitution recognition method based on the zero-shot learning with the knowledge of TCM. To further improve the performance, a new zero-shot learning method is proposed by grouping attributes and learning discriminant latent features, which can better solve the imbalance problem of constitution categories. Experimental results on our constructed databases validate the proposed methods.
Traditional Chinese medicine (TCM) diagnosis is a unique disease diagnosis method with thousands of years of TCM theory and effective experience. Its thinking mode in the process is different from that of modern medicine, which includes the essence of TCM theory. From the perspective of clinical application, the four diagnostic methods of TCM, including inspection, auscultation and olfaction, inquiry, and palpation, have been widely accepted by TCM practitioners worldwide. With the rise of artificial intelligence (AI) over the past decades, AI based TCM diagnosis has also grown rapidly, marked by the emerging of a large number of data-driven deep learning models. In this paper, our aim is to simply but systematically review the development of the data-driven technologies applied to the four diagnostic approaches, i.e. the four examinations, in TCM, including data sets, digital signal acquisition devices, and learning based computational algorithms, to better analyze the development of AI-based TCM diagnosis, and provide references for new research and its applications in TCM settings in the future.
Machine learning offers automatic and objective approaches for disease detection based on biomedical data. However, 1) the percentage of patients in the real world is smaller than that of healthy people; 2) the annotations by medical experts are costly. Therefore, medical datasets are often imbalanced and partially labeled. To address this problem, we propose balanced evolutionary semi-stacking (BESS) for disease detection using partially labeled imbalanced (PLI) data. BESS aims to detect illnesses by considering the input of the color, texture, and geometry features extracted from tongue images. Specifically, the proposed method first mitigates the class imbalance problem and leverages the unlabeled data through the so-called balanced evolutionary co-training approach. Then BESS exploits both the data and classifier diversity obtained by balanced evolutionary co-training to improve the performance of the stacking ensemble. We quantitatively evaluate the proposed algorithm based on the PLI tongue image database. BESS achieves the best performance in detecting diabetes mellitus, chronic kidney disease, breast cancer, and chronic gastritis, compared to other state-of-the-art methods. The results of the experiments substantiate the superiority and effectiveness of the proposed algorithm. Codes and datasets have been made publicly available at url:https://github.com/CUHKSZ-NING/Balanced-Evolutionary-Semi-Stacking.
