Figure - available from: Biomedical Optics Express
This content is subject to copyright. Terms and conditions apply.
Heartrate measurement with snapshot-hyperspectral-camera-based multispectral imaging system. (a) The overlaid face skin image of raw image with the derived map of hemoglobin absorption information (coded according to the color bar shown in the right). The red dotted box area on the cheek is the target area for extracting heartrate from blood absorption information content derived from the hyperspectral imaging. (b) The frequency spectrum of temporal profile of blood absorption information content summed within the red-box region in (a). (c) The heartrate reference from the PowerLab pulse sensor.
Source publication
We propose a snapshot hyperspectral imaging system and methods for skin morphological feature analysis and real-time monitoring of skin activities. The analysis method includes a strategy using weighted subtractions between sub-channel images to extract absorption information due to specific chromophores within skin tissue, for example hemoglobin a...
Similar publications
Peripunctal tumors usually account for as low as 6.3% of all punctal lesions. Peripunctal nevus is benign in nature, but it can cause epiphora by its sheer mass effect or cosmetic blemish. Anterior segment optical coherence tomography has recently been used in various lower punctal pathologies. A 60‑year‑old lady presented with a painless, benign p...
Citations
... This is accomplished through cutting-edge optical designs that record data concurrently for all spectral ranges. Snapshot sensors enable quick data acquisition and are ideal for dynamic scenarios or situations needing promptly available spectral information [42]. ...
Hyperspectral imaging (HSI) has become a very compelling technique in different scientific areas; indeed, many researchers use it in the fields of remote sensing, agriculture, forensics, and medicine. In the latter, HSI plays a crucial role as a diagnostic support and for surgery guidance. However, the computational effort in elaborating hyperspectral data is not trivial. Furthermore, the demand for detecting diseases in a short time is undeniable. In this paper, we take up this challenge by parallelizing three machine-learning methods among those that are the most intensively used: Support Vector Machine (SVM), Random Forest (RF), and eXtreme Gradient Boosting (XGB) algorithms using the Compute Unified Device Architecture (CUDA) to accelerate the classification of hyperspectral skin cancer images. They all showed a good performance in HS image classification, in particular when the size of the dataset is limited, as demonstrated in the literature. We illustrate the parallelization techniques adopted for each approach, highlighting the suitability of Graphical Processing Units (GPUs) to this aim. Experimental results show that parallel SVM and XGB algorithms significantly improve the classification times in comparison with their serial counterparts.
... Existing bioassay systems can inspect the chemical content in the sweat of finger marks; however, this approach requires the samples to be treated with costly reagent kits that destroy metabolites. Active research is exploring a contactless manner with which to acquire the sweat metabolites from fingertips via hyperspectral imaging (HSI) [31,32]. HSI has gained a lot of interest in various fields, including agriculture and medical research [33][34][35]. ...
AI-empowered sweat metabolite analysis is an emerging and open research area with great potential to add a third category to biometrics: chemical. Current biometrics use two types of information to identify humans: physical (e.g., face, eyes) and behavioral (i.e., gait, typing). Sweat offers a promising solution for enriching human identity with more discerning characteristics to overcome the limitations of current technologies (e.g., demographic differential and vulnerability to spoof attacks). The analysis of a biometric trait’s chemical properties holds potential for providing a meticulous perspective on an individual. This not only changes the taxonomy for biometrics, but also lays a foundation for more accurate and secure next-generation biometric systems. This paper discusses existing evidence about the potential held by sweat components in representing the identity of a person. We also highlight emerging methodologies and applications pertaining to sweat analysis and guide the scientific community towards transformative future research directions to design AI-empowered systems of the next generation.
... Hyperspectral approaches for skin inspection have the same operation principle as MSI systems, but the main difference is related to the much higher number of narrow spectral bands involved in the acquisition process. The absorption information about chromophores can be extracted by a snapshot hyperspectral camera with only one exposure, minimizing the risk of motion artifacts in images [29]. Besides skin cancer detection, the possibility of monitoring skin tissue activities, such as a response to a vascular occlusion, wound examination, and heartrate assessment, was also investigated and provided good preliminary results [29]. ...
... The absorption information about chromophores can be extracted by a snapshot hyperspectral camera with only one exposure, minimizing the risk of motion artifacts in images [29]. Besides skin cancer detection, the possibility of monitoring skin tissue activities, such as a response to a vascular occlusion, wound examination, and heartrate assessment, was also investigated and provided good preliminary results [29]. ...
Skin optical inspection is an imperative procedure for a suspicious dermal lesion since very early skin cancer detection can guarantee total recovery. Dermoscopy, confocal laser scanning microscopy, optical coherence tomography, multispectral imaging, multiphoton laser imaging, and 3D topography are the most outstanding optical techniques implemented for skin examination. The accuracy of dermatological diagnoses attained by each of those methods is still debatable, and only dermoscopy is frequently used by all dermatologists. Therefore, a comprehensive method for skin analysis has not yet been established. Multispectral imaging (MSI) is based on light–tissue interaction properties due to radiation wavelength variation. An MSI device collects the reflected radiation after illumination of the lesion with light of different wavelengths and provides a set of spectral images. The concentration maps of the main light-absorbing molecules in the skin, the chromophores, can be retrieved using the intensity values from those images, sometimes even for deeper-located tissues, due to interaction with near-infrared light. Recent studies have shown that portable and cost-efficient MSI systems can be used for extracting skin lesion characteristics useful for early melanoma diagnoses. This review aims to describe the efforts that have been made to develop MSI systems for skin lesions evaluation in the last decade. We examined the hardware characteristics of the produced devices and identified the typical structure of an MSI device for dermatology. The analyzed prototypes showed the possibility of improving the specificity of classification between the melanoma and benign nevi. Currently, however, they are rather adjuvants tools for skin lesion assessment, and efforts are needed towards a fully fledged diagnostic MSI device.
... The number and value of wavelengths were fixed, which means that either the application must be specific or the SRDA must be custom-made, both limited the efficiency of research. Nevertheless, SRDA systems found many applications in bioimaging; we found systems that used SRDA in fluorescence microscopy, 23 fluorescence endoscope imaging, 160,161 skin imaging, 40 and fundus imaging. 54 Recovering the full spectral image from the acquired mosaiced image is not a simple task since the mosaic image shows a sparse representation of the captured data. ...
Significance:
Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.
Aim:
With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.
Approach:
We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.
Results:
There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.
Conclusions:
Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.
... In this respect, the development of snapshot imaging technologies capable of acquiring a hypercube in a single-shot manner has been an active area of research (summarized in Table S1). The most common configuration used for snapshot imaging involves simultaneously capturing multiple images with different spectral bands using a large-area image sensor (26)(27)(28)(29)(30)(31)(32). In particular, large-area image sensor-based snapshot imaging is beneficial for reducing the acquisition time (2,33,34). ...
Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral trade-off. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from a red–green–blue (RGB) image without complete hyperspectral measurements. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral trade-off, offering simple hardware requirements and potential applications of various machine learning techniques.
... Multispectral imaging provides morphological and physiological information of examined lesions, related to the absorption and scattering properties of skin chromophores, such as hemoglobin, melanin and bilirubin content, tissue oxygenation, etc. [25]. MSR imaging for skin cancer diagnostics has been repeatedly demonstrated and clinically proven [26]. ...
In this work, we propose to use an artificial neural network to classify limited data of clinical multispectral and autofluorescence images of skin lesions. Although the amount of data is limited, the deep convolutional neural network classification of skin lesions using a multi-modal image set is studied and proposed for the first time. The unique dataset consists of spectral reflectance images acquired under 526 nm, 663 nm, 964 nm, and autofluorescence images under 405 nm LED excitation. The augmentation algorithm was applied for multi-modal clinical images of different skin lesion groups to expand the training datasets. It was concluded from saliency maps that the classification performed by the convolutional neural network is based on the distribution of the major skin chromophores and endogenous fluorophores. The resulting classification confusion matrices, as well as the performance of trained neural networks, have been investigated and discussed.
... In many cases, the microvasculature can be used as a biomarker for studying diseases/conditions with microvascular involvements, such as aging-related skin problems [2], diabetes [3], and wound injuries [4]. Therefore, developing non-invasive and cost-effective imaging tools [5][6][7][8] to facilitate the study of microvasculature and hemodynamics in cutaneous tissue beds is crucial for the understanding, diagnosis, and treatment of diseases, including skin conditions. ...
We report a method to image facial cutaneous microvascular perfusion using wide-field imaging photoplethysmography (iPPG) and handheld swept-source optical coherence tomography (OCT). The iPPG system employs a 16-bit-depth camera to provide a 2D wide-field blood pulsation map that is then used as a positioning guidance for OCT imaging of cutaneous microvasculature. We show the results from iPPG and OCT to demonstrate the ability of guided imaging of cutaneous microvasculature, which is potentially useful for the assessment of skin conditions in dermatology and cosmetology.
... From the data cube, we selected bacteria-contaminated tissue and endogenous tissue as the targets to compare their spectral appearance, thus realizing bacteriatargeted spectral analyses. According to their spectral performance of fluorescence, we applied weighted subtraction between different wavebands to extract and contrast porphyrins from the tissue beds [29] . The processing can be depicted as: ...
We propose a smartphone-enabled system and method to realize multispectral autofluorescence imaging for analyzing bacterial infection within skin and oral cavity. The system consists of an unmodified and intact commercial smartphone and a home-made cell phone case with built-in black light LEDs. We use Wiener estimation method to calibrate the RGB-mode smartphone camera and transform the acquired autofluorescence photographs into pseudo-multispectral data cubes with 15 wavebands ranging from 420 nm to 700 nm. Then, we extract and compare the spectral performance of emissions from bacteria-produced porphyrins and endogenous background tissue. Based on the extracted autofluorescence spectra, we apply weighted subtraction between wavebands of interest to realize bacteria targeting and feature mapping. We conduct analysis of autofluorescence on facial skin and dental plaques to demonstrate the performance of the proposed system and methods. Further, with this strategy, we realize quantitative analysis of the bacterial infection in the combination and oily types of skin. Compared to traditional bacteria assessment strategies, we provide a method with the features of real-time visualization, label-free molecular identification, and feature mapping. Meanwhile, differing from the most conventional multispectral imaging systems, the proposed smartphone-based system works in a snapshot mode, thus improving its immunity to motion artifacts. Considering the popularity of smartphones in today's world, it is expected a relatively easy acceptance of the proposed cost-effective method by the community, making an impact on skin and oral care in general and in rural areas with low resource settings in particular.
... In our previous study, we have developed smartphone-enabled transdermal optical imaging techniques to investigate the spatial blood perfusion on the human skin [34]. This work has shown that snapshot RGB-mode photographs of the skin can be reconstructed into multispectral images to map the contents of bio-chromophores [35], which proves it possible to integrate the smartphone-enabled transdermal optical imaging technique with rPPG to measure and monitor the cutaneous hemodynamics in both time and spatial scales. ...
We propose a smartphone-enabled remote multispectral photoplethysmography (SP-rmPPG) system and method to realize spatiotemporal monitoring of perfusion changes and pulsations of the oxyhemoglobin (HbO2) and deoxyhemoglobin (Hb) information of the effective blood volume within light interrogated skin tissue beds. The system is implemented on an unmodified smartphone utilizing its built-in camera and flashlight to acquire videos of the skin reflectance. The SP-rmPPG method converts the RGB video into multispectral cubes, upon which to decouple the dynamic changes in HbO2 and Hb information using a modified Beer-Lambert law and the selective wavelength bands of 500 nm and 650 nm. Blood pulsation amplitudes are then obtained by applying a window-based lock-in amplification on the derived spatiotemporal changes in HbO2 or Hb signals. To demonstrate the feasibility of proposed method, we conduct two experiments on the skin tissue beds that are conditioned by occlusive maneuver of supplying arteries: one using the popular blood cuff pressure maneuver on the upper arm, and another artificially inducing a transient ischemic condition on the facial skin tissue beds by finger pressing on the supplying external carotid artery. The cuff experiment shows that the measured dynamic information of HbO2 and Hb in the downstream agrees well with the parallel measurements of oxygenation saturation given by the standard pulse oximeter. We also observe the expected imbalance of spatiotemporal changes in the HbO2 and Hb between the right and left cheeks when the transient ischemic condition is induced in the one side of facial skin tissue beds. The results from the two experiments sufficiently demonstrate the feasibility of the proposed method to monitor the spatiotemporal changes in the skin hemodynamics, including blood oxygenation and pulsation amplitudes. Considering the ever-growing accessibility and affordability of the smartphone to the general public, the proposed strategy promises the early screening of vascular diseases and improving general public health particularly in rural areas with low resource settings.
... Considering the ever-growing utility of the smartphones in the community, mobile health (mHealth) is playing an increasingly important role in nowadays medical market, even for personal amusements. Heart/respiration rate monitoring [38], blood pressure surveillance [39], morphological features analysis and hemodynamics monitoring [40,41] are all successful applications of mHealth to date. More opportunities of medical applications by leveraging the advances in the development of smartphones would help improve the medical environment in undeveloped areas, making it possible for remote monitoring of health conditions and realizing household health monitoring economically. ...
We propose a robust non-contact method to accurately estimate peripheral oxygen saturation (SpO2) using a smartphone-based imaging photoplethysmography. The method utilizes the built-in color camera as a remote sensor and the built-in flashlight as illumination to estimate the SpO2. Following the ratio of ratios between green and red channels, we introduce a multiple linear regression algorithm to improve the SpO2 estimation. The algorithm considers the ratio of ratios and the reflectance images recorded at the RGB channels during a calibration process to obtain a set of weighting coefficients to weigh each contributor to the final determination of SpO2. We demonstrate the proposed smartphone-based method of estimating the SpO2 on five healthy volunteers whose arms are conditioned by a manual pressure cuff to manipulate the SpO2 between 90∼100% as detected simultaneously by a medical-grade pulse oximeter. Experimental results indicate that the overall estimated error between the smartphone and the reference pulse oximeter is 0.029 ± 1.141%, leading to a 43% improvement over the conventional ratio of ratios method (0.008 ± 2.008%). In addition, the data sampling time in the current method is 2 seconds, similar to the sampling cycle used in the commercial medical-grade pulse oximeters.
