Yu Liu

Hefei University of Technology · Department of Biomedical Engineering

Medical image fusion integrates multi-modal images with complementary information to enhance the image quality in clinical diagnosis. This process typically involves three steps including feature extraction, feature fusion, and image reconstruction. However, most image fusion methods have limitations in simultaneously extracting specific and shared...

In the field of medical imaging, the fusion of data from diverse modalities plays a pivotal role in advancing our understanding of pathological conditions. Sparse representation (SR), a robust signal modeling technique, has demonstrated noteworthy success in multi-dimensional (MD) medical image fusion. However, a fundamental limitation appearing in...

Multi-modal image fusion (MMIF) can provide more comprehensive scene characteristics by synthesizing a single image from multi-sensor images of the same scene, which works out the limitation of single-type hardwares. To handle MMIF tasks, current deep learning-based methods usually employ convolutional neural networks or combine transformer to extr...

As an effective technique to extend the depth-of-field (DOF) of optical lenses, multi-focus image fusion has recently become an active topic in image processing community. However, a major problem remaining unsolved in this field is the lack of universal criteria in selecting objective evaluation metrics. Consequently, the metrics utilized in diffe...

To minimize parallax errors and achieve high spatial resolution positron emission tomography (PET) systems, developing depth-of-interaction (DOI) encoding detectors has become a significant research topic. In this paper, we investigated a dual-ended readout PET detector based on the multi-voltage threshold sampling method combined with a convolutio...

Accurate segmentation of brain tumors in multimodal MRI plays a crucial role in clinical quantitative assessments, diagnostic processes, and the planning of therapeutic strategies. Both convolutional neural networks (CNNs) with strong local information extraction capacities and Transformers with excellent global representation capacities have achie...

Multi-focus image fusion aims to merge source images with distinct focused areas into a single, fully focused fused image. Sparse representation (SR) stands out as a robust signal modeling technique that has achieved remarkable success in multi-focus image fusion. Numerous SR-based fusion methods have been proposed over the years, underscoring the...

Multi-focus image fusion aims to generate an all-in-focus image from multiple partially focused images of the same scene captured with different focal settings. In this paper, we present a coupled convolutional sparse representation (CCSR) model for multi-focus image fusion. Instead of being solved by an iterative thresholding algorithm, the propos...

In recent years, although significant progress has been made in infrared and visible image fusion, existing methods typically assume that the source images have been rigorously registered or aligned prior to image fusion. However, the difference in modalities of infrared and visible images poses a great challenge to achieve strict alignment automat...

The aim of camouflaged object detection (COD) is to find objects that are hidden in their surrounding environment. Due to the factors like low illumination, occlusion, small size and high similarity to the background, COD is recognized to be a very challenging task. In this paper, we propose a general COD framework, termed as MSCAF-Net, focusing on...

Imaging ballistocardiography (iBCG) is a novel technique that utilizes video-based technology to measure heart rate (HR). This innovative method is based on the detection of subtle mechanical head movements that are caused by heartbeats and does not require direct physical contact with the body. However, the iBCG signals can be easily contaminated...

The fusion of infrared and visible images aims to generate a composite image that can simultaneously contain the thermal radiation information of an infrared image and the plentiful texture details of a visible image to detect targets under various weather conditions with a high spatial resolution of scenes. Previous deep fusion models were general...

Multi-modal image fusion (MMIF) aims to synthesize one more informative image from complementary multi-modal images. Due to the absence of ground truth, most MMIF works utilize unsupervised loss functions to preserve specific source information into the fused image, suffering from subjective information preservation and high demand for a large trai...

Emotion recognition is a key component of human-computer interaction technology, for which facial electromyogram (fEMG) is an important physiological modality. Recently, deep-learning-based emotion recognition using fEMG signals has drawn increased attention. However, the ability of effective feature extraction and the demand of large-scale trainin...

The steady-state visual evoked potential (SSVEP) has been widely used in building multi-target brain-computer interfaces (BCIs) based on electroencephalogram (EEG). However, methods for high-accuracy SSVEP systems require training data for each target, which needs significant calibration time. This study aimed to use the data of only part of the ta...

In this letter, a deep learning (DL)-based multi-exposure image fusion (MEF) method via multi-scale and context-aware feature learning is proposed, aiming to overcome the defects of existing traditional and DL-based methods. The proposed network is based on an auto-encoder architecture. First, an encoder that combines the convolutional network and...

Recently, non-contact monitoring of respiratory rate (RR) based on consumer-level cameras has gained increasing attention. However, motion artifacts with large amplitudes usually contaminate subtle video-extracted respiratory signals, challenging the performance of video-based RR estimation. To address this problem, we propose to estimate RR by fus...

In recent years, deep learning has gained widespread attention in electroencephalogram (EEG)-based emotion recognition. However, deep learning methods are usually time-consuming with a large amount of memory usage, which obstructs their practical usage on resource-constrained devices. In this paper, we propose a binary capsule network (Bi-CapsNet)...

Deep learning has recently achieved remarkable success in emotion recognition based on Electroencephalogram (EEG), in which convolutional neural networks (CNNs) are the mostly used models. However, due to the local feature learning mechanism, CNNs have diffilty in capturing the global contextual information involving temporal domain, frequency doma...

In this paper, an unrolling algorithm of the iterative subspace-based optimization method (SOM) is proposed for solving full-wave inverse scattering problems (ISPs). The unrolling network, named SOM-Net, inherently embeds the LippmannSchwinger physical model into the design of network structures. The SOM-Net takes the deterministic induced current...

Brain tumor segmentation in multimodal MRI volumes is of great significance to disease diagnosis, treatment planning, survival prediction and other relevant tasks. However, most existing brain tumor segmentation methods fail to make sufficient use of multimodal information. The most common way is to simply stack the original multimodal images or th...

In this paper, an unrolling algorithm of the iterative subspace-based optimization method (SOM) is proposed for solving full-wave inverse scattering problems (ISPs). The unrolling network, named SOM-Net, inherently embeds the Lippmann- Schwinger physical model into the design of network structures. The SOM-Net takes the deterministic induced curren...

Convolutional neural networks (CNNs) have achieved better performance than traditional algorithms in electroencephalogram (EEG)-based emotion recognition tasks in the recent years. However, as the number of convolution layers increases, the number of network parameters increases sharply. Furthermore, emotional labels are not fully utilized by most...

Dear Editor, In recent years, multi-modal medical image fusion has received widespread attention in the image processing community. However, existing works on medical image fusion methods are mostly devoted to pursuing high performance on visual perception and objective fusion metrics, while ignoring the specific purpose in clinical applications. I...

Owing to the limitations of imaging sensors, it is challenging to obtain a medical image that simultaneously contains functional metabolic information and structural tissue details. Multimodal medical image fusion, an effective way to merge the complementary information in different modalities, has become a significant technique to facilitate clini...

In this package, we collect 20 pairs of grayscale multi-focus images that are widely used in multi-focus image fusion. There is no repetition with the "Lytro" dataset and the "MFFW" dataset. The dataset is named "Classic".

Medical image fusion aims to integrate the complementary information captured by images of different modalities into a more informative composite image. However, current study on medical image fusion suffers from several drawbacks: 1) existing methods are mostly designed for 2-D slice fusion, and they tend to lose spatial contextual information whe...

In the field of image fusion, great progress has been achieved in sparse representation (SR)-based methods with the advent of considerable efficacious algorithms being developed. The improvement of SR-based image fusion methods mainly focuses on pursuing better signal representation as well as effective measurement to sparse coefficients. However,...

Emotion recognition based on electroencephalography (EEG) plays an increasingly important role in the field of brain-computer interfaces (BCI). Recently, deep learning (DL) has been widely used in EEG decoding due to its powerful automatic feature extraction capabilities. Transformer holds great superiority in processing time series signals due to...

A large number of deep learning classification methods for emotion recognition tasks based on electroencephalogram (EEG) have achieved excellent performance, and it is implicitly assumed that all labels are correct. However, humans have natural bias, subjectiveness and inconsistencies in their judgement, which would lead to noisy label for the EEG...

Deep learning (DL) technologies have recently shown great potential in emotion recognition based on electroencephalography (EEG). However, existing DL-based EEG emotion recognition methods are built on single-task learning, \emph{i.e.}, learning arousal, valence, and dominance individually, which may ignore the complementary information of differen...

Automatic segmentation of brain tumor regions from multimodal MRI scans is of great clinical significance. In this letter, we propose a “Segmentation-Fusion” multi-task model named SF-Net for brain tumor segmentation. In comparison to the widely-used multi-task model that adds a variational autoencoder (VAE) decoder to reconstruct the input data, u...

Medical image fusion aims to derive complementary information from medical images with different modalities and is becoming increasingly important in clinical applications. The design of fusion strategy plays a key role in achieving high-quality fusion results. Existing methods usually employ handcrafted fusion rules or convolution-based networks t...

Sparse unmixing (SU) of hyperspectral image (HSI), as a semisupervised approach, aims to find the optimal subset of the spectral library known in advance to represent each pixel in HSI. However, most of the existing SU methods cannot take full advantage of spatial information and mixed noise in HSI. To this end, we propose a superpixel-based noise...

With the flourishing development of deep learning (DL) and the convolution neural network (CNN), electroencephalogram-based (EEG) emotion recognition is occupying an increasingly crucial part in the field of brain-computer interface (BCI). However, currently employed architectures have mostly been designed manually by human experts, which is a time...

In deep learning (DL)-based multifocus image fusion, effective multiscale feature learning is a key issue to promote fusion performance. In this article, we propose a novel DL model named multiscale feature interactive network (MSFIN), which can segment the source images into focused and defocused regions accurately by sufficient interaction of mul...

In this letter, a convolutional sparsity regularization (CSR) is introduced into the framework of nonlinear iterative methods for solving inverse scattering problems (ISPs). The permittivity image of scatterers is sparsely represented in a convolutional form with pre-learned dictionary filters. The CSR is then incorporated with the subspace-based o...

Cardiovascular disease (CVD) is one of the most serious diseases threatening human health. Arterial blood pressure (ABP) waveforms, containing vivid cardiovascular information, are of great significance for the diagnosis and the prevention of CVD. This paper proposes a deep learning model, named ABP-Net, to transform photoplethysmogram (PPG) signal...

In cell and molecular biology, the fusion of green fluorescent protein (GFP) and phase contrast (PC) images aims to generate a composite image, which can simultaneously display the functional information in the GFP image related to the molecular distribution of biological living cells and the structural information in the PC image such as nucleus a...

Medical image fusion, which aims to combine multi-source information captured by different imaging modalities, is of great significance to medical professionals for precise diagnosis and treatment. In the last decade, sparse representation (SR)-based approach has emerged as a very active direction in the field of medical image fusion, due to its po...

Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitati...

Non-contact and low-cost heart rate (HR) measurement based on imaging photoplethysmography (iPPG) technology is commonly desired for health care monitoring. However, the usually employed Red-Green-Blue (RGB) cameras are sensitive to illumination variations and can not work under dark situations. In this study, we propose a novel framework of applyi...

Imaging ballistocardiography (iBCG) is a video-based non-contact technique to detect heart rate (HR) from weak mechanical head movements caused by heart beating. However, rigid motions caused by voluntary movements and non-rigid motion resulted from facial expressions can easily distort the iBCG measurements. In this paper, we propose a novel metho...

Multi-modal medical imaging has emerged as a general trend in clinical diagnosis and treatment planning. In recent years, great efforts have been made to investigate and develop dual-modality scanners, among which PET/CT is the most widespread one in clinical practice. In this paper, we propose a simple yet effective PET/CT data visualization metho...

Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ign...

Sign language is an important communication tool between the deaf and the external world. As the number of the Chinese deaf accounts for 15% of the world, it is highly urgent to develop a Chinese sign language recognition (CSLR) system. Recently, a novel phonology- and radical-coded CSL, taking advantages of a limited and constant number of coded g...

Heart rate (HR) measurement and monitoring is of great importance to determine the physiological and mental status of individuals. Recently, it has been demonstrated that HR can be remotely retrieved from facial video-based photoplethysmographic signals captured using consumer-grade cameras. However, in existing studies, subjects are mostly require...

In recent years, deep learning (DL) techniques, and in particular convolutional neural networks (CNNs), have shown great potential in electroencephalograph (EEG)-based emotion recognition. However, existing CNN-based EEG emotion recognition methods usually require a relatively complex stage of feature pre-extraction. More importantly, the CNNs cann...

Multi-focus image fusion is an effective technique to extend the depth-of-field of optical lenses by creating an all-in-focus image from a set of partially focused images of the same scene. In the last few years, great progress has been achieved in this field along with the rapid development of image representation theories and approaches such as m...

Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitati...

Recently, deep neural networks (DNNs) have been applied to emotion recognition tasks based on electroencephalog-raphy (EEG), and have achieved better performance than traditional algorithms. However, DNNs still have the disadvantages of too many hyperparameters and lots of training data. To overcome these shortcomings, in this paper, we propose a m...

Image fusion technique is an effective way to merge the information contained in different imaging modalities by generating a more informative composite image. Fusion of green fluorescent protein (GFP) and phase contrast images is of great significance to the subcellular localization, the functional analysis of protein, and the expression of gene....

In the field of cell and molecular biology, green fluorescent protein (GFP) images provide functional information embodying the molecular distribution of biological cells while phase-contrast images maintain structural information with high resolution. Fusion of GFP and phase-contrast images is of high significance to the study of subcellular local...

Sparse unmixing has long been a hot research topic in the area of hyperspectral image (HSI) analysis. Most of the traditional sparse unmixing methods usually assume to only take the Gaussian noise into consideration. However, there are also other types of noise in real HSI, i.e., impulse noise, stripes, dead lines and so on. In addition, the intens...

Recently, compressed sensing (CS) has been an effective data compression technique for telemonitoring of mul-tichannel electroencephalogram (EEG) signals through wireless body-area networks. Most of the existing multichannel EEG CS methods ignore the noise or only consider the Gaussian noise. However, there are also some other types of noise, such...

Electroencephalogram (EEG) signals are often contaminated with diverse artifacts, such as electromyogram (EMG), electrooculogram (EOG) and electrocardiogram (ECG) artifacts. These artifacts make subsequent EEG analysis inaccurate and prevent practical usage. Recently, the use of wearable EEG devices in ambulatory systems has been developed. For pra...

Bone age assessment (BAA) has various clinical applications such as diagnosis of endocrine disorders and prediction of final adult height for adolescents. Recent studies indicate that deep learning techniques have great potential in developing automated BAA methods with significant advantages over the conventional methods based on handcrafted featu...

In this letter, a sparse representation (SR) model named convolutional sparsity based morphological component analysis (CS-MCA) is introduced for pixel-level medical image fusion. Unlike the standard SR model which is based on single image component and overlapping patches, the CS-MCA model can simultaneously achieve multi-component and global spar...

Heart rate (HR) estimation and monitoring is of great importance to determine a person's physiological and mental status. Recently, it has been demonstrated that HR can be remotely retrieved from facial video-based photoplethysmographic signals captured using professional or consumer-level cameras. Many efforts have been made to improve the detecti...

Generalized bilinear model (GBM) has received extensive attention in the field of hyperspectral nonlinear unmixing. Traditional GBM unmixing methods are usually assumed to be degraded only by additive white Gaussian noise (AWGN), and the intensity of AWGN in each band of hyperspectral image (HSI) is assumed to be the same. However, the real HSIs ar...

By integrating the information contained in multiple images of the same scene into one composite image, pixel-level image fusion is recognized as having high significance in a variety of fields including medical imaging, digital photography, remote sensing, video surveillance, etc. In recent years, deep learning (DL) has achieved great success in a...

The fusion of infrared and visible images of the same scene aims to generate a composite image which can provide a more comprehensive description of the scene. In this paper, we propose an infrared and visible image fusion method based on convolutional neural networks (CNNs). In particular, a siamese convolutional network is applied to obtain a wei...

Electroencephalography (EEG) recordings are frequently contaminated by both ocular and muscle artifacts. These are normally dealt with separately, by employing blind source separation (BSS) techniques relying on either second-order or higher-order statistics (SOS & HOS respectively). When HOS-based methods are used, it is usually in the setting of...

In the past few years, convolutional neural networks (CNNs) have exhibited great potential in the field of image classification. In this paper, we present a novel strategy named cross-level to improve the existing networks’ architecture in which different levels of feature representation in a network are merely connected in series. The basic idea o...

As is well known, activity level measurement and fusion rule are two crucial factors in image fusion. For most existing fusion methods, either in spatial domain or in a transform domain like wavelet, the activity level measurement is essentially implemented by designing local filters to extract high-frequency details, and the calculated clarity inf...

As a popular signal modeling technique, sparse representation (SR) has achieved great success in image fusion over the last few years with a number of effective algorithms being proposed. However, due to the patch-based manner applied in sparse coding, most existing SR-based fusion methods suffer from two drawbacks, namely, limited ability in detai...

Convolutional neural networks (CNNs) have exhibited great potential in the field of image classification in the past few years. In this paper, we present a novel strategy named cross-level to improve the existing CNNs’ architecture in which different levels of feature representation in a network are merely connected in series. The basic idea of cro...

Chessboard corner detection is a necessary procedure of the popular chessboard pattern-based camera calibration technique, in which the inner corners on a two-dimensional chessboard are employed as calibration markers. In this study, an automatic chessboard corner detection algorithm is presented for camera calibration. In authors' method, an initi...

In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the...

Multi-focus image fusion technique is an important approach to obtain a composite image with all objects in focus. The key point of multi-focus image fusion is to develop an effective activity level measurement to evaluate the clarity of source images. This paper proposes a novel image fusion method for multi-focus images with dense scale invariant...

In this study, a novel adaptive sparse representation (ASR) model is presented for simultaneous image fusion and denoising. As a powerful signal modelling technique, sparse representation (SR) has been successfully employed in many image processing applications such as denoising and fusion. In traditional SR-based applications, a highly redundant d...

Chessboard corner detection is a fundamental work of the popular chessboard pattern-based camera calibration technique. In this paper, a fast and robust algorithm for chessboard corner detection is presented. In our method, an initial corner set is obtained with an improved Hessian corner detector. And then, a novel strategy which takes both textur...

In this paper, we present a novel medical image fusion method by taking the complementary advantages of two powerful image representation theories: nonsubsampled contourlet transform (NSCT) and sparse representation (SR). In our fusion algorithm, the NSCT is firstly performed on each of the pre-registered source images to obtain the low-pass and hi...

Owing to convenience and naturalness, hand gesture recognition has been widely used in various human-computer interaction (HCI) systems. In this paper, we address the problem from the perspective of system, and present a static hand gesture recognition algorithm based on Krawtchouk moments. The effect of the order and number of Krawtchouk moments o...

Pan-sharpening is an important remote sensing image pre-processing technique, which aims at obtaining a high-resolution multispectral (HRM) image by integrating the spectral information of a low-resolution multispectral (LRM) image and the spatial details of a high-resolution panchromatic (HRP) image. This paper proposes a new pan-sharpening method...

Sparse representation (SR) has been widely used in many image processing applications including image fusion. As the contents vary significantly across different images, a highly redundant dictionary is always required in the sparse model, which reduces the algorithm stability and efficiency. This paper proposes a multi-focus image fusion method ba...