Fig 1 - uploaded by Jianchao Yang
Content may be subject to copyright.
Image formation process illustration. Typical image formation process includes: (1) low-pass filtering and (2)
Source publication
This paper presents a non-local kernel regression (NL-KR) method for image and video restoration tasks, which exploits both
the non-local self-similarity and local structural regularity in natural images. The non-local self-similarity is based on
the observation that image patches tend to repeat themselves in natural images and videos; and the loca...
Contexts in source publication
Context 1
... and video restoration aims to estimate the high-quality version of the low-quality observations, which are typically noisy and of low resolution. Figure 1 depicts the typical image formation process assumed in restoration literatures, where the low-quality observations are obtained by blurring and subsampling from the high-quality underlying image with sensing noise. This process can be modeled as follows: ...
Context 2
... further make more comparisons with state-of-the-art methods on real images, where the input LR image is zoomed by a factor of 4, as shown in Figure 9 and Figure 10. Note that these methods are designed specifically to work on single images. ...
Context 3
... Figure 9, it can be seen that the proposed method can preserve more details than Fattal's method [35] and is comparable with Kim's method [34] and the more recent work [14]. In Figure 10, however, our algorithm outperforms both [35] and [14], where our result is free of the jaggy artifacts on the edges and the characters generated by our method is more realistic. ...
Context 4
... average PSNR and SSIM results on these three test sequences are given in Table V. As shown, the proposed method achieves better reconstruction accuracy than GNL-Means and BM3D. 3 In Figure 11, we further show the PSNR results on Foreman 3 The PSNR results of 3D-KR are not listed, because they are not numerically available in their original papers (plotted in a figure). However, compared with their figure, our method improves over GNL-Means by a larger margin than the 3D-KR method. ...
Context 5
... compared with their figure, our method improves over GNL-Means by a larger margin than the 3D-KR method. Miss America sequences are given in Figure 12 and Figure 13 respectively for visual comparison. Note that GNL-Means sometimes generates severe block artifacts (see the Mouth part in Figure 12 and Eye part in Figure 13). ...
Context 6
... compared with their figure, our method improves over GNL-Means by a larger margin than the 3D-KR method. Miss America sequences are given in Figure 12 and Figure 13 respectively for visual comparison. Note that GNL-Means sometimes generates severe block artifacts (see the Mouth part in Figure 12 and Eye part in Figure 13). ...
Context 7
... America sequences are given in Figure 12 and Figure 13 respectively for visual comparison. Note that GNL-Means sometimes generates severe block artifacts (see the Mouth part in Figure 12 and Eye part in Figure 13). The 3D-KR method, on the other hand, will generate some ghost effects, due to overfitting of the regression and inaccurate estimation of the 3D kernel (see the Mouth part in Figure 12). ...
Context 8
... America sequences are given in Figure 12 and Figure 13 respectively for visual comparison. Note that GNL-Means sometimes generates severe block artifacts (see the Mouth part in Figure 12 and Eye part in Figure 13). The 3D-KR method, on the other hand, will generate some ghost effects, due to overfitting of the regression and inaccurate estimation of the 3D kernel (see the Mouth part in Figure 12). ...
Context 9
... that GNL-Means sometimes generates severe block artifacts (see the Mouth part in Figure 12 and Eye part in Figure 13). The 3D-KR method, on the other hand, will generate some ghost effects, due to overfitting of the regression and inaccurate estimation of the 3D kernel (see the Mouth part in Figure 12). Furthermore, the 3D-KR method has to employ a motion pre-compensation procedure in order for good 3D kernel estimation, while our model does not require this step. ...
Context 10
... the 3D-KR method has to employ a motion pre-compensation procedure in order for good 3D kernel estimation, while our model does not require this step. Finally, the BM3D method generates severe artifacts at edge areas, as shown in Figure 12 (second row), while our method can recover the edge structure much better. Our method is also better than BM3D in terms of objective evaluation. ...
Similar publications
The alternating direction method is widely applied in total variation image restoration. However, the search directions of the method are not accurate enough. In this paper, one method based on the subspace optimization is proposed to improve its optimization performance. This method corrects the search directions of primal alternating direction me...
This paper proposes a new k Nearest Neighbor (kNN) algorithm based on sparse learning, so as to overcome the drawbacks of the previous kNN algorithm, such as the fixed k value for each test sample and the neglect of the correlation of samples. Specifically, the paper reconstructs test samples by training samples to learn the optimal k value for eac...
As the matrix formed by nonlocal similar patches in a natural image is of a low rank, the nuclear norm minimization (NNM) has been widely studied for image processing. Since the singular values have clear meanings and should be treated differently, NNM regularizes each of them equally, which often restricts its capability and flexibility. Recent ad...
High qualityimage reconstruction fromsparse projecting data taken within a limited observation
angle is in general not possible due to a problem ill-posedness. Wepropose here the verified mathematical
background and numerical procedure for a special class of objects, namely the objects with bi-connected
domain, which classical examples are pipe-...
![Fig. 3: Reconstructed images comparison on the PICMUS [17] dataset...](profile/Yuxin-Zhang-11/publication/378306559/figure/fig2/AS:11431281230651574@1711045363320/Reconstructed-images-comparison-on-the-PICMUS-17-dataset-using-various-approaches-All_Q320.jpg)
Ultrasound image reconstruction can be approximately cast as a linear inverse problem that has traditionally been solved with penalized optimization using the \(l_1\) or \(l_2\) norm, or wavelet-based terms. However, such regularization functions often struggle to balance the sparsity and the smoothness. A promising alternative is using learned pri...
Citations
... With respect to regression-based methods, we can cite the work of Zhang et al. [41], who proposed a non-local kernel regression framework that takes profit of non-local selfsimilarity of image patches that tend to repeat themselves in natural images and the local structural regularity properties in image patches. Sun et al. [26] exploited the use of gradient profiles describing the shape and sharpness of image gradients to estimate a high-resolution image from a low resolution image. ...
Image deblurring and super resolution attempts to restore images that have been degraded. We propose a joint technique for super resolution and deblurring to solve the problem of blur and low resolution in text images. This joint technique is based on the use of a Deep Convolutional Neural Network (Deep CNN). Deep CNN has achieved promising performance for single image super-resolution. In particular, the Deep CNN skip Connection and Network in Network (DCSCN) architecture has been successfully applied to natural images super-resolution. In this work we propose a model that jointly performs super-resolution and deblurring of low-resolution blurry text images based on DCSCN. Our model uses subsampled blurry images in the input and original sharp images as ground truth. The proposed architecture consists of a higher number of filters in the input CNN layer to a better analysis of the text details. The experimental results have achieved state-of-the-art performance in the peak-signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), the information fidelity criterion (IFC) and Visual Information Fidelity (VIF) metrics. Thus, we confirm that DCSCN provides satisfactory results for enhancement tasks on low blurry images. The quantitative and qualitative evaluation on different datasets proves the high performance of our model to reconstruct high-resolution and sharp text images with PSNR= 20.406, SSIM= 0.877, VIF= 0.351, IFC= 2.868 for scale 4 compared to DCSCN with PSNR= 15.553, SSIM= 0.621, VIF= 0.166 IFC= 1.129. In addition, in terms of computational time, our proposed method gives competitive performance compared to state-of-the-art methods.
... The loss is given as Total variation loss:. To encourage smoothness at high resolution, we follow former works [JAF16,ZYZH10] to utilize total variation regulation [AD05].x is used to represent G(x, φ(r)) to simplify the expression of the regulation, which is formulated as ...
Automatic sketch colourization is a highly interestinged topic in the image‐generation field. However, due to the absence of texture in sketch images and the lack of training data, existing reference‐based methods are ineffective in generating visually pleasant results and cannot edit the colours using text tags. Thus, this paper presents a conditional generative adversarial network (cGAN)‐based architecture with a pre‐trained convolutional neural network (CNN), reference‐based channel‐wise attention (RBCA) and self‐adaptive multi‐layer perceptron (MLP) to tackle this problem. We propose two‐step training and spatial latent manipulation to achieve high‐quality and colour‐adjustable results using reference images and text tags. The superiority of our approach in reference‐based colourization is demonstrated through qualitative/quantitative comparisons and user studies with existing network‐based methods. We also validate the controllability of the proposed model and discuss the details of our latent manipulation on the basis of experimental results of multi‐label manipulation.
... With respect to regression-based methods, we can cite the work of Zhang et al [9], who proposed a non-local kernel regression framework that takes profit of non-local selfsimilarity of image patches that tend to repeat themselves in natural images and the local structural regularity properties in image patches. Sun et al. [10] exploited the use of gradient profiles describing the shape and sharpness of image gradients to estimate a high-resolution image from a low resolution image. ...
p>Deep convolutional neural networks (Deep CNN) have achieved hopeful performancefor single image super-resolution. In particular, the Deep CNN skip Connection andNetwork in Network (DCSCN) architecture has been successfully applied to naturalimages super-resolution. In this work we propose an approach called SDT-DCSCN thatjointly performs super-resolution and deblurring of low-resolution blurry text imagesbased on DCSCN. Our approach uses subsampled blurry images in the input and origi-nal sharp images as ground truth. The used architecture is consists of a higher numberof filters in the input CNN layer to a better analysis of the text details. The quantitativeand qualitative evaluation on different datasets prove the high performance of our modelto reconstruct high-resolution and sharp text images. In addition, in terms of computa-tional time, our proposed method gives competitive performance compared to state ofthe art methods.</p
... With respect to regression-based methods, we can cite the work of Zhang et al Zhang et al. (2010), who proposed a nonlocal kernel regression framework that takes profit of non-local self-similarity of image patches that tend to repeat themselves in natural images and the local structural regularity properties in image patches. Sun et al. ...
Deep convolutional neural networks (Deep CNN) have achieved hopeful performance for single image super-resolution. In particular, the Deep CNN skip Connection and Network in Network (DCSCN) architecture has been successfully applied to natural images super-resolution. In this work we propose an approach called SDT-DCSCN that jointly performs super-resolution and deblurring of low-resolution blurry text images based on DCSCN. Our approach uses subsampled blurry images in the input and original sharp images as ground truth. The used architecture is consists of a higher number of filters in the input CNN layer to a better analysis of the text details. The quantitative and qualitative evaluation on different datasets prove the high performance of our model to reconstruct high-resolution and sharp text images. In addition, in terms of computational time, our proposed method gives competitive performance compared to state of the art methods.
... Earlier approaches rely on pure image processing such as bilinear interpolation, nearest-neighbour interpolation, bicubic interpolation, and other interpolation-based [16]. Some other approaches like natural image statistic [17] [18], Pre-defined Models [19]. A deep convolutional network has recently shown explosive popularity and powerful capability in mapping LR image to HR image [20] [21]. ...
License Plate Recognition (LPR) is an important implemented application of Artificial Intelligence (AI) and deep learning in the past decades. However, due to the low image quality caused by the fast movement of vehicles and low-quality analogue cameras, many plate numbers cannot be recognised accurately by LPR models. To solve this issue, we propose a new deep learning architecture called D_GAN_ESR (Double Generative Adversarial Networks for Image Enhancement and Super Resolution) used for effective image denoising and super-resolution for license plate images. In this paper, we show the limitation of the existing networks for image enhancement and image super-resolution. Furthermore, a features based evaluation metric called Peak Signal to Noise Ratio Features (PSNR-F) is used to evaluate and compare performance between different methods. It is shown that the use of PSNR-F has a better performance indicator than the classical PSNR-pixel-to-pixel (PSNR-pixel) evaluation metric. The results show that using D_GAN_ESR to enhance the license plate images increases the LPR accuracy from 30% to 78% when blur images are used and increases the accuracy from 59% to 74.5% when low-quality images are used.
... Image super-resolution aims at recovering HR images from the LR counterparts, which has become a significant task in the field of computer vision for its widespread application in surveillance [1,2], image enhancement [17], medical imaging [18], face recognition [4,5], etc. Earlier works utilized prediction-based methods [19], edge-based methods [20,21], statistical methods [22,23], and patch-based methods [24,25] to reconstruct HR images. Recently, with the rapid development of deep learning techniques, a large number of deep-learning based super-resolution methods [6,7,26,27] have been proposed and shown impressive performance. ...
Face super-resolution aims to recover high-resolution face images with accurate geometric structures. Most of the conventional super-resolution methods are trained on paired data that is difficult to obtain in the real-world setting. Besides, these methods do not fully utilize facial prior knowledge for face super-resolution. To tackle these problems, we propose an end-to-end unsupervised face super-resolution network to super-resolve low-resolution face images. We propose a gradient enhancement branch and a semantic guidance mechanism. Specifically, the gradient enhancement branch reconstructs high-resolution gradient maps, under the restriction of two proposed gradient losses. Then the super-resolution network integrates features in both image and gradient space to super-resolve face images with geometric structure preservation. Moreover, the proposed semantic guidance mechanism, including a semantic-adaptive sharpen module and a semantic-guided discriminator, can reconstruct sharp edges and improve local details in different facial regions adaptively, under the guidance of semantic parsing maps. Qualitative and quantitative experiments demonstrate that our proposed method can reconstruct high-resolution face images with sharp edges and photo-realistic details, outperforming the state-of-the-art methods.
... When the spatial resolution of images is reduced, some information is permanently lost and cannot be recovered. Most SR techniques obtain information within the image itself [41], or from outside [42], [43] to restore the lost information. Recently, deep convolutional neural networks (DCN) have been used for many SR problems and they have shown impressive performance improvement. ...
Super-resolution (SR) is an ill-posed problem. Generating high-resolution (HR) images from low-resolution (LR) images remains a major challenge. Recently, SR methods based on deep convolutional neural networks (DCN) have been developed with impressive performance improvement. DCN-based SR techniques can be largely divided into peak signal-to-noise ratio (PSNR)-oriented SR networks and generative adversarial networks (GAN)-based SR networks. In most current GAN-based SR networks, the perceptual loss is computed from the feature maps of a single layer or several fixed layers using a differentiable feature extractor such as VGG. This limited layer utilization may produce overly textured artifacts. In this paper, a new edge texture metric (ETM) is proposed to quantify the characteristics of images and then it is utilized only in the training phase to select an appropriate layer when calculating the perceptual loss. We present experimental results showing that the GAN-based SR network trained with the proposed method achieves qualitative and quantitative perceptual quality improvements compared to many of the existing methods.
... With respect to regression-based methods, we can cite the work of Zhang et al [9], who proposed a non-local kernel regression framework that takes profit of non-local selfsimilarity of image patches that tend to repeat themselves in natural images and the local structural regularity properties in image patches. Sun et al. [10] exploited the use of gradient profiles describing the shape and sharpness of image gradients to estimate a high-resolution image from a low resolution image. ...
We suggest a novel approach that performs jointlysuper-resolution and deblurring from a low blurry image. Theexperimental results have achieved state-of-the-art performancein PSNR and SSIM metrics. Thus, we confirm that DCSCNprovides satisfactory results for enhancement tasks on low blurryimages.
... With respect to regression-based methods, we can cite the work of Zhang et al [9], who proposed a non-local kernel regression framework that takes profit of non-local selfsimilarity of image patches that tend to repeat themselves in natural images and the local structural regularity properties in image patches. Sun et al. [10] exploited the use of gradient profiles describing the shape and sharpness of image gradients to estimate a high-resolution image from a low resolution image. ...
p>Deep convolutional neural networks (Deep CNN) have achieved hopeful performancefor single image super-resolution. In particular, the Deep CNN skip Connection andNetwork in Network (DCSCN) architecture has been successfully applied to naturalimages super-resolution. In this work we propose an approach called SDT-DCSCN thatjointly performs super-resolution and deblurring of low-resolution blurry text imagesbased on DCSCN. Our approach uses subsampled blurry images in the input and origi-nal sharp images as ground truth. The used architecture is consists of a higher numberof filters in the input CNN layer to a better analysis of the text details. The quantitativeand qualitative evaluation on different datasets prove the high performance of our modelto reconstruct high-resolution and sharp text images. In addition, in terms of computa-tional time, our proposed method gives competitive performance compared to state ofthe art methods.</p
... The problem of SISR has been widely studied. Early approaches either rely on natural image statistics [23,45] or predefined models [18,14,35]. Later, mapping functions between LR images and HR images are investigated, such as sparse coding based SR methods [44,39]. ...
Several methods have recently been proposed for the Single Image Super-Resolution (SISR) problem. The current methods assume that a single low-resolution image can only yield a single high-resolution image. In addition, all of these methods use low-resolution images that were artificially generated through simple bilinear down-sampling. We argue that, first and foremost, the problem of SISR is an one-to-many mapping problem between the low resolution and all possible candidate high-resolution images and we address the challenging task of learning how to realistically degrade and down-sample high-resolution images. To circumvent this problem, we propose SR-NAM which utilizes the Non-Adversarial Mapping (NAM) technique. Furthermore, we propose a degradation model that learns how to transform high-resolution images to low-resolution images that resemble realistically taken low-resolution photos. Finally, some qualitative results for the proposed method along with the weaknesses of SR-NAM are included.
