Figure 13 - available via license: Creative Commons Attribution 2.0 Generic
Content may be subject to copyright.
Inpainting example 1 obtained by applying the proposed method to larger missing areas. (a) Original image (342×512 pixels, 11.2% loss), (b) flag image of (a), and (c) results obtained by the proposed method.
Source publication
This paper presents an image inpainting method based on sparse representations optimized with respect to a perceptual metric. In the proposed method, the structural similarity (SSIM) index is utilized as a criterion to optimize the representation performance of image data. Specifically, the proposed method enables the formulation of two important p...
Similar publications
Spatial Light Modulators currently available lack of resolution to display large hologram, it is then necessary to find some technique to broaden the size of the reconstructed image. For example, time multiplexing can be use to tile several elementary holograms via a high speed scanning system. However, the best compromise to exploit the bandwidth...
Depth image super-resolution (SR) is a technique which can reconstruct a high-resolution (HR) depth image from a low-resolution (LR) depth image. Its purpose is to obtain HR details to meet the needs of various applications in computer vision. In general, conventional depth image SR methods often cause edges in the final HR image to be blurred or r...
SliceGAN is used to generate a 3D microstructure image from one or three 2D images in ferrite‐martensite dual phase steels. This method accelerates 3D visualization of microstructure to be hourly task. The 2D images sliced from the SliceGAN‐3D image seem to be quite similar with the original 2D images. The volume fraction of martensite in SliceGAN‐...
In order to obtain higher quality super-resolution reconstruction (SRR) of remote sensing images, an improved sparse representation remote sensing images SRR method is proposed in this paper. First, low-resolution image is processed by improved feature extract operator. The high-resolution image and low-resolution image blocks have the same sparse...
In this study, an efficient self‐learning method for image super‐resolution (SR) is presented. In the proposed algorithm, the input image is divided into equal size patches. Using these patches, a dictionary is learned based on K‐SVD, referred to as high resolution (HR) dictionary. Then, by down‐sampling, the columns of the dictionary, called atoms...
Citations
... The basic idea of this approach is to represent an image by the sparse combination of an overcomplete set of transforms; after that, the missing pixels are inferred by adaptively updating this sparse representation. This method is categorized as follows: constrained optimization strategy, the greedy strategy approximation, homotopy algorithm-based sparse representation, and proximity algorithm-based optimization strategy [23][24][25][26][27][28]. ...
... where 1 is a matrix with all elements of 1 and the size of the matrix is the same Ψp and is set to be 0.01 as the balancing strength of the constraints in (20) to (25). The constraint can be rewritten in the following way: ...
The approach described in this research is an exemplar‐based inpainting problem that combines a two‐stage structure tensor and image sparse representation to fill in any missing pixels. An important step is to select the filling order and local intensity smoothness, as well as to ensure that the structure is not destroyed. We employ a two‐stage structure tensor‐based priority for the filling order: finding the candidate patches and determining the appropriate weight of each candidate patch under the constraint of local patch consistency, then applying a blend of a sparse linear combination of candidate patches to fill in the missing region of the image. In addition, this technique may also be used for object removal. The proposed method yields results that are visually natural and qualitative.
... Over the past few years, the sparse representation of images has attracted researchers due to some of its unique advantages [25]. The sparse approach of an image has been successfully used in different image processing applications like super-resolution, inpainting, denoising, encryption, etc. of greyscale [25][26][27] and colored images [25,[28][29][30]. An image in the sparse domain can be represented by the linear combination of a few sparse vectors or atoms containing a few nonzero or significant coefficients [31]. ...
In this work, we have used the standard symmetric key block cipher data encryption algorithm RC5(32,16,8) for the encryption of digital greyscale images. For this, we have only considered the nonzero elements of the sparse matrix of the images which have been generated using the sparse representation technique. The security strength of the proposed technique is verified through different quality parameters, e.g. information entropy, correlation coefficients, NPCR, UACI, and NIST test. Our proposed algorithm achieved the maximum information entropy value of 7.9977 for the cameraman image and nearly zero correlation coefficient values establish the robustness of the cryptosystem. The highest NPCR and UACI values obtained in our work are 99.6916 and 33.9605, which are closer to the maximum theoretical values. The proposed technique is also compared with some of the contemporary works to validate the credibility of our proposed algorithm.
... where the Hadamard products with M in the data-fidelity term avoid trivial solutions (i.e., solutions where the unknown pixels' values are kept equal to 0). In this scenario, the data-fidelity term can take different forms, like the Mean Squared Error (MSE) or the Structural Similarity Index Measure (SSIM) [36]. The regularization term can assume a number of formulations as well, such as the L2 norm [37], the L1 norm [38], the Total Variation (TV) norm [39], and so forth. ...
This paper presents an extensive evaluation of the Deep Image Prior (DIP) technique for image inpainting on Synthetic Aperture Radar (SAR) images. SAR images are gaining popularity in various applications, but there may be a need to conceal certain regions of them. Image inpainting provides a solution for this. However, not all inpainting techniques are designed to work on SAR images. Some are intended for use on photographs, while others have to be specifically trained on top of a huge set of images. In this work, we evaluate the performance of the DIP technique that is capable of addressing these challenges: it can adapt to the image under analysis including SAR imagery; it does not require any training. Our results demonstrate that the DIP method achieves great performance in terms of objective and semantic metrics. This indicates that the DIP method is a promising approach for inpainting SAR images, and can provide high-quality results that meet the requirements of various applications.
... They are used to produce better results than diffusion-based techniques. The inpainting process fills the missing region from the neighbour surrounding pixels within the same patch (Ogawa and Haseyama 2013) and (Awati 2016). ...
Over the decades' researchers have studied the image inpainting problem intensively due to its high significance and effectiveness in various image processing applications such as people and object security, object removal, face editing applications. Image inpainting is defined as the process of completing or removing a missing region in images. It is considered one of the most challenging topics in the image processing field, although, it requires a deep understanding of the image details in terms of texture and structure. In this paper, a survey of most image inpainting techniques is presented and summarized with comparisons including the merits and demerits of each technique which could help researchers in evaluating their proposed techniques against.
... Zhang et al. (2014) introduced the concept of group-based sparse representation (GSR) where similar patches are grouped for sparse representation yielding high-quality image restoration. Ogawa and Haseyama (2013) introduced a structural similarity index metric (SSIM) for the calculation of coe±cients of sparse representation and for updating the dictionary. This method can¯ll large holes without blurring but it leaves some artifacts in the output. ...
Image inpainting is a process of reconstructing an incomplete image from the available information in a visually plausible way. In the proposed framework, existing image inpainting methods are classified in a new perspective. The information which is referred to, while reconstructing an image, is a critical factor of inpainting algorithms. Source of this information can be host image itself or an external source. The proposed framework broadly classifies inpainting algorithms into introspective and extrospective categories based on the source of information. Various parameters influencing the algorithms under these categories are identified in the proposed framework. A comprehensive list of all publicly available datasets along with the references are also summarized. Additionally, an in-depth analysis of the results obtained with the surveyed techniques is performed based on quantitative and qualitative parameters. The proposed framework aids the user in identifying the most suitable algorithm for various inpainting scenarios.
... Afterward, these techniques were expedited by Barnes et al. [9] who proposed PatchMatch, a fast randomized patch search algorithm that could handle the high computational and memory cost. Later such patch-based image completion techniques were improved by Darabi et al. [10] by incorporating gradient-domain image blending, He et al. [11] by computing the statistics of patch offsets and Ogawa et al. [19] by optimizing sparse representations w.r.t. SSIM perceptual metric. ...
In this paper, we present a novel image inpainting technique using frequency domain information. Prior works on image inpainting predict the missing pixels by training neural networks using only the spatial domain information. However, these methods still struggle to reconstruct high-frequency details for real complex scenes, leading to a discrepancy in color, boundary artifacts, distorted patterns, and blurry textures. To alleviate these problems, we investigate if it is possible to obtain better performance by training the networks using frequency domain information (Discrete Fourier Transform) along with the spatial domain information. To this end, we propose a frequency-based deconvolution module that enables the network to learn the global context while selectively reconstructing the high-frequency components. We evaluate our proposed method on the publicly available datasets CelebA, Paris Streetview, and DTD texture dataset, and show that our method outperforms current state-of-the-art image inpainting techniques both qualitatively and quantitatively.
... To evaluate the inpainting results we have used two objective criteria, PSNR [31] and SSIM [32] which are used commonly in similar works. It is noteworthy; PSNR and SSIM can be utilized when the reference image is available. ...
In this paper, we present a novel and efficient algorithm for image inpainting based on the structure and texture components. In our method, after decomposing the image into its texture and structure components using Principal Component Analysis (PCA), these components are inpainted separately using the proposed algorithm. Finally, the inpainted image is simply acquired by adding the two inpainted images. For structure inpainting we used quadtree concept to identify the importance of each pixel located on the boundary of the target region. Subsequently, we detect the correct path for filling so that this path demonstrates an orientation for the better structure inpainting. It is noteworthy that structure inpainting is more important because human vision is sensitive to the coherence of structure. For texture inpainting, we use Euclidean distance in the texture component for patch selection. Also, the geometric feature is considered by Local Steering Kernel (LSK) in the original image to assist chooing a better patch candidate. The experimental results of our algorithm demonstrate the effectiveness of the proposed method. © 2020 Materials and Energy Research Center. All rights reserved.
... Consequently, introducing objective metrics is one of the challenges in this scope. The PSNR [39], SSIM [30] are quality metrics that are used for determining algorithm performance when the reference image is available. ...
... Another prior knowledge as non-local self-similarity is considered in the patch-based inpainting algorithm so that the statistics of patch offsets [20] and geometric transform [12] are proposed in an objective function. Sparsity prior [30] is introduced in sparsity-based inpainting and low-rank property [4,27] which is applied in natural images. By using lowrank assumption, an image can be considered as a low-rank matrix or low-rank tensor, as well as a simplified assumption are image patches represented by a low-rank matrix. ...
... Zhang et al. proposed a sparse representation model for a group of similar structure patches instead of considering a patch as a basic unit of sparse representation [41]. Ogawa and Haseyama applied the SSIM metric for computing sparse representation coefficients and dictionary updating [30]. Fan and Zhang considered adaptive patch size based on patch sparsity also proposed new metrics for patches difference to acquire rotation invariant matched patches [14]. ...
In this paper, we present a new algorithm for image inpainting using structure and texture information. Our image decomposition to texture and structure is accomplished by the SVD method in the primary step, and then an algorithm for texture inpainting is applied. At the next level, edge detection is used in target region related to inpainted texture component. The detected edges demonstrate border of different textures in the target region, and the boundary pixels are ignored from mask temporarily. The other target pixels should be primarily inpainted, and then border pixels would be filled subsequently. Experimental results of this algorithm show better consistency in comparison with state of the art methods.
... Differential based inpainting uses the concept of variational methods and PDEs. Exemplar based inpainting focuses on filling absent information from nearby surrounding pixels at patch level (Ogawa and Haseyama, 2013;Amasidha et al., 2016;Vreja and Brad, 2014). The performance assessment of image inpainting can be judged on the basis of the produced subspaces and linear coefficients for estimating linear combination (Ogawa and Haseyama, 2013). ...
... Exemplar based inpainting focuses on filling absent information from nearby surrounding pixels at patch level (Ogawa and Haseyama, 2013;Amasidha et al., 2016;Vreja and Brad, 2014). The performance assessment of image inpainting can be judged on the basis of the produced subspaces and linear coefficients for estimating linear combination (Ogawa and Haseyama, 2013). ...
... This technique not only works on geometric methods but also on hybrid and texture methods. A simple flow chart of the technique is given in Fig. 7. Ogawa and Haseyama (2013) presented a technique based upon sparse representations improved concerning with perpetual metric. The technique presented uses Structural Similarity (SSIM) index for the better conduct of the image data. ...
... Thus, to obtain a virtual restored image which is faithful to the original as much as possible, we recently proposed an improvement to this algorithm. In this improved version, the identified bleed-through pattern is inpainted in continuity with its surrounding, by using techniques of image inpainting based on sparse representation [32], [33]. ...
In recent years, extensive campaigns of digitization of the documental heritage conserved in libraries and archives have been performed, with the primary goal to ensure the
preservation and fruition of this important part of the human cultural and historical patrimony. Besides protecting conservation, the availability of high quality digital copies has increasingly stimulated the use of image processing techniques, to perform a number of
operations on documents and manuscripts, without harming the often precious and fragile originals. Among those, virtual restoration tasks are crucial, as they facilitate the traditional work of philologists and paleographers, and constitute a first step towards an automatic analysis of the written contents. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most
frequent and impairing degradations affecting ancient manuscripts, i.e., the bleed-through distortion. We show that techniques of blind source separation are versatile tools to either cancel these unwanted interferences or isolate specific features for content analysis goals. Specialized algorithms, based on recto-verso models and sparse image representation, are then shown to be able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.
