No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
NormalNet: Learning-Based Mesh Normal Denoising via Local Partition Normalization
Abstract
Mesh denoising is a critical technology in geometry processing that aims to recover high-fidelity 3D mesh models of objects from noise-corrupted versions. In this work, we propose a learning-based mesh normal denoising scheme, called
NormalNet
, which employs deep networks to find the correlation between the volumetric representation and denoised face normal. Overall,
NormalNet
follows the iterative framework of filtering-based mesh denoising. During each iteration, firstly, a local partition normalization strategy is applied to split the local structure around each face into dense voxels, in which both the structure and face normal information can be preserved during this transformation. Benefiting from the thorough information preservation, we can use simple residual networks, which employ the volumetric representation as the input and produce the learned denoised face normal, to achieve satisfactory results. Finally, the vertex positions are updated according to the denoised normals. Besides introducing normalization into mesh denoising, our main contributions include a classification-based training faces selection strategy for balancing the training set and a mismatched-faces rejection strategy for removing the mismatched faces between noisy mesh and ground truth. Compared to state-of-the-art works,
NormalNet
can effectively remove noise while preserving the original features and avoiding pseudo-features.
... Wang et al. [30] proposed a data-driven mesh denoising approach using cascaded regression functions, in which geometry features were extracted from neighborhood faces and mapped to denoise face normal vectors by using neural networks trained on clustered faces. Zhao et al. [31] proposed the NormalNet, which uses convolutional neural networks (CNNs) to obtain denoised normal vectors and follows an iterative framework of a filtering-based scheme. DNF-Net [32] uses a combination of encoder and decoder neural networks to regress the face normal vectors based on the vertex positions. ...
... In this case, an iterative process updates the final vertex positions according to the information on the vertex positions and face normal vectors. Most of the current techniques predicted noise-free face normal vectors using many input data, such as manually crafted local geometric features [30,31,35] or learned features encoded by a neural network [31][32][33]36]. After obtaining the noise-free normal, the traditional normal filtering methods typically use the standard iterative vertex updating approach [37] to produce output vertex positions [22,23]. ...
... In this case, an iterative process updates the final vertex positions according to the information on the vertex positions and face normal vectors. Most of the current techniques predicted noise-free face normal vectors using many input data, such as manually crafted local geometric features [30,31,35] or learned features encoded by a neural network [31][32][33]36]. After obtaining the noise-free normal, the traditional normal filtering methods typically use the standard iterative vertex updating approach [37] to produce output vertex positions [22,23]. ...
... These issues prevent the deployment of CNNs in mesh denoising. Accordingly, different from 2D image denoising in which CNNs-based strategy has become the basic methodology (Zhang et al. 2017), to the best of our knowledge, there are only a few deep learning based schemes for mesh denoising (Wang, Liu, and Tong 2016;Li et al. 2020a,b;Armando, Franco, and Boyer 2020;Zhao et al. 2021;Shen et al. 2021). ...
... • Learning-based schemes: 1) Cascaded normal regression (CNR) (Wang, Liu, and Tong 2016), 2) Facet graph convolutions (FGC) (Armando, Franco, and Boyer 2020), 3) Normalf-net (NFN) (Li et al. 2020b), 4) Nor-malNet (NNT) (Zhao et al. 2021), 5) GCN-Denoiser (GCN) (Shen et al. 2021). • Traditional schemes: 6) Guided normal filtering (GNF) (Zhang et al. 2015), 7) Non-local low-rank normal filtering (NLF) (Li et al. 2018). ...
3D meshes are widely employed to represent geometry structure of 3D shapes. Due to limitation of scanning sensor precision and other issues, meshes are inevitably affected by noise, which hampers the subsequent applications. Convolultional neural networks (CNNs) achieve great success in image processing tasks, including 2D image denoising, and have been proven to own the capacity of modeling complex features at different scales, which is also particularly useful for mesh denoising. However, due to the nature of irregular structure, CNNs-based denosing strategies cannot be trivially applied for meshes. To circumvent this limitation, in the paper, we propose the local surface descriptor (LSD), which is able to transform the local deformable surface around a face into 2D grid representation and thus facilitates the deployment of CNNs to generate denoised face normals. To verify the superiority of LSD, we directly feed LSD into the classical Resnet without any complicated network design. The extensive experimental results show that, compared to the state-of-the-arts, our method achieves encouraging performance with respect to both objective and subjective evaluations.
... In deep learning, the abnormal distribution of data scales can lead to Internal Covariate Shift, where normalization layers play a crucial role in controlling and constraining data scales and distributions to facilitate model training. Common normalization methods include Batch Normalization (BN) [37], Instance Normalization (IN) [38], Region Normalization (RN) [39], Local Partition Normalization [40], and Spatially-Adaptive Normalization (SPADE) [41]. These methods typically involve operations such as addition, subtraction, multiplication, and division on the input to achieve mean subtraction, standard deviation division, multiplication by learnable parameter gamma, and addition of learnable parameter beta, resulting in the output. ...
The image harmonization task endeavors to adjust foreground information within an image synthesis process to achieve visual consistency by leveraging background information. In academic research, this task conventionally involves the utilization of simple synthesized images and matching masks as inputs. However, obtaining precise masks for image harmonization in practical applications poses a significant challenge, thereby creating a notable disparity between research findings and real-world applicability. To mitigate this disparity, we propose a redefinition of the image harmonization task as “Unified Image Harmonization,” where the input comprises only a single image, thereby enhancing its applicability in real-world scenarios. To address this challenge, we have developed a novel framework. Within this framework, we initially employ inharmonious region localization to detect the mask, which is subsequently utilized for harmonization tasks. The pivotal aspect of the harmonization process lies in normalization, which is accountable for information transfer. Nonetheless, the current background-to-foreground information transfer and guidance mechanisms are limited by single-layer guidance, thereby constraining their effectiveness. To overcome this limitation, we introduce Region Augmented Attention Normalization (RA2N), which enhances the attention mechanism for foreground feature alignment, consequently leading to improved alignment and transfer capabilities. Through qualitative and quantitative comparisons on the iHarmony4 dataset, our model exhibits exceptional performance not only in unified image harmonization but also in conventional image harmonization tasks.
Mesh denoising is a fundamental yet challenging task. Most of the existing data‐driven methods only consider the zero‐order information (vertex location) and first‐order information (face normal). However, higher‐order geometric information (such as curvature) is more descriptive for the shape of the mesh. Therefore, in order to impose such high‐order information, this paper proposes a novel Curvature‐Driven Multi‐Stream Graph Convolutional Neural Network (CDMS‐Net) architecture. CDMS‐Net has three streams, including curvature stream, face normal stream and vertex stream, where the curvature stream focuses on the high‐order Gaussian curvature information. Moreover, CDMS‐Net proposes a novel block based on residual dense connections, which is used as the core component to extract geometric features from meshes. This innovative design improves the performance of feature‐preserving denoising. The plug‐and‐play modular design makes CDMS‐Net easy to be implemented. Multiple sets of ablation study are carried out to verify the rationality of the CDMS‐Net. Our method establishes new state‐of‐the‐art mesh denoising results on publicly available datasets.
Self-supervised denoising frameworks have recently been proposed to learn denoising models without noisy-clean image pairs, showing great potential in various applications. The denoising model is expected to produce visually pleasant images without noise patterns. However, it is non-trivial to achieve this goal using self-supervised methods because 1) the self-supervised model is difficult to restore the perceptual information due to the lack of clean supervision, and 2) perceptual quality is relatively subjective to users’ preferences. In this paper, we make the first attempt to build an interactive self-supervised denoising model to tackle the aforementioned problems. Specifically, we propose an interactive two-branch network to effectively restore perceptual information. The network consists of a denoising branch and an interactive branch, where the former focuses on efficient denoising, and the latter modulates the denoising branch. Based on the delicate architecture design, our network can produce various denoising outputs, allowing the user to easily select the most appealing outcome for satisfying the perceptual requirement. Moreover, to optimize the network with only noisy images, we propose a novel two-stage training strategy in a self-supervised way. Once the network is optimized, it can be interactively changed between noise reduction and texture restoration, providing more denoising choices for users. Existing self-supervised denoising methods can be integrated into our method to be user-friendly with interaction. Extensive experiments and comprehensive analyses are conducted to validate the effectiveness of the proposed method.
This study proposes a deep-learning framework for mesh denoising from a single noisy input, where two graph convolutional networks are trained jointly to filter vertex positions and facet normals apart. The prior obtained only from a single input is particularly referred to as a self-prior. The proposed method leverages the framework of the deep image prior (DIP), which obtains the self-prior for image restoration using a convolutional neural network (CNN). Thus, we obtain a denoised mesh without any ground-truth noise-free meshes. Compared to the original DIP that transforms a fixed random code into a noise-free image by the neural network, we reproduce vertex displacement from a fixed random code and reproduce facet normals from feature vectors that summarize local triangle arrangements. After tuning several hyperparameters with a few validation samples, our method achieved significantly higher performance than traditional approaches working with a single noisy input mesh. Moreover, its performance is better than the other methods using deep neural networks trained with a large-scale shape dataset. The independence of our method of either large-scale datasets or ground-truth noise-free mesh will allow us to easily denoise meshes whose shapes are rarely included in the shape datasets. Our code is available at: https://github.com/astaka-pe/Dual-DMP.git.
KeywordsMesh denoisingSelf-priorGraph convolutional network
We examine the problem of mesh denoising, which consists of removing noise from corrupted 3D meshes while preserving existing geometric features. Most mesh denoising methods require a lot of mesh-specific parameter fine-tuning, to account for specific features and noise types. In recent years, data-driven methods have demonstrated their robustness and effectiveness with respect to noise and feature properties on a wide variety of geometry and image problems. Most existing mesh denoising methods still use hand-crafted features, and locally denoise facets rather than examine the mesh globally. In this work, we propose the use of a fully end-to-end learning strategy based on graph convolutions, where meaningful features are learned directly by our network. It operates on a graph of facets, directly on the existing topology of the mesh, without resampling, and follows a multi-scale design to extract geometric features at different resolution levels. Similar to most recent pipelines, given a noisy mesh, we first denoise face normals with our novel approach, then update vertex positions accordingly. Our method performs significantly better than the current state-of-the-art learning-based methods. Additionally, we show that it can be trained on noisy data, without explicit correspondence between noisy and ground-truth facets. We also propose a multi-scale denoising strategy, better suited to correct noise with a low spatial frequency.
This paper presents a deep normal filtering network, called DNF-Net, for mesh denoising. To better capture local geometry, our network processes the mesh in terms of local patches extracted from the mesh. Overall, DNF-Net is an end-to-end network that takes patches of facet normals as inputs and directly outputs the corresponding denoised facet normals of the patches. In this way, we can reconstruct the geometry from the denoised normals with feature preservation. Besides the overall network architecture, our contributions include a novel multi-scale feature embedding unit, a residual learning strategy to remove noise, and a deeply-supervised joint loss function. Compared with the recent data-driven works on mesh denoising, DNF-Net does not require manual input to extract features and better utilizes the training data to enhance its denoising performance. Finally, we present comprehensive experiments to evaluate our method and demonstrate its superiority over the state of the art on both synthetic and real-scanned meshes.
Recent advances in 3D scanning technology have enabled the deployment of 3D models in various industrial applications like digital twins, remote inspection and reverse engineering. Despite their evolving performance, 3D scanners, still introduce noise and artifacts in the acquired dense models. In this work, we propose a fast and robust denoising method for dense 3D scanned industrial models. The proposed approach employs conditional variational autoencoders to effectively filter face normals. Training and inference are performed in a sliding patch setup reducing the size of the required training data and execution times. We conducted extensive evaluation studies using 3D scanned and CAD models. The results verify plausible denoising outcomes, demonstrating similar or higher reconstruction accuracy, compared to other state-of-the-art approaches. Specifically, for 3D models with more than 1e4 faces, the presented pipeline is twice as fast as methods with equivalent reconstruction error.
The most challenging problem in mesh denoising is to distinguish features from noise. Based on the robust guided normal estimation and alternate vertex updating strategy, we investigate a new feature-preserving mesh denoising method. To accurately capture local structures around features, we propose a corner-aware neighborhood (CAN) scheme. By combining both overall normal distribution of all faces in a CAN and individual normal influence of the interested face, we give a new consistency measuring method, which greatly improves the reliability of the estimated guided normals. As the noise level lowers, we take as guidance the previous filtered normals, which coincides with the emerging rolling guidance idea. In the vertex updating process, we classify vertices according to filtered normals at each iteration and reposition vertices of distinct types alternately with individual regularization constraints. Experiments on a variety of synthetic and real data indicate that our method adapts to various noise, both Gaussian and impulsive, no matter in the normal direction or in a random direction, with few triangles flipped.
This paper presents a simple and effective two-stage mesh denoising algorithm, where in the first stage, the face normal filtering is done by using the bilateral normal filtering in the robust statistics framework. \textit{Tukey's bi-weight function} is used as similarity function in the bilateral weighting, which is a robust estimator and stops the diffusion at sharp edges which helps to retain features and removes noise from flat regions effectively. In the second stage, an edge-weighted Laplace operator is introduced to compute the differential coordinate. This differential coordinate helps the algorithm to produce a high-quality mesh without any face normal flips and also makes the method robust against high-intensity noise.
Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising. However, these methods mostly learn a specific model for each noise level, and require multiple models for denoising images with different noise levels. They also lack flexibility to deal with spatially variant noise, limiting their applications in practical denoising. To address these issues, we present a fast and flexible denoising convolutional neural network, namely FFDNet, with a tunable noise level map as the input. The proposed FFDNet works on downsampled sub-images to speed up the inference, and adopts orthogonal regularization to enhance the generalization ability. In contrast to the existing discriminative denoisers, FFDNet enjoys several desirable properties, including (i) the ability to handle a wide range of noise levels (i.e., [0, 75]) effectively with a single network, (ii) the ability to remove spatially variant noise by specifying a non-uniform noise level map, and (iii) faster speed than benchmark BM3D even on CPU without sacrificing denoising performance. Extensive experiments on synthetic and real noisy images are conducted to evaluate FFDNet in comparison with state-of-the-art denoisers. The results show that FFDNet is effective and efficient, making it highly attractive for practical denoising applications.
We propose a robust and effective mesh denoising approach consisting of three steps: vertex pre-filtering, -median normal filtering, and vertex updating. Given an input noisy mesh model, our method generates a high quality model that preserves geometric features. Our approach is more robust than state of the art approaches when denoising models with different levels of noise and can handle models with irregular surface sampling.
We present a data-driven approach for mesh denoising. Our key idea is to formulate the denoising process with cascaded non-linear regression functions and learn them from a set of noisy meshes and their ground-truth counterparts. Each regression function infers the normal of a denoised output mesh facet from geometry features extracted from its neighborhood facets on the input mesh and sends the result as the input of the next regression function. Specifically, we develop a filtered facet normal descriptor (FND) for modeling the geometry features around each facet on the noisy mesh and model a regression function with neural networks for mapping the FNDs to the facet normals of the denoised mesh. To handle meshes with different geometry features and reduce the training difficulty, we cluster the input mesh facets according to their FNDs and train neural networks for each cluster separately in an offline learning stage. At runtime, our method applies the learned cascaded regression functions to a noisy input mesh and reconstructs the denoised mesh from the output facet normals.
Our method learns the non-linear denoising process from the training data and makes no specific assumptions about the noise distribution and geometry features in the input. The runtime denoising process is fully automatic for different input meshes. Our method can be easily adapted to meshes with arbitrary noise patterns by training a dedicated regression scheme with mesh data and the particular noise pattern. We evaluate our method on meshes with both synthetic and real scanned noise, and compare it to other mesh denoising algorithms. Results demonstrate that our method outperforms the state-of-the-art mesh denoising methods and successfully removes different kinds of noise for meshes with various geometry features.
This paper presents a tensor multiplication based smoothing algorithm that follows a two step denoising method. Unlike other traditional averaging approaches, our approach uses an element based normal voting tensor to compute smooth surfaces. By introducing a binary optimization on the proposed tensor together with a local binary neighborhood concept, our algorithm better retains sharp features and produces smoother umbilical regions than previous approaches. On top of that, we provide a stochastic analysis on the different kinds of noise based on the average edge length. The quantitative and visual results demonstrate the performance our method is better compared to state of the art smoothing approaches.
In recent years researchers have made noticeable progresses in mesh denoising, that is, recovering high-quality 3D models from meshes corrupted with noise (raw or synthetic). Nevertheless, these state of the art approaches still fall short for robustly handling various noisy 3D models. The main technical challenge of robust mesh denoising is to remove noise while maximally preserving geometric features. In particular, this issue becomes more difficult for models with considerable amount of noise. In this paper we present a novel scheme for robust feature-preserving mesh denoising. Given a noisy mesh input, our method first estimates an initial mesh, then performs feature detection, identification and connection, and finally, iteratively updates vertex positions based on the constructed feature edges. Through many experiments, we show that our approach can robustly and effectively denoise various input mesh models with synthetic noise or raw scanned noise. The qualitative and quantitative comparisons between our method and the selected state of the art methods also show that our approach can noticeably outperform them in terms of both quality and robustness.
Image restoration is a long-standing problem in low-level computer vision with many interesting applications.
We describe a flexible learning framework to obtain simple but effective models for various image restoration problems.
The proposed approach is based on the concept of nonlinear reaction diffusion, but we
extend conventional nonlinear reaction diffusion models by highly parametrized linear filters
as well as highly parametrized influence functions. In contrast to previous nonlinear diffusion models,
all the parameters, including the filters and the influence functions, are learned from training data
through a loss based approach. We call this approach TNRD -- Trainable Nonlinear Reaction Diffusion.
The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force.
We demonstrate its capabilities with three representative applications, Gaussian
image denoising, single image super resolution and
JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training
of the parameters and finally lead to the best reported performance on common test datasets with respect to the
tested applications.
Our trained models retain the structural simplicity of diffusion models and
take only a small number of steps, thus are highly efficient.
Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.
Mesh surface denoising is a fundamental problem in geometry processing. The main challenge is to remove noise while preserving sharp features (such as edges and corners) and preventing generating false edges. We propose in this paper to combine total variation (TV) and piecewise constant function space for variational mesh denoising. We first give definitions of piecewise constant function spaces and associated operators. A variational mesh denoising method will then be presented by combining TV and piecewise constant function space. It is proved that, the solution of the variational problem (the key part of the method) is in some sense continuously dependent on its parameter, indicating that the solution is robust to small perturbations of this parameter. To solve the variational problem, we propose an efficient iterative algorithm (with an additional algorithmic parameter) based on variable splitting and augmented Lagrangian method, each step of which has closed form solution. Our denoising method is discussed and compared to several typical existing methods in various aspects. Experimental results show that our method outperforms all the compared methods for both CAD and non-CAD meshes at reasonable costs. It can preserve different levels of features well, and prevent generating false edges in most cases, even with the parameters evaluated by our estimation formulae.
With the increasing use of geometry scanners to create 3D models, there is a rising need for fast and robust mesh smoothing to remove inevitable noise in the measurements. While most previous work has favored diffusion-based iterative techniques for feature-preserving smoothing, we propose a radically different approach, based on robust statistics and local first-order predictors of the surface. The robustness of our local estimates allows us to derive a non-iterative feature-preserving filtering technique applicable to arbitrary "triangle soups". We demonstrate its simplicity of implementation and its efficiency, which make it an excellent solution for smoothing large, noisy, and non-manifold meshes. Singapore-MIT Alliance (SMA)
Normal filtering is a fundamental step of feature-preserving mesh denoising. Methods based on convolutional neural networks (CNNs) have recently made their debut for normal filtering. However, they require complicated voxelization and/or projection operations for regularization, and afford an overall denoising accuracy with few powers of preserving surface features. We devise a novel normal filtering neural network algorithm, which we call as NormalF-Net. NormalF-Net consists of two cascaded subnetworks with a comprehensive loss function. The first subnetwork learns mapping from non-local patch-group normal matrices (NPNMs) to their ground-truth low-rank counterparts for denoising, and the second subnetwork learns mapping from the recovered NPNMs to the ground-truth normals for normal refinement. Different from existing learning-based methods, NormalF-Net, which bridges the connection between CNNs and geometry domain knowledge of non-local similarity, can not only preserve surface features when removing different levels and types of noise, but be free of voxelization/projection. NormalF-Net has been validated on different datasets of meshes with multi-scale features yet corrupted by noise of different distributions. Experimental results consistently demonstrate clear improvements of our method over the state-of-the-arts in both noise-robustness and feature awareness.
Through the years, several works have demonstrated
high-quality 3D mesh denoising. Despite the high reconstruction
quality, there are still challenges that need to be addressed
ranging from variations in configuration parameters to high
computational complexity. These drawbacks are crucial especially
if the reconstructed models have to be used for quality check,
inspection or repair in manufacturing environments where we
have to deal with large objects resulting in very dense 3D meshes.
Recently, deep learning techniques have shown that are able to
automatically learn and find more accurate and reliable results,
without the need for setting manually parameters. In this work,
motivated by the aforementioned requirements, we propose a fast
and reliable denoising method that can be effectively applied for
reconstructing very dense noisy 3D models. The proposed method
applies conditional variational autoencoders on face normals.
Extensive evaluation studies carried out using a variety of 3D
models verify that the proposed approach achieves plausible
reconstruction outputs, very relative or even better of those
proposed by the literature, in considerably faster execution times.
Distinguishing between geometric features and noise is of paramount importance for mesh denoising. In this paper, a graph-based feature-preserving mesh normal filtering scheme is proposed, which includes two stages: graph-based feature detection and feature-aware guided normal filtering. In the first stage, faces in the input noisy mesh are represented by patches, which are then modelled as weighted graphs. In this way, feature detection can be cast as a graph-cut problem. Subsequently, an iterative normalized cut algorithm is applied on each patch to separate the patch into smooth regions according to the detected features. In the second stage, a feature-aware guidance normal is constructed for each face, and guided normal filtering is applied to achieve robust feature-preserving mesh denoising. The results of experiments on synthetic and real scanned models indicate that the proposed scheme outperforms state-of-the-art mesh denoising works in terms of both objective and subjective evaluations.
Depth cameras and 3D scanners significantly simplify the procedure of geometric modeling. 3D surfaces have become more widespread, leading to a great demand for noise removal with the expectation of the minimal disturbance of mesh geometry. We propose a novel two-step data-driven mesh denoising approach. The first step removes noise by learning normal variations from noisy models to their ground-truth counterparts. Unlike existing denoising methods, we present the second step to recover the mesh geometry lost in the first step. The second step understands the commonly used filters by learning the mapping from filtered models to their ground-truth counterparts. In addition, (1) to handle noise with large variations, we model normal estimation as a low-rank matrix recovery problem in similar-patch collaboration before the first-step learning; (2) to recover the real geometry of a denoised mesh, we reversely filter the denoised mesh to obtain more geometry cues before the second-step learning. The detailed quantitative and qualitative results on various data demonstrate that, our two-step learning algorithm competes favorably with the state-of-the-art methods in terms of mesh geometry preservation and noise-robustness.
Mesh denoising is of great practical importance in geometric analysis and processing. In this paper we develop a novel L0 sparse regularization method to robustly and reliably eliminate noises while preserving features with theoretic guarantee, and our assumption is that, local regions of a noise-free shape should be smooth unless they contain geometric features. Both vertex positions and facet normals are integrated into the L0 norm to measure the sparsity of geometric features, and are then optimized in a sparsity-controllable fashion. We design an improved alternating optimization strategy to solve the L0 minimization problem, which is proved to be both convergent and stable. As a result, our sparse regularization exhibits its advantage to distinguish features from noises. To further improve the computational performance, we propose a multi-layer approach based on joint bilateral upsampling to handle large and complicated meshes. Moreover, the aforementioned framework is naturally accommodating the need of denoising time-varying mesh sequences. Both theoretical analysis and various experimental results on synthetic and natural noises have demonstrated that, our method can robustly recover multifarious features and smooth regions of 3D shapes even with severe noise corruption, and outperform the state-of-the-art methods.
Image restoration is a long-standing problem in low-level computer vision with many interesting applications. We describe a flexible learning framework based on the concept of nonlinear reaction diffusion models for various image restoration problems. By embodying recent improvements in nonlinear diffusion models, we propose a dynamic nonlinear reaction diffusion model with time-dependent parameters (\ie, linear filters and influence functions). In contrast to previous nonlinear diffusion models, all the parameters, including the filters and the influence functions, are simultaneously learned from training data through a loss based approach. We call this approach TNRD -- \textit{Trainable Nonlinear Reaction Diffusion}. The TNRD approach is applicable for a variety of image restoration tasks by incorporating appropriate reaction force. We demonstrate its capabilities with three representative applications, Gaussian image denoising, single image super resolution and JPEG deblocking. Experiments show that our trained nonlinear diffusion models largely benefit from the training of the parameters and finally lead to the best reported performance on common test datasets for the tested applications. Our trained models preserve the structural simplicity of diffusion models and take only a small number of diffusion steps, thus are highly efficient. Moreover, they are also well-suited for parallel computation on GPUs, which makes the inference procedure extremely fast.
Mesh denoising is imperative for improving imperfect surfaces acquired by scanning devices. The main challenge is to faithfully retain geometric features and avoid introducing additional artifacts when removing noise. Unlike the existing mesh denoising techniques that focus only on either the first-order features or high-order differential properties, our approach exploits the synergy when facet normals and quadric surfaces are integrated to recover a piecewise smooth surface. In specific, we vote on surface normal tensors from robust statistics to guide the creation of consistent subneighborhoods subsequently used by moving least squares (MLS). This voting naturally leads to a conceptually simple way that gives a unified mesh-denoising framework for not only handling noise but also enabling the recovering of surfaces with both sharp and small-scale features. The effectiveness of our framework stems from: 1) the multiscale tensor voting that avoids the influence from noise; 2) the effective energy minimization strategy to searching the consistent subneighborhoods; and 3) the piecewise MLS that fully prevents the side effects from different subneighborhoods during surface fitting. Our framework is direct, practical, and easy to understand. Comparisons with the state-of-The-Art methods demonstrate its outstanding performance on feature preservation and artifact suppression.
The joint bilateral filter is a variant of the standard bilateral filter, where the range kernel is evaluated using a guidance signal instead of the original signal. It has been successfully applied to various image processing problems, where it provides more flexibility than the standard bilateral filter to achieve high quality results. On the other hand, its success is heavily dependent on the guidance signal, which should ideally provide a robust estimation about the features of the output signal. Such a guidance signal is not always easy to construct. In this paper, we propose a novel mesh normal filtering framework based on the joint bilateral filter, with applications in mesh denoising. Our framework is designed as a two-stage process: first, we apply joint bilateral filtering to the face normals, using a properly constructed normal field as the guidance; afterwards, the vertex positions are updated according to the filtered face normals. We compute the guidance normal on a face using a neighboring patch with the most consistent normal orientations, which provides a reliable estimation of the true normal even with a high-level of noise. The effectiveness of our approach is validated by extensive experimental results.
Most mesh denoising techniques utilize only either the facet normal field or the vertex normal field of a mesh surface. The two normal fields, though contain some redundant geometry information of the same model, can provide additional information that the other field lacks. Thus, considering only one normal field is likely to overlook some geometric features. In this paper, we take advantage of the piecewise consistent property of the two normal fields and propose an effective framework in which they are filtered and integrated using a novel method to guide the denoising process. Our key observation is that, decomposing the inconsistent field at challenging regions into multiple piecewise consistent fields makes the two fields complementary to each other and produces better results. Our approach consists of three steps: vertex classification, bi-normal filtering, and vertex position update. The classification step allows us to filter the two fields on a piecewise smooth surface rather than a surface that is smooth everywhere. Based on the piecewise consistence of the two normal fields, we filtered them using a piecewise smooth region clustering strategy. To benefit from the bi-normal filtering, we design a quadratic optimization algorithm for vertex position update. Experimental results on synthetic and real data show that our algorithm achieves higher quality results than current approaches on surfaces with multifarious geometric features and irregular surface sampling.
We present an algorithm for denoising triangulated models based on L0 minimization. Our method maximizes the flat regions of the model and gradually removes noise while preserving sharp features. As part of this process, we build a discrete differential operator for arbitrary triangle meshes that is robust with respect to degenerate triangulations. We compare our method versus other anisotropic denoising algorithms and demonstrate that our method is more robust and produces good results even in the presence of high noise.
Many geometry processing applications are sensitive to noise and sharp features. Although there are a number of works on detecting noise and sharp features in the literature, they are heuristic. On one hand, traditional denoising methods use filtering operators to remove noise, however, they may blur sharp features and shrink the object. On the other hand, noise makes detection of features, which relies on computation of differential properties, unreliable and unstable. Therefore, detecting noise and features on discrete surfaces still remains challenging.
In this article, we present an approach for decoupling noise and features on 3D shapes. Our approach consists of two phases. In the first phase, a base mesh is estimated from the input noisy data by a global Laplacian regularization denoising scheme. The estimated base mesh is guaranteed to asymptotically converge to the true underlying surface with probability one as the sample size goes to infinity. In the second phase, an ℓ1-analysis compressed sensing optimization is proposed to recover sharp features from the residual between base mesh and input mesh. This is based on our discovery that sharp features can be sparsely represented in some coherent dictionary which is constructed by the pseudo-inverse matrix of the Laplacian of the shape. The features are recovered from the residual in a progressive way. Theoretical analysis and experimental results show that our approach can reliably and robustly remove noise and extract sharp features on 3D shapes.
We present a new framework for point cloud denoising by patch-collaborative spectral analysis. A collaborative generalization of each surface patch is defined, combining similar patches from the denoised surface. The Laplace–Beltrami operator of the collaborative patch is then used to selectively smooth the surface in a robust manner that can gracefully handle high levels of noise, yet preserves sharp surface features. The resulting denoising algorithm competes favourably with state-of-the-art approaches, and extends patch-based algorithms from the image processing domain to point clouds of arbitrary sampling. We demonstrate the accuracy and noise-robustness of the proposed algorithm on standard benchmark models as well as range scans, and compare it to existing methods for point cloud denoising.
Denoising surfaces is a a crucial step in the surface processing pipeline. This is even more challenging when no underlying structure of the surface is known, id est when the surface is represented as a set of unorganized points. In this paper, a denoising method based on local similarities is introduced. The contributions are threefold: first, we do not denoise directly the point positions but use a low/high frequency decomposition and denoise only the high frequency. Second, we introduce a local surface parameterization which is proved stable. Finally, this method works directly on point clouds, thus avoiding building a mesh of a noisy surface which is a difficult problem. Our approach is based on denoising a height vector field by comparing the neighborhood of the point with neighborhoods of other points on the surface. It falls into the non-local denoising framework that has been extensively used in image processing, but extends it to unorganized point clouds.
A fast routine for testing whether a triangle and a box are overlapping in three dimensions is presented. The test is derived using the separating axis theorem, whereafter the test is simplified and the code is optimized for speed. We show that this approach is 2.3 vs. 3.8 (PC vs. Sun) times faster than previous routines. It can be used for faster collision detection and faster voxelization in inter active ray tracers. The code is available online.
Decoupling local geometric features from the spatial location of a mesh is crucial for feature-preserving mesh denoising. This paper focuses on first order features, i.e., facet normals, and presents a simple yet effective anisotropic mesh denoising framework via normal field denoising. Unlike previous denoising methods based on normal filtering, which process normals defined on the Gauss sphere, our method considers normals as a surface signal defined over the original mesh. This allows the design of a novel bilateral normal filter that depends on both spatial distance and signal distance. Our bilateral filter is a more natural extension of the elegant bilateral filter for image denoising than those used in previous bilateral mesh denoising methods. Besides applying this bilateral normal filter in a local, iterative scheme, as common in most of previous works, we present for the first time a global, noniterative scheme for an isotropic denoising. We show that the former scheme is faster and more effective for denoising extremely noisy meshes while the latter scheme is more robust to irregular surface sampling. We demonstrate that both our feature-preserving schemes generally produce visually and numerically better denoising results than previous methods, especially at challenging regions with sharp features or irregular sampling.
In this paper, we propose a new and powerful shape denoising technique for processing surfaces approximated by triangle meshes and soups. Our approach is inspired by recent non-local image denoising schemes and naturally extends bilateral mesh smoothing methods. The main idea behind the approach is very simple. A new position of vertex P of a noisy mesh is obtained as a weighted mean of mesh vertices Q with nonlinear weights reflecting a similarity between local neighborhoods of P and Q. We demonstrate that our technique outperforms recent state-of-the-art smoothing methods. We also suggest a new scheme for comparing different mesh/soup denoising methods.
This paper presents frameworks to extend the mean and median filtering schemes in image processing to smoothing noisy 3D shapes given by triangle meshes. The frameworks consist of the application of the mean and median filters to face normals on triangle meshes and the editing of mesh vertex positions to make them fit the modified normals. We also give a quantitative evaluation of the proposed mesh filtering schemes and compare them with conventional mesh smoothing methods such as Laplacian smoothing flow and mean curvature flow. The quantitative evaluation is performed in error metrics on mesh vertices and normals. Experimental results demonstrate that our mesh mean and median filtering methods are more stable than conventional Laplacian and mean curvature flows. We propose thee new mesh smoothing methods as one possible solution of the oversmoothing problem.
Efficient conversion from rotating matrix to rotation axis and angle by extending Rodrigues’ formula
- liang
Digital photography with flash and no-flash image pairs
- G Petschnigg
- R Szeliski
- M Agrawala
- M Cohen
- H Hoppe
- K Toyama
Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising
- K Zhang
- W Zuo
- Y Chen
- D Meng
- L Zhang
Bilateral mesh denoising
- S Fleishman
- I Drori
- D Cohen-Or