![Perception of illusory shapes is mostly independent of the colors of the inducers (inspired by Figure 2 of Grossberg and Mingolla [25]).](https://www.researchgate.net/profile/Jackie-Shen/publication/274375484/figure/fig1/AS:294709256835076@1447275570363/Perception-of-illusory-shapes-is-mostly-independent-of-the-colors-of-the-inducers_Q320.jpg)
![Stand-alone contours (left) or substantially standout contours (right) can barely induce the perception of illusory shapes (inspired by Figure 14 of Grossberg and Mingolla [25]).](https://www.researchgate.net/profile/Jackie-Shen/publication/274375484/figure/fig2/AS:294709256835080@1447275570398/Stand-alone-contours-left-or-substantially-standout-contours-right-can-barely-induce_Q320.jpg)
Content uploaded by Jackie Jianhong Shen
Author content
All content in this area was uploaded by Jackie Jianhong Shen on Apr 02, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Illusory Shapes via Corner Fusion
Abstract and Figures
We propose a novel method for constructing illusory foreground and background shapes from convex corners. We first introduce a new class of visual cues called corner bases, which expands 0-dimensional corner points to 2-dimensional micro structures. These corner bases are then fused together by the functionalized elastica energy imposed upon an admissible phase field. The optimal phase field segments the visual field into disjoint connected components, which are further fused via a simple connectivity principle to construct both foreground illusory shapes and background occluded shapes. Robust and efficient numerical schemes are developed, and several generic examples are presented. Cognitive implications are highlighted.
Figures - uploaded by Jackie Jianhong Shen
Author content
All figure content in this area was uploaded by Jackie Jianhong Shen
Content may be subject to copyright.
... Nitzberg, Mumford and Shiota [39] observed that line energies such as Euler's elastica can be used as regularization for the completion of missing contours in images by providing strong priors for the continuity of edges. Since then, Euler's elastica has been successfully used for image denoising [47,56], inpainting [35,45,54], segmentation [3,18,61], segmentation with depth [22,59] and illusory contour [28]. However, the numerical minimization of Euler's elastica is highly challenging due to its non-smoothness, nonlinearity and non-convexity. ...
... We express (28) in the form of the error differences u k e , v k e and Λ k e and obtain ...
... Therefore, the sequence {(u k , v k ; Λ k )} ∈N satisfies the optimality conditions (28) such that ...
The curvature regularities are well-known for providing strong priors in the continuity of edges, which have been applied to a wide range of applications in image processing and computer vision. However, these models are usually non-convex, non-smooth, and highly nonlinear, the first-order optimality condition of which are high-order partial differential equations. Thus, numerical computation is extremely challenging. In this paper, we estimate the discrete mean curvature and Gaussian curvature on the local \(3\times 3\) stencil, based on the fundamental forms in differential geometry. By minimizing certain functions of curvatures over the image surface, it yields a kind of weighted image surface minimization problem, which can be efficiently solved by the alternating direction method of multipliers. Numerical experiments on image restoration and inpainting are implemented to demonstrate the effectiveness and superiority of the proposed curvature-based model compared to state-of-the-art variational approches.
... One main advantage of the Euler's elastica energy is that it gives more natural appearance by minimizing the total curvature as well as lengths of the level lines, which can overcome the staircasing effect. Due to the great success in image inpainting, the Euler's elastica has been applied to other image processing tasks, such as segmenta-tion [5,16,26,54], segmentation with depth [18,53], illusory contour [28,35] and denoising [15,43]. ...
... converges to a limit point that satisfies the first-order optimality conditions of (28). ...
... for almost every point in . This derives that the limit point satisfies first-order optimality conditions (28). This completes the proof of the convergence theorem. ...
In view of the exceptional ability of curvature in connecting missing edges and structures, we propose novel sparse reconstruction models via the Euler’s elastica energy. In particular, we firstly extend the Euler’s elastica regularity into the nonlocal formulation to fully take the advantages of the pattern redundancy and structural similarity in image data. Due to its non-convexity, non-smoothness and nonlinearity, we regard both local and nonlocal elastica functional as the weighted total variation for a good trade-off between the runtime complexity and performance. The splitting techniques and alternating direction method of multipliers (ADMM) are used to achieve efficient algorithms, the convergence of which is also discussed under certain assumptions. The weighting function occurred in our model can be well estimated according to the local approach. Numerical experiments demonstrate that our nonlocal elastica model achieves the state-of-the-art reconstruction results for different sampling patterns and sampling ratios, especially when the sampling rate is extremely low.
... Its significant performance enhancements included shape reconstruction of occluded objects and determination of their ordering relation in a specific scene based on only one single image. Many other works illustrated that the curvature-related terms have played crucial roles in the boundary reconstruction [8,16] and image restoration [17][18][19] with the capacity of producing excellent edge and corner preservation results. All of these researches show the significant potential for curvature-based methods. ...
... Functional (10) gives a good example that incorporating different noise distributions into one formulation based on PHA can enhance the ability of handling strong noise better than applying them separately, and the improved performance is validated in Fig. 1 as well. In addition, Euler's elastica was studied for visual construction in [7,13,15,16], of which the major advantages lie in the effect of reconstruct-ing illusory contours or recovering occluded shapes. [6,8] found that this good property of Euler's elastica term also reflected is robust against noises. ...
... Traditional segmentation with depth for an image with two circles. a noisy images and results obtained by the standard multiphase segmentation model; b the initialization for two binary functions φ 0 h ; c results obtained by traditional segmentation with depth model[15,16] (a) Noisy image and results obtained from functional (55) (b) Initialφ 0 h for two binary functions (c) Our proposed model (28) results ...
Euler’s elastica-based unsupervised segmentation models have strong capability of completing the missing boundaries for existing objects in a clean image, but they are not working well for noisy images. This paper aims to establish a Euler’s elastica-based approach that can properly deal with the random noises to improve the segmentation performance for noisy images. The corresponding formulation of stochastic optimization is solved via the progressive hedging algorithm (PHA), and the description of each individual scenario is obtained by the alternating direction method of multipliers. Technically, all the sub-problems derived from the framework of PHA can be solved by using the curvature-weighted approach and the convex relaxation method. Then, an alternating optimization strategy is applied by using some powerful accelerating techniques including the fast Fourier transform and generalized soft threshold formulas. Extensive experiments have been conducted on both synthetic and real images, which displayed significant gains of the proposed segmentation models and demonstrated the advantages of the developed algorithms.
... In contrast to TV which is lengthbased, it involves a combination of length and curvature along image level lines. Hence, it has an advantage over TV by enforcing better curvature information and reducing the staircase artifacts [14,37]. This regularization is defined by ...
... Euler's Elastica regularization has advantages over the TV (b = 0) in enforcing better curvature information and reducing the staircase effect. Due to these attractive features, it has found a wide range of applications in computer vision and image processing such as image denoising [6,37], image inpainting [36], and image segmentation [28,39], and illusory contour [14]. ...
SENSitivity Encoding (SENSE) is an effective mathematical formulation for reconstructing under-sampled MRI data obtained in Parallel Magnetic Resonance Imaging (Parallel MRI). The functional model includes a regularization term and a data fidelity term which need to be minimized to obtain a high quality MRI result. The proper choice of regularization is essential for image quality. In this paper, we show that Euler’s elastica is an effective regularization for Parallel MRI data reconstruction, and has advantages over the Total Variation (TV) regularization in improving image signal to noise ratio and image relative error. The Euler’s elastica functional is however complex to minimize as it is non-convex, non-smooth, and highly nonlinear. In this paper, we propose a new numerical method to solve Euler’s elastica regularized SENSE efficiently. This algorithm is based on a variable splitting approach and proper relaxation of the functional. Numerical examples are presented to show the effectiveness of the proposed Euler’s elastica algorithm in comparison to Bregman Operator Splitting with Variable Stepsize, a TV based algorithm.
... One natural extension way is thus to introduce curve curvature term for regularization. For example, Euler's elastica which contains both lengths and curvatures was proposed for image inpainting (Chan et al. 2002;Yashtini and Kang 2016), denoising Duan et al. 2013), zooming Duan et al. 2013), illusory contour (Kang et al. 2014), image decomposition , and image reconstruction (Yan and Duan 2020). Such regularity can provide strong priors for the continuity of edges. ...
Variable splitting and augmented Lagrangian method are widely used in image processing. This chapter briefly reviews its applications for solving the total variation (TV) related image restoration problems. Due to the nonsmoothness of TV, related models and variants are nonsmooth convex or nonconvex minimization problems. Variable splitting and augmented Lagrangian method can benefit from the separable structure and efficient subsolvers, and has convergence guarantee in convex cases. We present this approach for a number of TV minimization models including TV-L2, TV-L1, TV with nonquadratic fidelity term, multichannel TV, high-order TV, and curvature minimization models.
... The term was first used in computer vision by Mumford [22], and has since proven to be effective in solving the problems present in the TV model. The Euler's elastica has also been widely applied in various fields of image processing such as image denoising [23][24][25], image segmentation [20,[26][27][28], inpainting [24,[29][30][31], illusory contour [32,33], and segmentation with depth [34][35][36]. Therefore, we believe it is important to design an efficient numerical solution for the combined model. ...
The Euler’s elastica energy regularizer has been widely used in image processing and computer vision tasks. However, finding a fast and simple solver for the term remains challenging. In this paper, we propose a new dual method to simplify the solution. Classical fast solutions transform the complex optimization problem into simpler subproblems, but introduce many parameters and split operators in the process. Hence, we propose a new dual algorithm to maintain the constraint exactly, while using only one dual parameter to transform the problem into its alternate optimization form. The proposed dual method can be easily applied to level-set-based segmentation models that contain the Euler’s elastic term. Lastly, we demonstrate the performance of the proposed method on both synthetic and real images in tasks image processing tasks, i.e. denoising, inpainting, and segmentation, as well as compare to the Augmented Lagrangian method (ALM) on the aforementioned tasks.
... One natural extension way is thus to introduce curve curvature term for regularization. For example, Euler's elastica which contains both lengths and curvatures was proposed for image inpainting (Chan et al. 2002;Yashtini and Kang 2016), denoising Duan et al. 2013), zooming Duan et al. 2013), illusory contour (Kang et al. 2014), image decomposition , and image reconstruction (Yan and Duan 2020). Such regularity can provide strong priors for the continuity of edges. ...
Variable splitting and augmented Lagrangian method are widely used in image processing. This chapter briefly reviews its applications for solving the total variation (TV) related image restoration problems. Due to the nonsmoothness of TV, related models and variants are nonsmooth convex or nonconvex minimization problems. Variable splitting and augmented Lagrangian method can benefit from the separable structure and efficient subsolvers, and has convergence guarantee in convex cases. We present this approach for a number of TV minimization models including TV-L2, TV-L1, TV with nonquadratic fidelity term, multichannel TV, high-order TV, and curvature minimization models.
... As it is not lower semicontinuous some relaxed versions have been proposed by Bellettini et al. (1993), Masnou and and Ballester et al. (2001), which are compatible with the amodal completion theory of Kanizsa (1991). In a work of Kang et al. (2014) that proposes a computational method for modal completion, the elastica is a key ingredient to obtain illusory contours. It is also used in a method, proposed by Citti and Sarti (2006), for both modal and amodal completion which uses geodesics in the group of rotations and translations. ...
In this thesis we aim at analyzing images and videos at the object
level, with the goal of decomposing the scene into complete objects
that move and interact among themselves. The thesis is divided in
three parts. First, we propose a segmentation method to decompose
the scene into shapes. Then, we propose a probabilistic method,
which works with shapes or objects at two different depths, to infer
which objects are in front of the others, while completing the ones
which are partially occluded. Finally, we propose two video related
inpainting methods. On one hand, we propose a binary video inpainting
method that relies on the optical flow of the video in order
to complete the shapes across time taking into account their motion.
On the other hand, we propose a method for optical flow inpainting
that takes into account the information from the frames.
Perception, 2008, volume 37, pages 651- 665
Janos Geier, Laszlo Bernath, Mariann Hudak, Laszlo Sera
Stereo Vision Ltd, Nadasdy Ka¨lman utca 34, H 1048 Budapest, Hungary; Institute of Psychology, Eotvos Lorand University, Izabella utca 46, H 1064 Budapest, Hungary; #Department of Psychology,
e-mail: janos@geier.hu
Received 5 April 2006, in revised form 4 July 2007; published online 1 May 2008
Abstract. The generally accepted explanation of the Hermann grid illusion is Baumgartner's hypothesis that the illusory effect is generated by the response of retinal ganglion cells with con-
centric ON ^ OFF or OFF ^ ON receptive fields. To challenge this explanation, we have introduced some simple distortions to the grid lines which make the illusion disappear totally, while all pre-
conditions of Baumgartner's hypothesis remain unchanged. To analyse the behaviour of the new versions of the grid, we carried out psychophysical experiments, in which we measured the distor-
tion tolerance: the level of distortion at which the illusion disappears at a given type of distortion for a given subject. Statistical analysis has shown that the distortion tolerance is independent of grid-line width within a wide range, and of the type of distortion, except when one side of each line remains straight.We conclude that the main cause of the Hermann grid illusion is the straightness of the edges of the grid lines, and we propose a theory which explains why the illusory spots occur in the original Hermann grid and why they disappear in curved grids.
doi:10.1068/p5622
High order derivative information has been widely used in developing variational models in image processing to accomplish more advanced tasks. However, it is a nontrivial issue to construct efficient numerical algorithms to deal with the minimization of these variational models due to the associated high order Euler-Lagrange equations. In this paper, we propose an efficient numerical method for a mean curvature based image denoising model using the augmented Lagrangian method. A special technique is introduced to handle the mean curvature model for the augmented Lagrangian scheme. We detail the procedures of finding the related saddle-points of the functional. We present numerical experiments to illustrate the effectiveness and efficiency of the proposed numerical method, and show a few important features of the image denoising model such as keeping corners and image contrast. Moreover, a comparison with the gradient descent method further demonstrates the efficiency of the proposed augmented Lagrangian method.
Algebraic Preliminaries: 1. Tensor products of vector spaces 2. The tensor algebra of a vector space 3. The contravariant and symmetric algebras 4. Exterior algebra 5. Exterior equations Differentiable Manifolds: 1. Definitions 2. Differential maps 3. Sard's theorem 4. Partitions of unity, approximation theorems 5. The tangent space 6. The principal bundle 7. The tensor bundles 8. Vector fields and Lie derivatives Integral Calculus on Manifolds: 1. The operator $d$ 2. Chains and integration 3. Integration of densities 4. $0$ and $n$-dimensional cohomology, degree 5. Frobenius' theorem 6. Darboux's theorem 7. Hamiltonian structures The Calculus of Variations: 1. Legendre transformations 2. Necessary conditions 3. Conservation laws 4. Sufficient conditions 5. Conjugate and focal points, Jacobi's condition 6. The Riemannian case 7. Completeness 8. Isometries Lie Groups: 1. Definitions 2. The invariant forms and the Lie algebra 3. Normal coordinates, exponential map 4. Closed subgroups 5. Invariant metrics 6. Forms with values in a vector space Differential Geometry of Euclidean Space: 1. The equations of structure of Euclidean space 2. The equations of structure of a submanifold 3. The equations of structure of a Riemann manifold 4. Curves in Euclidean space 5. The second fundamental form 6. Surfaces The Geometry of $G$-Structures: 1. Principal and associated bundles, connections 2. $G$-structures 3. Prolongations 4. Structures of finite type 5. Connections on $G$-structures 6. The spray of a linear connection Appendix I: Two existence theorems Appendix II: Outline of theory of integration on $E^n$ Appendix III: An algebraic model of transitive differential geometry Appendix IV: The integrability problem for geometrical structures References Index.
We propose a novel fast numerical method for denoising of 1D signals based on curvature minimization. Motivated by the primal-dual formulation for total variation minimization introduced by Chan, Golub, and Mulet, the proposed method makes use of some auxiliary variables to reformulate the stiff terms presented in the Euler-Lagrange equation which is a fourth-order differential equation. A direct application of Newton’s method to the resulting system of equations often fails to converge. We propose a modified Newton’s iteration which exhibits local superlinear convergence and global convergence in practical settings. The method is much faster than other existing methods for the model. Unlike all other existing methods, it also does not require tuning any additional parameter besides the model parameter. Numerical experiments are presented to demonstrate the effectiveness of the proposed method.
1. The direct method in the calculus of variations.- 2. Minimum problems for integral functionals.- 3. Relaxation.- 4. ?-convergence and K-convergence.- 5. Comparison with pointwise convergence.- 6. Some properties of ?-limits.- 7. Convergence of minima and of minimizers.- 8. Sequential characterization of ?-limits.- 9. ?-convergence in metric spaces.- 10. The topology of ?-convergence.- 11. ?-convergence in topological vector spaces.- 12. Quadratic forms and linear operators.- 13. Convergence of resolvents and G-convergence.- 14. Increasing set functions.- 15. Lower semicontinuous increasing functionals.- 16. $$ \bar{\Gamma } $$-convergence of increasing set functional.- 17. The topology of $$ \bar{\Gamma } $$-convergence.- 18. The fundamental estimate.- 19. Local functionals and the fundamental estimate.- 20. Integral representation of ?-limits.- 21. Boundary conditions.- 22. G-convergence of elliptic operators.- 23. Translation invariant functional.- 24. Homogenization.- 25. Some examples in homogenization.- Guide to the literature.