Figure 2 - uploaded by Yizhou Yu
Content may be subject to copyright.
Three types of textures: (a) highly random (b) semi-structured (c) regular repeated. Above and below are pairs for texture morphing.
Source publication
In this paper, we study texture metamorphosis, or how to generate texture samples that smoothly transform from a source texture image to a target. We propose a pattern- based approach to specify the feature correspondence be- tween two textures, based on the observation that many tex- ture images have stochastically distributed patterns which are s...
Context in source publication
Citations
... We have demonstrated above with the multiple texture framework the synthesis of homogeneously textured images of several classes. We have also experimented with varying the biases in a spatial fashion, so to synthesize globally varying textures using a texture-by-numbers approach. In Figure 4. texture blending, texture metamorphosis (Liu et. al. 2002), and texture interpolation (Ruiters et. al. 2010). In texture blending the goal is to create a novel texture based on two or more source textures, so that the resulting texture has features from the different source textures; the generated texture can be seen as a blend of the source textures. In texture metamorphosis, the task is to ge ...
... Many of the texture morphing and interpolation methods are based on using image warping methods, and in general image metamorphosis/morphing methods. One of the earliest of these approaches is that of Liu et al. [2002], who introduced the problem of texture metamorphosis/morphing. Similar to typical image morphing methods, the approach divides the problem of obtaining a smooth morphing sequence of frames into two problems (i) of making correspondences between the source and the target images, and (ii) the problem of estimating a warping function to accommodate the changes in a smooth manner. ...
... The warp function is then obtained by combining the pattern and the landmark correspondences, and is estimated by using a sparse points interpolation technique -a standard morphing path estimation algorithm. Matusik et al. [2005] also consider a warp-based method for morphing textures, but in contrast to the approach in Liu et al. [2002], the correspondence estimation for morphing is fully automatic. Key to this is the analysis of morphability/similarity between textures, as measured by the residual error of a warping method optimizing the feature alignment between the textures. ...
This thesis considers statistical modelling of natural image data. Obtaining advances in this field can have significant impact for both engineering applications, and for the understanding of the human visual system. Several recent advances in natural image modelling have been obtained with the use of unsupervised feature learning. We consider a class of such models, restricted Boltzmann machines (RBMs), used in many recent state-of-the-art image models. We develop extensions of these stochastic artificial neural networks, and use them as a basis for building more effective image models, and tools for computational vision. We first develop a novel framework for obtaining Boltzmann machines, in which the hidden unit activations co-transform with transformed input stimuli in a stable and predictable way throughout the network. We define such models to be transformation equivariant. Such properties have been shown useful for computer vision systems, and have been motivational for example in the development of steerable filters, a widely used classical feature extraction technique. Translation equivariant feature sharing has been the standard method for scaling image models beyond patch-sized data to large images. In our framework we extend shallow and deep models to account for other kinds of transformations as well, focusing on in-plane rotations. Motivated by the unsatisfactory results of current generative natural image models, we take a step back, and evaluate whether they are able to model a subclass of the data, natural image textures. This is a necessary subcomponent of any credible model for visual scenes. We assess the performance of a state- of-the-art model of natural images for texture generation, using a dataset and evaluation techniques from in prior work. We also perform a dissection of the model architecture, uncovering the properties important for good performance. Building on this, we develop structured extensions for more complicated data comprised of textures from multiple classes, using the single-texture model architecture as a basis. These models are shown to be able to produce state-of-the-art texture synthesis results quantitatively, and are also effective qualitatively. It is demonstrated empirically that the developed multiple-texture framework provides a means to generate images of differently textured regions, more generic globally varying textures, and can also be used for texture interpolation, where the approach is radically dfferent from the others in the area. Finally we consider visual boundary prediction from natural images. The work aims to improve understanding of Boltzmann machines in the generation of image segment boundaries, and to investigate deep neural network architectures for learning the boundary detection problem. The developed networks (which avoid several hand-crafted model and feature designs commonly used for the problem), produce the fastest reported inference times in the literature, combined with state-of-the-art performance.
... Bar et al. [BJEYLW01] proposed a semi-morphing approach in which 50/50 blends of textures are created by performing texture analysis using wavelets. The first approach for morphing structural textures was presented in [LLSY02] using a pattern-based approach in which the user specifies landmarks and their correspondences for two textures. The landmarks are used to create a warp field and cross-dissolving accommodates texture appearance changes. ...
In this paper, we introduce a new texture metamorphosis approach for interpolating texture samples from a source texture into a target texture. We use a new energy optimization scheme derived from optimal control principles which exploits the structure of the metamorphosis optimality conditions. Our approach considers the change in pixel position and pixel appearance in a single framework. In contrast to previous techniques that compute a global warping based on feature masks of textures, our approach allows to transform one texture into another by considering both intensity values and structural features of textures simultaneously. We demonstrate the usefulness of our approach for different textures, such as stochastic, semi‐structural and regular textures, with different levels of complexities. Our method produces visually appealing transformation sequences with no user interaction.
... These methods tend to produce clusters of features and offer little understanding about the mixing process and how to control it. Texture metamorphosis approaches [24,25,26] perform the mixing by finding correspondences between elementary features (or textons) between the textures, and progressively morphing between the shapes of the features. These methods are extended by Matusik et al. [27] to perform convex combination of textures, by warping patches and averaging 1D histograms. ...
This paper proposes a new definition of the averaging of discrete probability distributions as a barycenter over the Monge-Kantorovich optimal transport space. To overcome the time complexity involved by the numerical solving of such problem, the original Wasserstein metric is replaced by a sliced approximation over 1D distributions. This enables us to introduce a new fast gradient descent algorithm to compute Wasserstein barycenters of point clouds.
This new notion of barycenter of probabilities is likely to find applications in computer vision where one wants to average features defined as distributions. We show an application to texture synthesis and mixing, where a texture is characterized by the distribution of the response to a multi-scale oriented filter bank. This leads to a simple way to navigate over a convex domain of color textures.
... Les méthodes par recopie permettent d'effectuer des mélanges de textures à l'aide de patches (sousimages) extraits d'un ensemble d'images d'apprentissage, ce qui crée des textures non-homogènes, voir par exemple [7,5,8]. Les méthodes de métamorphose de textures [10,19,18] effectuent un mélange en trouvant des correspondances entre des éléments de l'image (les textons). Ces méthodes sont étendues par Matusik et al. [11] afin de calculer des combinaisons convexes de textures en déformant les patches et en effectuant un moyennage d'histogrammes 1D. ...
... There are a few different approaches to texture morphing that have been developed over the last few years amongst which are [5] and [7]. The first paper depends heavily on user input, and only works for textures that are composed of repeated similar patterns (for example cells of the interior of a bee hive). ...
In this paper we present a method for achieving real-time view interpolation in a virtual navigation application that uses a collection of pre-captured panoramic views as a representation of the environment. In this context, viewpoint interpolation is essential to achieve smooth and realistic viewpoint transition while the user is moving from one panorama to another. In this proposed approach, view interpolation is achieved by first computing the optical flow field between a pair of adjacent panoramas. This flow field can then be used by the view morphing algorithm to generate, on-the-fly, virtual viewpoints in-between existing views. Realistic interpolation is obtained by taking into account both scene geometry and color information. To achieve real-time viewpoint interpolation, a GPU implementation of the viewpoint interpolation algorithm has been developed. We ran our algorithm on multiple interior and exterior scenes and we were able to produce smooth and realistic viewpoint transitions by generating virtual views at a rate of more than 300 panoramas per second.
... However, the synthesis quality within the transition area decreased significantly. Liu et al. proposed a pattern-based approach that uses ideas of image morphing in order to generate metamorphosis sequences [15]. Both Zhang et al. and Tonietto and Walter used texton maps to support a pixel-based texture morphing [30,23]. ...
Texture synthesis and morphing are important techniques for efficiently creating realistic textures used in scientific and entertainment applications. In this paper we present a novel fast algorithm for multi-dimensional texture synthesis and morphing that is especially suitable for parallel architectures such as GPUs or direct volume rendering (DVR) hardware. Our proposed solution generalizes the synthesis process to support higher than three-dimensional synthesis and morphing.
We introduce several improvements to previous 2D synthesis algorithms, such as new appearance space attributes and an improved jitter function. We then modify the synthesis algorithm to use it for texture morphing which can be applied to arbitrary many 2D input textures and can be spatially controlled using weight maps. Our results suggest that the algorithm produces higher quality textures than alternative algorithms with similar speed. Compared to higher quality texture synthesis algorithms, our solution is considerably faster and allows the synthesis of additional channels, such as transparencies and displacement maps, without affecting the running time of the synthesis at all. The method is easily extended to allow fast 3D synthesis and we show several novel examples and applications for morphed solid 3D textures.
Overall the presented technique provides an excellent trade-off between speed and quality, is highly flexible, allows the use of arbitrary channels, can be extended to arbitrary dimensions, is suitable for a GPU-implementation, and can be effectively integrated into rendering frameworks such as DVR tools.
... However, the synthesis quality decreased significantly. Liu et al. proposed a pattern- based approach that uses ideas of image morphing to generate metamorphosis sequences ( Liu et al., 2002). ...
Texture synthesis and morphing are important techniques for efficiently creating realistic textures used in scientific and entertainment applications. In this paper we present a novel fast algorithm for multi-dimensional texture synthesis and morphing that is especially suitable for parallel architectures such as GPUs or direct volume rendering (DVR) hardware. Our proposed solution generalizes the synthesis process to support higher than three-dimensional synthesis and morphing. We introduce several improvements to previous 2D synthesis algorithms, such as new appearance space attributes and an improved jitter function. We then modify the synthesis algorithm to use it for texture morphing which can be applied to arbitrary many 2D input textures and can be spatially controlled using weight maps. Our results suggest that the algorithm produces higher quality textures than alternative algorithms with similar speed. Compared to higher quality texture synthesis algorithms, our solution is considerablly faster and allows the synthesis of additional channels without affecting the running time of the synthesis at all. The method is easily extended to allow fast 3D synthesis and as another novel contribution we show several examples for morphed solid 3D textures. Overall the presented technique provides an excellent trade-off between speed and quality, is highly flexible, allows the use of arbitrary channels such as transparencies and displacement maps, can be extended to arbitrary dimensions and is suitable for a GPU-implementation. 1
... This technique practices well at simple warp, but they do not address the warp between stochastic textures. Liu et al. [12] presented a texture metamorphasis technique that is able to resolve the warp between textures with stochastic patterns. Using this technique, the user needs to specify a pattern in the source and target textures to establish the local feature correspondence. ...
We present a new algorithm for synthesizing temporal textures, which is simple and requires only a static texture image as input to produce a continuous varying stream of realistic images. We first introduce the basis sequence generation procedure in which the chosen patches from the input texture image are stitched on the output frame via blended alpha mattes formed by minimum error cuts. All the frames in the generated basis sequence are toroidal. We then employ the probabilities of similarity and transition links to generate an inexhaustible sequence with quasi- periodic quality. To ensure the smoothness of frame-to-frame transition, we interpolate natural metamorphosis in each transition link using the efficient automatic morphing technique which solves the problem of morphing between chaotic textures with obscure features. In addition, the proposed method allows users to simply and promptly control the motion parameters and interactively render the landscape animations. We combine the proposed approach with the color transfer technique to augment the visual gratification. Several examples including scenes for cloud, fire, and water are presented to demonstrate that the proposed algorithm is simple, efficient, and controllable for synthesizing temporal textures.
... In [4,29] feature sizes, orientations and so forth can be modified. In [19,30], the difficult problem of smooth transitions between different types of textures is addressed. As for texture particles [4], [30] considers elementary texture components (textons) by using texton masks. ...
We present a novel texture synthesis technique designed to reproduce at real-time frame-rates example texture images, with a special focus on patterns characterized by structural arrangements. Unlike current pixel-, patch- or texton-based schemes, that operate in image space, our approach is structural. We propose to assimilate texture images to corresponding 2D geometric meshes (called texture meshes). Our analysis mainly consists in generating automatically these meshes, while synthesis is then based on the creation of new vertex/polygon distributions matching some arrangement map. The output texture image is obtained by rasterizing the previously generated polygons using graphics hardware capabilities, which guarantees high speed performance. By operating in geometry space instead of image/pixel space, the proposed structural approach has a major advantage over current techniques: beyond pure texture reproduction, it allows us to define various tools, which allow users to further modify locally or globally and in real-time structural components of textures. By controlling the arrangement map, users can substitute new meshes in order to completely modify the structural appearance of input textures, yet maintaining a certain visual resemblance with the initial example image.
... A general solution to the morphing problem for any two arbitrary input textures is very hard. In [3] Liu and colleagues presented a solution to this problem. Their solution synthesizes smooth morph sequences, however the user must specify a correspondence between a pattern in the source image and a pattern in the target image. ...
... In [22], a solution is presented for a related case where the images contain a single pattern element and the morphing sequence maintains the pattern coherence throughout the transformation (for example, a baby deer morphing into an adult deer). The closest approach to ours is the work of Liu et al. [3]. They presented a solution for morphing a source texture into a target texture with the user defining how a pattern from the source texture maps to a pattern into the target texture. ...
... A pattern is defined as a ''. . . semantic unit that composes a group of feature points'' [3]. Their solution suffers from a common drawback common to all feature-based transformations, which is the specification of features. ...
Image morphing has been extensively studied in computer graphics and it can be summarized as follows: given two input images, morphing algorithms produce a sequence of inbetween images which transforms the source image into the target image in a visually pleasant way. In this paper, we propose an algorithm, based on recent advances from texture-from-sample ideas, which synthesizes a metamorphosis sequence targeted specifically for textures. We use the idea of binary masks—or texton masks—to control and drive the morphing sequence. Our solution provides an automatic mapping from source texels into target texels and thus guarantees a coherent and visually smoothing transition from the source texture into the target texture. We compare our morphing results with prior work and with simple interpolation between corresponding pixels in the same source and target images.
