Fig 7 - uploaded by Gianluigi Ciocca
Content may be subject to copyright.
Collages comparison between the best ranked collages in Experiment I (top) and the collages created with the user preference modeling and learning procedure for Experiment II (bottom). Color and full size images can be found at http: // www. ivl. disco. unimib. it/ research/ collage/ .
Source publication
In this paper we consider how to automatically create pleasing photo collages
created by placing a set of images on a limited canvas area. The task is
formulated as an optimization problem. Differently from existing
state-of-the-art approaches, we here exploit subjective experiments to model
and learn pleasantness from user preferences. To this end...
Contexts in source publication
Context 1
... final collages obtained on each dataset using the above described procedure, are reported in Figure 7. We denote each new collage with the corresponding configuration of states S ds . ...
Similar publications
![Fig. 1. Example of an Interpersonal Circumplex [8]](profile/Catherine-Pelachaud/publication/280853055/figure/fig1/AS:284465805971470@1444833341385/Example-of-an-Interpersonal-Circumplex-8_Q320.jpg)
Human’s non-verbal behavior may convey different meanings. They can reflect one’s emotional states, communicative intentions but also his social relations with someone else, i.e. his interpersonal attitude. In order to determine the non-verbal behavior that a virtual agent should display to convey particular interpersonal attitudes, we have collect...
Citations
... However, the manual creation of collages usually requires strong design experience (see Fig. 1) and involves an extensive, tedious process, especially when searching for appropriate pictures and creating a visual appealing layout. Automatic collage synthesis methods also exist, which can be divided into three categories in our experience: (i) traditional collage methods [1][2][3][4][5][6][7][8], focusing on creating a visual and informative summary of a given image set, (ii) picture mosaic methods [9][10][11], composing multiple sub-pictures to realize a visual simulation of a source image, and (iii) photo montage methods [12,13], synthesizing a new image from several photographs by cutting, gluing, rearranging, and overlapping operations. ...
A collage is a composite artwork made from the spatial layout of multiple pictures on a canvas, collected from the Internet or user photographs. Collages, usually made by skilled artists, involve a complex manual process, especially when searching for component pictures and adjusting their spatial layout to meet artistic requirements. In this paper, we present a visual perception driven method for automatically synthesizing visually pleasing collages. Unlike previous works, we focus on how to design a collage layout which not only provides easy access to the theme of the overall image, but also conforms to human visual perception. To achieve this goal, we formulate the generation of collages as a mapping problem: given a canvas image, first, compute a saliency map for it and a vector field for each sub-region of it. Second, using a divide-and-conquer strategy, generate a series of patch sets from the canvas image, where the salient map and the vector field are used to determine each patch’s size and direction respectively. Third, construct a Gestalt-based energy function to choose the most visually pleasing and orderly patch set as the final layout. Finally, using a semantic-color metric, map the picture set to the patch set to generate the final collage. Extensive experimental and user study results show that this method can generate visual pleasing collages.
... Image aesthetic quality assessment (IAQA) is an important visual task, which represents an important criterion for visual content curation and lays the foundation in many multimedia applications such as image retrieval [1,2], photo enhancement [3], and image cropping and photo album creation [4][5][6]. The goal of IAQA is to design algorithms that automatically predict image aesthetic quality. ...
The automatic assessment of the aesthetic quality of a photo is a challenging and extensively studied problem. Most of the existing works focus on the aesthetic quality assessment of photos regardless of the depicted subject and mainly use features extracted from the entire image. It has been observed that the performance of generic content aesthetic assessment methods significantly decreases when it comes to images depicting faces. This paper introduces a method for evaluating the aesthetic quality of images with faces by encoding both the properties of the entire image and specific aspects of the face. Three different convolutional neural networks are exploited to encode information regarding perceptual quality, global image aesthetics, and facial attributes; then, a model is trained to combine these features to explicitly predict the aesthetics of images containing faces. Experimental results show that our approach outperforms existing methods for both binary, i.e., low/high, and continuous aesthetic score prediction on four different image databases in the state-of-the-art.
... 15 In the era of digital and mobile photography, people are taking 16 many, perhaps too many, photos whenever and wherever they 17 want, leading to an explosion of personal photo collections. Even 18 when photos are organized in file directories by time, event, or 19 * Corresponding author. other themes, the number, as well as heterogeneity , of photos per 20 directory may still be too great for a user to appreciate browsing 21 photos one at a time. ...
In order to select representative images from a large image set to construct an album, we propose a new album management method in a comic-like layout based on images classification. We mainly focus on three key problems: how to organize the input set of images reasonably; how to classify and select representative images rapidly and accurately according to the user intent; how to display the representative images clearly. For the first problem, we use the model organization idea of Quartet Analysis, analyzing the set of images by the multiple global features, and we realize the effective organization of the images through the construction of the categorization tree (C-tree). For the second problem, in order to express the user intent better, we perform fine classification by using active learning method based on the C-tree. Then we recommend representative images for the user to select. For the third problem, we display representative images in a compact comic-like layout with hierarchical ordering information. To enrich layout styles, we create a template library based on enumeration by comic layout rules and complete the template selection and automatic collage of the images based on the principle of presenting the maximum information content of the images to be displayed. Experiments demonstrate that our method can not only assist the user with album production but also involve users in the whole process of making an album.
... Without an aesthetic evaluation as guidance, they cannot ensure the visual attractiveness of the collage results, which is a key factor of visual summarization [15], [16]. To solve this issue, researchers have proposed interactive methods, which conduct subjective experiments to capture and learn aesthetic appeal from user preferences in terms of the placement of multiple photos on a canvas [17] or the composition of visual elements [18]. However, these methods either require laborious data annotation or ignore the utility of visual summarization in information presentation. ...
... Image collage From the generation strategy view, image layout methods can be roughly divided into two categories, namely, bottom-up and top-down layout generation. Bottom-up methods usually use well-defined objective functions and their parameters contain the position, orientation, scale, and layer index of each image on the canvas [17], [29], [30], [31], [32], [33], [34]. These techniques usually have limitations regarding avoiding overlaps and precisely controlling the correlations among images. ...
With the surge of images in the information era, people demand an effective and accurate way to access meaningful visual information. Accordingly, effective and accurate communication of information has become indispensable. In this work, we propose a content-based approach that automatically generates a clear and informative visual summarization based on design principles and cognitive psychology to represent image collections. We first introduce a novel method to make representative and nonredundant summarizations of image collections, thereby ensuring data cleanliness and emphasizing important information. Then, we propose a tree-based algorithm with a two-step optimization strategy to generate the final layout that operates as follows: (1) an initial layout is created by constructing a tree randomly based on the grouping results of the input image set; (2) the layout is refined through a coarse adjustment in a greedy manner, followed by gradient back propagation drawing on the training procedure of neural networks. We demonstrate the usefulness and effectiveness of our method via extensive experimental results and user studies. Our visual summarization algorithm can precisely and efficiently capture the main content of image collections better than alternative methods or commercial tools.
... Color theme can be also adopted as signatures (or feature vectors) for the indexing of images in a content-based image retrieval system [19,20,21]. A user can query the system by choosing a color, or a set of colors, and then retrieve a set of images that are relevant to that query, or in another way a set of images for which the colors of the query are representative [19,22]. ...
Color of food images play a key role in human perception of food quality and calories, as well as in food choice. In this paper we investigate the use of computational methods for color harmony analysis of food images. To this end we propose a computational pipeline that includes color segmentation, color palette extraction and color harmony prediction. Such a pipeline makes it possible to analyze the emotions elicited by pairs and multiple combinations of food dishes.
... Image cropping is an important step to improve the aesthetic quality of an image. It can be used in many applications such as efficient image transmission, image retargeting, or photo collages [1]. For choosing the "best" image cropping method it is important to consider on which datasets this method is validated and possibly trained. ...
Image cropping is a common image editing task that aims to improve the composition as well as the aesthetics of an image by extracting well-composed sub-regions of the original image. For choosing the “best” autocropping method it is therefore important to consider on which datasets this method is validated and possibly trained. In this work we conduct a detailed analysis of the main datasets in the state of the art in terms of statistics, diversity and coverage of the selected sub-regions, namely the ground-truth candidate views. An analysis of how much semantics of ground-truth candidate views is preserved with respect to original images and a comparison among dummy autocropping solutions and state of the art methods is also presented and discussed. Results show that each dataset models the cropping problem differently, and in some cases very high performance can be reached by using a dummy autocropping strategy.
... A key aspect of our work, however, is the ability to switch between exploitation and exploration strategy. Regarding the laying out of visual materials on a canvas, most scholars have attempted to automate or support collaging [3,12,39,45] by letting users specify abstract areas [12], adapt sizes to their actions [45], or automate it in line with preference models [3] or areas of interest [39]. While most previous papers use pre-defined aesthetics criteria that drive optimization, we assume that these criteria evolve during the process, and we aim to recognize and adapt to them without actively interfering: e.g., through image size and visibility or letting designers create their own spatial representation. ...
... A key aspect of our work, however, is the ability to switch between exploitation and exploration strategy. Regarding the laying out of visual materials on a canvas, most scholars have attempted to automate or support collaging [3,12,39,45] by letting users specify abstract areas [12], adapt sizes to their actions [45], or automate it in line with preference models [3] or areas of interest [39]. While most previous papers use pre-defined aesthetics criteria that drive optimization, we assume that these criteria evolve during the process, and we aim to recognize and adapt to them without actively interfering: e.g., through image size and visibility or letting designers create their own spatial representation. ...
Design ideation is a prime creative activity in design. However, it is challenging to support computationally due to its quickly evolving and exploratory nature. The paper presents cooperative contextual bandits (CCB) as a machine-learning method for interactive ideation support. A CCB can learn to propose domain-relevant contributions and adapt their exploration/exploitation strategy. We developed a CCB for an interactive design ideation tool that 1) suggests inspirational and situationally relevant materials ("may AI?"); 2) explores and exploits inspirational materials with the designer; and 3) explains its suggestions to aid reflection. The application case of digital mood board design is presented, wherein visual inspirational materials are collected and curated in collages. In a controlled study, 14 of 16 professional designers preferred the CCB-augmented tool. The CCB approach holds promise for ideation activities wherein adaptive and steerable support is welcome but designers must retain full outcome control. CCS CONCEPTS • Human-centered computing → Interactive systems and tools; • Computing methodologies → Machine learning ; • Applied computing → Arts and humanities.
... Automatic image aesthetic assessment is a challenging task due to its fuzzy definition and its highly subjective nature. It represents an important criterion for visual content curation and it is useful in many applications such as image retrieval [1,2], photo enhancement [3], and image cropping [4,5,6]. Aesthetic assessment of images with generic content has been addressed in [6,7,8]. ...
Recent research has widely explored the problem of aesthetics assessment of images with generic content. However, few approaches have been specifically designed to predict the aesthetic quality of images containing human faces, which make up a massive portion of photos in the web. This paper introduces a method for aesthetic quality assessment of images with faces. We exploit three different Convolutional Neural Networks to encode information regarding perceptual quality, global image aesthetics, and facial attributes; then, a model is trained to combine these features to explicitly predict the aesthetics of images containing faces. Experimental results show that our approach outperforms existing methods for both binary, i.e. low/high, and continuous aesthetic score prediction on four different databases in the state-of-the-art.
... The solutions can be represented as trees, lines of code, expressions in prefix or postfix notations, strings of variable length, etc. GP has been widely used for finding suitable solutions for a wide range of problems needing optimization. For example, in image processing and computer vision applications GP has been used for: image segmentation, enhancement, layouting, classification, feature extraction, and object recognition [2,4,7,1,13]. For our purposes, we feed GP the set of the binary foreground images that correspond to the outputs of the single change detection algorithms, and a set operators represented by unary, binary, and n-ary functions that are used to combine the outputs (via logical AND, logical OR, etc. . . ) as well as to perform post-processing (via filter operators). ...
Given the existence of many change detection algorithms, each with its own peculiarities and strengths, we propose a combination strategy, that we termed IUTIS (In Unity There Is Strength), based on a genetic Programming framework. This combination strategy is aimed at leveraging the strengths of the algorithms and compensate for their weakness. In this paper we show our findings in applying the proposed strategy in two different scenarios. The first scenario is purely performance-based. The second scenario performance and efficiency must be balanced. Results demonstrate that starting from simple algorithms we can achieve comparable results with respect to more complex state-of-the-art change detection algorithms, while keeping the computational complexity affordable for real-time applications.
