Example of detected face and corresponding rectified image 

Context in source publication

Context 1

... at least three couples of points the six values a, b, c, d, e, f can be calcuated by least square. Faces where the feature detector failed to work with an high degree of confidence were rejected. Unfortunately face detection as well as facial features detection are error prone then in many cases it is not possible to obtain meaningful faces from generic images. Even worse in some cases the SVMs estimate with high confidence wrong facial features leading to non significant face data. Fig. 2 shows an example of automatically detected face and the corresponding rectified image. In fig. 3 a few rectified faces are reported. Note that, even though faces are heavily distorted, the identity of depicted people is still evident and faces are reasonably aligned to allow for appearance-based similarity search. 14 Once a face has been detected and succesfully rectified and cropped, a 20-dimensional face descriptor is computed. The descriptor is a vector w containing the projection of the rectified and cropped face in a subspace of the global face space. In practice the average face Ψ is subtracted from the 100 × 80 cropped and rectified face Γ i and the obtained image Φ is then projected on the eigenspace to obtain w i = e T i Φ. The average face Ψ and the 16 eigenimages e i associated with the largest eigenvalues are shown in fig. 4. The face space, as well as the average face, is learned off-line on a significant subset of the image collection and it is not updated. At any time, if most of the faces present in the image collection differ significantly from the training set, it is possible to build a new face space and effortlessy recompute the projection of each detected, rectified and cropped face in the new face space. In our experience we learned the face space with about 500 images. Other information, such as the size and the position of the originally detected face in the image or the reliability of the rectification process, 3 are also stored for future use but are not exploited in the current version of the system. The largest part of semantic information in personal photo is conveyed in areas where faces appear, the remaining part of the image is the context of the scene. Each picture is processed with the face detector 13 selecting areas containing faces. These areas are approximated with bounding boxes and are dealt as seen in previous section. The remaining part of the image is then processed as background (see fig. 1). According the same approach, more complex or additional detectors can be added to the system to extract further objects of interest in the scene and operate a different figure/background segmentation (e.g. a detector for entire body could be easily integrated in the system). Background is processed to capture relevant patterns able to characterize, with a coarse classification, the context of the scene depicted in the image. The representation of background is dealt with visual symbols or token usually referred as visual terms . These terms, introduced by Duygulu et al., 15 are used to represent visual content in a similar way to what is done in documents versus words representation. The association of labels to background patterns, expressed as function of visual terms, is performed with a supervised approach using Maximal Figure of Merit(MFoM) classifier. 16 It is a classifier based on Linear Discriminant Function (LDF) that is trained to learn a chosen figure of merit (e.g. Precision, Recall, F1 measure,...) and has been employed in automatic image annotation in 17 . 4 A visual feature describes image content with a sequence of values that can be interpreted as the projection of the image in the feature space. The distribution of the feature values in feature space tends to have multimodal density in the vector space. The centroids corresponding to different modes are considered as forming a base for data representation and are called visual terms ; any image can be represented as function of these points. Visual terms can be used to map single feature values, using in this case a representation simply based on unigrams, or they can be used considering structured forms such as spatial bigrams or even more complex structures. The data-driven approach for the extraction of visual terms allows the visual terms to emerge from the data set and build generic sets of symbols with representation power that is limited only by the coverage of the training set. Although k -means is typically used for extraction of visual terms, 15, 18 in this work the extraction of the visual terms has been achieved applying the vector quantization to the entire set of the characteristic vectors. In particular the codebooks are produced by the LBG algorithm of Linde et al. 19 ensuring less computational cost and a limited quantization error. A single feature allows capturing particular information of the image dataset according to its characteristics. Feature statistics in the image are dependent from the feature itself and are function of its statistical occurrence in the image. For example, if A = { A 1 , A 2 , . . . , A M } is the set of M visual terms for the feature A, each image is represented by a vector V = ( v 1 , v 2 , . . . , v M ) where the i -th component takes into account the statistic of the term A i in the image. Furthermore, the representation of the visual content can be enriched exploiting the spatial information. In fig. 5 is shown the usage of bigrams for an image partitioned with a regular grid. Each element is represented with a visual term identified as X . Using bigrams leads to better results than using unigrams although at the cost of higher dimensionality for image representation. As example for bigram-based representation, considering a codebook for a single feature formed by M elements, the image representation can be built placing in a vector the unigram-based representation followed by the bigrams-based representation. The total dimension of the vector in this case will be, M ∗ M + M . For a codebook of 64 elements the total dimension of the representation is 4160, for 128 elements it is 16512 and so on. Obviously the complexity of the visual information is captured more reliably if more characteristics, as orthogonal as possible, are used together. A simple way to represent visual patterns is to extract features related to color information and to texture, create the composed features as juxtaposition of the values of the two features and extract a visual vocabulary from the entire set of features. In this case visual terms will take into account the composition of color and texture features for the described ...