Fig 1 - uploaded by Kun Guo
Content may be subject to copyright.
Examples of a male face image presented with varying facial expressions (from left to right: happiness, sadness, anger, fear, disgust, and surprise) at 3 different viewpoints (from top to bottom: frontal view, left mid-profile view, left profile view). The faces in the far right column show examples of local facial regions (eyes, nose and mouth) across different viewpoints that were used in the eye-tracking analyses.
Source publication
Recent studies measuring the facial expressions of emotion have focused primarily on the perception of frontal face images. As we frequently encounter expressive faces from different viewing angles, having a mechanism which allows invariant expression perception would be advantageous to our social interactions. Although a couple of studies have ind...
Contexts in source publication
Context 1
... and mouth) on both hemifaces, and had no distinctive facial marks (e.g., moles) on each hemiface. Each of these models posed six high- intensity facial expressions (happy, sad, fearful, angry, disgusted, and surprised) at three different horizontal viewing angles: full-face frontal view, a left 45° mid-profile view, and a left profile view (see Fig. 1 for examples). As a result, 180 expressive face images (10 models × 6 expressions × 3 viewing angles) were generated for the testing session. Although they may have real-world limitations, and categorization performance for some expressions could be subject to culture influence, these well-controlled face images were chosen for their ...
Context 2
... and eyebrows; the 'nose' or 'mouth' region consisted of the main body of the nose (glabella, nasion, tip-defining points, alar-sidewall, and supra-alar crease) or the 'mouth' and immediate surrounding area (up to 1°). The division line between the mouth and nose regions was the midline between the upper lip and the bottom of the nose (see Fig. 1 for examples). Each fixation was then characterized by its location among feature regions and its time of onset relative to the start of the trial. The number of fixations directed at each feature was normalized to the total number of fixations sampled in that ...
Context 3
... may increase local facial information ambiguity and consequently lead to longer fixation duration. This possibility was examined by a 3 (viewpoint) × 6 (expression type) ANOVA with the average fixation duration across all fixations directed at each face as the dependent variables. The analysis revealed significant main effect of viewpoint (F (1.24, 38.32) = 12.12, p = 0.001, η p 2 = 0.28; Fig. 3B) and expression type (F( (Fig. 3A) but increased individual fixation durations (Fig. 3B). Faces with frontal and mid-profile views, on the other hand, attracted indistinguishable number of fixations and fixation durations (all ps N 0.05). We then examined detailed fixation distribution to ...
Context 4
... control analysis revealed almost identical findings as those shown in Fig. 5. Across all the expressions, a 3 (viewpoint) × 3 (face region) ANOVA with normalised proportion of fixations directed at each facial region as the dependent variables demonstrated significant main effect of face region (F(1.49, 46.08) = 53.85, p b 0.001, η p 2 = 0.64) and viewpoint (F (1.13, 34.86) = 30.33, p b 0.001, η p 2 = 0.5), and significant interaction between face region and viewpoint (F(1.99, 61.8) = 5.36, p = 0.007, η p 2 = 0.15; Fig. 7A). Regardless of viewpoint, the eyes attracted the highest proportion of fixations, followed by the nose and then the ...
Context 5
... revealed almost identical findings as those shown in Fig. 5. Across all the expressions, a 3 (viewpoint) × 3 (face region) ANOVA with normalised proportion of fixations directed at each facial region as the dependent variables demonstrated significant main effect of face region (F(1.49, 46.08) = 53.85, p b 0.001, η p 2 = 0.64) and viewpoint (F (1.13, 34.86) = 30.33, p b 0.001, η p 2 = 0.5), and significant interaction between face region and viewpoint (F(1.99, 61.8) = 5.36, p = 0.007, η p 2 = 0.15; Fig. 7A). Regardless of viewpoint, the eyes attracted the highest proportion of fixations, followed by the nose and then the mouth region (all ps b 0.05). But quantitatively the proportion ...
Similar publications
INTRODUCTION: With the development of artificial intelligence, facial expression recognition has become a hot topic. Facial expression recognition has been widely applied to every field of our life. How to improve the accuracy of facial emotion recognition is an important research content.OBJECTIVES: In today's facial expression recognition, there...
Facial expressions are widely recognized as universal indicators of underlying internal states in most species of animals, thereby presenting as a non-invasive measure for assessing physical and mental conditions. Despite the advancement of artificial intelligence-assisted tools for automated analysis of voluminous facial expression data in human s...
It is well known that memory can be modulated by emotional stimuli at the time of encoding and consolidation. For example, happy faces create better identity recognition than faces with certain other expressions. However, the influence of facial expression at the time of retrieval remains unknown in the literature. To separate the potential influen...
Posed facial expressions of actors have often been used as stimuli to induce mental state inferences, in order to investigate “Theory of Mind” processes. However, such stimuli make it difficult to determine whether perceivers are using a basic or more elaborated mentalizing strategy. The current study used as stimuli covert recordings of target ind...
Citations
... Our second hypothesis is that the view angle will influence the mental effort required of the user to understand the robot's behavior. We expect this result, as work by Guo and Shaw [18] found that the viewpoint for the perception of facial expressions matters for the perceived intensity of the expression. Their research contends that decreased visibility of some facial features from a side-view angle may reduce the intensity of the judgment, making identifying the emotion more difficult. ...
... There were no statistically significant correlations between the mean fixation time on the robot's body and the consensus score of the participant [r(98) = -0. 18 ...
... Our second hypothesis that the view angle would influence the mental effort required for the robot to be understood was also supported by our findings. These findings align with work by Guo and Shaw [18] who found similar results when observing human faces from different angles. They found that the sideview angle might provide less information due to decreased visibility of features. ...
Robots are becoming part of our social landscape. Social interaction with humans must be efficient and intuitive to understand because nonverbal cues make social interactions between humans and robots more efficient. This study measures mental effort to investigate factors influencing the intuitive understanding of expressive nonverbal robot motions. Using an eye tracker to measure pupil response and gaze, fifty participants were asked to watch eighteen short video clips featuring three different types of robots performing expressive robot behaviors. Our findings indicate that the appearance of the robot, the viewing angle, and the expression shown by the robot all influence the cognitive load. Therefore, they may affect the intuitive understanding of expressive robot behavior. Furthermore, we found differences in the fixation time for different features of the various robots. With these insights, we identified possible improvement directions for making interactions between humans and robots more efficient and intuitive.
... We defined two regions of interest (ROIs): (a) the upper part of the face (eye region), encompassing the left and right eyes, including the eyebrows and the nasion area inbetween the eyes, and (b) the lower part of the face (mouth region), consisting of the partial body of the nose (tip-defining points, alar-sidewall, and supra-alar crease) or the "mouth" and immediate surrounding area (left and right cheeks). The dividing line between the upper and lower parts was a horizontal line along the tip of the nose [52]. The ROIs were defined as generously sized rectangles around the facial features; the face regions' shape was fixed relative to the face but changed according to the facial orientation; the regions of interest containing the two eyes were always visible. ...
Recent research on intense real-life faces has shown that although there was an objective difference in facial activities between intense winning faces and losing faces, viewers failed to differentiate the valence of such expressions. In the present study, we explored whether participants could perceive the difference between intense positive facial expressions and intense negative facial expressions in a forced-choice response task using eye-tracking techniques. Behavioral results showed that the recognition accuracy rate for intense facial expressions was significantly above the chance level. For eye-movement patterns, the results indicated that participants gazed more and longer toward the upper facial region (eyes) than the lower region (mouth) for intense losing faces. However, the gaze patterns were reversed for intense winning faces. The eye movement pattern for successful differentiation trials did not differ from failed differentiation trials. These findings provided preliminary evidence that viewers can utilize intense facial expression information and perceive the difference between intense winning faces and intense losing faces produced by tennis players in a forced-choice response task.
... Regarding the eye region, much popular literature refers to the eyes as "a window into the soul." Empirical research in psychology has consistently revealed that during face perception tasks, the eye region tends to be prioritized for a ention and receives the longest fixations within the facial structure [6,7]. Furthermore, the eye region represents one of the most informative sources for facial expression recognition, and people can accurately identify others' current emotions and motivations based on isolated eye region cues. ...
The eye region conveys considerable information regarding an individual's emotions, motivations , and intentions during interpersonal communication. Evidence suggests that the eye regions of an individual expressing emotions can capture a ention more rapidly than the eye regions of an individual in a neutral affective state. However, how a entional resources affect the processing of emotions conveyed by the eye regions remains unclear. Accordingly, the present study employed a dual-target rapid serial visual presentation task: happy, neutral, or fearful eye regions were presented as the second target, with a temporal lag between two targets of 232 or 696 ms. Participants completed two tasks successively: Task 1 was to identify which species the upright eye region they had seen belonged to, and Task 2 was to identify what emotion was conveyed in the upright eye region. The behavioral results showed that the accuracy for fearful eye regions was lower than that for neutral eye regions under the condition of limited a entional resources; however, accuracy differences across the three types of eye regions did not reach significance under the condition of adequate a entional resources. These findings indicate that preferential processing of fearful expressions is not automatic but is modulated by available a entional resources.
... Considering that appropriate facial expression recognition plays a crucial role in guiding our social interaction and behaviour, it is important to understand how consistently we can interpret the same expressive cues in different viewing conditions and what factors could affect or bias our interpretation of these cues. Previous studies have revealed comparable accuracies in categorizing common facial expressions displayed at different expression intensities (Guo, 2012), image qualities (Du & Martinez, 2011;Guo et al., 2019;Johnston et al., 2003), viewing perspectives (Guo & Shaw, 2015;Matsumoto & Hwang, 2011), and viewing distances (Guo, 2013). For instance, changing the viewing angle across frontal, mid-profile, and profile views (Guo & Shaw, 2015) or reducing image resolution to 64 × 48 pixels or less (up to 30 × 20 pixels for some expressions; Du & Martinez, 2011) had little deterioration impact on expression recognition performance. ...
... Previous studies have revealed comparable accuracies in categorizing common facial expressions displayed at different expression intensities (Guo, 2012), image qualities (Du & Martinez, 2011;Guo et al., 2019;Johnston et al., 2003), viewing perspectives (Guo & Shaw, 2015;Matsumoto & Hwang, 2011), and viewing distances (Guo, 2013). For instance, changing the viewing angle across frontal, mid-profile, and profile views (Guo & Shaw, 2015) or reducing image resolution to 64 × 48 pixels or less (up to 30 × 20 pixels for some expressions; Du & Martinez, 2011) had little deterioration impact on expression recognition performance. It seems that our brain has a reasonably good tolerance for interpreting these common prototypical facial expression signals. ...
In contrast to prototypical facial expressions, we show less perceptual tolerance in perceiving vague expressions by demonstrating an interpretation bias, such as more frequent perception of anger or happiness when categorizing ambiguous expressions of angry and happy faces that are morphed in different proportions and displayed under high- or low-quality conditions. However, it remains unclear whether this interpretation bias is specific to emotion categories or reflects a general negativity versus positivity bias and whether the degree of this bias is affected by the valence or category of two morphed expressions. These questions were examined in two eye-tracking experiments by systematically manipulating expression ambiguity and image quality in fear- and sad-happiness faces (Experiment 1) and by directly comparing anger-, fear-, sadness-, and disgust-happiness expressions (Experiment 2). We found that increasing expression ambiguity and degrading image quality induced a general negativity versus positivity bias in expression categorization. The degree of negativity bias, the associated reaction time and face-viewing gaze allocation were further manipulated by different expression combinations. It seems that although we show a viewing condition-dependent bias in interpreting vague facial expressions that display valence-contradicting expressive cues, it appears that the perception of these ambiguous expressions is guided by a categorical process similar to that involved in perceiving prototypical expressions.
... To investigate whether different types of negative emotional faces have varying degrees of priming effects, the study employed blocks of trials consisting of "happy" and "neutral" faces, and either "fearful" or "angry" faces. Additionally, faces were presented either in half-profile view or frontal view to test whether potential priming effects would be weaker for the former due to the reduced perceived facial expression intensity reported in previous studies (Guo and Shaw, 2015;Sutherland et al., 2017;Surcinelli et al., 2022). The study also included a fifth block of stimuli where only the eyes and surrounding area of a face (either neutral, happy, or fearful) were visible, as eyes are an important source of social information (Marsh et al., 2005) and have been linked to activation in the amygdala in response to fearful expressions (Whalen et al., 2004). ...
The strength of the affective priming effect is influenced by various factors, including the duration of the prime. Surprisingly, short-duration primes that are around the threshold for conscious awareness typically result in stronger effects compared to long-duration primes. The misattribution effect theory suggest that subliminal primes do not provide sufficient cognitive processing time for the affective feeling to be attributed to the prime. Instead, the neutral target being evaluated is credited for the affective experience. In everyday social interactions, we shift our gaze from one face to another, typically contemplating each face for only a few seconds. It is reasonable to assume that no affective priming takes place during such interactions. To investigate whether this is indeed the case, participants were asked to rate the valence of faces displayed one by one. Each face image simultaneously served as both a target (primed by the previous trial) and a prime (for the next trial). Depending on the participant’s response time, images were typically displayed for about 1–2 s. As predicted by the misattribution effect theory, neutral targets were not affected by positive affective priming. However, non-neutral targets showed a robust priming effect, with emotional faces being perceived as even more negative or positive when the previously seen face was emotionally congruent. These results suggest that a “correct attribution effect” modulates how we perceive faces, continuously impacting our social interactions. Given the importance of faces in social communication, these findings have wide-ranging implications.
... Previous studies using eye-tracking found that the total fixation time on the eyes is the longest 8 . It has also been reported that the eyes are initially observed multiple times before the line of sight subsequently moves to the mouth with a similar visual pattern 17,18 . Studies on eye-tracking have revealed that regarding the face, the area around the where statistically significant differences were obtained, mainly in the face, nose, and eye items, dentists gazed more frequently than patients A K . ...
... Studies on eye-tracking have revealed that regarding the face, the area around the where statistically significant differences were obtained, mainly in the face, nose, and eye items, dentists gazed more frequently than patients A K . [18][19][20] , and this fixation shifts depending on jaw deformation and malocclusion 8,21 . Furthermore, it was revealed that in the case of a well-proportioned facial image with left-right symmetry, fixation was seldom directed to a particular part and occurred only on the mouth as part of the entire face 6 . ...
This research aims to determine the differences between the patient and orthodontist in terms of their visual attention using an eye-tracking system-based evaluation system for assessing facial features before and post orthognathic surgery. The participants included patients who underwent orthognathic treatment （n＝26; mean age, 26.04±6.6 years） at the Showa University Dental Hospital and orthodontists with ≥5 years of experience at our department （n＝10, mean age, 31.4±2.2 years）. Visual attention was assessed using an eyeglass-type device （eye tracker）. Facial photographs of each patient, both frontal and side views, were shown on a monitor to the patients and orthodontists, and their respective visual attention was comparatively assessed. SPSS Statistics 25 was used for the study data statistical analysis. The results were contrasted between the patients and the orthodontists using a linear mixed model, with photographs labeled with the same number serving as repeat factors. Total fixation analysis demonstrated that patients focused more on the lower face postoperatively （P＝0.044）, while orthodontists focused more on the entire face both postoperative and preoperatively. The postoperative findings also demonstrated significant differences in many areas. There was a significant difference in how orthodontists and patients examined the face. This study found differences in visual attention between patients and orthodontists when they examined facial features before and after orthognathic surgery.
... Holistic processing appears at a very early stage, as fast as 17 ms (Liu & Tanaka, 2019;Tanaka & Xu, 2018;Tanaka et al., 2012). Recently, researchers discovered evidence of viewpoint-invariant gaze patterns in facial expression appraisal, demonstrating that holistic mechanisms manifest during early perceptual stages (Gregory, Tanaka, & Liu, 2021;Guo & Shaw, 2015). Accordingly, the conflicting expressions may be more likely to be processed as holistic, and holistic processing of expressions perhaps occurs at an early stage. ...
The rapid and efficient recognition of facial expressions is crucial for adaptive behaviors, and holistic processing is one of the critical processing methods to achieve this adaptation. Therefore, this study integrated the effects and attentional characteristics of the authenticity of facial expressions on holistic processing. The results show that both regulated and spontaneous expressions were processed holistically. However, the spontaneous expression details did not indicate typical holistic processing, with the congruency effect observed equally for aligned and misaligned conditions. No significant difference between the two expressions was observed in terms of reaction times and eye movement characteristics (i.e., total fixation duration, fixation counts, and first fixation duration). These findings suggest that holistic processing strategies differ between the two expressions. Nevertheless, the difference was not reflected in attentional engagement.
... The observation ratio of the eye on 3D is higher than that on 2D. Similar results were obtained in studies using eye trackers to observe pictures of real people, showing that the face and eyes are the main areas of the gaze [32]. ...
Soothing dolls are becoming increasingly popular in a society with a lot of physical and mental stress. Many products are also combined with soothing dolls to stimulate consumers’ desire for impulse buying. However, there is no research on the relationship between consumers’ purchasing behavior, consumers’ preference for soothing dolls, and visual preference. The purpose of this study was to examine the possible factors that affect the emotional and visual preferences of soothing dolls. Two local stores’ sales lists were used to extract three different types of dolls. The 2D and 3D versions of these three dolls were used. Subjective emotional preferences were examined by the self-assessment manikin (SAM) scale, with 5-point Likert scales for valence and arousal factors. An eye tracker was used to examine visual preferences, both before and after positive/negative emotion stimulation by the International Affective Picture System (IAPS). There were 37 subjects involved, with an age range of 20–28 years. The experimental results show that the average valence/arousal scores for 2D/3D dolls were (3.80, 3.74) and (2.65, 2.68), respectively. There was no statistical difference, but both 2D and 3D pictures had high valence scores. Eye tracker analysis revealed no gaze difference in visual preference between 2D and 3D dolls. After negative emotional picture stimulation, the observation time of the left-side doll decreased from 2.307 (std 0.905) to 1.947 (std 1.038) seconds, p < 0.001; and that of the right-side picture increased from 1.898 (std 0.907) to 2.252 (std 1.046) seconds, p < 0.001. The average observation time ratio of the eye on the 3D doll was 40.6%, higher than that on the 2D doll (34.3%, p = 0.02). Soothing dolls may be beneficial for emotion relaxation. Soothing dolls always have high valence features according to the SAM evaluation’s measurement. Moreover, this study proposes a novel research model using an eye-tracker and the SAM for the SOR framework.
... In the literature, fear is usually the least recognized emotion and it is highly confounded with surprise. [29][30][31] The reason why some participants fail to accurately identify emotions is still unclear. One of the most discussed and commented on reasons for confusion between emotions is that some of them share AUs. ...
... Other studies 34,35 have addressed the role these AUs play in emotion recognition. Also, as shown in previous works, 30,31,36 the presentation angle of the faces (frontal or in mid-profile view) affects the accuracy in emotion recognition, especially in the recognition of fear and increases its confusion with surprise. ...
... Guo et al. 30 obtained similar results for the emotions fear and sadness (from 52% to 50% and from 94% to 89%, respectively), while the accuracy was slightly better in the mid-profile view for anger (from 86% to 88%), surprise (from 90% to 91%), and equally for disgust (80% for both). This is partially consistent with our results in the desktop condition for anger (from 93.79% to 95.10%), surprise (from 91.55% to 94.52%), and disgust (from 82.64% to 88.19%). ...
The recognition of facial expression of emotions in others is essential in daily social interactions. The different areas of the face play different roles in decoding each emotion. To find out which ones are more important, the traditional approach has been to use eye-tracking devices with static pictures to capture which parts of the face people are looking at when decoding emotions. However, the ecological validity of this approach is limited because, unlike in real life, there is no movement in the face that can be used to identify the emotion. The use of virtual reality technology opens the door to new experiences in which the users perceive that they are in front of dynamic virtual humans. Therefore, our main aim is to investigate whether the user's immersion in a virtual environment influences the way dynamic virtual faces are visually scanned when decoding emotions. An experiment involving 74 healthy participants was carried out. The results obtained are consistent with previous works. Having confirmed the correct functioning of our solution, it is our intention to study whether emotion recognition deficits in patients with neuropsychiatric disorders are related to the way they visually scan faces.
... Similar findings were also confirmed by Bindemann & Lewis [58]. Moreover, an eye-tracking study revealed that when human faces were viewed in profile, the perceived intensity of the facial expression was reduced compared with frontal and diagonal conditions [59]. Presumably, this is connected with direct gaze. ...
Chimpanzees exhibit a variety of behaviours surrounding their dead, although much less is known about how they respond towards conspecific skeletons. We tested chimpanzees' visual attention to images of conspecific and non-conspecific stimuli (cat/chimp/dog/rat), shown simultaneously in four corners of a screen in distinct orientations (frontal/diagonal/lateral) of either one of three types (faces/skulls/skull-shaped stones). Additionally, we compared their visual attention towards chimpanzee-only stimuli (faces/skulls/skull-shaped stones). Lastly, we tested their attention towards specific regions of chimpanzee skulls. We theorized that chimpanzee skulls retaining face-like features would be perceived similarly to chimpanzee faces and thus be subjected to similar biases. Overall, supporting our hypotheses, the chimpanzees preferred conspecific-related stimuli. The results showed that chimpanzees attended: (i) significantly longer towards conspecific skulls than other species skulls (particularly in forward-facing and to a lesser extent diagonal orientations); (ii) significantly longer towards conspecific faces than other species faces at forward-facing and diagonal orientations; (iii) longer towards chimpanzee faces compared with chimpanzee skulls and skull-shaped stones, and (iv) attended significantly longer to the teeth, similar to findings for elephants. We suggest that chimpanzee skulls retain relevant, face-like features that arguably activate a domain-specific face module in chimpanzees' brains, guiding their attention.
