Figure - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
List of sources from the systematic literature review (in alphabetical order).
Source publication
In the late 1970s, analysis of facial expressions to unveil emotional states began to grow and flourish along with new technologies and software advances. Researchers have always been able to document what consumers do, but understanding how consumers feel at a specific moment in time is an important part of the product development puzzle. Because...
Similar publications
Since the launch of online video portals in 2005, museums have encouraged visitors to upload and share their visits online. Although much has been written about visitors’ experiences in museums, very little exists on the impact virtual visits have on viewers. In this qualitative pilot study, a total of 2035 emotional reactions were recorded and ana...
In human-centered product design and development, understanding the users is essential. Empathizing with the user can help designers to gain deeper insights into the user experience and their needs. However, capturing real time empathy during user interaction and the degree to which empathy enhances user understanding remains unclear. To narrow thi...
Human emotions recognization contributes to the development of human-computer interaction. The machines understanding human emotions in the real world will significantly contribute to life in the future. This paper will introduce the Affective Behavior Analysis in-the-wild (ABAW3) 2022 challenge. The paper focuses on solving the problem of the vale...
Automated emotional facial expression analysis (AEFEA) is used widely in applied research, including the development of screening/diagnostic systems for atypical human neurodevelopmental conditions. The validity of AEFEA systems has been systematically studied, but their test–retest reliability has not been researched thus far. We explored the test...
Facial expressions bring human interactions to life with nonverbal cues that convey hues of emotions, feelings, and cultural intent. Analyzing these expressions is essential in the age of digital transformation. Traditional approaches to real-time facial expression analysis have limitations in capturing the complexity of these expressions and consu...
Citations
... Few studies have considered the dynamic information of facial expression measurements (Zhi et al., 2018). Recently, studies have begun to use time-series data and temporal statistical analysis methods to characterize how facial expressions change over time (Kessler et al., 2020;Mahieu et al., 2019). Accordingly, the results of the multilevel analyses are presented in three parts. ...
Purpose
This study aims to investigate how message sidedness affects the impact of fake news posted on social media on consumers' emotional responses.
Design/methodology/approach
The study involves a face-tracking experiment in which 198 participants were exposed to different fake news messages concerning the COVID-19 vaccine. Specifically, participants were exposed to fake news using (1) a one-sided negative fake news message in which the message was entirely unfavorable and (2) a two-sided fake news message in which the negative message was mixed with favorable information. Noldus FaceReader 7, an automatic facial expression recognition system, was used to recognize participants' emotions as they read fake news. The authors sampled 17,450 observations of participants' emotional responses.
Findings
The results provide evidence of the significant influence of message sidedness on consumers' emotional valence and arousal. Specifically, two-sided fake news positively influences emotional valence, while one-sided fake news positively influences emotional arousal.
Originality/value
The current study demonstrates that research on fake news posted on social media may particularly benefit from insights regarding the potential but often overlooked importance of strategic design choices in fake news messages and their impact on consumers' emotional responses.
... By checking the reference lists of the studies cited, an additional 37 papers were identified that met the inclusion criteria. In total, 59 articles, including four review articles [7,[26][27][28] and 55 original research articles, were included in the final review. A flow chart summarising the study selection process is depicted in Figure 1. ...
A challenge in social marketing studies is the cognitive biases in consumers’ conscious and self-reported responses. To help address this concern, biometric techniques have been developed to obtain data from consumers’ implicit and non-verbal responses. A systematic literature review was conducted to explore biometric applications’ role in agri-food marketing to provide an integrated overview of this topic. A total of 55 original research articles and four review articles were identified, classified, and reviewed. It was found that there is a steady growth in the number of studies applying biometric approaches, with eye-tracking being the dominant method used to investigate consumers’ perceptions in the last decade. Most of the studies reviewed were conducted in Europe or the USA. Other biometric techniques used included facial expressions, heart rate, body temperature, and skin conductance. A wide range of scenarios concerning consumers’ purchase and consumption behaviour for agri-food products have been investigated using biometric-based techniques, indicating their broad applicability. Our findings suggest that biometric techniques are expanding for researchers in agri-food marketing, benefiting both academia and industry.
... En sus conclusiones destacan la necesidad de utilizar un enfoque multi-método para comprender completamente los diferentes procesos del comportamiento del consumidor. Kessler et al. (2020), por su parte, realizan una interesante revisión sistemática sobre el análisis automatizado de la expresión facial (AFEA) en el segmento de la alimentación y bebidas, proporcionando una lista de software comercial y una explicación de los métodos y prácticas de los estudios analizados. Utilizando el análisis automatizado de la expresión facial (AFEA), junto a pruebas de asociación implícita de objetivo único (ST-IAT) para capturar las emociones asociadas a varios tipos de envases de leche, Clark et al. (2021) comprobaron que el análisis de series temporales de los datos de AFEA proporcionó evidencia de diferencias significativas en la intensidad de las emociones. ...
Comprender cómo el consumidor percibe, evalúa y elige los productos permite a las empresas optimizar el diseño del packaging y agregar un valor que contribuye a las estrategias de branding. La evaluación del branding y el packaging es compleja e involucra procesos cognitivos y mecanismos cerebrales, como la atención, la memoria, las emociones, sistemas de recompensa y la toma de decisiones. El objetivo principal es realizar una revisión de las principales herramientas de la neurociencia aplicadas en la investigación de marketing del branding y el packaging, repasando su base teórica, métricas, ventajas, limitaciones y aplicaciones en el campo de estudio.
... Nevertheless, the representativeness of the population was not pursued, as the sample size was relatively small (N = 40), although it was large enough to find significant differences [69]. The number of participants in implicit measurement experiments is usually low due to the time required to individually gather the data from each participant, normally ranging from 30 to 75 respondents [34,70]. ...
Packaging is no longer a mere structural element that only aims to preserve foods, but it is also a powerful marketing tool able to affect product perception, purchase decision and consumers’ food choices. Incorporating consumers’ voices into packaging design through co-creation could maximise its impact on the market. The main goal of this exploratory study was to test the usefulness of co-creation with consumers for packaging design. For that purpose, a survey with 200 Spanish participants was conducted to find out which of the presented visual and textual packaging attributes were the most appropriate. A validation study with 40 participants using implicit (eye tracker, galvanic skin response and automatic facial expression analysis) and explicit measurements was used to test the packaging co-created by consumers against some of its possible competitors in the market. The co-creation process with consumers allowed for the identification of the visual and textual attributes, among the available options, that best fit their preferences, whereas the validation process confirmed that the packaging design co-created by consumers was equally or even preferred over the competitors. The information gathered might help designers and marketers to guide the packaging design for fish products in the Spanish market.
... Kessler et al. [5] examined the role of automated facial expression analysis (AFEA) on beverages. Despite the huge diversity in the literature about the methods used in the AFEA, these authors evidenced clear benefits to using AFEA in product and package development, leading to a more comprehensive understanding of the consumer. ...
In 2015, the journal Beverages (ISSN 2306-5710) was launched to provide insight into the beverage industry [...]
... The intensity variable was developed as a running mean to capture the maximum expression of negative and positive valence. This variable was based on an assumption that the expression would not last throughout the whole exposure time (7000 ms) and that facial expressions occur with different latencies (Kessler et al., 2020). The low threshold for the frequency variable (10%) was chosen with the purpose of detecting very early signs of emerging facial actions (Brand & Ulrich, 2019). ...
Pervasive exposure to a vast and varied food repertoire has contributed to the obesity epidemic. Within this issue, there is a need for a better understanding of the psychophysiological responses to food cues that precede food choice and food intake to establish how these responses contribute to the link between food availability and increasing obesity levels. Biometric measures such as eye tracking, electrodermal activity and facial expressions may separately or collectively provide deeper insight into psychophysiological processes underlying food reward and food intake. We examined how biometric responses differed in foods varying in fat and taste and explored how these biometric signatures to food cues were related to food preference behaviours, food choice, and food intake. We developed and tested a biometric food preference task designed to concurrently assess biometric responses (eye tracking, electrodermal activity and facial expressions) and food reward to visual food stimuli from different food categories in 100 normal weight adults. Food intake and selection was examined using a simultaneous choice ad libitum buffet. The results from this cross-sectional study showed significant differences in visual attention towards foods varying in fat content and taste prior to making rapid food choice decisions. Furthermore, the study found positive associations between maintained attention during a forced choice paradigm and subsequent food reward and food intake measures. Attention, arousal and facial expression during passive viewing were not associated with food reward or intake measures, except for an association between negative valence and explicit liking such that less liked foods elicited stronger negative facial expressions. The findings indicate that implicit, biometric responses to food cues predict both food reward and actual food intake.
... A face reader framework can have its excellence validated by qualitative and quantitative evaluations taken from attractive and smart perspectives of face reading (Shu et al., 2016(Shu et al., , 2017. iMotions, a software platform, allows for the use of various biometric devices in different research works as well as qualitative and quantitative measures for assessing the influence of behaviors on treatments (Kessler et al., 2020). The integration of quantitative biometric measures and experimental design research was undertaken by Lohmeyer and Meboldt (2016). ...
Hedonic valuation models enable us to make relevant decisions concerning the equipage of the built environment amenities and to establish more correct price zoning and land use management. These studies examining the utilitarian and hedonic aspects of property often face issues: how can we measure and assess multisensory experiences of property buyers (their affective attitudes, emotional states and physiological parameters) in an integrated manner and make a decision? This fact was a vital goal of the integrated Neutrosophic Degree of Project Utility and Investment Value Assessments along with Recommendation Provisions (INVAR) method. About 200 million multisensory data points have been collected so far and, as a result of their correlation analysis, several tens of thousands of average and robust correlations were determined. The new Neutrosophic INVAR method offers a bigger, more detailed picture of the hedonic-utilitarian value of the property. Another benefit of the Neutrosophic INVAR method is its ability to assist in assessing any project, determining its integrated hedonic-utilitarian value and offering tips on ways to make the project better. The Neutrosophic INVAR method can also optimise any criterion of your choice to make a specific project as competitive in the market and can determine the value that would propel the project to the top among the projects discussed. The analysis performed in this research is the basis for asserting that a sustainable development policy for housing and a surrounding environment is only possible by analysing the two-way process of interactions between a built environment and its neighbourhood residents in an integrated manner.
Food sensory evaluation mainly includes explicit and implicit measurement methods. Implicit measures of consumer perception are gaining significant attention in food sensory and consumer science as they provide effective, subconscious, objective analysis. A wide range of advanced technologies are now available for analyzing physiological and psychological responses, including facial analysis technology, neuroimaging technology, autonomic nervous system technology, and behavioral pattern measurement. However, researchers in the food field often lack systematic knowledge of these multidisciplinary technologies and struggle with interpreting their results. In order to bridge this gap, this review systematically describes the principles and highlights the applications in food sensory and consumer science of facial analysis technologies such as eye tracking, facial electromyography, and automatic facial expression analysis, as well as neuroimaging technologies like electroencephalography, magnetoencephalography, functional magnetic resonance imaging, and functional near‐infrared spectroscopy. Furthermore, we critically compare and discuss these advanced implicit techniques in the context of food sensory research and then accordingly propose prospects. Ultimately, we conclude that implicit measures should be complemented by traditional explicit measures to capture responses beyond preference. Facial analysis technologies offer a more objective reflection of sensory perception and attitudes toward food, whereas neuroimaging techniques provide valuable insight into the implicit physiological responses during food consumption. To enhance the interpretability and generalizability of implicit measurement results, further sensory studies are needed. Looking ahead, the combination of different methodological techniques in real‐life situations holds promise for consumer sensory science in the field of food research.
Lighting assessment in special operating environments like enclosed spaces is of great research significance and value. In addition to investigating the visual ergonomics of workers, the emotional monitoring and guidance of workers in enclosed spaces is also a research focus. Based on the Circumplex emotion model theory, this paper designs an experiment to assess emotions in an enclosed space with 6 different lighting settings (2 correlated color temperature (CCT) × 3 illuminance levels). From the perspective of subjective assessment, participants used a rapid sensory analysis method (Check-all-that-apply, CATA) and a Subjective Coordinate Scale (SCS) method for rapid ambience perception checking and emotional self-reporting of lighting settings. From the perspective of objective evaluation, the participants' facial expressions were recorded during the experiment using a camera, and then the recorded facial expressions were automatically analyzed and predicted using FaceReader (FRE) software. The CATA and SCS showed similar results, with the 3100 K × 600 lx, 3100 K × 1000 lx and 6500 K × 600 lx lighting settings creating a relaxed, pleasant emotion in participants, the 6500 K × 1000 lx setting creating an excited, tense atmosphere, and the low illumination level settings of 3100 K × 250 lx and 6500 K × 250 lx made participants feel tired and frustrated. The results of the objective emotion analysis indicate that the FRE was able to effectively identify differences in participants' emotions in response to different lighting settings and was consistent with the results of participants' subjective emotion reports. This laboratory study validates that the three methods can effectively assess the enclosed space lighting settings, and provides a reference for further research on the enclosed space lighting settings and emotional monitoring of workers in the future.
Sensorial perceptions change as people age and biometrics analysis can be used to explore the unconscious consumer
responses. Investigation was conducted of effects of consumer age (younger, 22–52 years; older, 60–76 years) on facial expression response (FER) during consumption of beef patties with varying firmness (soft, medium, hard) and taste (±plum sauce). Video images were collected and FERs analysed using FaceReader™. Younger people exhibited higher intensity for happy/sad/scared and lower intensity for neutral/disgusted, relative to older people. Interactions between age and texture/sauce showed little FER variation in older people, whereas younger people showed considerable FER variation. Younger people, but not older people, had lowest intensity
of happy FER and highest intensity of angry FER for the hard patty. Sauce addition resulted in higher intensity of
happy/contempt in younger consumers, but not older consumers. FER collected using FaceReader™ was successfully used to differentiate between the unconscious responses of younger and older consumers.
