Content uploaded by Filip Lievens
Author content
All content in this area was uploaded by Filip Lievens
Content may be subject to copyright.
A preview of the PDF is not available
Validity evidence for the situational judgment test paradigm in emotional intelligence measurement
Abstract and Figures
To date, various measurement approaches have been proposed to assess emotional intelligence (EI). Recently, two new EI tests have been developed based on the situational judgment test (SJT) paradigm: the Situational Test of Emotional Understanding (STEU) and the Situational Test of Emotion Management (STEM). Initial attempts have been made to examine the construct-related validity of these new tests; we extend these findings by placing the tests in a broad nomological network. To this end, 850 undergraduate students completed a personality inventory, a cognitive ability test, a self-report EI test, a performance-based EI measure, the STEU, and the STEM. The SJT-based EI tests were not strongly correlated with personality and fluid cognitive ability. Regarding their relation with existing EI measures, the tests did not capture the same construct as self-report EI measures, but corresponded rather to performance-based EI measures. Overall, these results lend support for the SJT paradigm for measuring EI as an ability.
Figures - uploaded by Filip Lievens
Author content
All figure content in this area was uploaded by Filip Lievens
Content may be subject to copyright.
... That EI involves skills related to evaluation (perhaps among other things) was a point made by Krishnakumar et al. (2016). In validating the NEAT, a situational judgment test of EI (Libbrecht & Lievens, 2012), the investigators administered a simple word evaluation task that asked individuals to evaluate (1 = extremely negative; 9 = extremely positive) 60 familiar words (e.g., gossip, illness, laughter, passion, world) that differed in normative valence (Bradley & Lang, 1999). Expertise in evaluating the words was scored in terms of discrepancies between the evaluations of a particular participant and norms that averaged across all participants (Legree et al., 2005). ...
... The NEAT is both reliable and valid. For example, it correlates at .69 with the Situational Test of Emotional Understanding and at .59 with the Situational Test of Emotion Management (Krishnakumar et al., 2016), which are well-established ability EI tests (Libbrecht & Lievens, 2012). The NEAT also correlates with personality traits (e.g., a −.24 correlation with neuroticism) and cognitive ability (e.g., a .35 ...
New efforts to understand the processes involved in ability-related emotional intelligence (ability EI) could reinvigorate this area of scholarship and research. It is proposed that participants with higher levels of ability EI are evaluation experts, which should be evident in the attitude domain. Study 1 (n = 148) probed for affective, cognitive, and behavioral reactions to a diverse set of attitude objects. In addition, attitude certainty ratings were collected. Higher levels of ability EI, but not self-reports of EI, were linked to attitudes that were more extreme, certain, and structurally integrated. In Study 2 (n = 602), participant employees completed standard personality and job satisfaction assessments. Higher levels of ability EI were predictive of greater polarization (as assessed in extremity-related terms) in both domains. The individual differences assessed by ability EI, these results suggest, also tend to support attitudes and opinions with stronger features.
... Responses are scored as correct (1) and or incorrect (0). The test-retest reliability of the full version of the test is 0.72 (Libbrecht and Lievens, 2012). Cronbach alpha was 0.47 and McDonald's omega was 0.63 in our sample. ...
... What action would be the most effective for Juno?," the most appropriate response is "Find out what is happening and discuss his concerns with his family." The test-rest reliability of the full version of the test is 0.85 (Libbrecht and Lievens, 2012). In our sample, Cronbach alpha was 0.62 and McDonald's omega was 0.65. ...
Emotion information processing (EIP) has been recently introduced as a new component of emotional intelligence. We present a task aiming at measuring a type of emotion information processing related to fine-grained discrimination of emotional expressions. We modified an existing task presenting morphed faces created from a blend of two prototypical emotional expressions. Participants’ (N = 154) ability-EI, in particular emotion recognition, understanding and management, as well as intelligence were evaluated. Results show that all facets of EI independently predicted accuracy in the discrimination task and that emotion recognition was the strongest predictor. When controlling for emotion recognition level, we found that emotion understanding still predicted accuracy for less difficult stimuli. Results support the idea that individuals high in EI have higher emotion processing skills at the emotion perception stage of information processing and suggest that the task employed in the current study might measure more spontaneous processing of emotional expressions. Implications regarding the use of the current task as a new measure of the EIP component are discussed.
... Another important limitation is the low reliability level, as measured by the Cronbach's alpha value of the ability EI measure (STEU). Although there is generalized consensus that this does not imply that situational judgment tests are unreliable (Libbrecht & Lievens, 2012), a replication of this study might provide further evidence that defends the results already obtained. ...
Context: Research on emotional intelligence (EI) shows this concept's decades-long positive influence on well-being, self-efficacy, employability, and academic and professional achievement. Indeed, several studies have demonstrated that students with high EI have better grades and quality of life, as well as are more employable than students with low EI. By considering the two conceptualizations of EI as either an ability or personality trait, the present study investigates EI's influence on training achievement in initial vocational education and training (IVET). We posited that the two types of EI positively influence training achievement in a complementary way: Ability EI relates to achievement in theoretical training , while trait EI relates to achievement in practical training. Furthermore, these links are mediated by apprentices' engagement at school and in learning. Method: To test our hypotheses, 92 dual IVET health and social care apprentices in their last year of vocational school completed an online survey composed of validated scales measuring EI as an ability, EI as a personality trait, personality traits, and school engagement. The apprentices' grades were also obtained with their permission.
... The STEU has 19 scenarios selected from 42-item version of STEU using an inter-response time analysis. 79,81 In each questionnaire item, a different interpersonal scenario presenting emotional situations is described, the participants are asked to select the more likely emotional response experienced by one of the persons in that situation from five of the given emotions. In this model, 14 emotions were evaluated in three separate contextsde-contextualization or context-reduction (n =14 items), work or workplace context (n =14 items) and private or personal life context (n =14 items). ...
Emotional intelligence (EI) is defined as ability of an individual to perceive, understand, regulate, control and utilise emotions. EI is a relatively nascent concept from the 1900s that attempts to connect emotion to general intelligence. Several clinical studies and survey research have confirmed positive association between EI and optimal mental and physical health, overall well-being, and on-aggressive behaviours. In this review, a brief overview on evolutional theory of EI, various models associated with EI, methods of classifying measures, and comparing various EI measurement methods are described. Indeed, EI assessment measures such as Multifactor EI Scale, Mayer-Salovey-Caruso Tests, Self-report Test, Trait EI Questionnaire, Bar-On Emotional Quotient Inventory, The Situational Test of Emotional Management, The Situational Test of Emotional Understanding, Emotional and Social competence Inventory are also discussed. The key purpose of this review is to encapsulate important components of EI to audiences, reap the benefits of evaluating EI and implying attributes to improve their emotional health and general well-being.
... The evidence for the internal consistency across emotion perception abilities is mixed. On the one hand, as Elfenbein and MacCann (2017) note, different emotion perception tasks positively relate to other facets of EI, such as emotion understanding (see Ferguson and Austin 2010;Libbrecht and Lievens 2012;Vonk et al. 2015). On the other hand, Joseph and Newman's (2010) meta-analysis found only moderate evidence for a link between emotion perception and intelligence (ρ = .10). ...
Emotion decoding accuracy (EDA) plays a central role within the emotional intelligence (EI) ability model. The EI-ability perspective typically assumes personality antecedents and social outcomes of EI abilities, yet, traditionally, there has been very limited research to support this contention. The present paper argues that the way in which EDA has been conceptualized and operationalized in EI research has ignored developments in social perception theory and research. These developments point, on one hand, to the importance of embedding emotion expressions in a social context and, on the other, to reformulating the definitions of emotion decoding accuracy. The present paper outlines the importance of context in the framework of a truth and bias model of the social perception of emotions (Assessment of Contextualized Emotions, ACE) for EI abilities.
The field of ability-related emotional intelligence (ability EI) could benefit from new perspectives concerning dynamic operations. According to a recent perspective, variations in ability EI are likely to be linked to variations in skills related to evaluation. This perspective contends, perhaps counterintuitively, that higher levels of ability EI are likely to be linked to higher levels of emotional reactivity, defined in terms of stronger event-emotion relationships. Two studies (total N = 245) pursue such ideas in the context of multilevel models involving event valence and emotional experience. Variations in ability EI modulated event-emotion relationships in the context of laboratory inductions involving hypothetical events (Study 1), affective images varying in valence (Study 1), and with respect to naturally occurring variations in positive and negative daily events (Study 2), such that higher levels of ability EI were linked to stronger event-emotion relationships, regardless of whether events and emotions were positive or negative in valence. These results provide new evidence for recent theorizing concerning ability EI while speaking to functional versus dysfunctional perspectives on emotional reactivity.
Emotion understanding (EU) ability is associated with healthy social functioning and psychological well-being. Across three studies, we develop and present validity evidence for the Core Relational Themes of Emotions (CORE) Test. The test measures people’s ability to identify relational themes underlying 19 positive and negative emotions. Relational themes are consistencies in the meaning people assign to emotional experiences. In Study 1, we developed and refined the test items employing a literature review, expert panel, and confusion matrix with a demographically diverse sample. Correctness criteria were determined using theory and prior research, and a progressive (degrees of correctness) paradigm was utilized to score the test. In Study 2, the CORE demonstrated high internal consistency and a confirmatory factor analysis supported the unidimensional factor structure. The CORE showed evidence of convergence with established EU ability measures and divergent relationships with verbal intelligence and demographic characteristics, supporting its construct validity. Also, the CORE was associated with less relational conflict. In Study 3, the CORE was associated with more adaptive and less maladaptive coping and higher well-being on multiple indicators. A set of effects remained, accounting for variance from a widely used EU test, supporting the CORE’s incremental validity. Theoretical and methodological contributions are discussed.
The question of whether ability-related emotional intelligence (ability EI) predicts important life outcomes has attracted considerably more attention than the question of what ability EI consists of. In the present paper, the authors draw from the attitude and emotion literatures to suggest that the evaluation dimension of meaning is likely key in understanding how ability EI operates. Measures of ability EI predict the extent to which individuals can accurately evaluate words and measures of the latter type act as emotional intelligence measures. Extending this analysis, the paper reviews recent sources of data linking ability EI to attitudinal processes, such as those involved in attitude–behavior relationships and affective bipolarity. Individuals with high EI appear to experience their affect in more bipolar terms and they display evidence of greater decisiveness in their evaluations. Pursuing links of the present type will allow researchers to generate new predictions concerning the ability EI construct.
A situational judgment test (SJT) is a psychological instrument typically used to assess the suitability of applicants in personnel selection or development. Interest in SJTs has increased over the past decades as research has shown considerable validity of SJTs and various other benefits. Researchers often provide information about internal consistency reliability of SJTs, but this estimate is known to be unsuitable for SJTs because they are heterogeneous tests; a better estimate for the reliability of SJTs may be retest reliability. Thus, this study represents the first meta-analysis investigating the retest reliability of SJTs. The results reveal a pooled retest reliability estimate of r = .698 based on k = 37 coefficients. Additionally, variables associated with test characteristics and variables containing information about the study and the participants were examined as potential moderators.
A method for investigating measurement equivalence across subpopulations is developed and applied to an instrument frequently used to assess job satisfaction (the Job Descriptive Index; JDI). The method is based on Jöreskog's simultaneous factor analysis in several populations. Several adaptations are necessary to overcome problems with violations of assumptions that occur with rating scale data. Two studies were conducted to evaluate the measurement equivalence of the JDI across different subpopulations. Investigation of five relatively homogeneous subpopulations within one industry revealed invariant measurement properties for the JDI. In the second study, measurement equivalence of the JDI was examined across health care, retailing, and military samples. Generally small violations of measurement equivalence were found. The results in both studies indicate that mean differences in JDI scores (i.e., differences in job satisfaction across groups) are due to group differences rather than lack of measurement equivalence.
This chapter addresses four interrelated questions: What are the appraisals (motive-relevant evaluations) that cause particular emotions? Why do these particular appraisals cause these particular emotions? How can this model of appraisal-emotion relationships be applied to understand and influence emotions and emotional behaviors? What research might be undertaken to extend and deepen our understanding of the process of emotion generation and emotional response? The current version of the appraisal model that my colleagues and I have been developing proposes that seven appraisals of events directly influence emotions:1 (1) unexpectedness: not unexpected/unexpected (whether the event violates one’s expectations); (2) situational state: motive-inconsistent/motive-consistent (whether the event is unwanted or is wanted by the person); (3) motivational state: aversive/ appetitive (whether the event is being related to a desire to get less of something punishing or a desire to get more of something rewarding); (4) probability: uncertain/ certain (whether the occurrence of motive-relevant aspects of the event is merely possible or is definite); (5) agency: circumstances/other person/self (what or who caused the motive-relevant event); (6) control potential: low/high (whether there is nothing one can do or something one can do about the motive-relevant aspects of an event); and (7) problem type: instrumental/intrinsic (whether a motive-inconsistent event is unwanted because it blocks attainment of a goal or unwanted because of some inherent characteristic).
Entwicklung von Gc nach der Schule (S. 143, siehe auch Ackerman, 1996, 234f):
,
One must not forget that nine-tenths of generalizations and theorizing about intelligence and intelligence tests are based on observations in school (p. 142)
In recent years, innovative schools have developed courses in what has been termed emotional literacy, emotional intelligence, or emotional competence. This volume evaluates these developments scientifically, pairing the perspectives of psychologists with those of educators who offer valuable commentary on the latest research. It is an authoritative study that describes the scientific basis for our knowledge about emotion as it relates specifically to children, the classroom environment, and emotional literacy. Key topics include: historical perspectives on emotional intelligence neurological bases for emotional development the development of social skills and childhood socialization of emotion. Experts in psychology and education have long viewed thinking and feeling as polar opposites reason on the one hand, and passion on the other. And emotion, often labeled as chaotic, haphazard, and immature, has not traditionally been seen as assisting reason. All that changed in 1990, when Peter Salovey and John D. Mayer coined the term emotional intelligence as a challenge to the belief that intelligence is not based on processing emotion-laden information. Salovey and Mayer defined emotional intelligence as the ability to monitor one’s own and others’ feelings and emotions, to discriminate among them, and to use motivated scientists, educators, parents, and many others to consider the ways in which emotions themselves comprise an intelligent system. With this groundbreaking volume, invited contributors present cutting-edge research on emotions and emotional development in a manner useful to educators, psychologists, and anyone interested in the unfolding of emotions during childhood. In recent years, innovative schools have developed courses in “emotional literacy” that making; these classes teach children how to understand and manage their feelings and how to get along with one another. Many such programs have achieved national prominence, and preliminary scientific evaluations have shown promising results. Until recently, however, there has been little contact between educators developing these types of programs and psychologists studying the neurological underpinnings and development of human emotions. This unique book links theory and practice by juxtaposing scientific explanations of emotion with short commentaries from educators who elaborate on how these advances can be put to use in the classroom. Accessible and enlightening, Emotional Development and Emotional Intelligence provides ample evidence about emotional intelligence as well as sound information on the potential efficacy of educational programs based on this idea.
This article presents a framework for emotional intelligence, a set of skills hypothesized to contribute to the accurate appraisal and expression of emotion in oneself and in others, the effective regulation of emotion in self and others, and the use of feelings to motivate, plan, and achieve in one's life. We start by reviewing the debate about the adaptive versus maladaptive qualities of emotion. We then explore the literature on intelligence, and especially social intelligence, to examine the place of emotion in traditional intelligence conceptions. A framework for integrating the research on emotion-related skills is then described. Next, we review the components of emotional intelligence. To conclude the review, the role of emotional intelligence in mental health is discussed and avenues for further investigation are suggested.
Emotional intelligence is a type of social intelligence that involves the ability to monitor one's own and others' emotions, to discriminate among them, and to use the information to guide one's thinking and actions (Salovey & Mayer, 1990). We discuss (a) whether intelligence is an appropriate metaphor for the construct, and (b) the abilities and mechanisms that may underlie emotional intelligence. © 1993.