Figure - uploaded by Gordon B Willis
Content may be subject to copyright.
Source publication
Vignette design has been largely neglected in anchoring vignette studies. This study aimed to contribute to the science of vignette design by developing and evaluating vignettes for measuring vision in rural China. Cognitive interviews were conducted among 36 participants in a Chinese middle school. The respondents either directly evaluated vision...
Context in source publication
Context 1
... out of the 36 participants answered the question regarding difficulty in responding to vignettes. Among those 31 participants, nine (29%) thought the two vignettes were equivalent in response difficulty, 12 (39%) thought that it was more difficult to answer the poor-vision vignette, and the other 10 (32%) thought the opposite (Table 4). ...Similar publications
Probabilistic integration provides a criterion for stopping a simulation when a specified error tolerance is satisfied with high confidence. We comment on some of the modeling assumptions and implementation issues involved in designing an automatic Bayesian cubature.
Citations
... We also found that the familiarity ratings of students who tried to translate the non-existing item into Czech or to define its meaning were about equally distributed across the familiarity scale. Cognitive interviews could reveal the way respondents understand non-existing items, contributing to the theory of foil item creation and selection (e. g., Su et al., 2017). Further studies should pay attention to foil construction and the identification of their measurement properties as their quality is fundamental for the OCT to work properly (e.g., Muszyński et al., 2021). ...
Student self-reports of knowledge are widely used by researchers and educators, though their accuracy has been questioned due to potential biases. The overclaiming technique (OCT), based on the familiarity ratings of existing (reals) and non-existing (foils) items, has been used to identify accuracy and exaggeration in respondents' self-reports. We developed an original English OCT measure specifically for students who learn English as a foreign language (EFL). Our sample consists of Czech lower secondary students (N = 1391). We show that students' foil claiming relates to their gender, EFL exposure at school, at home, in their free time, and through peers, but not to their school type. School type, however, is a relevant factor in the claiming of reals. Warning about foils relates to lower foil claiming. Further research could examine students' understanding of OCT items through cognitive interviews and extend the use of the OCT to other foreign languages.
... Our work builds on the work by Au and Lorgelly (2014), which focused on testing one of the assumptions (response consistency) using in-depth interviews, and made the first step in focusing on these assumptions in a qualitative way. Su et al. (2017) conducted cognitive interviews with Chinese students evaluating anchoring vignettes for measuring vision. They focused primarily on the comparison of vignette questions using noncomparative versus comparative judgments and they formulated recommendations for improving the design of vignette descriptions. ...
Self-assessment measures are commonly used in questionnaire surveys. However, one of the problems with self-reports is that they may be prone to differences in scale usage among respondents. The anchoring vignette method addresses this issue. It relies on two assumptions: response consistency and vignette equivalence. Here we aim to develop a framework for the examination of these assumptions using cognitive interviews and demonstrate it in the Information and Communications Technology (ICT) skills domain. First, we develop new anchoring vignettes describing various ICT skill domains. Second, we examine both assumptions using cognitive interviews with students who are studying different fields. Our analysis shows that the assumptions are indeed not adhered to, especially the assumption of vignette equivalence. Third, we develop a general framework for the examination of the method’s assumptions.
... Su et al., after discovering that many respondents were not sure how far "20 meters" was, edited their vision vignette to refer to distances in concrete but nonnumeric terms: "In the cafeteria, [Xiao Wang] can clearly recognize students sitting at his table, but not those sitting at the next table" (Su et al., 2017). This is a nice example of how a vignette can be concrete without depending on specialized knowledge. ...
Objective:
Anchoring vignettes appear with growing frequency in surveys of health and aging, but little research investigates how to optimize their wording. This study experimentally tests whether mentioning specific health conditions and/or medical procedures enhances or undermines vignette validity.
Methods:
Three series of general health anchoring vignettes were fielded to 2,550 respondents in the Wisconsin Longitudinal Study: one mentioning no specific health conditions or procedures, one mentioning heart disease-related ones, and one mentioning diabetes-related ones. Variations on hierarchical ordered probit models were used to test whether vignette wording affected adherence to the key measurement assumptions of vignette equivalence (VE) and response consistency (RC).
Results:
While all vignette series showed substantial violations of VE, violations were larger (especially by sex and education) when using disease-specific texts. RC violations appeared relatively minor, but somewhat larger in disease-specific texts.
Discussion:
These findings suggest that more general, universal vignette texts may be preferable to ones describing highly specific conditions/procedures. The common advice to prioritize specificity and concreteness in survey texts may be misguided if sociodemographic groups differ in their familiarity or associations with the presented details. Anchoring vignettes are a potentially useful survey tool, but further efforts are needed to optimize their wording.
The main question of this paper is how people may agree in their interpersonal comparisons of wellbeing. These comparisons are important in social ethics and for policy purposes. The paper firstly examines grounds for convergence in easy cases. Then comes a more difficult case of low convergence in order to explore a way to increase it. For this, concepts from the empirical subjective well-being literature are used: life satisfaction and vignettes. Ideas of John Harsanyi and Serge Kolm thereby receive a new look.
