No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Consistency in Interpretation of Probabilistic Phrases
Abstract
It is occasionally claimed in both applied decision analysis and in basic research that people can better use and understand probabilistic opinions expressed by nonnumerical phrases, such as “unlikely” or “probably,” than by numbers. It is important for practical and theoretical reasons to evaluate this claim. The available literature indicates that there is large variability in the mapping of phrases to numbers, but provides no indication as to its cause. This study asks (a) whether the variability can be attributed to how people interpret the phrases per se, rather than to how they use the number scale and (b) whether the variability is due primarily to between-subject or to within-subject factors. In order to answer these questions, 32 subjects ranked and compared 19 probability phrases on each of three occasions. The results show that individuals have a relatively stable rank ordering of the phrases over time, but that different individuals have different rank orderings. Practical and methodological implications of these data are discussed.
... Concurrently, individuals show an internal consistence in the use and interpretation of VPE [9,10]. Dhami and Wallsten (2005) linked those findings and argued, that individuals have a stable lexicon of VPE, which however may differ considerably among individuals [7]. ...
... According to [2] most individuals are unaware of both the ambiguity of VPE as well as the variability of VPE interpretations in the general population. Hence, various examinations have focused on providing a translation aid from verbal to numerical probabilities or vice versa [8,9,10,13,14,15]. ...
... Year Origin N Sample Context Bergenstrom [18] 2003 UK/USA 87 Medical students Medical Brun [2] 1988 Norway 16 Psychology students Neutral Budescu [9] 1985 USA 32 Psychology students Neutral Chee [11] 2006 Malaysia 32 Mixed (mainly students) Neutral Cohn [4] 2009 Mexico/USA 263 Mixed (mainly students) Medical Damrosch [19] 1983 USA 70 ...
Introduction:
Verbal probabilities such as "likely" or "probable" are commonly used to describe situations of uncertainty or risk and are easy and natural to most people. Numerous studies are devoted to the translation of verbal probability expressions to numerical probabilities.
Methods:
The present work aims to summarize existing research on the numerical interpretation of verbal probabilities. This was accomplished by means of a systematic literature review and meta-analysis conducted alongside the MOOSE-guidelines for meta-analysis of observational studies in epidemiology. Studies were included, if they provided empirical assignments of verbal probabilities to numerical values.
Results:
The literature search identified 181 publications and finally led to 21 included articles and the procession of 35 verbal probability expressions. Sample size of the studies ranged from 11 to 683 participants and covered a period of half a century from 1967 to 2018. In half of the studies, verbal probabilities were delivered in a neutral context followed by a medical context. Mean values of the verbal probabilities range from 7.24% for the term "impossible" up to 94.79% for the term "definite".
Discussion:
According to the results, there is a common 'across-study' consensus of 35 probability expressions for describing different degrees of probability, whose numerical interpretation follows a linear course. However, heterogeneity of studies was considerably high and should be considered as a limiting factor.
... Due to the uncertain nature of the entire decision problem, precise numerical values are generally avoided because they may imply a sense of precision that a decision-maker does not want (Huizingh & Vrolijk, 1997). For example, people mostly think and talk about uncertainty in terms of verbal phrases (e.g., likely, almost certainly not, etc.,) and are more skilled in using the rules of language as compared to employing the rules of probability (Budescu & Wallsten, 1985). Linguistic labels are used across different domains such as business, academia, intelligence, medicine and politics in order to express preferences and/or judgments (e.g., significance, probabilities, etc.). ...
... It can be claimed that the numerical interpretation of linguistic labels is not same for all individuals due to the fact that words possess different meanings for different individuals. It is important for practical and theoretical reasons to evaluate this claim in the context of AHP, analogous to the case of probabilistic phrases (e.g., (Kent, 1964), (Tavana et al., 1997), (Budescu & Wallsten, 1985)). Unfortunately, not many empirical studies are available in the literature. ...
Linguistic labels are effective means of expressing qualitative assessments because they account for the uncertain nature of human preferences. However, to perform computations with linguistic labels, they must first be converted to numbers using a scale function. Within the context of the Analytic Hierarchy Process (AHP), the most popular scale used to represent linguistic labels numerically is the linear 1-9 scale, which was proposed by Saaty. However, this scale has been criticized by several researchers, and various alternatives are proposed in the literature. There is a growing interest in scale individualization rather than relying on a generic fixed scale since the perceptions of the decision maker regarding these linguistic labels are highly subjective. The methods proposed in the literature for scale individualization focus on minimizing the transitivity errors, i.e., consistency. In this research, we proposed a novel, easy-to-learn, easy-to-implement, and computationally less demanding scale individualization approach based on compatibility. We also developed an experimental setup and introduced two new metrics that can be used by researchers that contribute to the theory of AHP. To assess the value of scale individualization in general, and the performance of the proposed novel approach in particular, numerical and two empirical studies are conducted. The results of the analyses demonstrate that the scale individualization outperforms the conventional fixed scale approach and validates the benefit of the proposed novel heuristic.
Keywords: Decision Analysis, Linguistic Labels, Scale Individualization, Pairwise Comparisons, Multi Criteria Decision Making (MCDM)
... (Huizingh & Vrolijk, 1997). For example, people mostly think and talk about uncertainty in terms of verbal phrases (e.g., likely, almost certainly not, etc.,) and are more skilled in using the rules of language as compared to employing the rules of probability (Budescu & Wallsten, 1985). Linguistic labels are used across different domains such as business, academia, intelligence, medicine and politics in order to express preferences and/or judgments (e.g., significance, probabilities, etc.). ...
... It can be claimed that the numerical interpretation of linguistic labels is not same for all individuals due to the fact that words possess different meanings for different individuals. It is important for practical and theoretical reasons to evaluate this claim in the context of AHP, analogous to the case of probabilistic phrases (e.g., (Kent, 1964); (Tavana et al., 1997); (Budescu & Wallsten, 1985). Unfortunately, not many empirical studies are available in the literature. ...
Linguistic labels are effective means of expressing qualitative assessments because they account for the uncertain nature of human preferences. However, to perform computations with linguistic labels, they must first be converted to numbers using a scale function. Within the context of the Analytic Hierarchy Process (AHP), the most popular scale used to represent linguistic labels numerically is the linear 1-9 scale, which was proposed by Saaty. However, this scale has been criticized by several researchers, and various alternatives are proposed in the literature. There is a growing interest in scale individualization rather than relying on a generic fixed scale since the perceptions of the decision maker regarding these linguistic labels are highly subjective. The methods proposed in the literature for scale individualization focus on minimizing the transitivity errors, i.e., consistency. In this research, we proposed a novel, easy-to-learn, easy-to-implement, and computationally less demanding scale individualization approach based on compatibility. We also developed an experimental setup and introduced two new metrics that can be used by researchers that contribute to the theory of AHP. To assess the value of scale individualization in general, and the performance of the proposed novel approach in particular, numerical and two empirical studies are conducted. The results of the analyses demonstrate that the scale individualization outperforms the conventional fixed scale approach and validates the benefit of the proposed novel heuristic.
... Numerical formats can facilitate probability comparison [46]. Yet because probability is not understood in the same way by everyone, studies have shown that numbers can provide illusory precision [7]. ...
... We aim to see if this effect can be found for visualizations of similar values, and whether different patterns are found in languages beyond English. Empirical studies on the numerical estimation of probability expressions have used elicitation methods comprising word-to-number translations [7], number-to-word conversion [36], and rank ordering of expressions [30,38]. In these studies, probability expressions are generally studied by giving people probabilistic outcomes for specifc scenarios. ...
Visualizations today are used across a wide range of languages and cultures. Yet the extent to which language impacts how we reason about data and visualizations remains unclear. In this paper, we explore the intersection of visualization and language through a cross-language study on estimative probability tasks with icon-array visualizations. Across Arabic, English, French, German, and Mandarin, n = 50 participants per language both chose probability expressions - e.g. likely, probable - to describe icon-array visualizations (Vis-to-Expression), and drew icon-array visualizations to match a given expression (Expression-to-Vis). Results suggest that there is no clear one-to-one mapping of probability expressions and associated visual ranges between languages. Several translated expressions fell significantly above or below the range of the corresponding English expressions. Compared to other languages, French and German respondents appear to exhibit high levels of consistency between the visualizations they drew and the words they chose. Participants across languages used similar words when describing scenarios above 80% chance, with more variance in expressions targeting mid-range and lower values. We discuss how these results suggest potential differences in the expressiveness of language as it relates to visualization interpretation and design goals, as well as practical implications for translation efforts and future studies at the intersection of languages, culture, and visualization. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/g5d4r/.
... Further, as language can express varying degrees of uncertainty, some authors have stressed that the epistemic modality of an utterance entails a probabilistic meaning (Nuyts, 2001). Yet, empirical findings indicate that speculative formulations including linguistic cues of uncertainty, such as "probably" or "could," can be identified only with a broad and vague instead of a precise spectrum of numerical probabilities (Brun & Teigen, 1988;Budescu & Wallsten, 1985;Lichtenstein & Newman, 1967). In line with this observation, the epistemic modality of an utterance is widely considered to carry a distinct communicative function, compared with uncertainty that is represented through numbers. ...
... Thus, theoretically, in order to cover a spectrum of uncertainty, an infinite number of discrete tags would be needed (e.g., "a little likely"; "likely"; "highly likely"; etc.). Yet, an efficient storage of uncertainty through its reduction to a discrete memory category does seem plausible in light of prior linguistic work: It aligns with the observation (outlined above) that linguistic uncertainty expressions carry a vagueness regarding their probabilistic meaning and, thus, correspond mostly to a broad instead of a precise range of probabilities in empirical studies (Brun & Teigen, 1988;Budescu & Wallsten, 1985;Lichtenstein & Newman, 1967). From this perspective, representing numerous nuances of uncertainty would be rendered obsolete, as they would lack an informative value. ...
Modern media report news remarkably fast, often before the information is confirmed. This general tendency is even more pronounced in times of an increasing demand for information, such as during pressing natural phenomena or the pandemic spreading of diseases. Yet, even if early reports correctly identify their content as speculative (rather than factual), recipients may not adequately consider the preliminary nature of such information. Theories on language processing suggest that understanding a speculation requires its reconstruction as a factual assertion first-which can later be erroneously remembered. This would lead to a bias to remember and treat speculations as if they were factual, rather than falling for the reverse mistake. In six experiments, however, we demonstrate the opposite pattern. Participants read news headlines with explanations for distinct events either in form of a fact or a speculation (as still being investigated). Both kinds of framings increased participants' belief in the correctness of the respective explanations to an equal extent (relative to receiving no explanation). Importantly, however, this effect was not mainly driven by a neglect of uncertainty cues (as present in speculations). In contrast, our memory experiments (recognition and cued recall) revealed a reverse distortion: a bias to falsely remember and treat a presented "fact" as if it were merely speculative. Based on these surprising results, we outline new theoretical accounts on the processing of (un)certainty cues which incorporate their broader context. Particularly, we propose that facts in the news might be remembered differently once they are presented among speculations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... Working against this possibility, however, is a large body of research showing that verbal expressions of uncertainty produce highly variable interpretations across individuals (e.g., Beyth-Marom, 1982;Budescu & Wallsten, 1985). Verbal confidence statements are inherently more vague and less precise than are numeric confidence statements. ...
... These findings contribute to a growing literature showing that eyewitnesses' verbal and numeric confidence statements are similarly informative regarding eyewitness accuracy (Dodson & Dobolyi, 2016;Mansour, 2020;Weber et al., 2008). Theoretically, these findings suggest that any potential interpretive difficulties associated with verbal communications of uncertainty (e.g., Beyth-Marom, 1982;Budescu & Wallsten, 1985) may be counteracted by the capacity of verbal statements to more accurately tap witnesses' underlying feelings of certainty. This is consistent with the idea that people are largely "pre-Bernoullian" in their reasoning about uncertainty and are naturally inclined to provide uncertainty estimates verbally rather than numerically (Windschitl & Wells, 1996, p. 161;Zimmer, 1983). ...
Objectives:
We assessed recent policy recommendations to collect eyewitnesses' confidence statements in witnesses' own words as opposed to numerically. We conducted an experiment to test whether eyewitnesses' free-report verbal confidence statements are as diagnostic of eyewitness accuracy as their numeric confidence statements and whether the diagnostic utility of eyewitnesses' verbal and numeric confidence statements varies across witnessing conditions.
Hypotheses:
We hypothesized that eyewitnesses' verbal and numeric confidence statements are both significantly associated with identification accuracy among choosers and that their diagnostic utility holds across varying witnessing conditions.
Method:
In the first phase of the experiment, eyewitnesses (N = 4,795 MTurkers; 48.8% female; 50.8% male; .3% other; age M = 36.9) viewed a videotaped mock-crime and made an identification decision from a culprit-present or culprit-absent lineup. We manipulated witnessing conditions at encoding and retrieval to obtain varied levels of memory performance. In the second phase of the experiment, evaluators (N = 456 MTurkers; 35.5% female; 62.7% male .4% other; age M = 36.5) translated witnesses' verbal confidence statements to a numeric estimate and we used calibration and confidence-accuracy characteristic analyses to compare the diagnosticity of witnesses' verbal and numeric confidence statements across the two levels of memory performance.
Results:
Witnesses' verbal and numeric confidence statements were significantly and nondifferentially diagnostic of eyewitness accuracy for both choosers and nonchoosers, and their diagnostic utility held across variations in witnessing conditions.
Conclusions:
These findings suggest the applied utility of collecting either verbal or numeric confidence statements from eyewitnesses immediately following an identification decision. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
... This is typically the case for stimuli that have a different meaning for different respondents. For instance, individuals are likely to think of different likelihoods when being confronted with verbal quantifiers such as "seldom", "often", or "likely" (Budescu & Wallsten, 1985;Harris et al., 2017;Karelitz & Budescu, 2004). Each particular quantifier can be thought of as a probability interval with which the occurrence of an event can be described. ...
In psychological research, respondents are usually asked to answer questions with a single response value. A useful alternative are interval responses like the dual-range slider (DRS) where respondents provide an interval with a lower and an upper bound for each item. Interval responses may be used to measure psychological constructs such as uncertainty in the domain of personality (e.g., self-ratings), estimation (e.g., forecasting), variability in and ambiguity in judgments (e.g., concerning the pragmatic use of verbal quantifiers). However, it is unclear whether respondents are sensitive to the requirements of a particular task and whether interval widths actually measure the constructs of interest. To test the discriminant validity of interval widths, we conducted a study in which respondents answered 92 items belonging to seven different tasks from the domains of personality, estimation, and judgment. We investigated the dimensional structure of interval widths by fitting exploratory and confirmatory factor models while using an appropriate multivariate-logit function to transform the bounded interval responses. The estimated factorial structure closely followed the theoretically assumed structure of the tasks, which varied in their degree of similarity. We did not find a strong overarching general factor, which speaks against a response style influencing interval widths across all tasks and domains. Overall, this indicates that respondents are sensitive to the requirements of different tasks and domains when using interval response formats.
... Type was a between-subjects manipulation, participants in both conditions saw the same base confidence statement, and half of them also read the accompanying featural justification. Third, participants only read verbal expressions of confidence, as real-life witnesses are encouraged to provide verbal statements and verbal statements should provide more variation in interpretations (Beyth-Marom, 1982;Budescu & Wallsten, 1985). ...
Although the confidence-accuracy relationship is now well established, confidence assessments are usually taken after the lineup identification procedure. Witnesses, however, often express confidence in their potential identification accuracy at other times, such as prior to seeing a lineup. Recent research has shown that these post-identification confidence statements are not consistently interpreted in the manner witnesses intend them. The present studies compare interpretations of pre-and post-identification confidence statements, and examine whether these interpretations are similarly affected by numerical statements and featural justifications. Across four studies, participants read eyewitness confidence statements and judged how confident and accurate they perceived witnesses to be. We manipulated expression type (verbal, numerical), statement type (confidence only, confidence paired with justification), and statement time (pre-and post-identification). Pre-identification confidence statements were perceived as less confident and less likely to be accurate. Unlike post-identification statements, pre-identification statements were not discounted when accompanied by featural justifications. K E Y W O R D S accuracy, confidence, confidence statements, eyewitness, perceptions
... In addition to biases associated with limited attention, managers are susceptible to underestimating the likelihood and magnitude of risks (Weinstein 1980), ignoring outcomes that are considered remote (March and Shapira 1987). Managers are also found to be more comfortable with verbal rather than numeric characterizations of risk, although numeric characterizations often provide context-dependent outcomes that may not be considered through verbal characterizations (Budescu and Wallsten 1985). Although executives can be concerned about nontax risks such as reputation, a verbal characterization of the "Wall Street Journal test" (Graham et al. 2014) is potentially inadequate to understand the full spectrum of risk. ...
Tax aggressiveness presents nontax risks to firms’ cash flow. Evaluating these risks requires information beyond the accounting function’s expertise, resulting in high processing costs to acquire and integrate risk information relevant to tax strategies. Managers can rationally adapt by making assumptions about risk information, potentially resulting in decision biases when evaluating the risk-reward tradeoff of tax aggressiveness. Using a novel regulatory setting in the U.S. insurance industry, I examine whether the adoption of mandated enterprise risk assessments updates managers’ prior beliefs about the nontax risks of tax aggressiveness. I find that as regulation requires managers to accept processing costs to acquire and integrate risk information, managers learn about previously underestimated nontax risks and significantly reduce tax aggressiveness. Results suggest that absent firm-wide internal risk information, managers can use aggressive tax positions without fully considering nontax risks.
Data Availability: Data used in this study are available from public sources identified in the paper.
JEL Classifications: G22; G32; H25; M41.
... The consensus seems to be that VPEs are generally vague, and characterise a fuzzy range of probabilities, rather than specific points on the 0-100% scale. Attempts have accordingly been made to map the numeric meanings of selected phrases as distributions with a characteristic shape and peak (Budescu & Wallsten, 1985Wallsten et al., 1986). Mapping the "membership function" of a single phrase requires a large number of responses from each participant, which in addition to the time and effort involved, demands considerable insight in the usage of such terms in a variety of contexts. ...
The paper reviews two strands of research on communication of uncertainty that usually have been investigated separately: (1) Probabilities attached to specific outcomes, and (2) Range judgments. Probabilities are sometimes expressed by verbal phrases (“rain is likely”) and at other times in a numeric format (“70% chance of rain”), whereas range judgments describe the potential amounts expected (“1–4 mm of rain”). Examination of previous research shows that both descriptions convey, in addition to the strength of expectations, pragmatic information about the communicative situation. For instance, so-called verbal probability expressions (VPE), as likely, unlikely, a chance, or not certain give some, albeit vague, probabilistic information, but carry in addition an implicit message about the sources of uncertainty, the outcome’s valence and severity, along with information about the speakers’ attitudes and their communicative intentions. VPEs are directional by drawing attention either to an outcome’s occurrence (“it is possible”) or to its non-occurrence (“it is doubtful”). In this sense they may be more informative than numbers. Uncertainties about outcomes in a distribution (continuous quantities) are alternatively expressed as interval estimates. The width of such intervals can function as a cue to credibility and expertise. Incomplete, one-sided intervals, where only one boundary is stated, imply directionality. “More than 100 people” suggests a crowd, while “less than 200” implies a shortfall. As with VPEs, directionally positive intervals are more frequent, and perhaps more neutral than negative ones. To convey expectancies and uncertainty in a balanced way, communicators may have to alternate between complementary frames.
... In the tourism domain, subjectively perceived risk gets more attention than its conception based on objectivity because tourists tend to select a limited number of possible negative outcomes related to themselves in their consideration rather than the total outcomes (Budescu & Wallsten, 1985;Wolff et al., 2019). It is generally believed that perceived risk is a major concern in the tourist decision-making process (Sönmez & Graefe, 1998). ...
Previous studies have reported insignificant, weak, or small-sized effects of perceptions of tourism-related risks and crises on tourist behavioral outcomes. In this article, we delicately posited perceived risk as a necessary-but-not-sufficient cause for the intention to travel. Without low perceived risk, there is no high travel intention. However, high perceived risk does not guarantee high travel intention. To test both average and necessity effects, we utilized SEM and necessary condition analysis (NCA), respectively. From the two separate studies in the context of North Korea tourism, we demonstrated that perceived risks have a necessity effect on travel intention whereas their average effects are either insignificant or small-sized. This article suggests repositioning the role of perceived risk as a “necessary condition” for travel decision-making and introduces a new methodology, NCA, that helps to build parsimonious theories in the tourism field.
... It has previously been experimentally demonstrated that pairwise comparisons are more precise than direct evaluations (Millet 1997), (Por and Budescu 2017), (Whitaker 2007). Generally, the comparisons are expressed on a verbal scale because it is more familiar to decision-makers and they understand it better than numerical evaluations (Budescu and Wallsten 1985). The verbal scale is only converted into numerical values to calculate priorities in a second step without the decision-maker. ...
Pairwise comparisons have been a long-standing technique for comparing alternatives/criteria and their role has been pivotal in the development of modern decision-making methods. The evaluation is very often done linguistically. Several scales have been544 proposed to translate the linguistic evaluation into a quantitative evaluation. In this paper, we perform an experiment to investigate, under our methodological choices, which type of scale provides the best matching of the decision-maker’s verbal representation. The experiment aims to evaluate the suitability of eight evaluation scales for problems of different sizes. We find that the inverse linear scale provides the best matching verbal representation whenever the objective data are measured by means of pairwise comparisons matrices and a suitable distance between matrices is applied for computing the matching error.
... Thus, integrating the proposed method with the existing debiasing techniques is necessary to make the results more reliable. Secondly, the 38 ambiguity of linguistic terms may also have impact on the accuracy of the results (Budescu and Wallsten, 1985;Wallsten et al., 1986). In the future, the rough set theory can be integrated to handle the ambiguity in the expert judgments. ...
Identifying the key factors of interconnected systems and their relationships is a typical task in system analysis. However, most previous methods consider less about the nuanced effects of positive and negative influences on the accuracy of identification of critical factors in systems. The initial importance and strength attenuation effects of factors are often omitted in previous studies. To solve these problems, a novel method considering heterogeneous influence and strength attenuation (HISA) effects is developed in this paper. The effectiveness of the proposed method is validated by applying it to design of smart home system. Comparisons between the proposed method and previous approaches are conducted. The results show that the proposed method can make full use of the limited information and discern subtle relationships between factors to obtain more accurate and reasonable results.
... This study also used numeric descriptions of the eyewitness' confidence, but oftentimes an eyewitness describes confidence in his or her own words. Prior research has shown that there is more variability in jurors' interpretations of verbal confidence judgments (Budescu & Wallsten, 1985;Dodson & Dobolyi, 2015). Would our results change if verbal descriptions of confidence had been used? ...
Recent research shows a strong positive relationship between eyewitness confidence and identification accuracy, assuming the confidence judgment results from a first, fair test of memory. The current study examines whether jurors understand this relationship, and the boundary conditions under which this understanding holds. Mock jurors read a trial transcript in which we manipulated the eyewitness’ level of confidence (high vs. low), the timing of the confidence judgment (initial, courtroom), and its consistency (if the eyewitness expressed initial and courtroom confidence, did the two judgments match). Mock jurors voted guilty more when confidence was high, regardless of when the confidence judgment was made, or whether there were inconsistencies in the confidence levels. Jurors need a more nuanced appreciation of the role of eyewitness confidence, and we discuss ideas for potential interventions that may aid jurors’ decision making.
... Experiment 2 made three methodological improvements to Experiment 1. First, bearing in mind that people vary in their individual interpretations of verbal quantifiers (Budescu & Wallsten, 1985) and interpreations across contexts (e.g., Patt & Schrag, 2003;Piercey, 2009;Weber & Hilton, 1990), we controlled for this by having participants provide their own numerical interpretations of verbal probabilities in each context (sun and rain). This ensured that participants would see the verbal and numerical quantifiers as equivalent. ...
When people are given quantified information (e.g., ‘there is a 60% chance of rain’), the format of quantifiers (i.e., numerical: ‘a 60% chance’ vs. verbal: ‘it is likely’) might affect their decisions. Previous studies with indirect cues of judgements and decisions (e.g., response times, decision outcomes) give inconsistent findings that could support either a more intuitive process for verbal than numerical quantifiers or a greater focus on the context (e.g., rain) for verbal than numerical quantifiers. We used two pre-registered eye-tracking experiments (n(1) = 148, n(2) = 133) to investigate decision-making processes with verbal and numerical quantifiers. Participants evaluated multiple verbally or numerically quantified nutrition labels (Experiment 1) and weather forecasts (Experiment 2) with different context valence (positive or negative), and quantities (‘low’, ‘medium’, or ‘high’ in Experiment 1 and ‘possible’, ‘likely’, or ‘very likely’ in Experiment 2) presented in a fully within-subjects design. Participants looked longer at verbal than numerical quantifiers, and longer at the contextual information with verbal quantifiers. Quantifier format also affected judgements and decisions: in Experiment 1, participants judged positive labels to be better in the verbal compared to the equivalent numerical condition (and to be worse for negative labels). In Experiment 2, participants decided on rain protection more for a verbal forecast of rain than the equivalent numerical forecast. The results fit the explanation that verbal quantifiers put more focus on the informational context than do numerical quantifiers, rather than prompting more intuitive decisions.
... The main drawback of communicating uncertainty via predefined categories is that the audience, especially non-experts, might not be aware of or even misinterpret the threshold criteria of the categories. Many studies have shown that although individuals have internally consistent interpretation of words for describing probabilities (e.g., likely, probably), these interpretations can vary substantially from one person to another [26,36,116]. Recently, Budescu et al. [25] investigated how the general public interpret the uncertainty information in the climate change report published by the Intergovernmental Panel on Climate Change (IPCC). ...
... Thus, integrating the proposed method with the existing debiasing techniques is necessary to make the results more reliable. Secondly, the 38 ambiguity of linguistic terms may also have impact on the accuracy of the results (Budescu and Wallsten, 1985;Wallsten et al., 1986). In the future, the rough set theory can be integrated to handle the ambiguity in the expert judgments. ...
Failure Mode and Effect Analysis (FMEA) is an efficient method for risk prevention and control, which has been applied to identify the potential risks and improve system reliability in various fields. Nevertheless, few attempts in previous research have been made to examine the impacts of the interactions among the failure modes on risk priority, which will influence the risk priority accuracy. To solve this problem, we propose a novel method to improve the conventional FMEA approaches. The proposed method not only considers the positive and negative influences of failure modes, but also the attenuation effect of such influences in the system. In addition, different with the previous FMEA approaches assuming that all failure modes play equally important roles in risk propagation, the new FMEA method considers the initial strength effect of failure mode on the risk propagation to ensures the accuracy of the risk priority. The final risk priority of a failure mode is computed recursively by the PageRank algorithm, which considers the effect of risk priority of other failure modes associated with this failure mode. Finally, to verify the effectiveness of the proposed method, we apply it to a case study of failure mode effect analysis for smart bracelets.
... Finally, intervals will of course be valuable in any case where the most correct and complete answer to a question is itself an interval, such as interpretation of probability phrases (cf. Budescu & Wallsten, 1985, 1995Harris et al., 2017;Karelitz & Budescu, 2004) or other linguistic terms (Navarro et al., 2016;Wu et al., 2012). Here, they will inherently improve the fidelity of the response. ...
Obtaining quantitative survey responses that are both accurate and informative is crucial to a wide range of fields. Traditional and ubiquitous response formats such as Likert and visual analogue scales require condensation of responses into discrete or point values—but sometimes a range of options may better represent the correct answer. In this paper, we propose an efficient interval-valued response mode, whereby responses are made by marking an ellipse along a continuous scale. We discuss its potential to capture and quantify valuable information that would be lost using conventional approaches, while preserving a high degree of response efficiency. The information captured by the response interval may represent a possible response range—i.e., a conjunctive set, such as the real numbers between 3 and 6. Alternatively, it may reflect uncertainty in respect to a distinct response—i.e., a disjunctive set, such as a confidence interval. We then report a validation study, utilizing our recently introduced open-source software (DECSYS), to explore how interval-valued survey responses reflect experimental manipulations of several factors hypothesised to influence interval width, across multiple contexts. Results consistently indicate that respondents used interval widths effectively, and subjective participant feedback was also positive. We present this as initial empirical evidence for the efficacy and value of interval-valued response capture. Interestingly, our results also provide insight into respondents’ reasoning about the different aforementioned types of intervals—we replicate a tendency towards overconfidence for those representing epistemic uncertainty (i.e., disjunctive sets), but find intervals representing inherent range (i.e., conjunctive sets) to be well-calibrated.
... On the one hand, the perceived meaning of verbal phrases can vary depending on context (Brun & Teigen, 1988;Weber & Hilton, 1990), base rates (Wallsten, Fillenbaum, & Cox, 1986b), kinds of events (Harris & Corner, 2011;Weber & Hilton, 1990), conversational rules and pragmatics (Bonnefon & Villejoubert, 2006;Honda & Yamagishi, 2017;Teigen & Brun, 1999), as well as the employed elicitation method (Hamm, 1991;Wallsten, Budescu, Rapoport, Zwick, & Forsyth, 1986a;Wallsten, Budescu, & Zwick, 1993). Generally, within-subject variation tends to be lower than between-subjects variability, indicating a fairly stable understanding of everyday uncertainty terms at the individual level, although different people may vary in their judgments (Budescu & Wallsten, 1985). On the other hand, between-subjects variation in the interpretation of particular terms notwithstanding, people can also be quite consistent in their quantitative judgments (Simpson, 1963). ...
Dealing with uncertainty and different degrees of frequency and probability is critical in many everyday activities. However, relevant information does not always come in the form of numerical estimates or direct experiences, but is instead obtained through qualitative, rather vague verbal terms (e.g., “the virus often causes coughing” or “the train is likely to be delayed”). Investigating how people interpret and utilize different natural language expressions of frequency and probability is therefore crucial to understand reasoning and behavior in real‐world situations. While there is considerable work exploring how adults understand everyday uncertainty phrases, very little is known about how children interpret them and how their understanding develops with age. We take a developmental and computational perspective to address this issue and examine how 4‐ to 14‐year‐old children and adults interpret different terms. Each participant provided numerical estimates for 14 expressions, comprising both frequency and probability phrases. In total we obtained 2856 quantitative judgments, including 2240 judgments from children. Our findings demonstrate that adult‐like intuitions about the interpretation of everyday uncertainty terms emerge fairly early in development, with the quantitative estimates of children converging to those of adults from around 9 years on. We also demonstrate how the vagueness of verbal terms can be represented through probability distributions, which provides additional leverage for tracking developmental shifts through cognitive modeling techniques. Taken together, our findings provide key insights into the developmental trajectories underlying the understanding of everyday uncertainty terms, and open up novel methodological pathways to formally model the vagueness of probability and frequency phrases, which are abundant in our everyday life and activities.
... The above method has not been considered in the literature, although the significance of verbal probability estimates has extensively been studied in the social sciences (Pepper and Prytulak, 1974;Hammerton, 1976;Budescu and Wallsten, 1985;Wallsten, Filenbaum, and Cox, 1986;Brun and Teigen, 1988;Teigen, 1988;Weber and Hilton, 1990;Hendrickx, 1991;Mullet and Rivet, 1991). In the reported experiments the respondents were usually requested to assign values to probability terms within the context of an uncertain situation or to select the most suitable probability term from a given list in order to characterize the situation. ...
... As such, there is often a large variation in the numeric meaning assigned to vague quantifiers (e.g. Budescu & Wallsten, 1985). The scales also have relative meaning, such as where on the scale a respondent believes they are in comparison to similar others (Schaeffer, 1991). ...
... Several studies have shown a "communication mode preference paradox" in which, on average, senders prefer verbal probabilities but receivers prefer numeric probabilities (Brun & Teigen, 1988;Erev & Cohen, 1990;Wallsten et al., 1993). In spite of senders' preferential tendency, extensive research has shown intrapersonal imprecision and interpersonal inconsistency in how people translate verbal probabilities into numeric equivalents (e.g., Beyth-Marom, 1982;Budescu & Wallsten, 1985;Dhami & Wallsten, 2005;Harris et al., 2013;Lichtenstein & Newman, 1967). ...
Organizations tasked
with communicating expert judgments couched in uncertainty often use
numerically bounded linguistic probability schemes to standardize the meaning
of verbal probabilities. An experiment (N = 1,202) was conducted to ascertain
whether agreement with such a scheme was better when probabilities were
presented verbally, numerically or in a combined “verbal + numeric” format.
Across three agreement measures, the numeric and combined formats outperformed
the verbal format and also yielded better discrimination between low and high
probabilities and were less susceptible to the fifty-fifty blip phenomenon. The
combined format did not confer any advantage over the purely numeric format.
The findings indicate that numerically bounded linguistic probability schemes
are an ineffective means of communicating information about probabilities to
others and they call into question recommendations for use of the combined
format for delivering such schemes.
... Many studies have shown that although individuals have internally consistent interpretation of words for describing probabilities (e.g., likely, probably), these interpretations can vary substantially from one person to another (cf. [24,34,109]). More recently, Budescu et al. [23] investigated how the general public interpret the uncertainty information in the climate change report published by the Intergovernmental Panel on Climate Change (IPCC). ...
Transparency of algorithmic systems entails exposing system properties to various stakeholders for purposes that include understanding, improving, and/or contesting predictions. The machine learning (ML) community has mostly considered explainability as a proxy for transparency. With this work, we seek to encourage researchers to study uncertainty as a form of transparency and practitioners to communicate uncertainty estimates to stakeholders. First, we discuss methods for assessing uncertainty. Then, we describe the utility of uncertainty for mitigating model unfairness, augmenting decision-making, and building trustworthy systems. We also review methods for displaying uncertainty to stakeholders and discuss how to collect information required for incorporating uncertainty into existing ML pipelines. Our contribution is an interdisciplinary review to inform how to measure, communicate, and use uncertainty as a form of transparency.
This study investigates the mechanisms driving the effectiveness of free-form communication in promoting cooperation within a sequential social dilemma game. We hypothesize that the self-constructing nature of free-form communication enhances the sincerity of messages and increases the disutility of dishonoring promises. Our experimental results demonstrate that free-form messages outperform both restricted promises and treatments where subjects select and use previously constructed free-form messages. Interestingly, we find that selected free-form messages and restricted promises achieve similar levels of cooperation. We observe that free-form messages with higher sincerity increase the likelihood of high-price and high-quality choices, thereby promoting cooperation. These messages frequently include promises and honesty, while threats do not promote cooperation. Our findings emphasize the crucial role of the self-constructed nature of free-form messages in promoting cooperation, exceeding the impact of message content compared to restricted communication protocols.
Behavioral Decision Research has provided a deep understanding of how humans form judgments and make choices, since its emergence in the 1950s. Underlying this body of research is the contrast of actual decision making with the paragons of rationality, which provide the idealized model on how humans should decide. Starting with the Ellsberg and Allais paradoxes and followed by Tversky & Kahneman’s seminal research on biases and heuristics in judgments, it has been clear that no real person is fully rational. Exploring deviations from rationality has been a major focus of Behavioral Decision Research. This research has two quite distinct branches: descriptive and prescriptive. Descriptive Behavioral Decision Research examines deviations from rationality and strives to develop theories or models to explain these deviations. Prescriptive Behavioral Decision Research also starts from observed deviations from rationality, but rather than developing theories of models to explain these deviations, it develops and tests tools or analytical methods to correct such deviations. The distinction between the descriptive and prescriptive perspectives in Behavioral Decision Research has often been implicit and blurred in the Decision Sciences literature. In this chapter we aim to address this conceptual lacuna, proposing a taxonomy for these two perspectives in Behavioral Decision Research and describing their main achievements.
In the field of human intelligence, officers use an alphanumeric scale, known as the Admiralty System, to rate the credibility of messages and the reliability of their sources (NATO AJP-2.1, 2016). During this evaluation, they are expected to estimate the credibility and reliability dimensions independently of each other (NATO STANAG, 2003). However, empirical results show that officers perceive these dimensions as strongly correlated (Baker et al., 1968). More precisely, they consider credibility as playing the leading role over reliability, the importance of which is only secondary (Samet, 1975). In this paper, we present a formal evaluative procedure, called L(intel), in line with these findings. We adapt dynamic belief revision to make credibility the main dimension of evaluation and introduce dynamic operators to update credibility ratings with the source's reliability. In addition to being empirically sound, we show that L(intel) provides an effective procedure to classify intelligence messages along the descriptive taxonomy presented in Icard (2023).
English speakers use probabilistic phrases such as likely to communicate information about the probability or likelihood of events. Communication is successful to the extent that the listener grasps what the speaker means to convey and, if communication is successful, individuals can potentially coordinate their actions based on shared knowledge about uncertainty. We first assessed human ability to estimate the probability and the ambiguity (imprecision) of twenty-three probabilistic phrases in a coordination game in two different contexts, investment advice and medical advice. We then had GPT-4 (OpenAI), a Large Language Model, complete the same tasks as the human participants. We found that GPT-4’s estimates of probability both in the Investment and Medical Contexts were as close or closer to that of the human participants as the human participants’ estimates were to one another. However, further analyses of residuals disclosed small but significant differences between human and GPT-4 performance. Human probability estimates were compressed relative to those of GPT-4. Estimates of probability for both the human participants and GPT-4 were little affected by context. We propose that evaluation methods based on coordination games provide a systematic way to assess what GPT-4 and similar programs can and cannot do.
Can the vague meanings of probability terms such as doubtful, probable, or likely be expressed as membership functions over the [0, 1] probability interval? A function for a given term would assign a membership value of zero to probabilities not at all in the vague concept represented by the term, a membership value of one to probabilities definitely in the concept, and intermediate membership values to probabilities represented by the term to some degree. A modified pair-comparison procedure was used in two experiments to empirically establish and assess membership functions for several probability terms. Subjects performed two tasks in both experiments: They judged (a) to what degree one probability rather than another was better described by a given probability term, and (b) to what degree one term rather than another better described a specified probability. Probabilities were displayed as relative areas on spinners. Task a data were analyzed from the perspective of conjoint-measurement theory, and membership function values were obtained for each term according to various scaling models. The conjoint-measurement axioms were well satisfied and goodness-of-fit measures for the scaling procedures were high. Individual differences were large but stable. Furthermore, the derived membership function values satisfactorily predicted the judgments independently obtained in task b. The results support the claim that the scaled values represented the vague meanings of the terms to the individual subjects in the present experimental context. Methodological implications are discussed, as are substantive issues raised by the data regarding the vague meanings of probability terms.
Risk is the situation under which the decision outcomes and their probabilities of occurrences are known to the decision-maker, and uncertainty is the situation under which such information is not available to the decision-maker. Research on decision-making under risk and uncertainty has two broad streams: normative and descriptive. Normative research models how decision should be made under risk and uncertainty, whereas descriptive research studies how decisions under risk and uncertainty are actually made. Descriptive studies have exposed weaknesses of some normative models in describing people’s judgment and decision-making and have compelled the creation of more intricate models that better reflect people’s decision under risk and uncertainty.
Visualizations today are used across a wide range of languages and cultures. Yet the extent to which language impacts how we reason about data and visualizations remains unclear. In this paper, we explore the intersection of visualization and language through a cross-language study on estimative probability tasks with icon-array visualizations. Across Arabic, English, French, German, and Mandarin,
$n=50$
participants per language both chose probability expressions —
e.g. likely, probable
— to describe icon-array visualizations (Vis-to-Expression), and drew icon-array visualizations to match a given expression (Expression-to-Vis). Results suggest that there is no clear one-to-one mapping of probability expressions and associated visual ranges between languages. Several translated expressions fell significantly above or below the range of the corresponding English expressions. Compared to other languages, French and German respondents appear to exhibit high levels of consistency between the visualizations they drew and the words they chose. Participants across languages used similar words when describing scenarios above 80% chance, with more variance in expressions targeting mid-range and lower values. We discuss how these results suggest potential differences in the expressiveness of language as it relates to visualization interpretation and design goals, as well as practical implications for translation efforts and future studies at the intersection of languages, culture, and visualization. Experiment data, source code, and analysis scripts are available at the following repository:
https://osf.io/g5d4r/
.
Organizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst's confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N = 481 and 624, respectively), uncertainty communication preferences of expert (n = 41 intelligence analysts in Experiment 1) and nonexpert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated verbal probability and confidence terms as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that although most nonexperts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative. Experts and nonexperts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately one‐fourth of experts and over one‐half of nonexperts provided incoherent numeric probability translations for the terms likely and unlikely when the elicitation of best estimates and lower and upper bounds were briefly spaced by intervening tasks.
Scientific assessments, such as those by the Intergovernmental Panel on Climate Change (IPCC), inform policymakers and the public about the state of scientific evidence and related uncertainties. We studied how experts from different scientific disciplines who were authors of IPCC reports, interpret the uncertainty language recommended in the Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties. This IPCC guidance note discusses how to use confidence levels to describe the quality of evidence and scientific agreement, as well likelihood terms to describe the probability intervals associated with climate variables. We find that (1) physical science experts were more familiar with the IPCC guidance note than other experts, and they followed it more often; (2) experts’ confidence levels increased more with perceptions of evidence than with agreement; (3) experts’ estimated probability intervals for climate variables were wider when likelihood terms were presented with “medium confidence” rather than with “high confidence” and when seen in context of IPCC sentences rather than out of context, and were only partly in agreement with the IPCC guidance note. Our findings inform recommendations for communications about scientific evidence, assessments, and related uncertainties.
A comprehensive and realistic assessment of impacts of health threats and health security actions is essential for any reasoned health security decision. However, the uncertainty that is present in the context of every health security decision poses a core challenge that must be considered in such assessments. Here we present how we can adequately assess both the impacts of health threats, and of the decision alternatives for improving health security. We suggest a decision-focused modeling of these impacts, which can foster a better understanding of the decision situation and promote a systematic analysis of consequences under uncertainty.
In this chapter we discuss the main challenges and complexities in making health security decisions. This analysis may explain why taking health security decisions is a challenging process and highlight the benefits of using more systematic decision processes and reasoned approaches when making these decisions. We also examine how health security decisions are typically analyzed in practice and discuss the shortcomings intrinsic to the common assessment methods. The complexities involved in making these decisions, combined with these inadequate assessment methods, establish the basis for the need to improve the quality of health security decisions and to apply Health Decision Analysis.
We have previously presented the ways in which a health decision analysis can be conducted to systematically and thoroughly evaluate health security actions and health threats. This chapter focuses on how high-value options can be selected in health security decision processes. We present a discussion concerning the sources of evidence that can be employed to assess the impacts of health threats and the performances of health security actions. We also detail the different types of decision situations and the different ways in which we can analyze these situations in order to improve health security or to manage health threats.
When giving advice, people seek to inform others, but also help them reach a decision. We investigate how the motivation to help affects the confidence people express when advising others. We propose that assuming the role of advisor instigates a desire to help the advisee decide more easily. This desire in turn leads advisors to communicate higher confidence than they actually feel, provided that the environment is sufficiently certain, and thus the risk of misleading the advisee is low. We test our predictions in five studies, using experimental tasks (Studies 1–3), a survey of experienced professionals (Study 4) and an organizational scenario (Study 5). We find that in high-certainty environments, people convey higher confidence when providing advice than private judgments. This effect is driven by the motivation of advisors to facilitate advisees’ decision making: the higher advisors’ desire to help, the more pronounced the effect on their stated confidence.
Historically, best practice recommendations suggested obtaining eyewitnesses’ lineup identification confidence reports in their own words. More recently, best practice recommendations call for the collection of confidence reports using scales containing either words or numbers. Clearly, historical and contemporary recommendations are inconsistent. This article provides a review of the existing relevant scientific literature and presents new data to empirically assess the effect of confidence assessment method on the confidence-accuracy relationship. Although small, the extant literature has consistently failed to show any significant difference in the confidence-accuracy relationship as a function of confidence assessment method: Verbal confidence reports are as diagnostic of identification accuracy as numeric confidence reports. We present data from a basic repeated face recognition paradigm in which participants (n = 634) each attempted 16 separate lineup identifications and were randomly assigned to indicate their confidence using either a numeric scale, a verbal scale, or in their own words. Consistent with our review of the scientific literature, both calibration and confidence-accuracy characteristic analyses demonstrated that (a) confidence is predictive of accuracy, including when witness confidence is obtained via open-ended responding, and (b) the confidence-accuracy relationship is not dependent on the method used to obtain confidence estimates. Thus, results support using any of the recommended methods (i.e., scales containing either words or numbers or a verbatim account in the witness’s own words) for collecting confidence statements. However, consideration of other criteria (e.g., subjectivity associated with the interpretation of verbal confidence reports) may support recent calls for scale-based confidence collection.
Forecasting has always been at the forefront of decision making and planning. The uncertainty that surrounds the future is both exciting and challenging, with individuals and organisations seeking to minimise risks and maximise utilities. The large number of forecasting applications calls for a diverse set of forecasting methods to tackle real-life challenges. This article provides a non-systematic review of the theory and the practice of forecasting. We provide an overview of a wide range of theoretical, state-of-the-art models, methods, principles, and approaches to prepare, produce, organise, and evaluate forecasts. We then demonstrate how such theoretical concepts are applied in a variety of real-life contexts.
We do not claim that this review is an exhaustive list of methods and applications. However, we wish that our encyclopedic presentation will offer a point of reference for the rich work that has been undertaken over the last decades, with some key insights for the future of forecasting theory and practice. Given its encyclopedic nature, the intended mode of reading is non-linear. We offer cross-references to allow the readers to navigate through the various topics. We complement the theoretical concepts and applications covered by large lists of free or open-source software implementations and publicly-available databases.
Introduction
Many health providers and communicators who are concerned that patients will not understand numbers instead use verbal probabilities (e.g., terms such as “rare” or “common”) to convey the gist of a health message.
Objective
To assess patient interpretation of and preferences for verbal probability information in health contexts.
Methods
We conducted a systematic review of literature published through September 2020. Original studies conducted in English with samples representative of lay populations were included if they assessed health-related information and elicited either (a) numerical estimates of verbal probability terms or (b) preferences for verbal vs. quantitative risk information.
Results
We identified 33 original studies that referenced 145 verbal probability terms, 45 of which were included in at least two studies and 19 in three or more. Numerical interpretations of each verbal term were extremely variable. For example, average interpretations of the term “rare” ranged from 7 to 21%, and for “common,” the range was 34 to 71%. In a subset of 9 studies, lay estimates of verbal probability terms were far higher than the standard interpretations established by the European Commission for drug labels. In 10 of 12 samples where preferences were elicited, most participants preferred numerical information, alone or in combination with verbal labels.
Conclusion
Numerical interpretation of verbal probabilities is extremely variable and does not correspond well to the numerical probabilities established by expert panels. Most patients appear to prefer quantitative risk information, alone or in combination with verbal labels. Health professionals should be aware that avoiding numeric information to describe risks may not match patient preferences, and that patients interpret verbal risk terms in a highly variable way.
Tourism is an escape door to a better, nicer, cleaner and most of all peaceful world—the world of dreamed and long-awaited vacation, which rejuvenates with positive emotional experiences for the time our next escape. In the era of global tourism, every whim can be satisfied at a very reasonable price, every destination is relatively affordable, and every tourist is a citizen of the world. Although the tourism development was on cloud nine, there were at least a dozen thunder clouds during the past few decades, such as economic, social and political crises and unrests, pandemics, terrorists’ attacks, natural and man-made disasters, climate change and last but certainly not least the current pandemic caused by the SARS-CoV-2, responsible for the outbreak COVID-19. Could this be the extinction of the global tourism or at least the end of its golden ages? Therefore, the aim of the report is to establish Bulgarian tourists’ attitudes towards travelling before and after the COVID-19 crisis. The elaborations are made by the computer language Python and a Web-based interactive computing environment for creating documents—Jupyter Notebook, in order to answer the above question, analysing a survey implementing statistical models and tools. The research data are primary and were derived of a questionnaire survey of 121 individual respondents. Consequently, the research results outline the extent to which customers incline to travel before, in the post-self-isolation period and after the crisis. The survey can be used as a model for data analysis of tourist’s attitudes towards during crises.
Importance
Despite increased concern about the health consequences of contact sports, little is known about athletes’ understanding of their own risk of sports-related injury.
Objective
To assess whether college football players accurately estimate their risk of concussion and nonconcussion injury and to identify characteristics of athletes who misestimate their injury risk.
Design, Setting, and Participants
In this survey study, questionnaires were given to 296 current college football players on 4 teams from the 3 of the 5 most competitive conferences of the US National Collegiate Athletic Association. Surveys were conducted between February and May 2017. Data were analyzed from June 2017 through July 2020.
Main Outcomes and Measures
Multiple approaches were taken to compare athlete perceptions of their risks of concussion and nonconcussion injury with individual probabilities of these risks, which were modeled using logistic regression.
Results
Of 296 male college-aged athletes from 4 football teams who participated in the survey, 265 (89%) answered all questions relevant for this study. Participating teams were similar to nonparticipating teams across nearly all measured characteristics. One hundred athletes (34%) had sustained 1 or more concussions, and 197 (68% of the 289 who responded to the question) had sustained 1 or more injuries in the previous football season. Logistic regression models of single-season injury and concussion had reasonably good fit (area under the curve, 0.75 and 0.73, respectively). Of the 265 participants for whom all relevant data were available, 111 (42%) underestimated their risk of concussion (χ² = 98.6; P = .003). A similar proportion of athletes (113 [43%]) underestimated their risk of injury, although this was not statistically significant (χ² = 34.0; P = .09). An alternative analytic strategy suggested that 241 athletes (91%) underestimated their risk of injury (Wilcoxon statistic, 7865; P < .001) and 167 (63%) underestimated their risk of concussion (Wilcoxon statistic, 26 768; P < .001).
Conclusions and Relevance
The findings of this survey study suggest that college football players may underestimate their risk of injury and concussion. The implications for informed participation in sport are unclear given that people generally underestimate health risks. It is necessary to consider whether athletes are sufficiently informed and how much risk is acceptable for an athlete to participate in a sport.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This 2004 book views risk analysis as one important basis for informed debate, policy decisions and governance regarding risk issues within societies. Its twelve chapters provide interdisciplinary insights about the fundamental issues in risk analysis for the beginning of a new century. The chapter authors are some of the leading researchers in the broad fields that provide the basis for the risk analysis, including the social, natural, medical, engineering and physical sciences. They address a wide range of issues, including: new perspectives on uncertainty and variability analysis, exposure analysis and the role of precaution, environmental risk and justice, risk valuation and citizen involvement, extreme events, the role of efficiency in risk management, and the assessment and governance of transboundary and global risks. The book will be used as a starting point for discussions at the 2003 First World Congress on Risk, to be held in Brussels.
This chapter examines individual's ability to express numerically what is internally represented. The chapter examines whether they represented in (1) a verbal propositional mode, (2) a numerical propositional mode, or (3) in an analogue mode of automatic frequency monitoring. It seems unlikely that the mathematically appropriate procedures with numerical estimates of uncertainty have become automatized. It is more likely that people handle uncertainty by customary verbal expressions and the implicit and explicit rules of conversation connected with them. The chapter analyzes the meaning of common verbal expressions for uncertain events. These expressions are interpreted as possibility functions and the procedures applicable to them are modeled in the possibility theory. This theory allows for a numerical interpretation by means of determining the elastic constraints on the usage of such expressions. The results gained by interpreting verbal expressions of uncertainty as possibility functions can be compared to the results of the studies, where subjects provide the numerical expressions themselves.
Maintains that Hakel (American Psychologist, 1968, 23, 533-534) finds it "amazing that we can communicate at all," given the variability with which his students assigned percentages to frequency words. However, his questionnaire did not control the context with respect to which the percentages were assigned.
Used magnitude estimation to find the numerical equivalents of 39 expressions of frequency ranging from never to always, and 44 expressions of amount ranging from none to all. Data were collected from 175 male and female night school master's degree students, adult undergraduates, and high school juniors. Results were generalizable across 3 age-occupation levels and unaffected by topic importance. Geometric means and appropriate variance measures are provided for each expression as well as suggestions for anchors for 4- to 9-point scales. The percentage of overlap in judgments for adjacent points on scales are also given. Results are related to earlier work on scaling, and the utility of the present approach is indicated. (16 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A group of 158 students taking general and educational psychology courses rated a collection of 48 words or phrases indicating frequency as a 9-point graphic scale from "None of the time" to "All of the time." Scale values and dispersions for the entire list are presented with a recommendation as to one possible selection of terms for a 9-step continuum. From Psyc Abstracts 36:01:1GH24S. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
100 university students completed a questionnaire by R. H. Simpson (1944) on the meaning of 20 words used to indicate different degrees of "oftenness" of events. Results point out how imprecisely frequency words are used.
A variety of researches are examined from the standpoint of information theory. It is shown that the unaided observer is severely limited in terms of the amount of information he can receive, process, and remember. However, it is shown that by the use of various techniques, e.g., use of several stimulus dimensions, recoding, and various mnemonic devices, this informational bottleneck can be broken. 20 references. (PsycINFO Database Record (c) 2006 APA, all rights reserved).
Worded forecasts, which generally consist of both verbal and numerical expressions, play an important role in the communication of weather information to the general public. However, relatively few studies of the composition and interpretation of such forecasts have been conducted. Moreover, the studies that have been undertaken to date indicate that many expressions currently used in public forecasts are subject to wide ranges of interpretation (and to misinterpretation) and that the ability of individuals to recall the content of worded forecasts is quite limited. This paper focuses on forecast terminology and the understanding of such terminology in the context of short-range public weather forecasts.
The results of previous studies of forecast terminology (and related issues) are summarized with respect to six basic aspects or facets of worded forecasts. These facets include: 1) events (the values of the meteorological variables): 2) terminology (the words used to describe the events); 3) words versus numbers (the use of verbal and/or numerical expressions); 4) uncertainty (the mode of expression of uncertainty); 5) amount of information (the number of items of information); and 6) content and format (the selection of items of information and their placement). In addition, some related topics are treated briefly, including the impact of verification systems, the role of computer-worded forecasts, the implications of new modes of communication, and the use of weather forecasts.
Some conclusions and inferences that can be drawn from this review of previous work are discussed briefly, and a set of recommendations are presented regarding steps that should be taken to raise the level of understanding and enhance the usefulness of worded forecasts. These recommendations are organized under four headings: 1) studies of public understanding, interpretation, and use; 2) management practices; 3) forecaster training and education; and 4) public education.
The personalistic theory of probability prescribes that a person should use personal probability assessments in decision-making and that these assessments should correspond with his judgments. Since the judgments exist solely in the assessor's mind, there is no way to prove whether or not this requirement is satisfied. De Finetti has proposed the development of methods which should oblige the assessor to make his assessments correspond with his judgments. An ideal Assessor is hypothesized and his behavior is investigated under a number of such methods (including those suggested by de Finetti and others). The implications of these methods for the theory of personal probability are discussed. Finally, although the present interest is primarily normative, the practicability of the methods is also discussed.
This paper briefly describes some results of operational and experimental programmes in the United States involving subjective probability forecasts of precipitation occurrence and of maximum and minimum temperatures. These results indicate that weather forecasters can formulate such forecasts in a reliable manner.
In the Bayesian framework, quantified judgments about uncertainty are an indispensable input to methods of statistical inference and decision. Ultimately, all components of the formal mathematical models underlying inferential procedures represent quantified judgments. In this study, the focus is on just one component, the prior distribution, and on some of the problems of assessment that arise when a person tries to express prior distributions in quantitative form. The objective is to point toward assessment procedures that can actually be used.One particular type of statistical problem is considered and several techniques of assessment are presented, together with the necessary instruction so that these techniques can be understood and applied. A questionnaire is developed and used in a study in which people actually assess prior distributions. The results indicate that, by and large, it is feasible to question people about subjective prior probability distributions, although this depends on the assessor and on the assessment technique(s) used. A revised questionnaire, which is aimed at potential users of the assessment procedures and future investigators in the area of probability assessment, is presented.
Judgment of the probable accuracy of intelligence data and products is held as integral to the intelligence rating process. Accurate communication of the probability of uncertainty implicit in such judgment is requisite to the effective production and utilization of intelligence. Twenty-eight subjects, 14 U. S. Army enlisted men and 14 extension college students, numerically encoded on a 0 to 100 scale each of 15 probability phrases in each of three sentence contexts. Subjects were relatively consistent in their own encoding of given phrases, but differed, often radically, from other subjects. Cluster analysis of the numbers assigned by subjects indicated an underlying asymmetric probability scale comprised of a small number of intervals.
188 SS GAVE NUMERICAL PROBABILITY ESTIMATES FOR EACH OF 41 PROBABILITY-RELATED WORDS AND PHRASES. WHILE THE RESPONSES WERE REASONABLY CONSISTENT, ASYMMETRY BETWEEN MIRROR-IMAGE PHRASES, E.G., QUITE LIKELY (.79) AND QUITE UNLIKELY (.11) WERE FOUND. THUS VERBAL LABELS ON NUMERICAL PROBABILITY RESPONSE SCALES MAY NOT BE PRACTICABLE. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The reported experiment took place in a professional forecasting organization accustomed to giving verbal probability assessments (‘likely’, ‘probable’, etc.). It attempts to highlight the communication problems caused by verbal probability expressions and to offer possible solutions that are compatible with the forecasters overall perspective on their jobs
Experts in the organization were first asked to give a numerical translation to 30 different verbal probability expressions most of which were taken from the organization's own published political forecasts. In a second part of the experiment the experts were given 15 paragraphs selected from the organization's political publications each of which contained at least one verbal expression of probability. Subjects were again asked to give a numerical translation to each verbal probability expression
The results indicate that (a) there is a high variability in the interpretation of verbal probability expressions and (b) the variability is even higher in context. Possible reasons for the context effect are discussed and practical implications are suggested.
In order to review the empirical literature on subjective probability encoding from a psychological and psychometric perspective, it is first suggested that the usual encoding techniques can be regarded as instances of the general methods used to scale psychological variables. It is then shown that well-established concepts and theories from measurement and psychometric theory can provide a general framework for evaluating and assessing subjective probability encoding. The actual review of the literature distinguishes between studies conducted with nonexperts and with experts. In the former class, findings related to the reliability, internal consistency, and external validity of the judgments are critically discussed. The latter class reviews work relevant to some of these characteristics separately for several fields of expertise. In die final section of the paper the results from these two classes of studies are summarized and related to a view of vague subjective probabilities. Problems deserving additional attention and research are identified.
Based on the notion that the process of assessing probabilities of technical success for R&D projects is composed of four phases--the perception phase, the evaluation phase, the transformation phase, and the review phase--three types of variables are identified as accounting for individual differences in probability assessments: personal, organizational, and situational variables. The empirical findings about the impact of some selected variables on subjective probabilities are described and discussed for their relevance to the problem at hand. It is found that both an assessor's specific relations towards the project to be evaluated and his organizational rank may be of use in explaining individual differences in R&D probability assessments.
Techniques for partitioning objects into optimally homogeneous groups on the basis of empirical measures of similarity among those objects have received increasing attention in several different fields. This paper develops a useful correspondence between any hierarchical system of such clusters, and a particular type of distance measure. The correspondence gives rise to two methods of clustering that are computationally rapid and invariant under monotonic transformations of the data. In an explicitly defined sense, one method forms clusters that are optimally “connected,” while the other forms clusters that are optimally “compact.”
Modal adjectives (e.g., “possible” and “probable”) have been the subject of much discussion by linguists and logicians. For
ordinary speakers, it was found that an important aspect of the meaning of modal adjectives is that they can all be used to
qualify the truth of a statement: Subjects sorted modal adjectives according to similarity of meaning and then ordered the
same adjectives solely according to their degree of qualification. The sorting data yielded a one-dimensional scaling solution
of low stress that correlated highly with the results from the ordering task. Using the same techniques, negation was found
to translate a modal adjective down the scale of qualification (e.g., “improbable” is more qualifying than “probable”) so
that the order of affirmative adjectives with respect to each other is preserved for the corresponding negated adjectives.
Negation in this domain is more analogous to a subtractive, rather than multiplicative, process. Also, affixal negation, as
in “improbable” was consistently found to be more qualifying than lexical negation, as in “not probable.
As part of a method for assessing health risks associated with primary National Ambient Air Quality Standards. T. B. Feagans and W. F. Biller (Research Triangle Park, North Carolina. EPA Office of Air Quality Planning and Standards, May 1981) developed a technique for encoding experts' subjective probabilities regarding dose--response functions. The encoding technique is based on B. O. Koopman's (Bulletin of the American Mathematical Society, 1940, 46, 763-764; Annals of Mathematics, 1940, 41, 269-292) probability theory, which does not require probabilities to be sharp, but rather allows lower and upper probabilities to be associated with an event. Uncertainty about a dose--response function can be expressed either in terms of the response rate expected at a given concentration or, conversely, in terms of the concentration expected to support a given response rate. Feagans and Biller (1981, cited above) derive the relation between the two conditional probabilities, which is easily extended to upper and lower conditional probabilities. These relations were treated as coherence requirements in an experiment utilizing four ozone and four lead experts as subjects, each providing judgments on two separate occasions. Four subjects strongly satisfied the coherence requirements in both conditions. and three more did no in the second session only. The eighth subject also improved in Session 2. Encoded probabilities were highly correlated between the two sessions, but changed from the first to the second in a manner that improved coherence and reflected greater attention to certain parameters of the dose--response function.
Imprecision in decision analysis is modeled using fuzzy-set theory. Fuzziness on the probabilities and utilities used in a decision analysis implies fuzziness on the outputs; a method is suggested for calculating imprecise, though informative, statements about the attractiveness of the different options in a decision tree, which depends on the imprecision of the inputs. A general discussion of the advantages and disadvantages of the approach is given.
The applicability of fuzzy set theory to decision analysis (DA) is examined. It extends the ideas of an earlier paper "Fuzzy decision analysis," by Watson et al. [33]. Particular emphasis is placed on justifying the use of Zadeh's fuzzy calculus to model impression, and an axiomatic system is suggested towards this end. This is seen as an attempt at extending Savage's axioms of subjective probability to produce "approximate probabilities." It is argued that the method proposed by Watson et al. for comparing decision options is unsatisfactory, and several alternative methods are developed. Some computational anomalies are pointed out which severely limit the potential of this methodology. It is suggested that, for individual decisionmaking, fuzzy decision analysis should be viewed as an automatic sensitivity analysis, but that fuzzy sets may be useful with another interpretation for group decisionmaking. The conclusions are that the methodology has too many limitations to be of use for isolated decisions but that it may have a value for often repeated generic decisions.
A note on monotonic transformations in the context of functional measurement and analysis of variance. Bulletin of the Psycho-nomic Society Zntra-and intersubject consistency in comparing probability phrases Theory of data
- D V Budescu
- T S Wallsten
Budescu, D. V., and Wallsten, T. S. (1979). A note on monotonic transformations in the context of functional measurement and analysis of variance. Bulletin of the Psycho-nomic Society, 14, 307-3 10. Budescu, D. V., & Wallsten, T. S. (1981). Zntra-and intersubject consistency in comparing probability phrases. Research Memo No. 43, Psychometric Laboratory, University of North Carolina, Chapel Hill, NC. Coombs, C. H. (1964). Theory of data. New York: Wiley.
How often is often? American Psychologist
- M Hakel
Hakel, M. (1968). How often is often? American Psychologist, 23, 533-534.
Reliabiality of subjective probability of precipi-tation and temperature
- A H Murphy
- R L Winkler
Murphy, A. H., & Winkler, R. L. (1977). Reliabiality of subjective probability of precipi-tation and temperature. Journal of the Royal Statistical Society, Series C, 26, 41-47.
Often is often Problems in the quantification of frequency expressions New directions for methodology of social and behavioral sciences (9): Problems with language imprecision
- A Parducci
Parducci, A. (1968). Often is often. American Psychologist, 23, 828. Pepper, S. (1981). Problems in the quantification of frequency expressions. In D. Fiske (Ed.), New directions for methodology of social and behavioral sciences (9): Problems with language imprecision (pp. 25-41). San Francisco: Jossey-Bass.
Intra- and intersubject consistency in comparing probability phrases
- Budescu
Problems in the quantification of frequency expressions
- Pepper