No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Between never and always
... Several authors have attempted to determine the meaning of some commonly used qualifiers. One study restricted to pathologists [10] evaluated perceptions of 21 common expressions. Like our study, those expressions closest to Fig. 2 Boxplots of the diagnostic scores that pathologists attach to the six diagnostic phrases. ...
... One possibility is to issue a comment in the report stating in words how certain the pathologist is, e.g., "although this lesion has some characteristics of melanoma, such a possibility is considered unlikely." Another possibility is to provide in the pathology report, in addition to the diagnosis, a number indicating the degree of certainty that the pathologist has regarding that specimen [5,10]. Some pathologists (Dr. ...
When not all the histopathologic and clinical features necessary for a pathology diagnosis are present in a particular specimen, pathologists may use modifying phrases to convey various degrees of certainty, e.g., “consistent with…” and “suggestive of….” However, it is unclear whether pathologists use such phrases consistently or whether treating physicians fully understand their intended meaning. A questionnaire concerning six common modifying phrases (“consistent with, suggestive of, suspicious for, highly consistent with, highly suggestive of, some features of”) was sent to all physicians from a single institution who either issued or routinely received surgical pathology reports. Physicians were asked to rank their understanding of each phrase on a printed scale between 1 (“no evidence of”) and 10 (“diagnostic of”). One hundred sixty physicians (74.3%) responded. Despite wide variation, there was a hierarchy (from more to less diagnostic): highly consistent > highly suspicious > consistent > suspicious > suggestive > some features (p < 1 × 10−7). There were no significant differences between pathologists and treating physicians (p = 0.72) or attendings and residents (p = 0.9). Pathologists and treating physicians share an overall common understanding of their hierarchical relationship, albeit with wide ranges. Based upon our results, we propose to use only three qualifying phrases to convey the degree of certainty for a particular diagnosis: “suggestive of” (> 25 ≤ 50% certainty), “suspicious for” (> 50 ≤ 75%), and “consistent with” (> 75%). The phrase “no evidence of” should probably be used only when there is ≤ 5% confidence in a diagnosis, and conversely, “diagnostic of” should probably be used only when there is ≥ 95% confidence in a diagnosis.
... This is especially the case with expressions in the central range such as possible or likely. Expressions for the extremes of the range, that is: impossible and certain, are much less variably interpreted and could be used [40]. A small number of carefully selected expressions seems best [21] [41], or a table could be presented codifying the numerical meaning of the verbal phrases (compare among others [42] [43]). ...
... This is especially the case with expressions in the central range such as possible or likely. Expressions for the extremes of the range, that is: impossible and certain, are much less variably interpreted and could be used [40]. A small number of carefully selected expressions seems best [21,41], or a table could be presented codifying the numerical meaning of the verbal phrases (compare among others [42,43]). ...
The number of knowledge-based systems that build on Bayesian belief networks is increasing. The construction of such a network however requires a large number of probabilities in numerical form. This is often considered a major obstacle, one of the reasons being that experts are reluctant to provide numerical probabilities. The use of verbal probability expressions as an additional method of eliciting probabilistic information may to some extent remove this obstacle. In this paper, we review studies that address the communication of probabilities in words and/or numbers. We then describe our own experiments concerning the development of a probability scale that contains words as well as numbers. This scale appears to be an aid for researchers and domain experts during the elicitation phase of building a belief network and might help users understand the output of the network.
... Clinicians have difficulty assigning numbers to their probability estimates in a given patient. A large array of numbers can result from each frequency determinant (e.g. the word 'probable' is expressed in probabilities from 0.3 to 0.95 by clinicians) [9,10]. Moreover, the strict application of Bayesian calculation when multiple tests are involved may be misleading, as we often ignore whether tests are interrelated and to which extent. ...
... Why log 10 and not natural logarithms? Even if natural logarithms possibly might come closer to 'real' orders of magnitude for test power, probability estimation intuitively goes by orders of 10, not by 'natural' order (1 in 10, 1 in 100, 1 in 1000). ...
Rationale From general practitioners to academic staff, clinicians continue to have difficulties in applying clinical epidemiology in their everyday work. They do not fully understand the logical rules behind the numbers and they do not recognize these rules in their work. We present a new model where the pre-test and the post-test probabilities are converted to log10 of odds, and the likelihood ratio (LR) to its own log10.
Methods Following Bayes’ theorem, adding the log10LR to the log10 pre-test odds gives the log10 post-test odds, which can easily be represented on a logarithmic scale. In addition, by rounding the log10LR to half the unit, we create classes of discriminative power of tests, close to intuitive estimation. This model generates also a user-friendly diagram, adding considerably to the understanding of Bayes’ theorem. We evaluated the effect of the rounding, the current use of the classical model and the acceptability of the new model.
Results Rounding 10 disease characteristics to half the unit gives an absolute error of less than half a unit. After six explanations of Bayes’ theorem, only 6/16 medical students were capable of answering simple questions about predictive value. When asked about weight of disease characteristics, no one of the 50 clinicians mentioned sensitivity, specificity, predictive value or LR. With the new model, more than 80% of trainees found medical decision making easier to understand and recognized the theory in their practice.
Conclusions We conclude that our model of diagnostic clinical epidemiology offers a logical environment for an easy and rapid assessment of the evolution of disease probability with consecutive tests, providing a scientific format for ‘qualitative’ clinical estimations.
... A succession of studies reported a serious mismatch between words and numbers as alternative descriptions of the same risk (Bryant and Norman, 1980;Kenney, 1981;Toogood, 1980). When some sort of consensus was detected, it was suggested that this could be the basis for codification so that health professionals could use certain words that carried a definitive numerical meaning (Kong et al., 1986). ...
In the closing decades of the 20th century, a method of calculating numerical probabilities based on populations-at-risk emerged in public health/epidemiology and then moved into clinical medicine. This new method had its own autonomous social life as it reorganised the fields of clinical perception and clinical practice. This paper documents that revolution in the epistemological basis of medicine by investigating, through primary sources, when and how the social life of a new method undermined the professional status of medicine and changed the doctor-patient relationship.
... Given the reliance on risk discourse in medical consultations, individuals' understanding of both qualitative and quantitative risk expressions within medical contexts has been the focus of a great deal of research. Many studies have looked at medical practitioners' and lay individuals' understanding of qualitative expressions of health risks (Mapes 1979;Toogood, 1980;Bryant and Norman, 1980;Kenney, 1981;Nakao and Axelrod, 1983;Robertson, 1983;Kong et al, 1986;O'Brien, 1989;Shaw and Dear, 1990). This research reveals considerable inter-and intragroup variation in the understanding of these terms, which has led researchers to conclude that health risks should always be described using quantitative formats (with qualitative qualification) if possible. ...
Using observations, questionnaires, and interviews, this study describes the formats used to present risk information during genetic counseling for breast and ovarian cancer. Counselees' preferences for different types of qualitative and quantitative presentation formats are also discussed. The data indicate that there is considerable variation in the presentation of risk information both within and between consultations. The counselees were positive about the way the counselors had described their risk. Seventy‐three percent of the sample expressed a preference for risk to be described using quantitative formats, and there was little difference in the number who stated a preference for percentages, proportions, or population comparisons. Comparing preferred formats with those used in the consultations indicated that in over 40% of cases, risk information was not presented in the counselees' preferred quantitative format. This descriptive study raises questions about the presentation of risk information which warrant further research.
... For example, the comment may state, ''due to the samples' small size, we cannot confidently rule out carcinoma.'' Any one of a number of factors could contribute to a pathologist being unable to render an unequivocal diagnosis including unusual histomorphology, ambiguous immunohistochemical staining results, lack of clinical information, uncertain criteria for diagnosis in the medical literature, lack of experience, or a desire to avoid encumbering legal liability from a medical error. 1 Although there have been studies in radiology to examine the terminology used to convey diagnostic certainty in radiologist's reports, [2][3][4][5][6][7] to date, few studies have attempted to examine surgical pathologists' objectives and attitudes in terms of ''percent certainty'' implied by specific phrases that serve as modifiers or adjuncts to common line surgical pathology diagnosis. How various diagnostic scenarios might set limitations on a surgical pathologist's ability to have, and to communicate, various levels of certainty via FTCs remains an open question. ...
In order to document perceptions of text comments appearing in surgical pathology reports, questionnaires were distributed to 4 groups of caregivers: university staff pathologists, resident pathologists, faculty clinicians (other than pathologists), and resident clinicians at a teaching hospital. Results of this pilot study showed a wide degree of variability existed within each group of surgical pathology report users, with respect to percent confidence assigned to various phrases, commonly used to express diagnostic uncertainty, appearing often as free-text comments in surgical pathology reports. The unavailability of immunohistochemistry tests, or ambiguous immunohistochemistry test results, was especially problematic. With respect to modes of communication between the surgical pathology laboratory and its service users, clinicians indicated they preferred to use tumor boards/interdisciplinary conferences, face-to-face meetings, and phone calls to clarify their interpretations of a pathologist’s diagnoses, as compared with simply reading free-text comments. On the other hand, surgical pathologists rely heavily on their use of the comment portion of a surgical pathology report to clarify, modify, or expand on the diagnoses they render. The majority of clinicians stated that they “always” read the free-text comment portion of a surgical pathology report, whereas some acknowledged they do not always read it. Pathology residents had significantly less confidence in the ability of a free-text comment on a surgical pathology report to clarify a diagnosis (χ2 = 46.36, P < .0001). Pathology departments should consider standardizing definitions and weighting the words and phrases they use in their free-text comment sections of surgical pathology reports.
... Some early studies, for example, investigated how clinicians interpret the meaning of hedge expressions and how this interpretation might in turn affect intra-clinician communication and distributed medical decision-making. [6][7][8][9][10][11][12][13][14] Recent studies explored the interpretation of probability expressions in clinical documents such as pathology reports [15][16][17] and radiology reports. 18 19 Similar investigations have also been conducted to study how patients and families interpret the meaning of hedge phrases. ...
In this study, we quantified the use of uncertainty expressions, referred to as 'hedge' phrases, among a corpus of 100,000 clinical documents retrieved from our institution's electronic health record system. The frequency of each hedge phrase appearing in the corpus was characterized across document types and clinical departments. We also used a natural language processing tool to identify clinical concepts that were spatially, and potentially semantically, associated with the hedge phrases identified. The objective was to delineate the prevalence of hedge phrase usage in clinical documentation which may have a profound impact on patient care and provider-patient communication, and may become a source of unintended consequences when such documents are made directly accessible to patients via patient portals.
... These authors referred to the former as " theoretical probability " and the latter as " estimated probability. " The ideas and activities included in that unit are clearly discernible in many of the curriculum materials that followed (e.g., Singer, Konold, & Rubin, 1996 ; Bright, Frierson, Tarr, & Thomas, 2003; Lappan, Fey, Fitzgerald, Friel, & Phillips, 1998). At some point, these frequency-based estimates of probability began being referred to as " experimental " or " empirical " probabilities and defined in a way that obscured the fact that observed ratios were estimates of the probabilities, rather than probabilities themselves. ...
A core component of informal statistical inference is the recognition that judgments based on sample data are inherently uncertain. This implies that instruction aimed at developing informal inference needs to foster basic probabilistic reasoning. In this article, we analyze and critique the now-common practice of introducing students to both “theoretical” and “experimental” probability, typically with the hope that students will come to see the latter as converging on the former as the number of observations grows. On the surface of it, this approach would seem to fit well with objectives in teaching informal inference. However, our in-depth analysis of one eighth-grader's reasoning about experimental and theoretical probabilities points to various pitfalls in this approach. We offer tentative recommendations about how some of these issues might be addressed.
... Most of the empirical literature on probability expressions has focused on the translation of verbal expressions to point numerical equivalents. The overwhelming result is great variability in the values assigned to words and large overlap among the ranges assigned to the various expressions (e.g., Beyth-Marom, 1982;Budescu & Wallsten, 1985;Foley, 1959;Hakel, 1968;Johnson, 1973;Kenney, 1981;Lichtenstein & Newman, 1967;Nakao & Axelrod, 1983;Simpson, 1944Simpson, , 1963. Some of these studies (Beyth-Marom, 1982;Budescu & Wallsten, 1985;Johnson, 1973) have also shown that the between-subjects variability in assigning numbers to expressions far exceeds the within-subjects variability, which itself is not minor. ...
A two-stage within subjects design was used to compare decisions based on numerically and verbally expressed probabilities. In Stage 1, subjects determined approximate equivalences between vague probability expressions, numerical probabilities, and graphical displays. Subsequently, in Stage 2 they bid for (Experiment 1) or rated (Experiment 2) gambles based on the previously equated verbal, numerical, and graphical descriptors. In Stage 1, numerical and verbal judgments were reliable, internally consistent, and monotonically related to the displayed probabilities. However, the numerical judgments were significantly superior in all respects because they were much less variable within and between subjects. In Stage 2, response times, bids, and ratings were inconsistent with both of two opposing sets of predictions, one assuming that imprecise gambles will be avoided and the other that verbal probabilities will be preferred. The entire pattern of results is explained by means of a general model of decision making with vague probabilities which assumes that in the present task, when presented with a vague probability word, people focus on an implied probability interval and sample values within it to resolve the vagueness prior to forming a bid or a rating. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... In medicine, verbal expressions denoting frequency and probability are widely used in connection with etiology, diagnosis, prognosis , treatment, and prevention, although there has been considerable discussion about the function, interpretation, and use of such expressions . A number of studies agree that verbal expressions of frequency words in medicine are ambiguous and highly imprecise (Kenney, 1981; Nakao & Axelrod, 1983; Robertson, 1983; Toogood, 1980) and should, wherever possible, be replaced by numbers. Less is known about the ambiguity of uncertainty (probability) words in a medical context (except for a small-scale study by Bryant & Norman, 1980 ). ...
To investigate the communication value of verbal probabilistic phrases, like “possibly,” “probably,” and “perhaps,” three experiments were conducted. Subjects were asked to judge the degree of probability expressed by such phrases in different contexts: in sentences reflecting opinions on current events, in a medical discussion of treatment effectiveness, and in videotaped news reports. Judgments of degree of probability were performed in the first study on a 0–100% probability scale and in the other two on 7-point rating scales. Results indicated that different contexts influence the interpretation of probability terms and in many cases, but not always, lead to higher between-subject variability than when the terms are judged in isolation, presumably because the interpretation of probability terms tends to be correlated with the judges' personal opinions on the topics. Special communication problems arise from the fact that most people are not fully aware of the ambiguity of these phrases and underestimate the variability of such ratings in the general population. Miscommunication between experts and the general public was illustrated by answers to a questionnaire given to general medical practitioners and to parents of small children. The latter preferred numerical probabilities to words, but thinking from an individual-oriented perspective, they often misunderstood the intended statistical meanings.
... It has been suggested that qualitative expressions of probability may be preferred because they seem to be natural and easy to use, while quantitative expressions of probability may be preferred because they are more precise [116]. A number of studies have demonstrated that there is considerable variation in how patients and physicians interpret verbal probability expressions [117][118][119]. Patients and the lay public differ as to their preference for quantitative or qualitative risk communication [120][121][122][123]. ...
Low numeracy is pervasive and constrains informed patient choice, reduces medication compliance, limits access to treatments, impairs risk communication, and affects medical outcomes; therefore, it is incumbent upon providers to minimize its adverse effects.
We provide an overview of research on health numeracy and discuss its implications in clinical contexts.
Low numeracy cannot be reliably inferred on the basis of patients' education, intelligence, or other observable characteristics. Objective and subjective assessments of numeracy are available in short forms and could be used to tailor health communication. Low scorers on these assessments are subject to cognitive biases, irrelevant cues (e.g., mood), and sharper temporal discounting. Because prevention of the leading causes of death (e.g., cancer and cardiovascular disease) depends on taking action now to prevent serious consequences later, those low in numeracy are likely to require more explanation of risk to engage in prevention behaviors. Visual displays can be used to make numerical relations more transparent, and different types of displays have different effects (e.g., greater risk avoidance). Ironically, superior quantitative processing seems to be achieved by focusing on qualitative gist and affective meaning, which has important implications for empowering patients to take advantage of the evidence in evidence-based medicine.
... Objectives: To determine (1) the effect of context, experience, age, gender, race, occupation, and education on patients' numerical interpretation of probability terms and (2) patient preferences for information about side effects (qualitative or numerical). ...
Physicians often use qualitative probability statements to compare treatment options or describe risks of treatment, especially if exact numerical information is not readily available.
To determine (1) the effect of context, experience, age, gender, race, occupation, and education on patients' numerical interpretation of probability terms and (2) patient preferences for information about side effects (qualitative or numerical).
Cross-sectional survey.
A university-based family practice in Ann Arbor, Mich.
Patients 18 years of age and older and parents of patients younger than 18 years of age seen during January and February 1993 for any reason except complete physical examination.
A questionnaire presented scenarios of minor and major complications related to four different medical conditions. Participants were asked to estimate how many people of 100 would have a complication if their physician described the risk for the complication in each scenario as unlikely. Participants were then asked whether they preferred receiving information from their physician about the risk for complications in words or numbers.
Of 345 questionnaires distributed, 307 patients (89%) completed them. The rates assigned to the minor complications were significantly higher than the rates assigned to the major complications (P = .0001). Participants who had experienced the described complication reported significantly higher rates for the minor complications of vaccination and surgery (P = .0001 and P = .0235, respectively). Education had a significant effect only on the rates assigned to vaccination complications (P = .0069). Occupation had a significant effect only on the rates assigned to antibiotic side effects (P = .0090).
When a physician uses qualitative probability statements, he or she must be sensitive to the patient's previous experience with that procedure or medication. Also, if one wants to convey the same potential rate of occurrence for major and minor side effects, then one needs to use different words for each.
Risk communications often contain uncertainty information associated with technologies, behaviors, medical procedures, etc. To be effective, such messages need to incorporate ideas, images, and logic that will promote comprehension by a lay public. This paper reviews research strategies designed to make explicit the implicit and frequently idiosyncratic elements of lay reasoning about uncertainty. General biases and heuristics are outlined as they may influence the communication process. Empirical data on variables of newly emerging interest are presented as an extension of earlier research paradigms, with special focus on preferences among natural language terms, causal logic, and modalities for the expression of uncertainty. It is suggested that risk communication incorporate multiple ways to communicate the same information.
No therapy that is effective is free of adverse effects. The detection of adverse effects of drugs and adverse reactions to drugs and other therapeutic interventions, the scientific basis of which has been delineated over the last 50 years, is more important than ever before, as therapy becomes increasingly complex and is used in increasingly ageing populations. Here I define pharmacovigilance, review its history, and discuss terminology and definitions in pharmacovigilance, medication errors, the pharmacological classification of adverse drug reactions, drug interactions, reporting suspected adverse drug reactions, causality assessment, frequencies of adverse drug reactions, risk perception and adverse drug reactions, class effects of drugs, unlicensed indications, off-label uses, and orphan drugs, preventing adverse drug reactions, and publishing accounts of adverse drug reactions.
The title of this paper invites one to raise an eyebrow. Quality of life is a notion which we want to measure with precision, but seemingly cannot. We want to measure it for a variety of reasons. As individuals we wish not just to live long, but to live well. In seeking health care, for example, we wish not simply to survive, but to ease our pain, to ameliorate our handicaps, to shorten the course of our illnesses, to return as quickly as possible to the pursuit of our life plans and valued activities. As a society we similarly formulate some conception of our collective well-being, which we then pursue through public policies promoting, for example, health, education, defense and national culture. In all these activities, we must rely on judgments about the quality of life. These judgments may be rough, and they may not even be explicit; yet, as individuals or collectively as a society we cannot appraise our current lives or make decisions about our future without at least determining which states of affairs are acceptable and which not, and which alternatives are preferable and worthy of pursuit, which to be avoided. Without such basic judgments of value and priority, we would drift aimlessly, making decisions arbitrarily and ineffectually, wasting effort and resources.
Identifying the adverse effects of drugs, thus transforming adverse events into adverse drug reactions, is a useful and necessary but complicated task. Objective proof of a causal relationship between a drug and a specific event is quite exceptional. In most cases, this relationship remains subjective and is no more than inner conviction.
Several means are at our disposal to achieve causal assessment: spontaneous reporting, clinical trials, cohorts with and without controls, and case-control studies, with each having advantages and limitations. The search for causality in pharmacovigilance is a necessary scientific goal, but a high degree of suspicion may be all that is necessary to withdraw a drug from the market if it is suspected of causing serious adverse effects.
Translations of verbal into numerical probability expressions play a central role in risk management, for instance when assessing risks or when choosing appropriate measures to handle them. Despite the fact that the issue of probability translations has been widely studied in the past, results have remained quite isolated. The present work sets out to provide a meta-analysis of previous work, focussing on the most frequently used expressions. It turns out that the numerical ratings for verbal expressions differ considerably between studies. Subsequent analysis is devoted to distances between the results and to differences in methodology. Although there is significant disagreement on which figure represents a particular verbal probability best, methodology was not found to be a decisive factor. The study clearly shows that there is no consensus about probability translations and that attempts to codify word-number-pairs come too early. For such general purposes, the present study helps to distinguish between similarly versus inconsistently rated expressions instead.
Verbal expressions of probability are used in daily conversations, physician-physician and physician-patient communications, and questionnaire and interview responses. To assess the degree of agreement among English-speaking Australian adults in allocating numerical probabilities to these verbal expressions of probability, 966 interviewees provided estimates for 60 isolated expressions of probability and a sequence of seven items placed in a sequence. Means and median scores appeared to be consistent with common sense and with findings from other countries. Mirror-imaged terms were neither symmetrical nor equidistant, with the means and medians for the positive terms being closer to the mid-points of the scale than the means and medians of the negative terms. Items in a sequence of probability terms showed greater symmetry and less variability than isolated expressions. For most items, there was an unacceptably high level of within-subject and between-subject variability. Although subjects with higher levels of education and/ or mathematics education showed less variability, these factors accounted for very little of the variance. The greater variability in Australian results relative to those reported elsewhere was partly attributed to the use of interviews rather than questionnaires. There were no particular stems that yielded greater consistency. It was concluded that the use of these expressions leads to very imprecise communication.
Physicians often use verbal expressions of probability to characterize their uncertainty about outcomes and the risks or side effects of proposed therapies. However, there is an extensive literature that documents the inherent vagueness of such expressions. Because of the potential importance of probability terms to physician-patient communication and decision-making, we asked patients to tell us the odds they thought applicable to the term “rare”, as used by their physician to discuss the likelihood of an adverse outcome from surgery. Patients were randomly assigned to one of three outcome groups: death, severe heart attack, or severe pneumonia. Demographic data were elicited from each subject, as were indicators of present health status, medical history for certain diseases and surgery, and life expectancy. Linear regression and ANOVA analyses of the responses indicate that patient age, education level, perceived health status, and recency of experience with disease and medical care influence patients' numeric interpretations. We discuss the implications of these results.
Many multiple-choice questions (MCQs) used in medical education in the UK contain undefined, imprecise terms. They are particularly common in true/false items and can be found in classroom tests, published examples of MCQs and, more importantly, in high-stakes examinations which determine a candidate's graduation or membership of a professional body. This study investigated imprecise terms used in some MB BS finals examinations and the Part 1 Membership Examination of the Royal College of Physicians. It revealed that imprecise terms occur commonly, yet there is a wide range of opinion among the examiners themselves about their meanings. The numbers and variety of imprecise terms which were found in high-stakes MCQ examinations are described in this paper and details are given concerning the lack of consensus about their meanings as reported by the responsible examiners. A second type of construction error – disproportionately large numbers of `true' branches – was also recorded. Exemplary practices do exist in MCQ quality assurance, but in the UK they are very much the exception rather than the rule. The findings of this investigation strongly indicate a need for change.
Study objectives:To assess the meanings ascribed by patients to qualitative expressions of probability commonly used in medical care and to determine patient preferences for obtaining information when communicating with their physicians about medical risk numerically and/or qualitatively.
Design:Cross-sectional survey of consecutive patients.
Setting:A university-based Department of Veterans Affairs medical center.
Participants:133 patients sequentially seen in a general medicine clinic.
Measurements and results:Subjects were given a randomly ordered list of 12 common terms, each a qualitative expression of probability. They were asked to indicate what they understood to be the numerical meaning of each word. The patients’ probability estimates were found to comprise two groups of five terms each, with high intercorrelations among the probabilities assigned to the terms in each group. Mean probabilities assigned to terms in the first group all were greater than 60%, and mean probabilities assigned to terms in the second group all were below 50%. When asked whether they wanted chance information to be provided in numerical or qualitative terms, 32% reported that they wanted it only numerically; 35.3% wanted it only qualitatively; 21.8% wanted the information either way; and 8.3% wanted the information both ways.
Conclusions:The numerical meanings that patients ascribe to probability terms fall into identifiable patterns. While patients vary in the actual values they assign to terms, the relative meanings of terms show consistent trends.
Using observations, questionnaires, and interviews, this study describes the formats used to present risk information during genetic counseling for breast and ovarian cancer. Counselees' preferences for different types of qualitative and quantitative presentation formats are also discussed. The data indicate that there is considerable variation in the presentation of risk information both within and between consultations. The counselees were positive about the way the counselors had described their risk. Seventy-three percent of the sample expressed a preference for risk to be described using quantitative formats, and there was little difference in the number who stated a preference for percentages, proportions, or population comparisons. Comparing preferred formats with those used in the consultations indicated that in over 40% of cases, risk information was not presented in the counselees' preferred quantitative format. This descriptive study raises questions about the presentation of risk information which warrant further research.
The frequency with which verbal uncertainty expressions are employed suggests that they play an important role in the communication
of states of uncertainty and may have an important role in emerging technologies such as Expert Systems. This article critically
reviews empirical studies of verbal uncertainty expressions spanning two decades of research between 1967 and 1987 with the
principal conclusions that: (1) People are highly internally consistent in their use of verbal uncertainty expressions; (2)
No conclusions about between-subject variability are justified principally because (a) there is currently no consensus as
to what is to count as consistent or inconsistent use and (b) there are several factors that confound purported analyses of
between-subject consistency such as the composition of the stimulus set and the scaling tasks themselves; (3) One study suggests
that assessments of the meaning of verbal uncertainty expressions may be conditioned by the prior perceived probabilities
of the events they describe. However, other interpretations of this study are open. The review also discusses the more general
epistemological question of whether the concept of uncertainty as manifest by verbal uncertainty expressions is really amenable
to the unidimensional framework within which empirical studies have been conceived.
A probabilistic network consists of a graphical representation (a directed graph) of the important variables in a
domain of application, and the relationships between them, together with a joint probability distribution over the
variables. A probabilistic network allows for computing any probability of interest. The joint probability
distribution factorises into conditional probability distributions such that for each variable represented in the graph
a distribution is specified conditional on all possible combinations of the variable's parents in the graph. Even for
a moderate sized probabilistic network, thousands of probabilities need to be specified. Often the only source of
probabilistic information is the knowledge and experience of experts. People, even experts, are known not be
very good at assessing probabilities, and often dislike expressing their estimates as numbers. To overcome this
problem, we propose two qualitative approaches to quantifying probabilistic networks. The first approach is
abstracting away from probabilities by using qualitative probabilistic networks. The second approach is to allow
the use of verbal expressions of probability during elicitation. In qualitative probabilistic networks, the arcs of the
directed graph are augmented with signs: `+',`-', `0', and `?', indicating the direction of shift in probability for the
variable at one end of the arc, given a shift in values of the variable at the other end of the arc. For example, a
positive influence of variable A on variable B indicates that higher values for B become more likely given higher
values for A. Qualitative probabilistic networks allow for reasoning with probabilistic networks in a qualitative
way, thereby enabling us to check the robustness of the network's structure before probabilities are assessed. In
addition, the qualitative signs provide constraints on the probabilities to be elicited. Qualitative networks are,
however, not very expressive and therefore easily result in uninformative answers (`?'s) during reasoning. We will
suggest several refinements of the formalism of qualitative probabilistic networks that enhance their
expressiveness and applicability. To make probability elicitation easier on experts, we allow them to state verbal
probability expressions, such as "probable" and "impossible", as well as numbers. To this end, we have
augmented a vertical probability elicitation scale with verbal expressions. These expressions, and their position
on the scale, are the result of several studies we conducted. The scale, together with other ingredients such as
text-fragments describing the probability to be assessed and grouping of the probabilities that should sum to 1, is
used in a newly designed probability elicitation method. The method provides for the elicitation of initial rough
assessments. Assessments for which the outcome of the network is very sensitive can be refined using additional
experts and/or the more conventional elicitation methods. Our method has been used with two experts in
oncology in the construction of a probabilistic network for oesophageal carcinoma and allows us to elicit a large
number of probabilities in little time. The experts felt comfortable with the method and evaluations of the resulting
network have shown that it performs quite well with the rough assessments.
This study considers the way in which medical writers talk about things which they deem to be true, possible, and untrue The
study considers research papers drawn from three leading medical journals, published since 1991 Three main types of truth
are identified contextualized truth, evidential truth, and interpreted truth These deal, respectively, with truth as the research
tradition states it to be, truth as the statistical evidence states it to be, and truth as a matter of deriving possible non-statistical
meaning from findings Writers also make frequent explicit reference to the extent to which they are committed to the propositions
expressed in statements about truth the manner in which they do so is discussed, with a distinction being drawn between propositions
and comments
The objective was to examine the way that cancer patients translate verbal descriptors of probability into numerical estimates. A list of words commonly used on consent forms to describe the likelihood for benefits or risks of therapies was provided to 100 cancer patients. Two formats, paper/pencil or computer, were used to provide the list of words. Two methods, magnitude estimation and linear analogue scaling, were used to obtain probability estimates for each word. In addition, two scenarios were developed to study 'context effects' on numerical interpretations of verbal descriptions of probability. All patients provided numerical values for the words on two occasions, separated by one week, and two interviewers collected the data. Regardless of method or format, each word elicited widely variable numerical interpretations. An ANOVA model, including patient, word, interviewer, time, method and format, indicated that patient and interviewer produced major effects on probability estimates. Agreement between methods and across time was good. Paper/pencil and computer formats yielded similar results. Context effects did not appear to influence the numerical probabilities elicited by the 2 scenarios. It was concluded that, within this group of patients, there was no consensus about numerical meaning of a given word, and that interviewers can systematically influence numerical interpretations. There appears to be a great deal of 'noise' in this particular line of communication between patients and health professionals.
One hundred mothers and 50 medical students and doctors were interviewed to examine their understanding of eight common probability expressions as they are used in statements of the kind that doctors regularly make to mothers of babies. Respondents were asked to translate each term into a number on a scale from 0 to 10. There was a wide range of interpretation of the expressions, and significant differences were found between the mothers and medical group for seven of the expressions. Most mothers expressed a preference for receiving information in numerical terms. We advise that verbal probability expressions should be restricted, and more information presented in numerical terms.
A sample of 56 general practitioners were asked to rate, on a percentage scale, 23 words or phrases which denote frequency or likelihood. The hypothetical context of the exercise was that of communicating to patients the probability of a side-effect (headache) arising from an unspecified prescription medicine. Median phrase ratings ranged from 'never' at 0% to 'certain' at 95% with a 50% rating given to the phrase 'reasonable chance'. Despite relatively large variance in ratings between respondents, the median ratings of a number of phrases were similar, and some identical, to other studies from different medical professionals. Although the clinical context in which a given expression of probability is used may affect its meaning, the results are encouraging and suggest that phrases denoting likelihood might be systematically codified to enhance communication between doctor and patient. To move towards this objective more research is needed to evaluate how patients interpret expressions of probability, and the relative effectiveness of different modes of communicating likelihood.
In his masterful sociologic dissection of the medical profession, Eliot Freidson [1] identifies five attributes that characterize the behavior of the typical practicing clinician:
He believes in what he is doing. He believes that he is doing good rather than harm. He has faith in the efficacy of his treatments, and believes that what he does makes the difference between success and failure. When things go right, he takes the credit.
He prefers action to inaction. Successful action is preferred to unsuccessful action, but action with little chance of success is preferred over no action at all.
He is pragmatic. He is prone to seeing apparent cause/effect relationships (even in the absence of any theoretical foundation), and is inclined to “tinker” with conventional methods—to bend the rules—if he isn’t getting the results that he expects. He is highly subjective. He depends more on his own first-hand experience and “gut feelings” than on abstract principles or “book knowledge.” He emphasizes uncertainty in his defense. He is prone to justify this pragmatic reliance on personal experience by citing the Lawlessness of Chance instead of the Laws of Science. When things go wrong, it’s not his fault.
The mental status examination (MSE) is an integral and universal tool of medicine. We studied the form and content of the MSE performed by emergency physicians. An 11-item questionnaire was developed to determine the indications, amount of time necessary to evaluate mental status, the content of MSEs utilized, and the ideal characteristics of a short, standardized MSE. A study group of a random sample of 120 of 1,174 American Board of Emergency Medicine-certified emergency physicians and a validation group were surveyed by questionnaire, which was returned by 80% and 62%, respectively. Emergency physicians agreed that there are numerous specific indications to evaluate mental status including head injury (99%), behavioral abnormality (98%), drug ingestion (96%), and psychiatric complaint (95%); however, few would test it in abdominal pain (21%) or extremity injury (17%). Seventy-two percent take less than five minutes to evaluate mental status in the emergency department. Most emergency physicians uniformly test the same set of items selected from the formal MSE, including level of consciousness (95%); orientation to time (87%), place (86%), and person (84%); spontaneous speech (80%); and behavioral observation (72%) to evaluate mental status. Almost all of the physicians (82%) perceived a need for, and would use, a short standardized MSE that would take less than five minutes to perform. Results from the validation group confirmed the accuracy of the survey technique used. Our study demonstrated a perceived need for a short, standardized MSE in emergency medicine.
Qualitative expressions of probability, such as "likely," have different numerical meanings to different people, which can lead to misunderstanding among physicians and between physicians and patients. In a study conducted through a nationwide interactive computer network based at Massachusetts General Hospital, we gathered information on the meaning of common expressions of probability. Three groups of medical professionals assigned percentage values to 12 expressions of the probability that a given symptom would appear in a patient with an unspecified disease. The median values assigned to these expressions by physicians, medical students, and other professionals were almost the same. Comparisons of the means for 7 of these 12 expressions with those found in an earlier study by other investigators showed that they were quantified in the same order, although they had not been assigned the same numerical values. This degree of agreement among professionals and between studies is encouraging for the future prospects of codifying the meaning of such expressions. The variation among five studies in the mean values assigned to 37 expressions in the medical literature and the variation among individual opinions show that such codification is necessary. In the meantime, the average numerical values presented here for various qualitative expressions of probability could well be used to enhance communication among medical professionals.
Verbal estimates of probability (such as likely or certain) commonly used in laboratory, radiological, and clinical reports may be a barrier to effective communication between health care professionals. In this study the investigators sought to determine whether a consensus of numerical meaning existed in a sample of nurses for each of 30 widely used verbal expressions of probability. Seventy female nurses enrolled in a graduate course were surveyed by means of a test-retest procedure using a questionnaire designed to elicit assignments of numerical probability (0% to 100%). The results showed wide inconsistency both between subjects and within subjects.
We were concerned about the precision (i.e., consensus as to meaning) of adjectives and adverbs used to express frequency in the medical literature. We asked 103 physicians and 106 nonphysicians to assign to each of 22 such modifiers a percentage representing their understanding of the term. As indexed by the standard deviations, the degree of imprecision for 17 terms was so great for both physicians and for laymen as to make their use unacceptable. Consensus was significantly less among laymen than among physicians for 10 of the terms. Greater consensus was shown by native English-speaking physicians than by those with other native languages. Our data suggest further that American graduates of American medical schools show more consensus than American graduates of foreign medical schools. Board-certified physicians did not show greater consensus than physicians who were not board-certified. Verbal expressions of frequency should be eliminated from medical communications; failing that, the author should specify numerically the frequency he intends when he uses any such expression.
Questions about how to improve human judgment and reasoning are of theoretical and practical interest, notwithstanding the continuing controversy over whether people are "rational". Improving judgment may involve modifying people's processes to fit their environments better, or vice versa. We illustrate the latter approach in a study of diagnostic reasoning in which subjects learned to distinguish two fictitious diseases. Prior findings suggest that people may judge the likelihood of a diagnostic category on the presence or absence of features that are typical of, rather than diagnostic of, the category. We varied the structure of the information provided to subjects without attempting to modify their judgmental processes. In an "independent" format, subjects learned about each disease separately; in a "contrastive" format, information about the two diseases was juxtaposed to highlight distinctive features. Subjects in the two conditions formed different disease concepts. Diagnoses following contrastive training were much closer to the statistically prescribed judgments based on likelihood ratios. Interventions that modify the environment may provide an alternative approach where it is difficult to modify people's processes. Effective design of such interventions is one motivation for directing research toward understanding how task characteristics affect the use of and the outcomes of judgment and reasoning processes.
The necessity for numerical probabilities in oral radiographic diagnosis is increasing, due to recent developments in computer-aided diagnosis, decision analysis, informed consent and medical litigation. These numerical probabilities are only partly available from current texts on oral radiology, where they are often expressed by ill-defined, semiquantitative phrases. Therefore, in this study 30 phrases expressing the probability of a relationship between a diagnosis and its symptoms were taken from a selected textbook on oral radiology. Seven oral radiologists from the USA and the Netherlands scored each of these probabilistic phrases on a 20-cm visual analogue scale. Low intraradiologist and high interradiologist variation was found. Because the high variation among authors of texts on oral radiology in interpreting probability information could have a negative influence on their ability to transfer unambiguous information to their readers, it is recommended that the use of semiquantitative phrases in oral radiology is restricted to five probability groups.
To assess the level of involvement patients want in decision making related to the acceptance or rejection of an invasive medical intervention and whether their preference for decision making is related to their preference for qualitative (verbal) or quantitative (numeric) information about the risks of the procedure.
A university-based Department of Veterans Affairs Medical Center.
Cross-sectional study using structured interviews of consecutive patients seen for continuity care visits in a general medicine clinic.
Four hundred and sixty-seven consecutive patients with a mean age of 65.2 years (SD 10.70 years, range 31-88 years) and with a mean of 12.6 years (SD 2.96 years, range 0-24 years) of formal education.
In the context of an invasive diagnostic or therapeutic intervention, patients were asked whether they preferred patient-based, physician-based, or shared patient-physician decision making. Patients were asked to give the ratio of patient-to-physician decision making they preferred, and whether they preferred discussions using words, numbers, or both. Of 467 subjects, 318 (68%) preferred shared decision making; 100 (21.4%) preferred physician-based decision making; and 49 (10.5%) preferred patient-based decision making. In terms of risk disclosure, 436 (93.4%) preferred that their physician disclose risk information to them. Of these 436 patients, 42.7% preferred disclosure of information about the probability of adverse outcomes using qualitative (verbal) expressions of probability; 35.7% preferred disclosure in terms of quantitative (numeric) expressions of probability; and 9.8% preferred disclosure in both qualitative and quantitative terms. Younger patients (odds ratio [OR] 0.96; confidence interval [CI] 0.93, 0.99), patients who had at least one stroke (OR 3.03; CI 1.03, 8.90), and patients who preferred to discuss risk information with their physicians in terms of numbers (OR 2.39; CI 1.40, 4.06) tended to prefer patient-based or shared decision making.
Male veterans consistently preferred shared patient-physician decision making in the context of invasive medical interventions.
Many multiple-choice questions (MCQs) used in medical education in the UK contain undefined, imprecise terms. They are particularly common in true/false items and can be found in classroom tests, published examples of MCQs and, more importantly, in high-stakes examinations which determine a candidate's graduation or membership of a professional body. This study investigated imprecise terms used in some MB BS final examinations and the Part 1 Membership Examination of the Royal College of Physicians. It revealed that imprecise terms occur commonly, yet there is a wide range of opinion among the examiners themselves about their meanings. The numbers and variety of imprecise terms which were found in high-stakes MCQ examinations are described in this paper and details are given concerning the lack of consensus about their meanings as reported by the responsible examiners. A second type of construction error--disproportionately large numbers of 'true' branches--was also recorded. Exemplary practices do exist in MCQ quality assurance, but in the UK they are very much the exception rather than the rule. The findings of this investigation strongly indicate a need for change.
We define an adverse drug reaction as "an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product, which predicts hazard from future administration and warrants prevention or specific treatment, or alteration of the dosage regimen, or withdrawal of the product." Such reactions are currently reported by use of WHO's Adverse Reaction Terminology, which will eventually become a subset of the International Classification of Diseases. Adverse drug reactions are classified into six types (with mnemonics): dose-related (Augmented), non-dose-related (Bizarre), dose-related and time-related (Chronic), time-related (Delayed), withdrawal (End of use), and failure of therapy (Failure). Timing, the pattern of illness, the results of investigations, and rechallenge can help attribute causality to a suspected adverse drug reaction. Management includes withdrawal of the drug if possible and specific treatment of its effects. Suspected adverse drug reactions should be reported. Surveillance methods can detect reactions and prove associations.
To examine whether there are differences between adolescents and adults in their interpretation of probability terms.
Participants were 20 fifth graders, 54 seventh graders, 45 ninth graders, and 34 young adults (mean adult age = 26.24 years, standard deviation = 2.09) from the San Francisco Bay area. They completed a self-administered survey asking them to assign percentage estimates (0% to 100%) to 30 randomly ordered probability terms.
Significant age differences in the mean percentage estimates for 8 of the 30 terms were shown. Moreover, we found large variation in the interpretation of most probability terms studied, with larger variation among the adolescents than adults. Finally, all age groups had some difficulty correctly differentiating between "possibly" and "probably".
Owing to wide variation in the interpretation of probability terms, both within and across age groups, we suggest health practitioners use percentages rather than probability terms to convey risk to both adolescents and adults.
In 3 studies we recorded gay men's estimates of the likelihood that HIV would be transmitted in various sexual behaviours. In Study 1 (data collected 1993, n=92), the men were found to believe that transmissibility is very much greater than it actually is; that insertive unprotected anal intercourse (UAI) by an HIV-infected partner is made safer by withdrawal before ejaculation, and very much safer by withdrawal before either ejaculation or pre-ejaculation; that UAI is very much safer when an infected partner is receptive rather than insertive; that insertive oral sex by an infected partner is much less risky than even the safest variant of UAI; that HIV is less transmissible very early after infection than later on; and that risk accumulates over repeated acts of UAI less than it actually does. In Study 2 (data collected 1997/8, n=200), it was found that younger and older uninfected men generally gave similar estimates of transmissibility, but that infected men gave somewhat lower estimates than uninfected men; and that estimates were unaffected by asking the men to imagine that they themselves, rather than a hypothetical other gay man, were engaging in the behaviours. Comparison of the 1993 and 1997/8 results suggested that there had been some effect of an educational campaign warning of the dangers of withdrawal; however, there had been no effect either of a campaign warning of the dangers of receptive UAI by an infected partner, or of publicity given to the greater transmissibility of HIV shortly after infection. In Study 3 (data collected 1999, n=59), men induced into a positive mood were found to give lower estimates of transmissibility than either men induced into a neutral mood or men induced into a negative mood. It is argued that the results reveal the important contribution made to gay men's transmissibility estimates by cognitive strategies (such as the 'availability heuristic' and 'anchoring and adjustment') known to be general characteristics of human information-processing. Implications of the findings for AIDS education are discussed.
This study was performed to assess the extent of agreement among radiologists and nonradiologists in perception of diagnostic certainty conveyed by words and phrases commonly used in radiology reports.
The study was performed in a large academic radiology department. To determine the commonly used terminology for conveying diagnostic certainty in radiology reports, 12 randomly selected radiologists from six different subspecialties were interviewed. The authors identified the 15 most commonly used words and phrases and included these in random order in a questionnaire sent to all staff radiologists (n = 45) and to 158 referring physicians. Physicians were asked to rank the 15 phrases in order of the diagnostic certainty conveyed by each, from 1 (most certain) to 15 (least certain), using each number only once. The kappa statistic was used to assess agreement in rank order among physicians.
The questionnaire response rate was 76% (n = 34) for radiologists and 49% (n = 78) for nonradiologists. There was excellent agreement among radiologists (kappa = 0.95) and nonradiologists (kappa = 0.93) in the rank order for the phrase diagnostic of. Although there was good agreement (kappa = 0.45) among radiologists for the word unlikely, agreement among nonradiologists was poor (kappa = 0.27). There was very poor agreement among all physicians for the rank order of the other 13 phrases.
Among radiologists and nonradiologists, concordance was poor regarding the diagnostic certainty associated with phrases commonly used in radiology reports. Because poor agreement could lead to suboptimal quality of care, the standardization of terminology would benefit all parties.
Clinical pathologists use descriptive terms or modifiers to express the probability or likelihood of a cytologic diagnosis. Words are imprecise in meaning, however, and may be used and interpreted differently by pathologists and clinicians.
The goals of this study were to 1) assess the frequency of use of 18 modifiers, 2) determine the probability of a positive diagnosis implied by the modifiers, 3) identify preferred modifiers for different levels of probability, 4) ascertain the importance of factors that affect expression of diagnostic certainty, and 5) evaluate differences based on gender, employment, and experience.
We surveyed 202 clinical pathologists who were board-certified by the American College of Veterinary Pathologists (Clinical Pathology). Surveys were distributed in October 2001 and returned by e-mail, fax, or surface mail over a 2-month period. Results were analyzed by parametric and nonparametric tests.
Survey response rate was 47.5% (n = 96) and primarily included clinical pathologists at veterinary schools (n = 58) and diagnostic laboratories (n = 31). Eleven of 18 terms were used "often" or "sometimes" by >/= 50% of respondents. Broad variability was found in the probability assigned to each term, especially those with median values of 75 to 90%. Preferred modifiers for 7 numerical probabilities ranging from 0 to 100% included 68 unique terms; however, a set of 10 terms was used by >/= 50% of respondents. Cellularity and quality of the sample, experience of the pathologist, and implications of the diagnosis were the most important factors affecting the expression of probability.
Because of wide discrepancy in the implied likelihood of a diagnosis using words, defined terminology and controlled vocabulary may be useful in improving communication and the quality of data in cytology reporting.
Nomenclature surrounding drug safety needs to be clear and unambiguous, so that patients, prescribers, manufacturers, and regulators can all understand each other. In particular, it needs to make it clear how adverse events and drug therapy are related to one another, how they are best classified, and their frequency, intensity and seriousness. In this article, we therefore discuss and define terms used in the field of drug safety, particularly terms that are sometimes misunderstood or misused, including medicinal product, pharmaceutical formulation, excipient, adverse event (or experience) and adverse drug reaction (or effect). We also discuss terms used to define the seriousness, intensity, and risk of adverse reactions, and their classification. Instead of creating definitions from scratch, as is commonly done, we have taken the novel approach of critically examining definitions that have been proposed or widely used and have formulated new or modified definitions based on a logical appraisal of their merits and demerits. We hope that these definitions will lead to discussion that will allow a corpus of satisfactory definitions to be widely agreed.
We completed a systematic review of information reported as included in decision aids (DAs) for adult patients, to determine if it is complete, balanced and accurate.
DAs were identified using the Cochrane Database of DAs and searches of four electronic databases using the terms: 'decision aid'; shared decision making' and 'patients'; 'multimedia or leaflets or pamphlets or videos and patients and decision making'. Additionally, publications reporting DA development and actual DAs that were reported as publicly available on the Internet were consulted. Publications were included up to May 2006.
Data were extracted on the following variables: external groups consulted in development of the DA, type of study used, categories of information, inclusion of probabilities, use of citation lists and inclusion of patient experiences.
68 treatment DAs and 30 screening DAs were identified. 17% of treatment DAs and 47% of screening DAs did not report any external consultation and, of those that did, DA producers tended to rely more heavily on medical experts than on patients' guidance. Content evaluations showed that (i) treatment DAs frequently omit describing the procedure(s) involved in treatment options and (ii) screening DAs frequently focus on false positives but not false negatives. About 1/2 treatment DAs reported probabilities with a greater emphasis on potential benefits than harms. Similarly, screening DAs were more likely to provide false-positive than false-negative rates.
The review led us to be concerned about completeness, balance and accuracy of information included in DAs.