Testing Strategies to Increase Source Credibility through Strategic Message Design in the Context of Vaccination and Vaccine Hesitancy

Abstract

Health communicators in the United States face substantial challenges in their efforts to increase parent uptake of the measles, mumps, and rubella (MMR) vaccine for their children. One major set of challenges involves low levels of trust in medical science behind vaccination safety and effectiveness, pharmaceutical companies who produce these vaccines, and government health agencies who promote vaccination. We conducted a two-wave randomized experiment (N = 1,000 at time 1, t1, N = 803 at time 2, t2) to test whether messages designed to convey the expertise, trustworthiness, or caring/goodwill of a governmental source of information (the Centers for Disease Control and Prevention) increased perceived source credibility, increased parent intentions to vaccinate their children, and/or reduced vaccine hesitancy. We found no support for any of the study’s original, pre-registered hypotheses. However, post-hoc analyses reveal a variety of promising directions for future work on strategic messaging to increase source credibility in the context of vaccine hesitancy. A message designed to convey source expertise produced greater perceived caring/goodwill among parents overall. Furthermore, among parents who were vaccine hesitant at baseline, a message originally designed to convey source expertise produced greater perceived trustworthiness and reduced vaccine hesitancy among this group.

Full text

Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=hhth20
Health Communication
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/hhth20
Testing Strategies to Increase Source Credibility
through Strategic Message Design in the Context
of Vaccination and Vaccine Hesitancy
Yiwei Xu, Drew Margolin & Jeff Niederdeppe
To cite this article: Yiwei Xu, Drew Margolin & Jeff Niederdeppe (2021) Testing Strategies
to Increase Source Credibility through Strategic Message Design in the Context of
Vaccination and Vaccine Hesitancy, Health Communication, 36:11, 1354-1367, DOI:
10.1080/10410236.2020.1751400
To link to this article: https://doi.org/10.1080/10410236.2020.1751400
Published online: 19 Apr 2020.
Submit your article to this journal
Article views: 1345
View related articles
View Crossmark data
Citing articles: 4 View citing articles

Testing Strategies to Increase Source Credibility through Strategic Message Design
in the Context of Vaccination and Vaccine Hesitancy
Yiwei Xu , Drew Margolin, and Jeff Niederdeppe
Department of Communication, Cornell University
ABSTRACT
Health communicators in the United States face substantial challenges in their efforts to increase parent
uptake of the measles, mumps, and rubella (MMR) vaccine for their children. One major set of challenges
involves low levels of trust in medical science behind vaccination safety and effectiveness, pharmaceutical
companies who produce these vaccines, and government health agencies who promote vaccination. We
conducted a two-wave randomized experiment (N= 1,000 at time 1, t1, N= 803 at time 2, t2) to test whether
messages designed to convey the expertise, trustworthiness, or caring/goodwill of a governmental source of
information (the Centers for Disease Control and Prevention) increased perceived source credibility,
increased parent intentions to vaccinate their children, and/or reduced vaccine hesitancy. We found no
support for any of the study’s original, pre-registered hypotheses. However, post-hoc analyses reveal
a variety of promising directions for future work on strategic messaging to increase source credibility in
the context of vaccine hesitancy. A message designed to convey source expertise produced greater
perceived caring/goodwill among parents overall. Furthermore, among parents who were vaccine hesitant
at baseline, a message originally designed to convey source expertise produced greater perceived trust-
worthiness and reduced vaccine hesitancy among this group.
Mass vaccination is one of the most successful health inter-
ventions in medical history. The measles, mumps and rubella
(MMR) vaccine, for example, are highly effective in protecting
children from getting these extremely contagious diseases and
is responsible for dramatic reductions in rates of morbidity
and mortality from these diseases across the globe (CDC,
2019a). Nevertheless, health communicators in the United
States (US) face substantial challenges in their efforts to
increase parent uptake of MMR vaccine for their children.
There have been several measles outbreaks in the US in 2019
alone, and large, well-organized anti-vaccine movements have
emerged both online and offline. Some parents are hesitant to
vaccinate their children for reasons that include low confi-
dence and trust in the efficacy and safety of vaccines them-
selves, medical science, vaccine researchers, healthcare
providers, pharmaceutical companies, and government agen-
cies who promote vaccination (Evans et al., 2001; Hilton et al.,
2007). Governmental sources of information, such as the US
Centers for Disease Control and Prevention (CDC), play an
important role in communicating about public health issues
like vaccination. Previous work has found that trust in the
CDC is associated with less negative attitudes toward vaccina-
tion (Krishna, 2018), and others have argued that a positive
relationship between the CDC and the general public is essen-
tial for people to follow health advice from the CDC (Chon &
Park, 2019).
The current study tested whether messages designed to
convey information we predicted to be relevant to specific
sub-components of source credibility of CDC can increase
perceived source credibility and reduce vaccine hesitancy
among parents living in the US. Specifically, we tested the
impact of messages designed to enhance (a) expertise, (b)
trustworthiness, and (c) caring/goodwill on perceived source
credibility and vaccine hesitancy. We did so by (1) describing
the science of how vaccines work to protect children (attempt-
ing to convey expertise by depicting the mechanics of vacci-
nation), (2) acknowledging side effects and describing
mechanisms for citizen reporting of adverse events (attempt-
ing to convey trustworthiness by legitimizing concerns and
depicting a two-way relationship), and (3) acknowledging
concerns about medical science/pharmaceutical companies/
the CDC while conveying the success of previous public/
private partnerships for vaccine safety and the public safety
mission of the CDC (attempting to convey care/goodwill by
showing that the CDC and pharmaceutical industry have
successfully worked together for public goodwill). We further
tested whether message effects differed based on prior atti-
tudes –whether or not respondents were vaccine hesitant at
baseline.
This study focused on parents with children because they
are a key audience for child vaccination and are responsible
for decisions about their child’s MMR vaccination. We
focused on credibility because perceptions of source credibil-
ity are an important factor in shaping whether or not people
adhere to health promotion recommendations (Keller &
Lehmann, 2008; Pornpitakpan, 2004). Source credibility has
CONTACT Yiwei Xu yx497@cornell.edu Department of Communication, Cornell University, 495A Mann Library Building, Ithaca, NY 14853, USA
HEALTH COMMUNICATION
2021, VOL. 36, NO. 11, 1354–1367
https://doi.org/10.1080/10410236.2020.1751400
© 2020 Taylor & Francis Group, LLC

been frequently studied in health communication and persua-
sion research, but these studies typically manipulate the
source’s identity (e.g., contrasting an expert with
a layperson) rather than attempting to increase the credibility
of a specific source through strategic messaging. Little is
known about how strategic message design can influence
perceived source credibility and subsequent effects on mes-
sage-targeted beliefs, attitudes, and intentions to perform
a targeted behavior like vaccination. Furthermore, two sub-
components of source credibility, source expertise, and trust-
worthiness, have been the focus of most source credibility
research. Yet McCroskey and Teven (1999) have argued for
greater attention to another sub-component of source cred-
ibility, caring/goodwill, which in turn shapes the likelihood of
message effects, so we included an explicit focus on this
dimension. Findings from this work are relevant for health
communication research and public health practice in that the
study identifies strategies to enhance source credibility toward
the larger goal of increasing the effectiveness of strategic
health messaging. Results also seek to enhance our theoretical
understanding of the role(s) of source credibility in effective
health communication.
The context: MMR vaccination and vaccine hesitancy
The MMR vaccine is highly effective in preventing three
infectious diseases: mumps, measles, and rubella. The CDC
(2019b) recommends parents to get their children two doses
of the MMR vaccine, which are about 97% effective at
preventing measles (CDC, 2019b). However, efforts to suc-
cessfully communicate the need for parents to get their
children the MMR vaccine faces many challenges. The US
has experienced more reported cases of measles in 2019 than
any year since 1992. Between January 1 and September 1,
2019, the CDC confirmed more than 1,200 individual cases
of measles across 31 US states. Most of these people who fell
ill with measles were not given the MMR vaccine (CDC,
2019c).
The World Health Organization (WHO) has listed
“vaccine hesitancy”among the top 10 threats to global
health for 2019. The WHO defines vaccine hesitancy as
“the reluctance or refusal to vaccinate despite the avail-
ability of vaccines”(WHO, 2019). The proliferation of
misinformation about vaccination has been accompanied
by evidence of declining confidence and trust in vaccina-
tion themselves and those who produce and promote
vaccines (Gardner et al., 2010; Guillaume & Bath, 2004).
Rates of vaccine hesitancy and vaccine refusal have
increased accordingly (Dubé et al., 2015). Prior work has
also found that negative attitudes toward vaccination can
impact both communication behaviors (like information
seeking and scanning) and vaccination intentions
(Krishna, 2018;McKeeveretal.,2016). Therefore, it is
important to adopt effective public health strategies to
enhance public beliefs in the safety and effectiveness of
the MMR vaccine, especially those who are hesitant about
vaccination.
Communication interventions to promote the MMR
vaccine
In order to improve the effectiveness of vaccine communica-
tion, researchers have developed and evaluated multiple com-
munication interventions and strategies. There is mixed
evidence for the effectiveness of messaging that seeks to cor-
rect misperceptions about the effects of the MMR vaccine. On
the one hand, one study found that highlighting scientific
consensus that the MMR vaccine is safe and is recommended
for children is a successful strategy to increase public support
for vaccination (Van der Linden et al., 2015). Furthermore,
another study showed that weight-of-evidence messages
(those that emphasize that the overwhelming evidence from
scientific studies has found no link between autism and the
MMR vaccine) also had positive effects on vaccine attitudes
through increased information certainty (Clarke et al., 2015).
On the other hand, other researchers have attempted to cor-
rect vaccine misinformation in health messages (i.e., to refute
the link between the MMR vaccine and autism), but found
these messages backfired (decreased vaccination intentions)
among parents who were least favorable of vaccine prior to
message exposure (Nyhan et al., 2014). In a similar study,
Horne et al. (2015) found that correcting the MMR-autism
link in health message was not helpful in promoting favorable
attitudes toward the MMR vaccine.
One reason for differences in these observed findings may
have to do with perceptions of the credibility of the sources
that convey the message. The Nyhan et al. (2014) and Horne
et al. (2015) studies attributed vaccine-related messaging lar-
gely to a governmental agency (the CDC) while the other
studies attributed vaccine-related information largely to scien-
tists or scientific journals. We reason that credibility percep-
tions of the CDC may have played a role in shaping responses
to subsequent messages. In support of this suggestion, one
study looked at the effects of an MMR vaccine public service
announcement (PSA) and found that perceived source cred-
ibility of both PSA sponsors and online commenters (through
direct source manipulation) can influence people’s attitude
(Kareklas et al., 2015). While this study makes clear that
source credibility matters in shaping message effects on vac-
cine-related attitudes and intentions, it does not offer gui-
dance on how strategic messaging might serve to increase
the credibility of a specific source.
In the context of vaccination in the United States, the CDC
is likely to remain an important messenger about infectious
diseases and strategies (like vaccination) to reduce their
spread. Government agencies like the CDC cannot change
their identity without massive shifts in organizational struc-
ture or branding; they can try to modify source credibility via
message design. Thus, there would be considerable practical
value in developing strategic message strategies to increase the
credibility of this specific source of health information that do
not rely on manipulations of source identity itself. Beyond the
context of resistance to vaccination, a better understanding of
how messages can shape dimensions of source credibility that
are most effective in addressing rejections of science in gen-
eral (e.g., communicating to audiences who are skeptical
about climate change) would be of broader value. Yet there
HEALTH COMMUNICATION 1355

is insufficient evidence on how to increase source credibility
through strategic health message design and how it can influ-
ence parents’beliefs and intentions about vaccination, parti-
cularly the MMR vaccine.
Source credibility in persuasion and health
communication research
Dimensions of source credibility
It is well established that source credibility is an important
factor that influences the effectiveness of persuasion (Slater &
Rouner, 1996). There are three core components in source
credibility: expertise, trustworthiness, and caring/goodwill
(McCroskey & Teven, 1999). Expertise is a measure of
whether the source/communicator is able to know the truth
(O’Keefe, 2015). Trustworthiness is a measure of whether the
source is likely to tell the truth (O’Keefe, 2015). Caring/good-
will is a measure of empathy, understanding, and responsive-
ness (McCroskey & Teven, 1999). Early research on source
credibility mostly focused on expertise and trustworthiness
(O’Keefe, 2015). McCroskey and Teven (1999) reexamined
credibility dimensions and argued that caring/goodwill had
been dismissed and should be considered as one of the source
credibility features. In order to deliver caring/goodwill, the
source needs to show understanding of people’s thoughts and
feelings (i.e., understanding), be able to identify and relate to
those thoughts and feelings (i.e., empathy), and let people
know they are being listened to (i.e., responsiveness)
(McCroskey & Teven, 1999).
Enhancing perceived source credibility of health messages
is a promising way to make health persuasion more effective.
In a systematic review of the persuasiveness of source cred-
ibility, researchers have concluded that high-credibility
sources are more persuasive than low-credibility sources in
promoting attitude and behavioral changes (Pornpitakpan,
2004). Other work suggests source credibility effects are
more conditional. Another meta-analysis concluded that
source credibility has greater impact on the formation of
new attitudes among those who do not have strong prior
attitudes, with less influence on attitudes among people who
have a preexisting attitude toward an issue (Kumkale et al.,
2010). However, these researchers concluded that more
research is needed to understand how source credibility may
work for individuals holding different levels or valence of
prior attitudes. Other research finds that high-credibility
sources, manipulated by simple statements of identity (e.g.,
comparing messages attributed to a doctor versus an ordinary
person), outperform low-credibility sources when the message
contains strong arguments (Tormala et al., 2006), or that
source credibility matters more when the audience is not
very involved in an issue (Keller & Lehmann, 2008; Stiff,
1986). Source credibility effects have been observed in
a variety of health-related topics as well (Berry & Shields,
2014; Wang et al., 2008).
Expertise and trustworthiness are two well-studied sub-
components in prior credibility research, and they both are
important in predicting persuasive success. According to the
systematic review, expertise and trustworthiness are two most
commonly identified dimensions of source credibility in
previous empirical research studies, and the persuasiveness
of these two components is confirmed by many previous
studies (Pornpitakpan, 2004). One of the meta-analyses on
source effects revealed that source expertise (expert versus
non-expert source manipulation) has the greatest impact on
persuasion among all types of source manipulation, including
trustworthiness, physical attractiveness, and ideological simi-
larity (Wilson & Sherrell, 1993). In another meta-analysis
about source credibility in persuasive communication cam-
paigns, it was concluded that source expertise is generally
effective in persuading, though some groups (e.g., women/
minorities) are influenced by source expertise more than
other groups (e.g., teenagers) (Kumkale et al., 2010). A meta-
analytic review of health information credibility looked at
how the manipulation of expertise and trustworthiness can
predict health information credibility and revealed that exper-
tise correlated more than trustworthiness with health infor-
mation credibility on the internet but not offline (Yang &
Beatty, 2016).
Caring/goodwill
In contrast, the caring/goodwill component of credibility has
received much less attention. In some cases, it has been sub-
sumed into other dimensions, such as in Yang and Beatty
(2016) meta-analysis where goodwill (using a single indicator)
was included as part of their measure of trustworthiness.
Nonetheless, McCroskey and Teven (1999) have validated
caring/goodwill as a separate, unique dimension of source
credibility, and they have suggested that future credibility
research should take caring/goodwill into consideration.
There are also theoretical reasons to believe perceptions of
goodwill may be particularly important for individuals who
distrust institutional science. A distinguishing feature of good-
will is its relational, rather than individual nature. Expertise is
an attribute of a source itself, as a source’s ability to know is
unrelated to with whom they are communicating with. In
theory, trustworthiness could be interpreted relationally as it
could vary within social relationships (Burt, 2005); however, it
is typically operationalized with questions that assess the
source only (e.g., do participants perceive the source as “trust-
worthy”,“honest”,or“moral”).
In contrast, caring/goodwill is assessed in a relational man-
ner. Instruments typically ask questions that include the
respondent as a specific audience or recipient in relation to
the source. For example, caring/goodwill instruments ask the
participants whether they perceive the source “cares about
me”or “concerned with me”. The definition of goodwill also
includes relational concepts such as empathy, which require
a target (i.e., have empathy to whom). When individuals
perceive caring/goodwill is lacking, they may reject informa-
tion because they believe it is intended to harm them or is
provided without concern for them.
There is evidence that these relational judgments play
a role in the rejection of institutional authority. For example,
some people use conspiracy theories to explain that some
events happened because powerful groups were seeking for
their own benefits (Uscinski & Parent, 2014). From this per-
spective, conspiracies are partially being motivated by a desire
to check elite power. In the face of threat from those in power
1356 Y. XU ET AL.

(like big institutions, including pharmaceutical companies),
powerless people are likely to be very vigilant toward them
(Fiske et al., 1996). From the perspective of the relational
communication literature, then, delivering a negative form
of control might have a negative impact on information
receiver (Burgoon & Hale, 1984). On the other hand, if the
audience perceives the institution conveys caring and has
goodwill instead of being threatening, it is likely that
a person’s aversion to a threat of power decreases and institu-
tional trust increases.
Limitations of the evidence base
As noted above, much health communication research testing
the effects of source credibility has directly manipulated the
source itself –for instance, comparing a medical doctor
(deemed a “credible”source) to a layperson (deemed a “non-
credible”source) as the message source (Kareklas et al., 2015;
Slater & Rouner, 1996).
Nevertheless, a few other studies have used simple descrip-
tions of the same source to manipulate the construct –for
example, describing a source as honest, sincere, and trust-
worthy versus devious, calculating, and self-centered to con-
trol for trustworthiness (McGinnies & Ward, 1980). Other
research has modified other elements of messages to increase
source credibility perceptions (Jensen & Hurley, 2012; Popper,
1961). For example, Jensen (2008) found that hedging scien-
tific claims –reporting and acknowledging limitations of
cancer research studies –can enhance perceived trustworthi-
ness. Subsequent research, however, questions the universality
of favorable effects of acknowledging scientific uncertainty.
Jensen and Hurley (2012) found that messages communicat-
ing greater scientific uncertainty (reporting conflicting scien-
tific results) increased perceived source credibility in one
context (dioxin regulation) but reduced perceived source
credibility in another context (wolf reintroduction).
Combined, these studies confirm that credibility assess-
ments of the same source are indeed malleable via strategic
messaging. Yet they leave many questions unanswered. For
instance, Jensen and Hurley (2012) work suggests that the
nature/direction of specific message effects on perceived
source credibility may be context-dependent, emphasizing
the need to test the impact of a variety of message strategies
to enhance source credibility across a variety of contexts,
including vaccination. Also, little is known about the less-
studied component of source credibility, caring/goodwill
(McCroskey & Teven, 1999). This component has not been
well examined by previous studies through either direct
source manipulation or message content manipulation since
previous research has focused exclusively on expertise and
trustworthiness (Pornpitakpan, 2004; Yang & Beatty, 2016).
Thus, an important goal of this study is to test specific
message strategies to increase source expertise, trustworthi-
ness, and caring/goodwill in the context of governmental
communication (from the CDC) about MMR vaccination.
Study hypotheses
Based on previous work showing that source credibility per-
ception can be modified by strategic message design (e.g.,
Jensen, 2008; Jensen & Hurley, 2012), we sought to develop
specific message strategies to increase perceived source cred-
ibility overall, as well as specific subcomponents (i.e., per-
ceived expertise, perceived trustworthiness, and perceived
care/goodwill). We thus designed strategic messages with the
goal of enhancing expertise, trustworthiness, and care/good-
will. The message designed to enhance expertise described
scientific knowledge about how vaccine works, which sought
to convey the source’s ability to know the truth (O’Keefe,
2015); past studies focused mostly on direct source manipula-
tion and there is a lack of evidence about how to increase
perceived expertise through message design. The message
designed to enhance trustworthiness acknowledged possible
side effects and uncertainty, which sought to convey that the
source is honest and likely to tell the truth (O’Keefe, 2015),
and past studies show that hedging is effective in increasing
perceived trustworthiness (Jensen, 2008). The message
designed to enhance caring/goodwill acknowledged and
sought to address concerns about the intentions of pharma-
ceutical companies and government agencies who developed
and promote the MMR vaccine, which sought to convey that
these groups are understanding and empathetic (McCroskey
& Teven, 1999). In light of prior work and arguments
described above about source credibility, we offer the follow-
ing hypotheses (preregistered via the Open Science
Framework (OSF) where noted, see link below):
Preregistered Hypothesis 1a (PH1a): Parents exposed to an MMR
vaccination message designed to increase source (i) expertise, (ii)
trustworthiness, or (iii) goodwill/caring, will perceive the source
of the message to have greater credibility than parents exposed to
an MMR vaccination message without any of these credibility
features.
We did not specify in our pre-registration that we expected
the effects of targeted source credibility dimensions to influ-
ence the specific dimension they targeted. Yet we also
hypothesize:
Hypothesis 1b (H1b): Compared to parents exposed to an MMR
vaccination message without any of strategically designed message
features, parents exposed to an MMR vaccination message
designed to increase source (i) expertise, (ii) trustworthiness, or
(iii) goodwill/caring will have greater (i) perceived expertise, (ii)
perceived trustworthiness, or (iii) perceived goodwill/caring of the
message source, respectively.
We hypothesized, in turn, that messages strategically designed
to increase perceived source credibility would translate into
the effects on intentions to vaccinate, as well as broader beliefs
about vaccination. Past literature and research studies have
established that higher source credibility makes health mes-
sages more persuasive and typically have a positive impact on
health-related attitude and behaviors (Berry & Shields, 2014;
Pornpitakpan, 2004). We hypothesize:
Preregistered Hypothesis 2 (PH2): Parents exposed to an MMR
vaccination message designed to increase source (i) expertise, (ii)
trustworthiness, or (iii) goodwill/caring will have higher inten-
tions to vaccinate their children than parents exposed to an MMR
vaccination message without any of these credibility features.
While we did not pre-register a hypothesis about expectations
that these messages would influence a broader set of beliefs or
HEALTH COMMUNICATION 1357

attitudes about vaccination, we did list “attitudes and beliefs
about the MMR vaccine”among our three dependent vari-
ables (credibility and intentions as the other two). We thus
also tested an additional hypothesis about message effects on
an index of all measured beliefs, attitudes, and intentions
related to vaccination, conceptualized as a continuous scale
from vaccine hesitancy (toward unfavorability) to vaccine
conviction (toward favorability):
Hypothesis 3 (H3): Parents exposed to an MMR vaccination
message designed to increase source (i) expertise, (ii) trustworthi-
ness, or (iii) goodwill/caring will have greater vaccine conviction
(i.e., lower hesitancy) than parents exposed to an MMR vaccina-
tion message without any of these credibility features.
Finally, it is unclear whether these effects are likely to differ
between those parents who are vaccine hesitant at baseline
and those who are not. Vaccine hesitant parents are less likely
to vaccinate their children, which in turn increases both their
own risk and can accelerate the spread of this highly infec-
tious disease (WHO, 2019). In the past, researchers also found
that health messages work differently among vaccine-hesitant
respondents versus non-hesitant respondents (e.g., Haglin,
2017; Horne et al., 2015; Nyhan et al., 2014), and the afore-
mentioned meta-analysis suggested that it is unclear how
source credibility works for individuals with different preex-
isting attitudes (Kumkale et al., 2010). Thus, we pose the
following research question:
Research Question 1 (RQ1): Do the effects in PH1a, H1b, PH2
and H3 differ by baseline levels of vaccine hesitancy/conviction?
Methods
Study procedure and message conditions
We collected data between July 5, 2019 and July 14, 2019
using Prolific Academic (Prolific). Prolific is an online crowd-
sourcing platform designed to facilitate academic research.
We restricted our sample to adults living in the U.S. who
had at least one child under 18 years of age. We invited
respondents to participate via the Prolific platform, obtained
informed consent, and then proceeded with data collection.
Participants received 0.75 USD at time 1 and an additional
1.00 USD at time 2, a rate beyond the U.S. minimum wage
given the average time for survey completion. Our institu-
tion’s Institutional Review Board deemed the study exempt
from review.
At time 1, 1,000 respondents completed a short survey
gauging their preexisting attitudes, beliefs, and intentions
about the MMR vaccine, along with a series of distraction
items focused on other controversial social issues that
included abortion, euthanasia, and consequentialism. They
also answered how much they trusted CDC, doctors, and
pharmaceutical companies. Participants then completed
a series of items assessing their demographic information.
About 1 week later, we re-contacted the same 1,000 respon-
dents through the Prolific Academic platform to complete
a second survey. We again obtained informed consent, after
which we randomly assigned them to one of the four message
conditions (described in further detail below). Before
displaying our stimuli, we explicitly told our respondents the
message was from CDC (“We’d like to ask for you to read
a short message from the Centers for Disease Control and
Prevention (CDC) about …”). After reading the messages,
respondents completed a survey gauging their perceived
source credibility (prefaced by a reminder that the message
was from the CDC) as well as attitudes, beliefs, and intentions
related to the MMR vaccine. A total of 803 respondents
completed both surveys; analyses excluded the 197 people
who completed time 1 but not time 2.
We highlight here that our hypotheses contain the phrase
“designed to elicit”specific source credibility dimensions.
This is because the exact message features that would elicit
these beliefs in relation to a specific source (in this case the
CDC) are not well understood from previous research which,
as noted earlier, has tended to manipulate source credibility
by simple exchanges of the sources themselves (e.g., a doctor
vs. a student), rather than try to modify credibility of a source
that is held constant (the CDC). We thus designed our mes-
sages based on our read of the literature on what is likely to
elicit these beliefs. In the sections below, in order to distin-
guish between manipulated message features and the out-
comes we sought to influence with them, we refer to these
messages by the features selected in their design. We then test
the effectiveness of each manipulated message in eliciting
these dimensions in our direct test of H1b.
The study included four message conditions, three of them
featuring content designed to shape different components of
credibility: (a) mechanistic –a message designed to increase
source expertise (N= 199, number of words = 405,
M
time reading
= 72.6 s), (b) hedging –a message designed to
increase source trustworthiness (N= 200, number of
words = 364, M
time reading
= 74.7 s), (c) acknowledgment of
concern –a message designed to increase source goodwill/
caring (N= 203, number of words = 497, M
time reading
= 97.2
s), and (d) a control message that did not include any source
components (N= 201, number of words = 198, M
time reading
= 50.1 s). The three manipulations began with content
designed to increase the relevant credibility dimension. The
expertise-targeted condition presented scientific knowledge
about herd immunity –that the MMR vaccine can protect
the whole community if more than a certain percentage of
people get vaccinated. This was intended to convey the
source’s ability to know the truth by describing the mechanics
of how vaccines work (O’Keefe, 2015); as such, we refer to this
condition as the “mechanistic message”. The trustworthiness-
targeted condition described the potential side effects of the
MMR vaccine and emphasized that parents can report any
vaccine-related injury case to a website monitored by CDC
anytime. This sought to convey that the source is honest and
likely to tell the truth (O’Keefe, 2015), and past studies have
shown that hedging (acknowledging uncertainty) is effective
in increasing perceived trustworthiness (Jensen, 2008). We
thus refer to this condition as the “hedging message”. The
caring/goodwill-targeted condition acknowledged the rele-
vance of parents’concerns about the MMR vaccine and con-
veyed that it understood why they might be skeptical about
pharmaceutical companies and government agencies given
past behavior. It then tried to address this concern by
1358 Y. XU ET AL.

describing how pharmaceutical companies and government
agencies have worked in collaboration to enhance the public
good. It also emphasized the CDC’s official mandate and
mission to protect the health of the public. This sought to
show the source’s understanding, empathy, and responsive-
ness (McCroskey & Teven, 1999). We thus refer to this con-
dition as the “acknowledgement message”, in the sense that it
acknowledges audience concerns about the vaccine and var-
ious stakeholders involved in its production and distribution.
Each message concluded with additional paragraphs that
introduced basic facts of MMR and described that the MMR
vaccine can prevent these contagious diseases. The control
condition included no source credibility information but
included the same basic facts about MMR and the protective
value of the MMR vaccine. The supplemental appendix con-
tains full text of all message conditions.
Measures
The full study instrument, including all item wordings and
response categories, can be found at the OSF preregistration
website (study URL: https://osf.io/3ub7x). Here we describe
details for items that are the focus of our pre-registered
hypotheses and the current analysis.
Perceived credibility at time 2
After exposure, participants rated the credibility of the mes-
sage source. We measured three components of credibility –
expertise, trustworthiness, and caring/goodwill –using the 18-
item credibility scale developed and refined by McCroskey
and Teven (1999). The scale features six response categories
using a semantic differential format. Six items gauged exper-
tise (e.g., expert versus inexpert, trained versus untrained) that
were averaged to form a perceived expertise scale (α= 0.94,
M= 5.54, SD = 0.74). Six items gauged trustworthiness (e.g.,
trustworthy versus untrustworthy, honest versus dishonest) to
form a perceived trustworthiness scale (α= 0.95, M= 5.29,
SD = 0.93). Six items gauged caring/goodwill (e.g., care about
me versus not care about me, concerned with me versus
unconcerned with me) to form a perceived caring scale
(α= 0.93, M= 5.13, SD = 0.96). We also created an overall
perceived credibility scale with all 18 items (α= 0.97,
M= 5.32, SD = 0.83).
MMR vaccination intentions at time 1 and time 2
A series of questions measured respondents’behavioral
beliefs, normative beliefs, control beliefs, attitudes, and beha-
vioral intentions toward MMR vaccination, both at time 1 and
time 2. We worded these items based on guidance offered by
the Theory of Planned Behavior (Ajzen, 1985). To assess
parents’behavioral intentions, we asked them to consider,
on a 6-point scale from very unlikely (1) to very likely (6),
how likely they would be to have their next child vaccinated
against MMR if they were to have another child. This measure
served as the dependent variable in testing PH2a.
MMR vaccination hesitancy/conviction at time 1 and time 2
We also included intentions in a broader scale of MMR
vaccination cognitions that also included belief and attitude
items since these items were very highly correlated with one
another and meta-analysis suggests that persuasive message
effects on attitudes, intentions, and behavior are equivalent
regardless of the measure used (O’Keefe, 2013). We used
semantic differentials with 6-item response scales to assess
attitudes toward vaccinating children against MMR by asking
whether vaccinating their next child against MMR is “wise (6)
versus foolish (1),”“a good idea (6) versus a bad idea (1),”
and “beneficial (6) versus harmful (1).”We also asked a series
of questions about behavioral beliefs, normative beliefs, and
control beliefs using 6-point Likert scales from strongly dis-
agree (1) to strongly agree (6). Consistent with previous work,
we averaged these 12 intention, attitude, and belief items to
form an MMR vaccine hesitancy/conviction scale (time 1
α= 0.95, M=5.43,SD = 0.85; time 2 α=0.95,M=5.50,
SD = 0.77). We modeled this item as a continuous variable in
all analyses and coded it such that lower scores reflect more
hesitancy and higher scores reflect more conviction. At base-
line, this variable ranged from 1.67 (highest hesitancy/lowest
conviction) to 6 (lowest hesitancy/highest conviction).
17.9 percent of the sample held baseline scores on this scale
below 5.00, indicating some level of vaccine hesitancy (since
a 5 indicates “agree”while a 4 indicates “somewhat agree;”we
reasoned that the choice to say “somewhat”reveals some
degree of hesitancy).
Analytic approach
We conducted all analyses using the SPSS and R statistical
packages. We began the analysis by removing respondents
who did not meet pre-registered analytic criteria and examin-
ing the distribution of key items (e.g., vaccine hesitancy/con-
viction). We conducted a randomization check to see whether
random assignment produced balanced groups on measured
variables. We then used ordinary least squares (OLS) regres-
sion to test study hypotheses by creating dummy variables for
each randomized condition except the control group and
assessing whether these dummy variables predicted each of
the hypothesized outcomes. Finally, we created interaction
terms between vaccine hesitancy/conviction and dummy vari-
ables for randomized condition to test whether the effects of
condition differed by baseline levels of vaccine hesitancy. We
probed significant interactions using the Johnson-Neyman
technique to test whether messages designed to increase
source credibility had different effects by variations in baseline
vaccine hesitancy.
Results
Participant demographics and randomization check
As stated in the pre-registered analysis plan, we excluded
participants who spent less than 60 s in both waves to make
sure they paid reasonable amount of attention to the ques-
tions and stimuli. This reduced the analytic sample to 803
respondents, and all subsequent analyses use this sample.
Participants were majority female (59%) and ranged in age
from 19 to 72 years old (M= 37, SD = 14.76). A majority of
participants identified themselves as White (83%), followed by
HEALTH COMMUNICATION 1359

African American (10%), Hispanic or Latino (8%), Asian
(5%), and other (1%).
F-tests and chi-square tests revealed that randomization
produced balanced groups on measured demographic and
potential moderating variables (using p<.05 as the statistical
criterion throughout). Each condition also had comparable
levels of trust in the CDC and comparable levels of vaccine
hesitancy in the time 1 survey.
Effects of strategic message design on perceived source
credibility
We hypothesized that parents exposed to an MMR vaccina-
tion message designed to increase source (i) expertise (i.e.,
mechanistic message), (ii) trustworthiness (i.e., hedging mes-
sage), or (iii) goodwill/caring (i.e., acknowledgment message)
would perceive the source of the message to have greater
credibility (PH1a) and have higher intentions (PH2) or
lower hesitancy (H3) to vaccinate their children than parents
exposed to an MMR vaccination message without any of these
credibility features. OLS regressions tested whether parents’
perceived source credibility (PH1a), vaccination intentions
(PH2), and vaccine hesitancy/conviction (H3) differed
between conditions. Compared to the control condition, no
message increased perceived credibility, vaccination inten-
tions, or vaccine hesitancy/conviction (Table 1). Therefore,
all tests of PH1a (PH1a
i
, PH1a
ii
, and PH1a
iii
), PH2 (PH2
i
,
PH2
ii
, and PH2
iii
) and H3 (H3
i
,H3
ii
, and H3
iii
) were not
supported.
We also conducted three additional OLS regressions to test
whether any of the three conditions produced significantly
higher perceived (i) expertise, (ii) trustworthiness, or (iii)
caring/goodwill compare to the control group (H1b).
Contrary to expectations, the mechanistic message produced
greater perceived caring/goodwill (M= 5.26, SD = 0.92) than
the control condition (M= 5.06, SD = 0.97, B= 0.21, p< .05)
(Table 2). None of the messages influenced their intended
credibility domain (e.g., the mechanistic message influencing
perceived expertise), leading us to reject H1b
i
, H1b
ii
, and
H1b
iii.
Post-hoc mediation analyses
In light of the results from the previous analysis (i.e., the
mechanistic message produced higher perceived caring/good-
will than the control condition), and considering that past
research has found an association between perceived source
credibility and targeted beliefs, we tested in a post-hoc analy-
sis whether there was evidence that message type (i.e., the
mechanistic message relative to control, controlling for the
other two conditions) was indirectly associated with t2 vacci-
nation intentions or vaccine hesitancy/conviction via per-
ceived caring/goodwill, which was the only dimension of
credibility perception that changed via strategic message
design. We conducted statistical mediation analysis in R and
used the bootstrapping technique recommended by Preacher
and Hayes (2004,2008) with 10,000 iterations. The indirect
effect estimate between the mechanistic message and vaccina-
tion intentions was 0.13, and the 95% confidence interval (CI)
for the indirect effect was 0.01 to 0.25. The indirect effect
estimate between the mechanistic message and vaccine hesi-
tancy/conviction was 0.11 [95% CI 0.01 to 0.21]. The fact that
the 95% CI does not overlap zero in either case means the
evidence is consistent with significant, indirect associations
between the expertise condition and both vaccination inten-
tions and vaccine hesitancy/conviction at time 2 via perceived
caring/goodwill.
Post-hoc analyses conditional on vaccine hesitancy/
conviction
As described above, there is also interest in understanding
whether the effects of strategic messaging to improve source
credibility differ based on baseline levels of vaccine hesitancy/
Table 1. Ordinary least squares (OLS) regressions predicting perceived source credibility, vaccination intentions, and vaccination hesitancy/conviction
by message condition.
Source credibility (PH1a)
Vaccination intentions
(PH2) Vaccine hesitancy/conviction (H3)
BSEpBSEp B SE p
Constant 5.30 0.06 5.59 0.08 5.47 0.05
Mechanistic message 0.12 0.08 0.14 0.12 0.11 0.28 0.09 0.08 0.23
Hedging message −0.03 0.08 0.69 0.08 0.11 0.48 0.04 0.08 0.60
Acknowledgment message −0.02 0.08 0.85 0.07 0.11 0.53 0.03 0.08 0.74
The control group is the omitted category and thus the comparison group.
Table 2. Ordinary least squares (OLS) regressions predicting sub-dimensions of source credibility by message conditions.
Perceived
expertise (H1b
i
) Perceived trustworthiness (H1b
ii
) Perceived caring/goodwill (H1b
iii
)
BSEp B SE p B SE p
Constant 5.58 0.05 5.26 0.07 5.06 0.07
Mechanistic message 0.02 0.07 0.79 0.15 0.09 0.11 0.21* 0.10 0.03
Hedging message −0.11 0.07 0.13 −0.01 0.09 0.94 0.02 0.10 0.83
Acknowledgment message −0.09 0.07 0.21 −0.03 0.09 0.77 0.07 0.09 0.47
The control group is the omitted category and thus the comparison group. *Denotes p< .05.
1360 Y. XU ET AL.

conviction. Previous research on vaccine communication
found that vaccine promotion messages work differently
among vaccine-hesitant respondents compared to respon-
dents who are not hesitant (Haglin, 2017; Horne et al., 2015;
Nyhan et al., 2014). We thus conducted a series of hierarchical
OLS regression models in which we included baseline vaccine
hesitancy/conviction, dummy coded condition variables, and
interaction terms between message conditions and preexisting
hesitancy/conviction to predict perceived source credibility,
credibility subcomponents, vaccination intentions, and post-
message vaccine hesitancy/conviction.
We begin with overall perceived credibility. In step one, we
regressed perceived credibility on the three dummy coded
condition variables and continuous vaccine hesitancy/convic-
tion variable –functionally a test of whether baseline vaccine
hesitancy/conviction predicted perceived credibility at time 2.
Step 2 added the interaction terms between vaccine hesitancy/
conviction and all three manipulated conditions into the
model. Adding the interaction items of vaccine hesitancy/
conviction and all manipulated conditions did not improve
model fit (p= .07), indicating that (null) message effects on
overall perceived credibility did not differ by vaccine hesi-
tancy/conviction (not presented in tables).
We repeated these regression analyses for each sub-
dimension of credibility. Baseline vaccine hesitancy/convic-
tion was a significant (p< .001) predictor of each variable,
so we included the interaction items of hesitancy/conviction
status and all three manipulated conditions into the model for
all three dependent variables.
For perceived expertise, the second model fits significantly
better than the previous model (Rsquare change is 0.01,
Fchange is 39.56, p< .05). The coefficient for the interaction
between the acknowledgment message and vaccine hesitancy
was again positive (b= 0.22, p< .01), indicating that this
message manipulation was significantly less effective among
parents with more baseline vaccine hesitancy than those with
more conviction (Table 3). The Johnson-Neyman technique
(Figure 1) showed that this message was significantly less
effective than the control condition in increasing perceived
expertise (p< .05) when the value of hesitancy/conviction was
less than 5.4, but not statistically significant with higher vac-
cine conviction. Thus, we conclude that this message had
backfire effects in reducing perceived source expertise
among parents with more vaccine hesitancy.
For perceived trustworthiness, the second model fits sig-
nificantly better than the previous model (Rsquare change is
0.01, Fchange is 46.04, p< .01). The coefficients for the
mechanistic message (B=−0.20, p< .05) and hedging mes-
sage (B=−0.32, p< .001) were negative and significant,
indicating that these manipulations were significantly more
effective among parents with more baseline vaccine hesitancy
than those with more conviction (Table 4). The Johnson-
Neyman technique (Figure 2) showed that the mechanistic
Table 3. Ordinary least squares (OLS) regressions predicting perceived expertise
by message conditions, baseline hesitancy/conviction, and their interaction.
BSEp
Step 1
Constant 2.92 0.14
Mechanistic message −0.03 0.06 0.66
Hedging message −0.13* 0.06 0.03
Acknowledgment message −0.12 0.06 0.05
Vaccine hesitancy/conviction 0.49*** 0.03 <.001
R
2
,F (df) 0.33, 97.33 (4, 794)
Step 2
Constant 3.54 0.28
Mechanistic message −0.74 0.40 0.07
Hedging message −0.56 0.40 0.16
Acknowledgment message −1.33*** 0.38 <.001
Vaccine hesitancy/conviction 0.38*** 0.05 <.001
Mechanistic X hesitancy/conviction 0.13 0.07 0.07
Hedging X hesitancy/conviction 0.08 0.07 0.27
Acknowledgment X hesitancy/conviction 0.22** 0.07 <.01
R
2
,F (df) 0.34, 57.77 (7, 791)
ΔR
2
0.01*
The control group is the omitted category and thus the comparison group.
*Denotes p< .05; **denotes p< .01; ***denotes p< .001.
Figure 1. Johnson-Neyman plot: Interaction between the acknowledgment message and baseline hesitancy/conviction on perceived expertise.
X-axis: value of baseline hesitancy/conviction; y-axis: coefficient of the acknowledgment message designed to increase caring/goodwill; reference group: control
condition. The slope of acknowledgment message is p<.05 when baseline hesitancy/conviction is outside this interval: [5.4, 7.05] (range of observed values of
baseline hesitancy/conviction: [1.67, 6]).
HEALTH COMMUNICATION 1361

message was significantly more effective than the control
condition in increasing perceived trustworthiness (p< .05)
when the value of baseline hesitancy/conviction was less than
4.98, but not significant with higher values of conviction. The
Johnson-Neyman technique (Figure 3) also showed that the
hedging message was significantly more effective than the
control condition in increasing perceived trustworthiness
when the value of baseline hesitancy/conviction was less
than 4.69, but was significantly less effective than control
condition in increasing perceived trustworthiness (p< .05)
when the value of baseline hesitancy/conviction was above 5.8
(essentially those with the strongest possible baseline convic-
tions toward MMR vaccination). Thus, we conclude that these
two messages had favorable effects on perceived trustworthi-
ness among more vaccine-hesitant parents, but the hedging
message had unfavorable effects on perceived trustworthiness
among parents with very strong convictions toward support
of MMR vaccination.
For perceived caring/goodwill, adding the interaction items
of vaccine hesitancy/conviction and all manipulated condi-
tions only marginally improved model fit (p= .09). However,
the coefficient for the interaction between the hedging mes-
sage and baseline hesitancy/conviction (B=−0.20, p< .05)
was negative and significant, indicating that this message was
significantly more effective among more vaccine-hesitant par-
ents than parents with more vaccine conviction (see Table 5).
The Johnson-Neyman technique (Figure 4) showed that this
condition was significantly more effective than the control
condition in increasing perceived caring/goodwill (p< .05)
when the value of baseline hesitancy/conviction was less than
2.58, but not significant with higher values of conviction.
Thus, we conclude that this message manipulation had favor-
able effects on increasing perceived caring among parents very
hesitant at baseline about getting their children MMR vaccine.
We now turn to models interacting baseline hesitancy/con-
viction and message conditions in predicting vaccine intentions
and vaccine hesitancy/conviction post-message exposure.
For vaccination intentions, baseline hesitancy/conviction
was a strong predictor of vaccination intentions, such that
those with higher baseline conviction had much stronger
vaccination intentions at follow-up, regardless of whether or
not they were exposed to a message, or (if exposed) which
message they read (B= 1.03, SE = 0.03, p< .001). Adding the
interaction items of baseline vaccine hesitancy/conviction and
all manipulated conditions did not improve model fit
(p= .10), indicating that the (null) effects on vaccination
intentions were similar among baseline vaccine-hesitant par-
ents and those with more baseline vaccination conviction.
For vaccine hesitancy/conviction post-message exposure,
baseline hesitancy/conviction was a strong and significant
(p< .001) predictor –on average, those with high vaccine
conviction at baseline were far more likely than those with
Table 4. Ordinary least squares (OLS) regressions predicting perceived trust-
worthiness by message conditions, baseline hesitancy/conviction, and their
interaction.
BSEp
Step 1
Constant 1.70 0.18
Mechanistic message 0.08 0.08 0.28
Hedging message −0.04 0.07 0.64
Acknowledgment message −0.06 0.07 0.42
Vaccine hesitancy/conviction 0.66*** 0.03 <.001
R
2
,F (df) 0.37, 114.8 (4, 795)
Step 2
Constant 0.93 0.34
Mechanistic message 1.15* 0.49 0.02
Hedging message 1.68*** 0.48 <.001
Acknowledgment message 0.34 0.47 0.46
Vaccine hesitancy/conviction 0.81*** 0.06 <.001
Mechanistic X hesitancy/conviction −0.20* 0.09 0.03
Hedging X hesitancy/conviction −0.32*** 0.09 <.001
Acknowledgment X hesitancy/conviction −0.08 0.09 0.38
R
2
,F (df) 0.38, 68.76 (7, 792)
ΔR
2
0.01***
The control group is the omitted category and thus the comparison group.
*Denotes p< .05; ***denotes p< .001.
Figure 2. Johnson-Neyman Plot: Interaction between the Mechanistic Message and Baseline Hesitancy/Conviction on Perceived Trustworthiness.
X-axis: value of baseline hesitancy/conviction; y-axis: coefficient of the mechanistic message designed to increase expertise; reference group: control condition. The
slope of mechanistic message is p< .05 when baseline hesitancy/conviction is outside this interval: [4.98, 9.76] (range of observed values of baseline hesitancy/
conviction: [1.67, 6]).
1362 Y. XU ET AL.

more hesitancy to have high vaccine conviction after expo-
sure. Adding interaction terms significantly improved model
fit (Rsquare change is 0.01, Fchange is 349.7, p< .001). The
coefficient for the interaction between the mechanistic mes-
sage and baseline hesitancy/conviction was negative
(B=−0.10, p< .05), indicating that this message was signifi-
cantly more effective among vaccine-hesitant parents than
those with greater conviction (Table 6). The Johnson-
Neyman technique (Figure 5) showed that this message
manipulation was significantly more effective than the control
condition in reducing vaccine hesitancy (p< .05) when the
value of preexisting belief was less than 4.83, but not statisti-
cally significant with higher values of baseline conviction.
Thus, we conclude that the mechanistic message had favorable
effects on reducing vaccine hesitancy among those parents
who were more hesitant at baseline.
Discussion
We begin the discussion of results by stating, unequivocally,
that we found no support for any of the study’s original, pre-
registered hypotheses. We observed no main effects of any
message condition on overall source credibility; only 1 of 9
tests examining message effects on the sub-dimensions of this
variable (and that test was not consistent with the targeted
sub-dimension of the manipulation); none of our hypothe-
sized manipulations influenced the sub-dimensions they tar-
geted (overall); we did not find effects on parental intentions
to vaccinate their children. We thus acknowledge that the
interpretations of findings that follow are based on post-hoc
analyses that are more susceptible to statistical chance and
failure to replicate down the line. We nevertheless believe that
findings from this research are worthy of further considera-
tion by health communication researchers and public health
practitioners in thinking about possible strategies for enhan-
cing source credibility and considering ways that messages
can convey both intended and unintended effects to both
targeted (e.g., those who need convincing to vaccinate their
children) and secondary audiences (those who are largely
committed to doing so already).
Our exploratory, post-hoc analyses suggest that enhancing
perceived source credibility through the strategic design of
health messages has both promise and potential pitfalls. On the
promising side, although none of the messages worked as
intended among parents overall, post-hoc analyses suggest that
perceived source caring can be enhanced through strategic mes-
sage design. We found some evidence that a strategic message
(albeit one hypothesized as being likely to enhance source exper-
tise) increased perceptions of caring and goodwill toward
a public health institution (CDC), which, in turn, were asso-
ciated with lower vaccine hesitancy overall. This message also
appeared to reduce vaccine hesitancy directly among parents
who expressed such hesitancy at baseline. This suggests that
Figure 3. Johnson-Neyman plot: Interaction between the hedging message and baseline hesitancy/conviction on perceived trustworthiness.
X-axis: value of baseline hesitancy/conviction; y-axis: coefficient of the hedging message designed to increase trustworthiness; reference group: control condition. The
slope of hedging message is p< .05 when baseline hesitancy/conviction is outside this interval: [4.69, 5.8] (range of observed values of baseline hesitancy/conviction:
[1.67, 6]).
Table 5. Ordinary least squares (OLS) regressions predicting perceived caring/
goodwill by message conditions, baseline hesitancy/conviction, and their
Interaction.
BSEp
Step 1
Constant 1.76 0.19
Mechanistic message 0.15 0.08 0.07
Hedging message −0.01 0.08 0.94
Acknowledgment message 0.04 0.08 0.62
Vaccine hesitancy/conviction 0.61*** 0.03 <.001
R
2
,F (df) 0.30, 86.52 (4, 795)
Step 2
Constant 1.31 0.37
Mechanistic message 0.93 0.53 0.08
Hedging message 1.06* 0.52 0.04
Acknowledgment message 0.09 0.50 0.86
Vaccine hesitancy/conviction 0.70*** 0.07 <.001
Mechanistic X hesitancy/conviction −0.14 0.10 0.14
Hedging X hesitancy/conviction −0.20* 0.10 0.04
Acknowledgment X hesitancy/conviction −0.01 0.09 0.92
R
2
,F (df) 0.31, 50.59 (7, 792)
ΔR
2
0.01
The control group is the omitted category and thus the comparison group.
*Denotes p< .05; ***denotes p< .001.
HEALTH COMMUNICATION 1363

the relational nature of audience perceptions of goodwill may be
particularly important for individuals who distrust institutional
science. In retrospect, the fact that the mechanism through
which vaccines work involves the protection of other people
may have also conveyed a sense that the CDC has empathy for,
understanding of, and responsiveness to, other people
(McCroskey & Teven, 1999). We also found some evidence
that the expertise condition increased perceived trustworthiness
among vaccine-resistant parents. This suggests that explaining
the science behind vaccination could enhance beliefs among
even skeptical audiences that a public health information source
like the CDC is likely to tell the truth.
It is also noteworthy that acknowledging potential side effects
and illuminating pathways for the public to report on such side
effects has potential to increase perceived trustworthiness among
vaccine-hesitant parents and increase perceived caring among
parents with substantial vaccine hesitancy. The fact that we
observed some evidence for effects among vaccine-hesitant par-
ents is noteworthy, if not predicted a priori. That a form of
acknowledging uncertainty (that side effects can happen)
increased trustworthiness judgments among key audiences is
consistent with previous work which has found that other forms
of acknowledging uncertainty (like scientists describing study
limitations) can enhance perceived trustworthiness (Jensen,
2008). It could be that delivering information on potential vaccine
side effects and mechanisms for reporting adverse cases shows
that the CDC has empathy for, and responsiveness to, parents
with concerns about vaccine safety (McCroskey & Teven, 1999). It
also highlights the need to consider preexisting attitudes toward
the targeted issue as a potential effect moderator.
Results also caution about some potential pitfalls of at least
some strategies designed to enhance source credibility. The
acknowledgment message, which was originally designed to
improve perceived caring/goodwill, failed to produce favor-
able effects on credibility among vaccine-hesitant parents. In
retrospect, it is possible that this message may have been
interpreted by respondents as sounding like an attempt to
“correct”a false belief among those who hold firm beliefs
against MMR vaccination and institutions that promote it.
Although we tried to convey understanding and empathy by
acknowledging people’s concern about pharmaceutical com-
panies and government agencies, we also attempted to address
this concern by telling them that pharmaceutical companies
actually have good intentions. This may have been interpreted
as akin to an attempt to correct misinformation, and prior
work suggests that such correction can cause backfire effects
as it is challenging or even counterproductive to correct
people’s existing beliefs (Nyhan et al., 2014; Walter &
Murphy, 2018). It is possible that other strategies, like perso-
nal stories about dedicated people who work at these institu-
tions, could enhance caring or goodwill without sounding like
a correction. Future research should explore this possibility.
Figure 4. Johnson-Neyman plot: Interaction between the hedging message and baseline hesitancy/conviction on perceived caring/goodwill.
X-axis: value of baseline hesitancy/conviction; y-axis: coefficient of the hedging message designed to increase trustworthiness; reference group: control condition. The
slope of hedging message is p< .05 when baseline hesitancy/conviction is outside this interval: [2.58, 7.7] (range of observed values of baseline hesitancy/conviction:
[1.67, 6]).
Table 6. Ordinary least squares (OLS) regressions predicting post-message hes-
itancy/conviction by message conditions, baseline hesitancy/conviction, and
their interaction.
BSEp
Step 1
Constant 1.08 0.08
Mechanistic message 0.02 0.03 0.57
Hedging message 0.01 0.03 0.87
Acknowledgment message −0.01 0.03 0.67
Vaccine hesitancy/conviction 0.82*** 0.01 <.001
R
2
,F (df) 0.81, 852.1 (4, 794)
Step 2
Constant 0.97 0.15
Mechanistic message 0.56* 0.22 0.01
Hedging message 0.39 0.22 0.08
Acknowledgment message −0.39 0.21 0.06
Vaccine hesitancy/conviction 0.84*** 0.03 <.001
Mechanistic X hesitancy/conviction −0.10* 0.04 0.01
Hedging X hesitancy/conviction −0.07 0.04 0.08
Acknowledgment X hesitancy/conviction 0.07 0.04 0.07
R
2
,F (df) 0.82, 502.4 (7, 791)
ΔR
2
0.01***
The control group is the omitted category and thus the comparison group.
*Denotes p< .05; ***denotes p< .001.
1364 Y. XU ET AL.

Beyond the backfire effects among vaccine-hesitant parents, it
is also noteworthy that there was some indication that acknowl-
edging side effects could produce less trust among those parents
with the most favorable initial beliefs. Jensen and Hurley (2012)
found that hedging increased perceived source credibility in
dioxin regulation context but reduced perceived source credibil-
ity in another context (wolf reintroduction). Collectively, find-
ings to date question the universality of favorable effects of
acknowledging caveats and limitations to scientific research
and suggest that modifying perceived trustworthiness via these
caveats is not only context-dependent but the effects can also
vary among different audiences. Future work should continue to
explore when hedging works to improve perceived trustworthi-
ness, and when it does not.
Leaving aside the ineffective acknowledgment message, the
larger pattern of (post hoc) analytic results suggests that deliver-
ing information about how vaccination works and how vaccina-
tion can help to protect the community has some potential to
increase perceived caring overall and increase perceived trust-
worthiness among vaccine-hesitant parents. Health communi-
cators might consider a strategy that involves explaining the
process of vaccination in an attempt to increase the perceived
trustworthiness and caring of health information sources. Health
communication practitioners might also try to increase source
trustworthiness among vaccine-hesitant groups by acknowled-
ging possible side effects and offering people a platform to report
adverse events, but be cautious about such acknowledgments
among non-hesitant groups. Future work might consider the
tradeoffs of this approach in practice –whether gains in trust-
worthiness among one group may be offset by reductions in
trustworthiness among another.
More broadly, this study offers some hope that strategic mes-
saging could be used to build dimensions of source credibility
among key audiences. As noted before, most source credibility
studies directly manipulate the source itself; there is a relatively
limited literature on effective strategies to increase source cred-
ibility through strategic message design, particularly in health
contexts. More research needs to be conducted to find out other
effective strategic design strategies that can improve caring/good-
will because it has not been well studied in the past.
Study limitations
Several limitations are worth noting in this study. First, as
noted earlier, the majority of study participants were not
vaccine hesitant, perceived the CDC to be a very credible
source, and had strong vaccination intentions. Limited varia-
tion may have hindered efforts to detect message effects on
these outcomes. Future research should explore these and
other design strategies to enhance perceived credibility using
different message sources. These might include, for example,
multiple forms of media (e.g., news media and social media),
government officials and bodies, pharmaceutical companies,
anti-vaccination websites and organizations, and non-
governmental organizations like the World Health
Organization (WHO). This line of research would also benefit
from studying different topics, especially topics with more
diversified baseline attitudes. Second, the caring/goodwill con-
dition was significantly longer than the other three conditions.
This may have influenced the effectiveness of this message
because it required more time and longer attention from the
audience. Third, a limited number of highly vaccine resistant
parents in all conditions produced limited statistical power to
detect effects of message design at more extreme values of
vaccine hesitancy. Fourth, the crowdsourcing online data col-
lection platform used here (Prolific) is not nationally
Figure 5. Johnson-Neyman plot: Interaction between the mechanistic message and baseline hesitancy/conviction on vaccine hesitancy/conviction post-message
exposure.
X-axis: value of baseline hesitancy/conviction; y-axis: coefficient of the mechanistic message designed to increase expertise; reference group: control condition. The
slope of mechanistic message is p< .05 when baseline hesitancy/conviction is outside this interval: [4.83, 7.30] (range of observed values of baseline hesitancy/
conviction: [1.67, 6]).
HEALTH COMMUNICATION 1365

representative. The sample overrepresented females, political
liberals, and White non-Hispanic respondents relative to the
U.S. population. Fifth, we used simple text messages without
any kind of graphic design layout as stimuli instead of visual
or audio forms of media messages. Sixth, since we tested
different message strategies to increase the credibility of an
existing source, the phrasing and writing style choices of the
stimuli might be different from CDC’s voice, which may have
influenced message effectiveness. Finally, we are limited to
a single-message design in a single context from a single
(attributed) source (the CDC). These limitations should be
taken into account when considering the potential general-
izability of the current study to other populations, messages,
settings, and media/message channels.
Conclusion
The present study contributed to source credibility research
by investigating message design strategies to increase per-
ceived source credibility and exploring its under-studied
dimension, caring/goodwill. Perceptions of caring/goodwill
may be particularly important for individuals who distrust
institutional science because of its relational nature. Results
suggest that perceived caring/goodwill can be modified
through strategic message design, though not necessarily in
ways that were predicted a priori by the research team. It is
promising to further study how to increase the audience’s
perceived caring/goodwill as an effective health message
design strategy. Further, identifying how to avoid backfire
effects while designing strategic health messages, especially
among people with unfavorable initial attitudes, will be
critical as researchers continue to investigate source cred-
ibility in communication and persuasion research.
ORCID
Yiwei Xu http://orcid.org/0000-0002-2225-7806
References
Ajzen, I. (1985). From intentions to actions: A theory of planned beha-
vior. In J. Kuhl & J. Beckmann (Eds.), Action control: From cognition
to behavior (pp. 11–39). Springer Berlin Heidelberg. https://doi.org/
10.1007/978-3-642-69746-3_2
Berry, T. R., & Shields, C. (2014). Source attribution and credibility of
health and appearance exercise advertisements: Relationship with
implicit and explicit attitudes and intentions. Journal of Health
Psychology,19(2), 242–252. https://doi.org/10.1177/
1359105312468190
Burgoon, J. K., & Hale, J. L. (1984). The fundamental topoi of relational
communication. Communication Monographs,51(3), 193–214. https://
doi.org/10.1080/03637758409390195
Burt, R. S. (2005). Brokerage and closure: An introduction to social
capital. OUP Oxford.
CDC. (2019a, February 25). Measles | History of Measles | CDC.https://
www.cdc.gov/measles/about/history.html
CDC. (2019b, March). About measles vaccination. Vaccination and
Immunizations. http://www.cdc.gov/vaccines/vpd/measles/index.html
CDC. (2019c, September 9). Measles cases and outbreaks.https://www.
cdc.gov/measles/cases-outbreaks.html
Chon, M.-G., & Park, H. (2019). Predicting public support for govern-
ment actions in a public health crisis: Testing fear, organization-public
relationship, and behavioral intention in the framework of the situa-
tional theory of problem solving. Health Communication,1–11.
https://doi.org/10.1080/10410236.2019.1700439
Clarke, C. E., McKeever, B. W., Holton, A., & Dixon, G. N. (2015). The
influence of weight-of-evidence messages on (vaccine) attitudes:
A sequential mediation model. Journal of Health Communication,20
(11), 1302–1309. https://doi.org/10.1080/10810730.2015.1023959
Dubé, E., Gagnon, D., & MacDonald, N. E. (2015). Strategies intended to
address vaccine hesitancy: Review of published reviews. Vaccine,33
(34), 4191–4203. https://doi.org/10.1016/j.vaccine.2015.04.041
Evans, M., Stoddart, H., Condon, L., Freeman, E., Grizzell, M., & Mullen, R.
(2001). Parents’perspectives on the MMR immunisation: A focus group
study. The British Journal of General Practice, 51(472), 904–910. Retrieved
from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1314147/
Fiske, S. T., Morling, B., & Stevens, L. E. (1996). Controlling self and
others: A theory of anxiety, mental control, and social control.
Personality and Social Psychology Bulletin,22(2), 115–123. https://
doi.org/10.1177/0146167296222001
Gardner, B., Davies, A., McAteer, J., & Michie, S. (2010). Beliefs under-
lying UK parents’views towards MMR promotion interventions:
A qualitative study. Psychology, Health & Medicine,15(2), 220–230.
https://doi.org/10.1080/13548501003623963
Guillaume, L. R., & Bath, P. A. (2004). The impact of health scares on
parents’information needs and preferred information sources: A case
study of the MMR vaccine scare. Health Informatics Journal,10(1),
5–22. https://doi.org/10.1177/1460458204040664
Haglin, K. (2017). The limitations of the backfire effect. Research &
Politics,4(3), 2053168017716547. https://doi.org/10.1177/
2053168017716547
Hilton, S., Petticrew, M., & Hunt, K. (2007). Parents’champions vs. vested
interests: Who do parents believe about MMR? A qualitative study. BMC
Public Health,7(1), 42. https://doi.org/10.1186/1471-2458-7-42
Horne, Z., Powell, D., Hummel, J. E., & Holyoak, K. J. (2015).
Countering antivaccination attitudes. Proceedings of the National
Academy of Sciences,112(33), 10321–10324. https://doi.org/10.1073/
pnas.1504019112
Jensen, J. D. (2008). Scientific uncertainty in news coverage of cancer
research: Effects of hedging on scientists’and journalists’credibility.
Human Communication Research,34(3), 347–369. https://doi.org/10.
1111/j.1468-2958.2008.00324.x
Jensen, J. D., & Hurley, R. J. (2012). Conflicting stories about public
scientific controversies: Effects of news convergence and divergence
on scientists’credibility. Public Understanding of Science (Bristol,
England),21(6), 689–704. https://doi.org/10.1177/0963662510387759
Kareklas, I., Muehling, D. D., & Weber, T. J. (2015). Reexamining health
messages in the digital age: A fresh look at source credibility effects.
Journal of Advertising,44(2), 88–104. https://doi.org/10.1080/
00913367.2015.1018461
Keller, P. A., & Lehmann, D. R. (2008). Designing effective health com-
munications: A meta-analysis. Journal of Public Policy & Marketing,
27(2), 117–130. https://doi.org/10.1509/jppm.27.2.117
Krishna, A. (2018). Poison or prevention? Understanding the linkages
between vaccine-negative individuals’knowledge deficiency, moti-
vations, and active communication behaviors. Health
Communication,33(9), 1088–1096. https://doi.org/10.1080/
10410236.2017.1331307
Kumkale, G. T., Albarracín, D., & Seignourel, P. J. (2010). The effects of
source credibility in the presence or absence of prior attitudes:
Implications for the design of persuasive communication campaigns.
Journal of Applied Social Psychology,40(6), 1325–1356. https://doi.org/
10.1111/j.1559-1816.2010.00620.x
McCroskey, J. C., & Teven, J. J. (1999). Goodwill: A reexamination of the
construct and its measurement. Communication Monographs,66(1),
90–103. https://doi.org/10.1080/03637759909376464
McGinnies, E., & Ward, C. D. (1980). Better liked than right:
Trustworthiness and expertise as factors in credibility. Personality
and Social Psychology Bulletin,6(3), 467–472. https://doi.org/10.
1177/014616728063023
McKeever, B. W., McKeever, R., Holton, A. E., & Li, J.-Y. (2016). Silent
majority: Childhood vaccinations and antecedents to communicative
1366 Y. XU ET AL.

action. Mass Communication and Society,19(4), 476–498. https://doi.
org/10.1080/15205436.2016.1148172
Nyhan, B., Reifler, J., Richey, S., & Freed, G. L. (2014). Effective messages
in vaccine promotion: A randomized trial. Pediatrics,133(4), peds.-
2013–2365. https://doi.org/10.1542/peds.2013-2365
O’Keefe, D. J. (2013). The relative persuasiveness of different message
types does not vary as a function of the persuasive outcome assessed:
Evidence from 29 meta-analyses of 2,062 effect sizes for 13 message
variations. Annals of the International Communication Association,37,
221–249. https://doi.org/10.1080/23808985.2013.11679151
O’Keefe, D. J. (2015). Persuasion: Theory and research (3rd ed.). SAGE
Publications, Inc.
Popper, K. (1961). The logic of scientific discovery. Science Editions.
Pornpitakpan, C. (2004). The persuasiveness of source credibility: A critical
review of five decades’evidence. Journal of Applied Social Psychology,34
(2), 243–281. https://doi.org/10.1111/j.1559-1816.2004.tb02547.x
Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for
estimating indirect effects in simple mediation models. Behavior
Research Methods, Instruments, & Computers,36(4), 717–731.
https://doi.org/10.3758/BF03206553
Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling
strategies for assessing and comparing indirect effects in multiple
mediator models. Behavior Research Methods,40(3), 879–891.https://
doi.org/10.3758/BRM.40.3.879
Slater, M. D., & Rouner, D. (1996). How message evaluation and source
attributes may influence credibility assessment and belief change.
Journalism & Mass Communication Quarterly,73(4), 974–991.
https://doi.org/10.1177/107769909607300415
Stiff, J. B. (1986). Cognitive processing of persuasive message cues:
A meta-analytic review of the effects of supporting information on
attitudes. Communication Monographs,53(1), 75–89. https://doi.org/
10.1080/03637758609376128
Tormala, Z. L., Briñol, P., & Petty, R. E. (2006). When credibility attacks:
The reverse impact of source credibility on persuasion. Journal of
Experimental Social Psychology,42(5), 684–691. https://doi.org/10.
1016/j.jesp.2005.10.005
Uscinski, J. E., & Parent, J. M. (2014). American conspiracy theories.
Oxford University Press.
van der Linden, S. L., Clarke, C. E., & Maibach, E. W. (2015).
Highlighting consensus among medical scientists increases public
support for vaccines: Evidence from a randomized experiment. BMC
Public Health,15(1), 1207. https://doi.org/10.1186/s12889-015-2541-4
Walter, N., & Murphy, S. T. (2018). How to unring the bell: A
meta-analytic approach to correction of misinformation.
Communication Monographs,85(3), 423–441. https://doi.org/10.
1080/03637751.2018.1467564
Wang, Z., Walther, J. B., Pingree, S., & Hawkins, R. P. (2008). Health
information, credibility, homophily, and influence via the Internet:
Web sites versus discussion groups. Health Communication,23(4),
358–368. https://doi.org/10.1080/10410230802229738
WHO. (2019). Ten health issues WHO will tackle this year.https://www.
who.int/emergencies/ten-threats-to-global-health-in-2019
Wilson, E. J., & Sherrell, D. L. (1993). Source effects in communication
and persuasion research: A meta-analysis of effect size. Journal of the
Academy of Marketing Science,21(2), 101. https://doi.org/10.1007/
BF02894421
Yang, Q., & Beatty, M. (2016). A meta-analytic review of health informa-
tion credibility: Belief in physicians or belief in peers? Health
Information Management Journal,45(2), 80–89. https://doi.org/10.
1177/1833358316639432
HEALTH COMMUNICATION 1367