Content uploaded by Oscar H. Gandy, Jr.
Author content
All content in this area was uploaded by Oscar H. Gandy, Jr.
Content may be subject to copyright.
PREVENTING DOMESTIC VIOLENCE IN THE AFRICAN AMERICAN COMMUNITY:
THE IMPACT OF A DRAMATIC RADIO SERIAL
Robert Hornik
Oscar Gandy
Ricardo Wray
Jo Stryker*
The Annenberg School for Communication
University of Pennsylvania
3620 Walnut Street
Philadelphia PA 19104
Phone: (215) 898-7041
Fax: (215) 898-2024
Paper submitted to the Health Communication Division
50th International Communication Association Annual Conference
May 2000
“It’s Your Business” Impact Evaluation, 10/99 Version 1
* Robert Hornik Ph.D. and Oscar Gandy Ph.D. are Professors and Ricardo Wray and Jo Stryker are Ph.D.
candidates at the University of Pennsylvania. Respective e-mail addresses are: RHornik@asc.upenn.edu,
OGandy@asc.upenn.edu, Rwray@.asc.upenn.edu, and JStryker@pobox.asc.upenn.edu.
“It’s Your Business” Impact Evaluation, 10/99 Version 2
PREVENTING DOMESTIC VIOLENCE IN THE AFRICAN AMERICAN COMMUNITY:
THE IMPACT OF A DRAMATIC RADIO SERIAL
Abstract
The paper reports on the evaluation of “It’s Your Business,” a dramatic radio serial promoting domestic
violence prevention in the African American community. Despite commitments from radio stations in
four study sites, the series was played for the requested 12 weeks in only one city. Consequently, only
cross-sectional data from one city could be used to assess impact. Only nine percent of the sample could
confidently be called exposed, answering a recall question correctly and claiming to hear more than five
episodes. These moderately exposed respondents scored higher than non-exposed respondents on 21 out
of 27 anti-domestic violence beliefs and behaviors; ten differences were statistically significant.
However, for only two out of the 27 outcomes did the moderate exposure group display stronger
outcomes than a group who claimed exposure but could not recall much about the program at a
statistically significant level. We conclude that the association of moderate exposure and anti-domestic
violence outcomes was most likely an artifact of selective perception, and not a result of impact. The
negative result is attributed to the low levels of exposure, not that the series could not work at a higher
level of exposure. The evaluation points to the need to better understand how exposure can be achieved
to complement our work on developing messages.
“It’s Your Business” Impact Evaluation, 10/99 Version 3
1. Introduction
Concern with violence and crimes of passion has reached historic levels in the United States in response
to a number of widely reported stories in the news involving youngsters in schools, over committed
investors, and hate-filled racists (Newsweek, August 23, 1999; New York Times, May 29, 1999; Time,
August 9, 1999]. Yet, in the United States, violent crimes are more often committed within families than
among strangers (Straus and Gelles, 1990). Evidence suggests that family members or acquaintances
commit nearly half (47%) of all homicides (Alpert, Cohen and Sege, 1997). Among experts, domestic
violence is considered one of the most insidious and pervasive forms of violence in America today.
Defined as intentional violent or controlling behavior by a person in an intimate relation with the victim
(Alpert, Cohen and Sege, 1997), adult intimate-partner violence has been documented in every race,
religion, class and level of education (Straus and Gelles, 1986).
The prevalence of domestic violence is difficult to estimate due to the covert nature of the behavior, as
well as the norms that tolerate abuse. Published surveys suggest a wide range, with estimates of between
one and four million American women suffering from intimate partner abuse each year (Alpert, Cohen
and Sege, 1997). A national survey suggests that one in four (26%) American women of all races and
classes has at some time been a victim of domestic abuse (Lieberman Research, 1996). Scant information
about the prevalence of domestic violence in the African American community is available. One survey
estimated that sixteen percent of African American women has been physically abused by a husband or
partner within the last five years (Falik and Collins, 1996). Another study found that black women were
the victims in more than half (53%) of the violent deaths occurring in the homes of female victims
(Bailey, Kellerman, Somes, Banton, Rivara and Rushforth, 1997).
In recent years, the problem of violence has been recast from the legal to the public health domain (Cole
and Flanagin, 1998, 1999). By setting violence prevention as a public health priority, national and
international health agencies have signaled this shift, including the Centers for Disease Control and
Prevention (Foege, Rosenberg and Mercy, 1995), the World Health Organization (World Health
Assembly, 1996) and the American Medical Association (Marwick, 1998). This critical shift in
perspective transforms the research we conduct to better understand the problem (Rosenberg, Fenley,
Johnson and Short, 1997; Wallace and Wallace, 1998), and changes the character of the solutions that are
proposed to address it (American College of Physicians, 1998; Morbidity and Mortality Weekly Report,
1997).
“It’s Your Business” Impact Evaluation, 10/99 Version 4
Recent large-scale initiatives build on earlier efforts by the domestic violence prevention community to
offer shelter and services to victims, as well as to influence how courts, enforcement agencies, and other
policy actors respond to incidents of abuse (Rosenberg, et al., 1997). This emerging perspective adopts
an analytical approach informed by public health models that emphasizes the social and cultural contexts
of abuse, and highlights prevention in addition to treatment (Cole and Flanagin, 1998). Among other
approaches, such as mandatory alternative treatment for batterers, and professional training for service
providers who come into contact with victims, this perspective is increasingly reflected in communication
interventions that are designed to change beliefs, norms and social practices related to abuse (Rosenberg,
et al., 1997). The public health model also highlights the role of evaluation in testing the effectiveness of
new and alternative interventions (Rosenberg, et al., 1997).
Proponents for domestic violence prevention activities suggest that above and beyond the important
efforts to influence the behavior of abusers and victims, successful interventions must also address the
social norms, beliefs and practices related to domestic violence in the individuals living around and
interacting with those directly involved in abuse. Both advocates and researchers argue that transforming
social norms from those of silence and toleration to intervention and condemnation is essential to the
long-term reduction of domestic violence (Klein, Campbell, Soler and Ghez, 1997; Rosenberg et al.,
1997). It is in the light of this argument that a radio-based intervention called “It’s Your Business” was
designed. This article describes its evaluation.
2. Background
2.1. The origins of “It’s Your Business”
The “It’s Your Business” campaign built upon previous efforts by the Family Violence Prevention Fund
(Fund). This San Francisco-based agency founded in 1980 works to improve the health, judicial, law
enforcement and public policy responses to domestic violence. In recent years the Fund has developed a
series of media campaigns and community-mobilization efforts to promote community action and
prevention of domestic abuse (Klein et al., 1997)
1
. Among other achievements, the Fund developed a
nationally broadcast series of television spots entitled “There’s no excuse for domestic violence” in
collaboration with the Advertising Council. Focus group discussions held in relation to this campaign
suggested that a culturally specific initiative might be more effective than a general campaign on
domestic violence in reaching the African American community. With the sponsorship of the Advertising
Council, a group of freelance scriptwriters with experience in writing dramatic materials for an African-
1
The website for the Fund (www.fvpf.org) describes their mission and current activities.
“It’s Your Business” Impact Evaluation, 10/99 Version 5
American audience, and the Uniworld Group, Inc., the nation’s largest African American marketing
communications agency, a sophisticated radio campaign was developed.
Unlike traditional public service announcements, the Fund pursued the development of a social drama, an
innovative approach that had been used extensively outside the United States (Advocates for Youth,
1998; Nariman, 1993). These programs use entertaining formats such as the soap opera to deliver social
messages. The success of these interventions has caught the attention of behavioral researchers and
public health programs in developing countries over the past two decades (Maibach and Holtgrave, 1995;
Montgomery, 1990).
Sporadic public health messages have been included in television programs in the US (Montgomery,
1990). To the best of our knowledge, no dramatic series distributed at the national level has been
dedicated to the delivery of a specific health message. The cost of media production and the value of
broadcast time in comparison with those in developing countries may be one explanation for the lack of
experimentation in the United States (Advocates for Youth, 1998). The “It’s Your Business” campaign
developed a series of twelve long-form (ninety-second) public service announcements for radio. While
each episode in the series offered a specific educational message linked to the overall campaign theme,
the series was built around a dramatic story line designed to capture listener interest and involvement with
the characters and their circumstances. Such involvement was believed to be critical to the impact of the
campaign (Bandura, 1986; Hoffner, 1996; Rubin and Perse, 1988; Slater and Rouner, 1997).
The characters, relationships, interactions and outcomes dramatized throughout the series were selected
on the basis of assumptions about the relations between attitudes and behaviors that might be affected by
exposure to, and involvement with the series. These assumptions were based on the existing literature
about domestic violence, related investigations, and specific pre-test surveys. Included in the conceptual
framework were beliefs about becoming involved and taking actions with regard to domestic violence.
These beliefs included costs and benefits, or consequences of action and inaction in the face of knowledge
or suspicion regarding domestic violence. They also included assessments of social norms, and perceived
self-efficacy with regard to the primary goal of the project, increasing public willingness to talk about
domestic violence and specifically to offer support to women thought to be victims of abuse. Evidence
from research over the years has suggested that these types of beliefs serve as prominent cognitive
determinants and facilitators of a number of health behaviors (Bandura, 1986; Fishbein, Bandura,
Triandis, Kanfer, Becker, and Middlestadt, 1991; Fishbein and Middlestadt, 1995.).
“It’s Your Business” Impact Evaluation, 10/99 Version 6
The series featured a central character, Ma Bea, who was the host of a community affairs radio call-in
show that began each of the twelve campaign segments. At the beginning of each episode Ma Bea would
provide an “update” about a local domestic violence trial. This update served as a framing device that
would allow other “characters” in Ma Bea’s fictional audience to discuss, and thereby reinforce the
relevant lesson of that particular episode from the perspectives of their own lives. Different episodes
promoted specific elements of the overall theme. The first episode encouraged listeners to speak out
about domestic violence, to “air their dirty laundry,” true to the campaign theme-- “It is your business.”
Other episodes modeled characters offering help to a victim and providing information about where to
seek help. Within this fictional audience, a set of recurring characters made up an extended family that
struggled over the course of the series to convince a young woman in the family to leave her abusive
husband. Each episode concluded with an 800 number where listeners could call for information about
how to get involved in their communities.
The program was made available to a wide range of radio stations across the country through the
American Urban Radio Network. In some sites, particularly in the four cities where the evaluation was
undertaken, a special effort was undertaken to encourage local stations to broadcast the series.
2.2. Evaluation design
The evaluation was designed to determine the effectiveness of the “It’s Your Business” radio campaign in
achieving several cognitive and behavioral goals. At one level, the evaluation was concerned with
determining the impact of the campaign within the population of experimental cities, in comparison to
control cities. Cities were selected as potential sites for evaluation based on two criteria:1) having a
substantial African American population, and 2) having a single radio station that attracted a large
proportion of the African American listening audience. Four cities that matched these criteria were
chosen for the study -- Charlotte, North Carolina; Dayton, Ohio; Kansas City, Missouri; and Louisville,
Kentucky.
The design involved measurement by telephone survey at five times during the project period:
O1 O3 O5 X5 O6 X8 O8
“It’s Your Business” Impact Evaluation, 10/99 Version 7
In the above schematic, O signifies measurement wave, with the subscript indicating the month of
activity; X5 signifies the beginning of the radio series, and X8 the end of the series, three months later;
O1, O3 and O5 indicate three pre-broadcast surveys; O6 a mid-broadcast survey; and O8 an immediate
post-broadcast survey.
Successive cross-sectional samples were drawn. A random digit dial sample was purchased from Survey Sampling,
Inc. focusing on specific telephone exchanges where the likelihood was greater than 60% that an answering
respondent would be African American. Telephone interviews were undertaken by the DataStat survey company of
Ann Arbor, Michigan. All chosen households were contacted at least ?? times. Respondents were screened to
ensure inclusion of only those respondents who identified themselves as African American and as regular listeners
to the radio stations slated to broadcast the program. Thus the sampling strategy was designed to maximize the
likelihood of finding people who had access to the series. A power analysis suggested the need for surveying
approximately 600 respondents per wave before the broadcast (150 per city), and 900 respondents during and after
the broadcast (225 per city). The duration of the completed pre-broadcast telephone interview was approximately 15
minutes; the post-broadcast interview was about three minutes longer. The pre-broadcast instrument included
measures of demographics, experience with domestic violence, racial identification, media use, and a series of
measures designed to capture beliefs, attitudes, intentions and behaviors related to the goals of the campaign.
Exposure measures included a general item which was asked in the pre-broadcast surveys to assess the level of false
positives, the tendency to recall exposure to the program even before it was broadcast, and further questions were
added for the mid- and post-broadcast survey waves.
2.3 Plan of analysis
The preliminary design supported two analytical approaches: longitudinal and cross-sectional.
The longitudinal analysis would ask whether or not there was an effect on the population: it examined
what level of exposure was achieved and whether the overall population changed on the outcomes as a
result of the campaign. As indicated by the design schematic above, the longitudinal analysis was
intended to have four replications (one in each of the test cities), with comparisons focusing on
differences within each city between pre-, mid- and post-broadcast measures. As in all longitudinal field
experiments, confidence in a causal interpretation would be threatened by occurrence of an unplanned,
but relevant event in any city (Cook and Campbell, 1979), but the inclusion of four different cities was
expected to make such an event an unlikely explanation for any observed effects replicated in each site.
The cross-sectional component was designed to provide a post-hoc dose response analysis to complement
the longitudinal analysis, and would examine the association between reported exposure and attitudinal
“It’s Your Business” Impact Evaluation, 10/99 Version 8
and behavioral outcome measures at the post-broadcast wave only. A causal inference would be
contingent on observing a statistically significant and substantial association between exposure to the
program, and higher levels of targeted attitudes and behaviors, after possible confounders were
statistically controlled. Confidence in this inference depends upon confidence in our measures of
exposure as well as in self-report measures of attitudinal and behavioral response.
While this original research design would have allowed for fairly strong inferences about the nature of
effects of the “It’s Your Business” campaign, it could not ultimately be applied because there were many
difficulties associated with achieving adequate broadcast exposure to the series. Ultimately, the analyses
actually performed reflect the influence of a number of unplanned events.
Communication professionals acknowledge that interventions that are dependent upon the good will of
broadcasters are always at high risk and some argue for the need to pay for airtime (Advocates for Youth,
1998; Cosper, 1997).
2
The “It’s Your Business” campaign was at a particularly high level of risk
because: 1) it represented an unusual approach to public service advertising; and 2) it required an unusual
commitment of valuable time (90 second spots, 12 different segments, each to run a single week, over 12
weeks.).
Despite commitments from stations in each of the selected cities, no station actually presented the
campaign as originally agreed. In one city, the series received little airplay at all. In two other cities, the
stations played the series over the course of 12 days rather than 12 weeks. In only one city was the series
played for the requested 12 weeks. Even in this city, the series was shifted mid-way to a sister station due
to an unanticipated shift in program formats. The result was a dramatic reduction in the levels of
exposure that were possible. And, given the design of the series as an integrated serial drama, with a
cumulative impact based upon involvement with the story and its characters, maximal impact could not be
expected.
Given that three of the four test cities did not fully air the “It’s Your Business” series, we could not
execute our original plan of analysis, which would have included comparisons across all four cities, both
over time and post-campaign only. However, the lack of sufficient exposure across the test cities in airing
the campaign answers one of our original evaluation questions: there was no general impact of the
campaign on the populations across the four cities.
2
At the same time, the Advertising Council (1999) reported a 25% increase from 1997 to 1998 in donated radio time
for their campaigns, to a total of over $700 million. Radio alone made up almost 60% of total media support to the
“It’s Your Business” Impact Evaluation, 10/99 Version 9
The only other question we could ask is about the one city that did manage to air the series in its entirety,
Louisville. Was there evidence, there, that the campaign produced changes in beliefs, attitudes, norms, or
behavior with respect to domestic violence? Thus, our original plan of analysis was modified to include
both longitudinal and cross-sectional comparisons for Louisville only, to assess both population and
individual level changes within this city. In Louisville, 385 respondents were surveyed before the
broadcast, and 698 afterwards (we were able to supplement the planned Louisville ‘after’ sample, since
we were aware of the failure to fully broadcast the program in the other cities). The analyses that follow
were conducted using only data from this city.
3. Results
Looking at the data for Louisville, the first research question was as follows:
Research Question 1: Was the Louisville population as a whole affected by the “It’s Your Business”
Campaign?
In order to detect changes in the outcome variables of interest in the Louisville population from pre- to
post-campaign, it would be necessary for a substantial proportion of the target audience to have been
exposed to the series. Thus, a preliminary question to ask is whether or not the reach and frequency of
“It’s Your Business” was large enough that we might expect to detect changes at the population level, if
exposure was effective?
3.1 Exposure Measures
In addition to a general exposure question asked both pre- and post- broadcast,
3
respondents surveyed
post-broadcast were given a second chance to report exposure to the campaign with a more prompted
version of the question.
4
Anyone who reported exposure based on either of these questions were then
asked to recall what the series was about
5
, how many times each segment was heard, and how many
different episodes were heard.
Advertising Council totaling $1.2billion in 1998 alone.
3
“In the PAST MONTH, did you hear any dramatic advertisements against domestic violence featuring the
character Ma Bea?”
4
“In the PAST MONTH, did you hear a series of radio advertisements on that told the story of a domestic violence
situation? In the series, CHARLISE is the name of the victim and JAMES is the abuser; the tagline and theme were
IT'S YOUR BUSINESS; and the advertisements took the form of a series, like a soap opera?”
5
I am going to read 3 short descriptions of what the series featuring Ma Bea was about. Can you tell me which one
of them best describes what happens on the series? Was it MOSTLY about...
“It’s Your Business” Impact Evaluation, 10/99 Version 10
An examination of pre-broadcast exposures revealed a substantial number of false positives, or claimed
exposure to “any dramatic advertisements against domestic violence featuring the character Ma Bea on
Station --” before the series actually aired: nearly 16% of the respondents made such claims. While
significantly more respondents, 22%, claimed exposure after the campaign aired (Χ
2
= 5.87, p < .05), we
were reluctant to rely upon this somewhat faulty measure as the primary basis for claims of the
campaign’s impact.
Thus, we constructed a more refined measure of campaign exposure based on all of the exposure
measures, but which was only applicable to those who were interviewed post-broadcast. To try to
distinguish individuals who claimed exposure from those who we could be more confident were exposed
to the series, we constructed a more stringent measure of respondents’ exposure, based on three
conditions, all of which needed to be satisfied for a respondent to have qualified as exposed to the series
(See Table 1). A respondent was categorized as exposed if s/he 1) reported hearing the campaign
generally; 2) answered a multiple choice question about the plot of the series correctly; 3) reported
hearing an episode at least 3 times OR reported hearing at least 3 different episodes. Individuals who
reported exposure to the campaign, but could not correctly identify what the series was about, OR did not
report hearing at least 3 different episodes, OR did not report hearing one episode at least 3 times were
classified as ambiguously exposed. Individuals who did not claim to have heard any dramatic
advertisements against domestic violence were classified as not exposed to the campaign.
“It’s Your Business” Impact Evaluation, 10/99 Version 11
Table 1: Distribution of variables used in refined exposure measure
Question Values % N
In the PAST MONTH, did you hear any dramatic advertisements against domestic violence
featuring the character Ma Bea?
No
Yes
77.8%
22.2%
527
150
Asked only of those who said “No” above
In the PAST MONTH, did you hear a series of radio advertisements on that told the story
of a domestic violence situation? In the series, CHARLISE is the name of the victim and
JAMES is the abuser; the tagline and theme were IT'S YOUR BUSINESS; and the
advertisements took the form of a series, like a soap opera?”
No
Yes
83.7%
16.3%
448
86
Asked only of those who said “Yes” to either question above
I am going to read 3 short descriptions of what the series featuring Ma Bea was about. Can you tell me
which one of them best describes what happens on the series? Was it MOSTLY about...
1. A man who had been in prison and returned home and the problems he and his wife faced?
(OR)
2. How people in a family and community reacted to protect a woman from beating by her
husband? (OR)
3. A young child who had to tell a teacher in school the problems between her mother and
her mother's boyfriend? (OR)
None/Other/DK
14.4%
52.5%
18.2%
14.8%
34
124
43
35
The advertisements took the form of a series, like a soap opera. About how many different
episodes of the series did you hear?
1-2
3-5
6-9
13+
73.0%
23.3%
3.3%
.5%
157
50
7
1
About how many times did you hear each episode? 1
2
3-4
5+
39.0%
31.4%
19.3%
10.3%
87
70
43
23
Refined exposure measure
Includes either general exposure measure, story check, and how many different episodes.
“not exposed” = “No” to either general exposure measure
“exposed” = “yes” to either general exposure, “2” to story check, and “>=3” to # different
episodes or “>=2” to # times heard episode
Not Exposed
Exposed
Ambiguous (Not Included)
68.5%
9.2%
22.3%
455
61
148
“It’s Your Business” Impact Evaluation, 10/99 Version 12
Exposure to the campaign was extremely limited. By the criteria mentioned above, only 9.2% of the
‘after’ sample could be classified as being exposed to the campaign. It seems clear that the series did not
reach its audience as intended, with few individuals hearing many episodes of “It’s Your Business” on
more than one occasion. Barely 1% of respondents in Louisville reported hearing even half (6) of the
episodes. Only 61 individuals out of 698 surveyed could describe what the series was about, and claimed
to have heard an episode more than twice, or to have heard more than two episodes. This means that
there is a strong first conclusion: even in Louisville, “It’s Your Business” was not able to reach its
audience, except marginally. With such a small proportion of the total target audience being exposed to
the campaign beyond superficial levels, it is unrealistic to assume that the campaign could have had an
impact on the population, or that these effects could be detected.
3.2 Hypotheses
We were then left with asking the more limited question: if the program had been able to achieve
exposure at a higher level would it have been effective? This is a relevant question from a policy
perspective, since it allows us to consider whether an alternative strategy to assure exposure, for example,
buying time, might be worthwhile. Ultimately, the low level of exposure achieved by the campaign
allowed us to test only one evaluation hypothesis:
Research Hypothesis 1: The “It’s Your Business” series was effective among the few individuals
reached by the campaign.
In other words, was the series effective for individuals who were classified as exposed to the campaign,
based on the criteria discussed above? Did it lead them to express stronger anti-domestic violence beliefs,
attitudes, norms, or behavior with regard to intervention than individuals who were not exposed to the
campaign?
This hypothesis can only be tested by looking at differences among individuals measured after the
campaign who claimed more or less exposure to the series, estimating the association between exposure
and outcome. The primary competing hypothesis to a causal interpretation of these associations is one of
selective exposure: that individuals with pre-existing interest in the issue, or with some other
characteristic, were more likely to be exposed and recall exposure to the campaign as well as to report
anti-domestic norms, beliefs, attitudes, and behavior. If so, any observed differences between the
exposed and non-exposed groups would not be attributable to the campaign.
“It’s Your Business” Impact Evaluation, 10/99 Version 13
To increase our confidence that any observed differences between those classified as exposed to the series
and those not exposed were genuine, we made several assumptions about the patterns of results we would
expect to see if the campaign really did have an impact, taking advantage of the fact that we had three
exposure groups: (1) those who passed our stringent test of exposure and who we were reasonably sure
had been somewhat exposed to the series, although in most cases in a quite limited way – we’ll call them
‘moderately exposed’; (2) those who had claimed exposure but did not satisfy the stringent criterion for
exposure, and whom we classified as ‘ambiguously exposed’; and (3) the largest group, who made no
claim of exposure, called ‘no exposure’.
Hypothesis 1a: Individuals in the moderate exposure category should have higher anti-domestic
violence outcomes than individuals in the non-exposed category.
This is a basic test of the campaign’s impact, and does not address the primary competing hypothesis of
selective exposure. However, we needn’t worry about selective exposure if there is no evidence that
respondents with higher levels of reported exposure to “It’s Your Business” had stronger anti-domestic
violence outcomes than those who did not report exposure.
To test the primary competing hypothesis, we proposed that Hypothesis 1a above was a necessary but not
sufficient condition for demonstrating effects. Additionally, it would have to be established that:
Hypothesis 1b: Individuals in the moderate exposure category should have higher anti-domestic
violence outcomes than individuals in the ambiguously exposed category.
We are not necessarily claiming that the ‘ambiguously exposed’ individuals were falsely reporting
exposure, only that their exposure to the series was so limited that it is unrealistic to expect that it could
have produced an effect. For example, if a respondent cannot provide a general description of the series
in response to a multiple choice question, it is unlikely that he or she was paying enough attention so that
his/her attitude about domestic violence could have been affected.
A first test of selective exposure, then, is to establish that those who were definitively exposed to the
campaign had clearer anti-domestic violence outcomes than did those either mistakenly reporting
exposure or scarcely exposed to the series. Both of these groups were claiming exposure to the series. If
the association between exposure and outcomes was merely an artifact of prior interest in the issue, one
might hypothesize that both groups would be similarly affected, and one would expect them to look the
“It’s Your Business” Impact Evaluation, 10/99 Version 14
same on the outcomes. In contrast, if there were an effect of hearing the program then the moderately
exposed group would be expected to have higher outcomes. While this would not completely eliminate
the possibility of selective exposure, it would certainly help in the process. Once we could establish the
existence of such differences, we could then determine if there were other background variables, such as
demographic variables, including age and education; prior experience with domestic violence; and
different forms of racial identification that somehow accounted for the exposure-outcome relationship.
3.3 Outcome Measures
Several sets of outcome variables were identified for analysis on the basis of the literature and preliminary
surveys. The measures included the following broad categories: 1) general beliefs and opinions about
domestic violence; 2) beliefs about talking with victims; 3) intentions regarding talking with victims; 4)
actual talk with victims; 5) beliefs and opinions regarding talk with others about domestic violence; and
6) actual talk with others about domestic violence. The results of the analysis that follow are divided into
the three categories of outcomes -- general beliefs about domestic violence, and outcomes relating to the
two behaviors of talking to a victim, and general talk condemning domestic violence. All of the beliefs
were measured on a 1 to 5 scale, ranging from strongly disagree to strongly agree. All of the intentions
and behaviors were dichotomous measures.
3.4 Analysis Procedures
To test Hypothesis 1a and b, two separate sets of analyses were conducted, one for presentation purposes,
and one to conduct statistical tests of the hypotheses. For presentation purposes, the means or percentages
(in the case of dichotomous variables) for the three separate exposure groups are presented in the table
below. However, because none of the outcome variables were truly interval (some of the beliefs were
measured on a scale ranging from 1 to 5; otherwise the values were dichotomous), the more appropriate
statistical test relied upon the gamma statistic, a measure of ordinal association. For tests using gamma,
two separate tests were run: one comparing the moderate exposure group with the non-exposed group,
and one comparing the moderate group with the ambiguously exposed group. For any given outcome
variable, if gamma was statistically significant and in the right direction for the moderate/non-exposed
comparison (with the moderate exposure group displaying higher anti-domestic beliefs, attitudes, or
behavior), Hypothesis 1a would be supported. If gamma was statistically significant and in the right
direction for the comparison of the moderate and ambiguously exposed groups Hypothesis 1b would be
supported. For any given outcome variable, if both hypotheses 1a and b were supported, the general
Research Hypothesis would be supported.
“It’s Your Business” Impact Evaluation, 10/99 Version 15
3.5 Results
Research Hypothesis 1a
First, there is evidence consistent with an effect considering the comparison between the moderate and no
exposure groups only, or Research Hypothesis 1a (See Table 2). For 10 of the 27 outcomes, respondents
in the moderate exposure category expressed stronger anti-domestic beliefs, intentions, and/or behaviors
than those who were not exposed to the campaign at a statistically significant level (p < .05). If we make
a correction for the multiple tests conducted, and assume that 2 of the 27 tests produced a statistically
significant result by chance, we would still find 8 statistically significant results. Moreover, if we ignore
statistical significance in favor of examining the overall trend of the results, we find that Research
Hypothesis 1a is supported for 21 of the 27 outcomes; this number far exceeds what we would expect by
chance (about 13 or 14).
However, before we can conclude that the “It’s Your Business” campaign was effective, at least for those
who were reasonably exposed to the campaign, we must first address the concerns which prompted
Research Hypothesis 1b: that the results reported above represent actual differences due to the campaign,
rather than differences due to selective exposure/attention.
Research Hypothesis 1b
This analysis suggests that the differences between the “moderate” and “no” exposure groups are more
likely a function of selective exposure rather than the campaign itself. We would expect that genuine
effects of the campaign would be reflected in the fact that individuals who knew what the series was
about, and who heard multiple episodes or heard the same episode multiple times (the moderate exposure
category) should have stronger anti-domestic violence outcomes than individuals who claimed to have
heard the series, but either could not provide a minimal description, or did not report multiple exposures
(the ambiguously exposed). This was not the case.
For only 2 of the 27 outcomes did the moderate exposure group display stronger anti-domestic outcomes
than the ambiguously exposed at a statistically significant level (p < .05). Since we conducted so many
tests, we cannot be confident that these results are not a function of chance. Looking at the overall trend
of results while ignoring statistical significance, we see 18 of 27 outcomes for which the moderately
exposed had greater anti-domestic violence beliefs, intentions or behavior. While this is a clear majority
of outcomes, it is only about 4 more, and not significantly more, than we might expect to see due to
“It’s Your Business” Impact Evaluation, 10/99 Version 16
chance alone (assuming that chance would produce stronger anti-domestic violence outcomes among the
moderately exposed about half (13 or 14 times out of 27) of the time).
These results suggest that there is little reliable difference with respect to anti-domestic violence beliefs,
intentions, or behavior among individuals whom we believe to be moderately exposed to the “It’s Your
Business” and those who were less exposed; Research Hypothesis 1b was not supported.
“It’s Your Business” Impact Evaluation, 10/99 Version 17
Table 2: Comparisons between Moderate Exposure, No Exposure, and Ambiguous Exposure
Outcomes
Grand
Mean/%
SE
NE
*
Mean/%
SE
(N)
ME
*
Mean/%
SE
(N)
AE
*
Mean/%
SE
(N)
Gamma
Value
ME vs.
NE
Approx.
sign
ME>NE
*= ME
> NE at
p < .05
Gamma
Value
ME vs.
AE
Approx.
sign
ME>AE
*=ME >
AE at p
< .05
General Beliefs about Domestic Violence (1-5 scale—1 strongly disagree to 5 strongly agree)
Domestic violence is one of the most
important problems in your community
3.40
.08
3.13
.07
(446)
3.49
.20
(61)
3.57
.13
(148)
.203
.055
Yes .022
.856
Yes
You do not like talking with others about their
private lives
3.35
.08
3.16
.07
(450)
.372
.20
(61)
3.19
.13
(148)
-.259
.008
Yes* -.258
.017
Yes*
You don't really know what you can do to help
reduce domestic violence in your community
2.99
.09
2.72
.08
(450)
3.36
.21
(61)
2.88
.14
(148)
-.306
.001
Yes* -.241
.023
Yes*
Beliefs about Talking to a Victim of Domestic Violence (1-5 scale—1 strongly disagree to 5 strongly agree)
Talking to an abused woman will help her
improve her situation
4.18
.06
4.19
.06
(453)
4.20
.16
(61)
4.16
.10
(148)
.045
.710
Yes .008
.950
Yes
You know how to begin a conversation with an
abused woman about her situation
3.73
.08
3.52
.07
(450)
3.92
.19
(61)
3.76
.12
(148)
.203
.049
Yes* .038
.749
Yes
If a woman's partner found out you spoke to
her, he might abuse her more
2.11
.07
2.02
.06
(450)
2.15
.17
(61)
2.15
.11
(148)
-.092
.389
Yes -.046
.695
Yes
If you spoke to a woman about her abuse she
might get angry with you
2.27
.07
2.33
.06
(450)
2.18
.17
(61)
2.31
.11
(148)
.103
.339
No .093
.441
No
You would ask a woman about her abuse even
if you thought it would make her feel badly
3.59
.08
3.24
.07
(451)
3.90
.20
(61)
3.63
.13
(147)
.343
.001
Yes* .112
.344
Yes
People who are important to you expect you to
talk to an abused woman about her situation
3.76
.08
3.53
.07
(447)
3.89
.19
(61)
3.87
.12
(148)
.181
.080
Yes -.021
.860
No
“It’s Your Business” Impact Evaluation, 10/99 Version 18
Intentions to Talk to a Victim/ Say the Right Thing (% who said yes)
Imagine that you suspect a woman is being physically abused by her
partner BUT SHE HAD NEVER TALKED TO YOU ABOUT IT. Would you raise the issue with her. .
A. If she were a co-worker? 74.7%
.02
64.3%
.02
(443)
78.7%
.06
(61)
81.1%
.04
(148)
.344
.015
Yes* -.074
.697
No
B. if she were a neighbor who you didn't
know very well?
44.3%
.03
37.6%
.02
(449)
47.5%
.06
(61)
44.6%
.04
(148)
.200
.149
Yes .059
.698
Yes
C. if she were a stranger you noticed in a
supermarket?
19.9%
.02
19.5%
.02
(447)
21.3%
.05
(61)
18.9%
.03
(148)
.057
.740
Yes .074
.697
Yes
* NE = not exposed ME= moderately exposed AE = ambiguously exposed
“It’s Your Business” Impact Evaluation, 10/99 Version 19
Table 2(cont.): Comparisons between Moderate Exposure, No Exposure, and Ambiguous Exposure
Outcomes
Grand
Mean/%
SE
NE
*
Mean/%
SE
(N)
ME
*
Mean/%
SE
(N)
AE
*
Mean/%
SE
(N)
Gamma
Value
ME vs.
NE
Approx.
sign
ME>NE
*= ME
> NE at
p < .05
Gamma
Value
ME vs.
AE
Approx.
sign
ME>AE
*=ME >
AE at p
< .05
Suppose you are having a conversation with a friend who is being abused by her husband or boyfriend. Please tell me if you would
say any of the following statements to her about her situation.
A. It's not your fault. There is no excuse for
his hitting you.
87.4%
.02
82.7%
.02
(451)
90.2%
.05
(61)
89.2%
.03
(148)
.314
.081
Yes .053
.832
Yes
B. You can't make a big deal about it, he
probably had a hard day.
3.31%
.01
3.54%
.01
(451)
1.63%
.02
(61)
4.76%
.02
(147)
-.376
.304
Yes -.500
.194
Yes
C. There are people in the community who
you can turn to for support
92.5%
.02
88.4%
.01
(455)
96.7%
.04
(61)
92.5%
.03
(147)
.591
.004
Yes* .409
.183
Yes
D. Stop doing whatever is making him so
angry.
9.59%
.02
12.0%
.02
(450)
6.56%
.04
(61)
10.2%
.03
(147)
-.320
.127
Yes -.236
.367
Yes
Behavior- Talking to a Victim (% who said yes)
Asked only of people who said they had strong reason to believe that a woman they knew had been physically abused by her
husband or boyfriend In the PAST THREE MONTHS
Did you talk to other people about her
situation?
77.2%
.04
64.6%
.04
(127)
90.9%
.10
(22)
76.2%
.06
(63)
.692
.002
Yes* .515
.077
Yes
Some people have a chance to talk to victims
and others don't. How about you – did you
talk to the woman about her situation?
74.4%
.04
70.9%
.04
(127)
68.2%
.09
(22)
84.1%
.06
(63)
-.063
.802
No -.424
.151
No
Who first brought up the subject, you or the
woman?
(% saying respondent)
51.0%
.05
54.4%
.05
(90)
46.7%
.13
(15)
51.9%
.07
(52)
-.155
.578
No -.105
.720
No
Beliefs about General Talk Condemning Domestic Violence (1-5 scale—1 strongly disagree to 5 strongly agree)
If more people told each other they
disapproved of domestic violence, it would go
a long way to stopping the abuse
4.24
.07
3.96
.07
(452)
4.48
.18
(61)
4.28
.11
(148)
.266
.014
Yes* .039
.780
Yes
It is important for men to talk with each other
about domestic violence in order to solve the
problem
4.36
.07
4.13
.07
(449)
4.44
.16
(61)
4.51
.11
(148)
.252
.039
Yes* -.018
.908
No
There's no point in arguing with people about 3.89 3.57 4.18 3.92 -.323 Yes* -.173 Yes
“It’s Your Business” Impact Evaluation, 10/99 Version 20
domestic violence because talking won't
change what people do
.09 .08
(454)
.21
(61)
.13
(148)
.004 .195
People who are important to you expect you to
say that domestic violence is wrong
4.34
.07
4.27
.06
(453)
4.21
.16
(61)
4.53
.11
(148)
-.007
.957
No -.269
.097
No
You would feel badly if someone said
something which excused domestic violence
and you kept quiet
3.55
.09
3.58
.08
(452)
3.51
.22
(61)
3.55
.14
(148)
-.002
.986
No -.040
.750
No
Behavior- General Talk Condemning DV (% saying yes)
In the PAST MONTH did you talk with anyone
about domestic violence?
Asked only of those who said “yes”
56.0%
.03
48.8%
.02
(455)
59.0%
.06
(61)
60.1%
.04
(148)
.204
.133
Yes -.023
.881
No
Were any of these conversations about
domestic violence concerning something you
heard on the radio?
35.9%
.03
28.3%
.03
(219)
44.4%
.08
(36)
34.8%
.05
(89)
.339
.075
Yes .199
.323
Yes
In total, how many conversations about
domestic violence did you have in the PAST
MONTH?
(% saying more than 2)
40.4%
.03
46.4%
.03
(220)
44.4%
.08
(36)
30.3%
.05
(89)
-.039
.830
No .295
.145
Yes
* NE = not exposed ME= moderately exposed AE = ambiguously exposed
4. Summary and Conclusions
In many ways this was an innovative and important program, breaking ground in the creation of public
service announcements. Nonetheless, our evaluation leads to two negative summary statements. The
“It’s Your Business” Campaign was not able to convince the target radio stations to play the program as
scheduled; as a result in three cities there was virtually no exposure. In the fourth, where it was broadcast
in only approximate conformity with the proposed schedule, there was minimal exposure, and less than
10% of the people who claimed to be listeners to the stations where it was broadcast established that they
were credibly exposed. Second, even among those who were exposed to the program, the apparent
advantage they held in desirable beliefs, attitudes, norms and behaviors, was most likely an artifact of
selective perception. The exposure they achieved was not likely to have been enough to affect these
outcomes.
So, how can these results be interpreted? On the one hand, they add one more reason to be skeptical about
the promise of donated broadcast time as a means of achieving substantial exposure to social messages.
In this situation, the Fund obtained the endorsement of and some distribution assistance from the
Advertising Council, and the American Urban Radio Network. The programs were innovative, and
“It’s Your Business” Impact Evaluation, 10/99 Version 21
produced with a high level of professional skill by a group of experienced script writers and by the
Uniworld advertising agency. The Fund directly and repeatedly contacted public service directors in each
of the four local markets and obtained promises of participation. Nonetheless, the schedule of broadcasts
needed to make the program work could not be achieved.
While it is difficult to know for sure why the planned schedule was not realized, the simplest explanation
may be a financial one. The stations were asked to donate 90 seconds of ‘good’ time several times a day,
every day for 12 weeks, and that was more than they were able to do. Other complementary explanations
are also viable: the 90 second format and the use of a continuing story line was unusual and difficult to
program; there wasn’t enough local support for the broadcasts to complement the requests from outside
groups; the delivered product was less appealing to local stations than it had promised to be. Still, if an
advocacy agency wants to consider unpaid public service announcements as a primary channel to its
audience these results (and others) would have to leave an agency wary.
The second issue is what to conclude about the failure to find evidence that the exposure which was
achieved affected behavior. Here we are more reluctant to come to a clear conclusion. We cannot say
that “if only they could have bought time, they would have been effective.” We don’t have evidence for
that. But, contrarily, we are not ready to go in the opposite direction, either. The great majority of 61
individuals whom we classified as “moderately” exposed were not exposed to the series with the reach or
frequency intended by the campaign’s creators, and they were not exposed in a social context where the
program was the subject of local discussion. Only eight respondents in the entire Louisville ‘after’
sample of 663 individuals claimed that they had heard even half of the total episodes. Because so few
people heard the program, there was no likelihood that it would be the subject of discussion among social
networks; there could be no multiplication and reinforcement of messages that was central to the
underlying model of effect. Our conclusion that there was no effect on the moderately exposed group
does not preclude the possibility that such effects would have been seen at higher individual doses, or if
exposure had been widespread.
Thus, this evaluation points only to the failure to achieve exposure as the explanation for the failure of the
It’s Your Business series. We suspect that, too often, the failures of mass mediated public health
interventions have been attributed to faulty message design, or to the supposed ability of communication
to affect only awareness but not persuade people to adopt new beliefs or behaviors. Often, such failures
ought to be attributed first to a failure to achieve adequate exposure; in reality very few individuals were
ever exposed to the messages with any substantial frequency (Hornik, 1996; Hornik, 1998). Public health
“It’s Your Business” Impact Evaluation, 10/99 Version 22
communicators have become very able at developing good messages which have some hope of
influencing their audiences, but they may be far less effective at achieving the exposures needed. Perhaps
we need better work on understanding how exposure can be achieved, and how to maximize effects from
limited exposure, to complement our work on developing messages and their theory. We cannot
understand the real-world conditions under which messages might be effective if individuals are never
exposed to them.
“It’s Your Business” Impact Evaluation, 10/99 Version 23
REFERENCES
The Advertising Council. 1999. Media support – 1998 calendar year. New York: The Advertising
Council.
Advocates for Youth. 1998. The use of mainstream media to encourage social responsibility: The
international experience. Menlo Park CA: Kaiser Family Foundation.
Alpert EJ, Cohen S and Sege RD. 1997. “Family violence: An overview.” Academic Medicine, 72
(1,Suppl.): S3-6.
American College of Physicians. 1998. “Firearm injury prevention: Clinical guidelines.” Annals of
Internal Medicine, 128(3): 236-41.
Bailey JE, Kellerman AL, Somes GW, Banton JG, Rivara FP and Rushforth NP. 1997. "Risk factors for
violent death of women in the home." Archives of internal medicine, 157, April 14, 1997.
Bandura A. 1986. Social foundations of thought and action: A social cognitive theory. Englewood
Cliffs, NJ: Prentice-Hall.
Cole TB and Flanagin A. 1998. “Violence – ubiquitous, threatening and preventable.” The Journal of
the American Medical Association, 280(5): 468.
Cole TB and Flanagin A. 1999. “What can we do about violence?” The Journal of the American
Medical Association, 282(5): 481.
Cook TD and Campbell DT. 1979.Quasi-experimentation design and analysis issues for field settings.
Boston: Houghton Mifflin Co.
Cosper D. 1997.“Shock value.” Print, 51(6):38-41.
Falik MM and Collins KS (Eds.). 1996. Women's health: The Commonwealth Fund Survey. Baltimore
MD: The Johns Hopkins University Press.
Fishbein M, Bandura A, Triandis HC, Kanfer FH, Becker MH and Middlestadt SM. 1991. Factors
influencing behavior and behavior change. Final report. Theorists’ Workshop. Washington DC.
“It’s Your Business” Impact Evaluation, 10/99 Version 24
Fishbein M and Middlestadt S. 1995. “Noncognitive effects on attitude formation and change: Fact or
artifact?” Journal of Consumer Psychology, 4(2): 181-202.
Hoffner C. 1996. “Children’s wishful identification and parasocial interaction with favorite television
characters.” Journal of Broadcasting and Electronic Media, 40:389-402.
Hornik, R. 1996. "Health Communication: making sense of contradictory evidence." Paper presented at
the Annenberg Public Policy Center Public Health Communication Meeting, 1997.
Hornik, R. and Woolf K. 1998. "Complementing message theory with exposure theory: Is it worth it?"
Paper presented at the Kentucky Health Communication Conference, April, 1998.
Klein E, Campbell J,Soler E and Ghez M. 1997. Ending domestic violence. Thousand Oaks CA: Sage.
Lieberman Research Inc.. 1996. Domestic violence advertising campaign tracking survey (Wave IV).
Conducted for the Advertising Council and the Family Violence Prevention Fund, July-October1996.
Maibach E and Holtgrave DR.1995. “Advances in public health communication.” Annual Review of
Public Health. 16:219-238.
Marwick C. 1998.“Domestic violence recognized as a world problem.” The Journal of the American
Medical Association, 279(19):1510.
Montgomery K. 1990.“Promoting health through entertainment television.” In C Atkin and L Wallack
(Eds.) , Mass communication and public health. Newbury Park CA: Sage.
Morbidity and Mortality Weekly Report. 1997. “Perceptions of child sexual abuse as a public health
problem – Vermont September 1995.”46(34): 801-3.
Nariman HN. 1993. Soap operas for social change, toward a methodology for entertainment-education
television. Westport and London: Praeger Publishers.
Newsweek. 1999.“What must be done.” August 23, p22.
“It’s Your Business” Impact Evaluation, 10/99 Version 25
New York Times. 1999. “After Littleton, a change in principals' worlds.” May 29, pA1.
Rosenberg ML, Fenley MA, Johnson D and Short L. 1997. “Bridging prevention and practice: Public
health and family violence.” Academic Medicine, 72 (1, Suppl.): S13-18.
Rubin AM and Perse EM. 1988.“Audience activity and soap opera involvement.” Human
Communication Research, 14(2): 246-268.
Sabido M. 1981. Towards a social use of soap operas. Paper presented at the International Institute of
Communication, Strasbourg, France.
Slater MD and Rouner D. 1997. “The processing of narrative fiction containing persuasive content about
alcohol use: Effects of gender and outcome.” Presented to the Annual Conference of the International
Communication Association, Information Systems Division, Montreal, Canada, May.
Straus MA and Gelles RJ. 1986. “Societal changes and changes in family violence from1975 to 1985 as
revealed by two national surveys. Journal of Marriage and the Family, 48(3): 465-79.
Straus MA and Gelles RJ. 1990. Physical violence in American families: Risk factors and adaptations
to violence in 8,145families. New Brunswick NJ: Transaction Publishers.
Time. 1999. “A portrait of the killer: What triggered the rampage? A 1995 document may offer clues to
the mystery of Mark Barton.” August 9, 154(6): 22.
Wallace D and Wallace R. 1998. “Scales of geography, time and population: The study of violence as a
public health problem.” American Journal of Public Health,88(12): 1853-8.
