Content uploaded by Clea McNeely
Author content
All content in this area was uploaded by Clea McNeely on May 20, 2016
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=hjsr20
Download by: [University of Tennessee, Knoxville] Date: 20 May 2016, At: 07:55
The Journal of Sex Research
ISSN: 0022-4499 (Print) 1559-8519 (Online) Journal homepage: http://www.tandfonline.com/loi/hjsr20
Reliability of self‐reported contraceptive use and
sexual behaviors among adolescent girls
R. Sieving , W. Hellerstedt , C. McNeely , R. Fee , J. Snyder & M. Resnick
To cite this article: R. Sieving , W. Hellerstedt , C. McNeely , R. Fee , J. Snyder & M. Resnick
(2005) Reliability of self‐reported contraceptive use and sexual behaviors among adolescent
girls, The Journal of Sex Research, 42:2, 159-166, DOI: 10.1080/00224490509552269
To link to this article: http://dx.doi.org/10.1080/00224490509552269
Published online: 11 Jan 2010.
Submit your article to this journal
Article views: 87
View related articles
Citing articles: 27 View citing articles
Reliability
of
Self-Reported Contraceptive
Use and
Sexual Behaviors
Among Adolescent Girls
R. Sieving,
W.
Hellerstedt,
C.
McNeely,
R. Fee, J.
Snyder,
and M.
Resnick
University
of
Minnesota
This study examines
two
issues relevant
to
adolescents' self-reported sexual
and
contraceptive
use
behaviors: reliability
of partner-referent reports versus
3- and
6-month reports,
and
test-retest reliability
of
reports
completed over
a
2-week peri-
od.
Data are from
196 13- to
18-year-old girls recruited into this study while they sought reproductive care from health clin-
ics
in a
metropolitan area.
All
participants reported having
had
sexual intercourse during
the
past
6
months. Twice over
a
2-week interval, participants completed
the
same paper-and-pencil surveys.
The
survey presented questions about sexual
behavior
and
contraceptive
use
using
3
sequential frames
of
reference:
within
the
past
6
months, within
the
past
3
months,
and
by
specific sexual partners
in the
past
6
months. Findings demonstrate that adolescent girls
can
reliably report sexual
behavior
and
contraceptive
use
over
a
6-month interval. Study findings have implications
for
future research utilizing
ado-
lescents' self-reported sexual
and
contraceptive
use
behaviors.
Self-report data
on
contraceptive
use and
sexual behav-
ior
are
used
to
identify adolescents
at
risk
for
unintended
pregnancy
and
sexually transmitted diseases (STDs)
and
to evaluate programs aimed
at
preventing these outcomes.
The most efficient, feasible
and
commonly used approach
to studying sexual behaviors
is
self-reported recall
of
those behaviors over some time frame. Errors
in self-
report
can
bias prevalence estimates
of
high-risk behav-
iors
and
thus lead
to
misclassification
of
individuals
at
risk, thereby hampering prevention efforts. Important
intervention components
may be
overlooked
or
overem-
phasized
in
situations where measurement error leads
to
biased estimates
of
relationships between variables
(Catania, Chitwood, Gibson
&
Coates, 1990). Thus,
min-
imizing error related
to the
measurement
of
contraceptive
and sexual behaviors
is of
interest
to
both adolescent
health service providers
and
researchers.
Many questions related
to
adolescents' self-reported
sexual behaviors
and
contraceptive
use
remain unan-
swered.
For
example,
it is
unclear what time intervals
ado-
lescents
can
recall
in a
reliable fashion (e.g., 3-month peri-
od, 6-month period)
and
whether using partner-specific
assessments decreases bias
in
reporting
of
sexual
and con-
traceptive behaviors over
a
given time interval.
Measurement error related
to
self-reports
may
stem from
respondents themselves
or
measurement instruments used.
Respondent variables that influence
the
task
of
providing
information include those related
to
memory
and
recall
(Catania, 1999).
The
length
of the
recall period
may
influ-
ence
the
consistency
and
accuracy
of
respondents' reports
This study
was
supported
by
grant #U48/CCU513331,
the
National Teen
Pregnancy Prevention Research Center, from
the
Centers
for
Disease Control
and
Prevention.
Address correspondence
to
Renee Sieving, Ph.D.,
RNC,
University
of
Minnesota School
of
Nursing,
5-160
Weaver-Densford Hall,
308
Harvard Street
S.E., Minneapolis,
MN,
55455; e-mail: sievi001@umn.edu.
(Catania
et al.,
1990). Adolescents
may
base short-term
reports
on
counts
of
recent events,
but
they
may use
esti-
mation
to
recall over longer periods
of
time (McFarlane
&
St. Lawrence, 1999). Although research consistently docu-
ments that reliability
of
reported sexual behaviors decreas-
es with increasing duration
of
recall period (Catania
et al.,
1990;
Kauth,
St.
Lawrence,
&
Kelly, 1991),
few
studies
have explicitly compared reliability
of
teenagers'
self-
reports using short
and
longer-term recall periods
(for an
exception,
see
McFarlane
& St.
Lawrence, 1999).
The frequency
of
behaviors being assessed
may
also
influence
the
consistency
and
accuracy
of
adolescents'
reports.
A
study with
12- to
19-year-olds (McFarlane
& St.
Lawrence,
1999)
showed that estimates
of
yearly behavior
based
on
2-week, 2-month,
or
12-month recall periods
pro-
duced discrepant conclusions about adolescents' sexual
activity.
In
this study, discrepancies between recall periods
were larger with estimates
of
relatively frequent behaviors,
such
as
condom-protected sexual intercourse, than with
infrequent behaviors, such
as
unprotected oral
and
anal
sex.
Complex patterns
of
sexual activity
may
also influence
adolescents' ability
to
recall behavior.
For
example, indi-
viduals
in
monogamous relationships with
a
routine
pat-
tern
of
sexual activity
may
give highly reliable estimates,
but adolescents with more complex sexual patterns (e.g.,
multiple sexual partners
and
contraceptive
use
patterns
that vary between partners)
may
have more difficulty
in
providing overall behavioral estimates.
It is
assumed that
partner-by-partner estimates simplify respondents' task
and thus improve
the
consistency
and
accuracy
of
reports,
but
few
studies have examined consistency between
ado-
lescents' partner-based estimates
and
temporal estimates
of sexual
and
contraceptive behavior (Catania
et al.,
1990).
Another respondent variable that
may
influence
self-
reports
of
sexual activity relates
to
self-presentation. Fear
of reprisal, social desirability,
and
other personal factors
The Journal
of
Sex Research Volume
42,
Number
2, May
2005:
pp.
159-166159
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
160
Reliability
of
Teens'
Contraceptive Self-Reports
may bias adolescents' reports
of
sexual behavior. Young
people
who
fear reprisal
may
under-report sexual activity,
or others
who
want
to
project
an
adult image
may
overes-
timate sexual activity. Adolescents
who
seek approval
of
health care providers
may
over-report their contraceptive
use (Catania,
1999;
McFarlane
& St.
Lawrence,
1999;
Shew
et al.,
1997).
Sex and
ethnic affiliation
may
influence
self-presentation since sexual norms
and
values
can
vary
between genders
and
ethnic groups. Data from
a
nation-
wide sample
of
sexually experienced teens (Upchurch,
Lillard, Aneshensel,
&
Fang, 2002) showed that girls were
less inconsistent than boys
in
reports
of
their sexual behav-
iors;
however, patterns
of
response inconsistency
did not
differ between teens from different racial
and
ethnic
groups. Several studies suggest that self-presentation bias
can
be
minimized under conditions that allow more priva-
cy than face-to-face interviewing typically permits, such
as
self-administered questionnaires
and
computer-assisted
surveys (Romer
et al., 1997;
Turner, Lessler,
&
Devore,
1992;
Turner
et al.,
1998).
Instrument variables, including question order, question
structure
and
terminology,
may
also influence
the
task
of
providing information about sexual behavior.
It is com-
monly assumed that more sensitive questions should
be
asked later
in a
survey; respondents become gradually
desensitized
to
more intimate items. Another consideration
in ordering questions
is
that respondents
may
lose interest
in answering questions over
the
course
of
time, leading
to
greater measurement error
at the end of
long surveys.
Self-
administered questionnaires
and
computer-assisted surveys
that allow respondents
to
pace themselves,
as
well
as
inter-
view methods tailored
to
individuals' sexual histories,
may
be less susceptible
to
order effects than traditional inter-
view methods (Catania
et al., 1990;
Hearn, O'Sullivan,
&
Dudley, 2003). Survey instruments that provide detailed
explanations
of
sexual terms
and
frame sexual questions
using non-judgmental
and
developmentally appropriate
wording
may
also help minimize self-report bias (Catania
et
al., 1990;
Hearn
et al.,
2003;
Romer
et al.,
1997).
Pre-
testing
an
instrument with members
of a
target population
-
in
which
the
focus
is on
respondents' perceptions
of the
questions
and the
tasks
of
survey completion
-
provides
valuable information
on
appropriate survey length, accept-
ability
of
question order
and
structure,
and
understanding
of sexual terms employed (Romer
et al.,
1997).
The current study
was
undertaken
to
develop evaluation
measures
for
Prime Time,
a
multi-component intervention
study
to
increase
the
consistency
of
contraceptive
use and
reduce sexual risk behaviors among sexually active
ado-
lescent girls (Sieving
et al.,
2003).
The
overall goal
of the
current study
was to
examine
the
reliability
of
responses
to
questions about sexual behavior
and
contraceptive
use
using
two
frames:
a
temporal referent
and a
partner-spe-
cific referent.
The
current study addressed
two
core
research questions:
1.
How
reliable
are
adolescent girls' self-reports
of
contraceptive
use
using partner-referent questions
and time-referent questions within
the
same survey?
To address this question,
we
compared measures
of
contraceptive
use
based
on
adolescents' partner-
specific reports with their reports
of
contraceptive
use over
the
past
3
months
and
their reports
of con-
traceptive
use
over
the
past
6
months.
We
also
compared reliability
of 3- and
6-month time referent
questions. Given that adolescent sexual behavior
can
be episodic,
we
assumed that self-reports using longer
recall period,
and
thus
a
more comprehensive picture
of adolescent sexual experience,
may be
justified
if
such reports were highly reliable.
2.
What
is the
test-retest reliability
of
self-reported
contraceptive
use and
sexual behaviors
as
measured
twice over
a 1- to
2-week period?
To
address this
question,
we
analyzed
the
consistency
of
responses
over time
for
both partner-referent
and
time-referent
measures.
METHOD
Design
and
Participants
Data
for
this study were from
a
sample
of 13- to
18-year-
old girls seeking reproductive health care services
at six
school-
and
community-based adolescent clinics
in the
Minneapolis-St. Paul metropolitan area. Clinic staff identi-
fied
274
potential participants based
on
study eligibility
cri-
teria (e.g., ages 13-18, seeking reproductive health
ser-
vices,
English-speaking). Research staff invited
all
girls
who
met
eligibility criteria
to
participate
in the
study. Girls
were informed about
the
study
and
were told that participa-
tion would involve completing
one or two
paper-and-pen-
cil surveys that included questions about their first sexual
experience, along with their recent sexual behavior
and
contraceptive
use
patterns.
In
accordance with Minnesota's
1971 Medical Bill
for
Minors
Act,
which gives youth under
the
age of 18 the
right
to
seek confidential reproductive
ser-
vices,
girls
who
agreed
to
participate
in the
study
(n = 225;
82%
of
girls invited) provided their
own
written consent
for
study participation. After providing written consent
for
study participation, participants completed
a
paper-and-
pencil survey
(Tl
survey) which took
an
average
of 20
minutes
to
complete. Participants were paid
$10 to com-
plete
the Tl
survey.
Of 225
study participants,
196
(87.1%)
reported having
had
consensual vaginal intercourse
in the
previous
6
months
on the Tl
survey. These
196
participants
comprise
the
baseline sample
for
this study.
All participants
who
reported having vaginal
sex in the
past
6
months were classified
as
sexually active
and
invit-
ed
to
return
to the
clinic
in 1 to 2
weeks
to
complete
a sec-
ond identical survey
(T2
survey). Girls were offered
$20 to
complete
the T2
survey.
Of 196
eligible participants,
156
(79.6%) completed
the T2
survey.
Of
those completing
T2
surveys, three participants (1.9%) included information
on
new sexual partners between surveys,
and one
participant
(0.6%) reported
not
having
sex in the
past
six
months
on
the
T2
survey; data from these youth were excluded from
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
Sieving, Hellerstedt, McNeely, Fee, Snyder, and Resnick161
follow-up analyses. The remaining 152 participants com-
prise the follow-up sample for this study. For this sample,
the mean interval between Tl and T2 surveys was 9.5 days.
All study protocols were reviewed and approved by the
Institutional Review Board at the University of Minnesota.
Instrument and Measures
Tl and T2 survey instruments included identical questions
about timing of sexual intercourse, sexual partners and
sexual relationships, and contraceptive use. Some of the
questions were derived specifically for this study, and oth-
ers were derived from previous studies of youth sexual
behavior and contraceptive use (Brindis, Peterson, Card &
Eisen, 1996; Sieving et al., 1997; Sieving et al., 2001).
Questions were asked in sequential survey sections refer-
ring to the last 3 months, the last 6 months, and recent sex-
ual partners (up to 3 partners). The Tl survey included
additional questions about participants' age, ethnic back-
ground, lifetime pregnancy history, education and current
living situation. Survey instruments are available from the
corresponding author upon request.
To maximize the reliability and validity of self-report
measures, we incorporated several methodologic recom-
mendations (Catania et al., 1990) into instrument develop-
ment. First, we wrote survey questions using a combina-
tion of standard and familiar sexual terms. For example,
throughout the instrument "sex" was defined as "a male's
penis inside your vagina." Participants were asked to
exclude experiences of rape, sexual assault, and same-sex
intercourse in answering survey questions. "Sex partner"
was consistently defined as "male you had sex with"; for
example, with questions related to last male sex partner,
participants were asked to "think about the last male you
had sex with (his penis in your vagina)." "Birth control"
was defined by a list of contraceptive methods including
condoms, spermicidal foam or film, diaphragm, oral con-
traceptive pills (OCPs; "birth control pills"), depot
medroxyprogesterone acetate (DMPA; Depo-Provera; "the
Depo-Provera shot"), and levonorgestrel implants
("Norplant"). The majority of survey questions about con-
traception focused specifically on OCP, DMPA, and con-
dom use because these methods are commonly used by
adolescents and are considered effective in reducing preg-
nancy risk. Second, anchor dates were used for reporting
periods in order to improve recall of behavior. Third, the
instrument was pilot tested with 2 focus groups of adoles-
cent girls (n = 16 teens). Focus group respondents indicat-
ed that they understood survey terminology and that both
the content of questions and the mode of survey adminis-
tration were acceptable to girls their age.
Time-Referent Items. In an initial section of the survey,
participants answered a series of questions about number
of sexual partners, level of condom use, duration of OCP
use and duration of DMPA use referring to the preceding 6
months. In the next survey section, participants were asked
the same series of questions referring to the preceding 3
months. The following is an example of the format used in
these sections: "In the last 3 months, how often did you use
condoms when you had sex?" (response options: Never,
Less than half the time, About half the time, More than half
the time, Always).
Partner-Referent Items. After completing survey sections
using temporal references, participants were asked a third
series of questions referring to specific sexual partners.
Participants were asked to respond to questions for each of
up to 3 most recent male sexual partners over the past 6
months. For each partner, participants were asked questions
about first and most recent sexual intercourse, partner age,
characteristics of the relationship, interval between first sex
and first contraceptive use, dual method contraceptive use
practices, level of alcohol or other drug use in combination
with sex, and reasons for not using contraception. The part-
ner-specific questions included a 6-month contraceptive cal-
endar that asked participants to indicate, by month, whether
they had sex with this partner, whether they used OCPs or
DMPA, and the level of condom use with this partner.
Participants' monthly reports of condom use were grouped
into one of 3 categories: "E" responses = always used con-
doms; "M" and "S" responses = sometimes used condoms;
No response = never used condoms.
For purposes of comparing partner-referent and time-
referent reports, we created composite measures of contra-
ceptive use based on calendar reports. Initially, we com-
pleted month-by-month counts of various forms of contra-
ceptive use (e.g., DMPA, OCP, condom use) using calen-
dar reports. For condom use counts, numeric values were
assigned according to above-mentioned categories (1 =
always used condoms; 0.5 = sometimes used condoms; 0 =
never used condoms). For a count of "any birth control
method," numeric values were assigned to each month (1
= used hormonal method or always used condoms; 0.5 =
did not use hormonal method and sometimes used con-
doms; 0 = did not use hormonal method and did not use
condoms).
For each form of contraceptive use, monthly
counts were summed; a proportion was created by divid-
ing this sum by the number of months the participant
reported having vaginal intercourse. To compare with
time-referent reports, proportions were grouped into 5 cat-
egories: Never used this method of contraception, Used
this method less than half the time (proportions with val-
ues of 0.1-0.3), Used this method about half the time (pro-
portions with values of 0.4-0.6), Used this method more
than half the time (proportions with values of 0.7-0.99),
Always used this method (proportions = 1.0).
For participants who completed contraceptive calendars
for 2 or 3 sexual partners (32.7% of follow-up sample), the
partner-specific condom use measure and "any birth con-
trol use" measure combined information from partners in
months when the participant reported having intercourse
with more than one partner. For example, if a participant
reported having sex with Partner A and Partner B in the
month of January, always using condoms with Partner A
and never using condoms with Partner B in that month, we
classified January as a "sometimes" condom use month
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
162Reliability of Teens' Contraceptive Self-Reports
(the average of "always" condom use with Partner A and
"never" condom use with Partner B).
Procedure
We administered all surveys in clinic settings, with partici-
pants completing the Tl survey at the time of their clinic
appointments. To maximize the validity and reliability of
participants' self-reports (Catania et al., 1990; Weinhardt et
al.,
1998), trained research staff introduced the survey to
each participant using a standardized, nonjudgmental expla-
nation of the purpose of the survey. Research staff reviewed
definitions of "sex" and "birth control," anchor dates, item
reporting periods, and sexual partner calendars with each
participant prior to administering the survey. Participants
were assured that their survey responses would remain con-
fidential, that they could choose not to answer questions that
made them uncomfortable, and that participation would in
no way affect their clinical care. To maximize privacy dur-
ing survey administration, participants completed surveys in
private rooms outside of public waiting areas.
We followed identical procedures during administration
of the T2 survey. For the T2 survey, research staff specifi-
cally instructed participants not to report on behaviors or
new relationships that had occurred since the administra-
tion of the first survey.
Data Analysis
To address Research Question 1, we examined within-sur-
vey consistency between categorical time-referent measures
and partner-referent measures of contraceptive use. We used
T2 survey data from the follow-up sample for these analy-
ses,
so results for the within-survey and between-survey
analyses are comparable. We restricted these analyses to
participants who reported 3 or fewer sex partners in the past
6 months. Parallel analyses were completed comparing part-
ner-referent measures to 3-month referent measures and to
6-month referent measures. We used a weighted kappa sta-
tistic to assess within-survey reliability (Rosner, 1990).
Compared to an unweighted kappa statistic, a weighted
kappa statistic allows for some error in grouping into cate-
gories. A kappa value of 0.0 suggests that the amount of
agreement between categorical measures is what would be
expected by chance alone; a kappa value of 1.0 suggests
perfect agreement between categorical measures. Using
established guidelines (Landis & Koch, 1977), we interpret-
ed kappa values of 0.01-0.20 to indicate slight agreement
between measures; kappa values of 0.21-0.40 to indicate
fair agreement; kappa values of 0.41-0.60 to indicate mod-
erate agreement; kappa values of 0.61-0.80 to indicate sub-
stantial agreement; and kappa values of 0.81-0.99 to indi-
cate almost perfect agreement between measures.
To address Research Question 2, we examined consisten-
cy of contraceptive use and sexual behavior measures as
reported at two points over a 1- to 2-week interval. Tl and
T2 survey data from the follow-up sample were used for
these analyses. We examined consistency over time of both
temporal and partner-referent measures. For categorical
measures, we assessed consistency using an unweighted
kappa statistic, because comparison categories were identi-
cal across surveys. For continuous measures, we assessed
consistency using Pearson's correlation coefficient and a
percent matching statistic (Rosner, 1990). Percent matching
refers to the percentage of participants whose Tl survey
responses match exactly with their T2 survey responses.
Since this measure takes only identical responses into
account, it is a more conservative measure of reliability than
a correlation coefficient, which takes into account similar
response patterns across surveys (Rosner, 1990).
RESULTS
Description of Samples
Descriptive characteristics of baseline and follow-up sam-
ples are listed in Table 1. Chi-square analyses (not shown)
were used to compare participants who did and did not
return for T2 surveys on these descriptive statistics. As
compared to participants who only completed baseline
surveys, a significantly lower percentage of the follow-up
sample reported having dropped out of school. There were
no other statistically significant differences in demograph-
ic characteristics or lifetime sexual history measures
between these two groups of participants.
The median age at which follow-up sample participants
first had vaginal intercourse was 15.0 years. Approximately
81.5%
of this sample reported that they or their partner
used some form of contraception at first intercourse.
Almost 35% of this sample reported having one lifetime
vaginal sex partner, and 37.5% reported having four or
more lifetime sexual partners.
Recent patterns of sexual relationships and contraceptive
use among the follow-up sample are listed in Table 2.
Approximately one third of this sample (32.5%) reported
having more than one sexual partner over the past 6 months.
In terms of timing of first sexual intercourse in a relation-
ship,
one quarter of this sample (24.7%) was either dating
for less than a week or not in a dating relationship when they
first had vaginal intercourse with their most recent sexual
partner. Only 26.3% of this sample reported using a condom
every time they had sex in the past six months.
Approximately one third of this sample (33.5%) reported
using OCPs during the past 6 months. Patterns of OCP use
suggest substantial risk for pregnancy and STDs. Of OCP
users,
one third (33.3%) missed pills on more than one occa-
sion in the past 6 months, and fewer than 1 in 5 (16.65%)
always used condoms while taking OCPs. Of this sample,
11.8%
used DMPA over the previous 6-month period.
Participants using DMPA were less likely than other partic-
ipants to use barrier contraception. Around 55.5% of DMPA
users reported never using condoms while on DMPA, com-
pared to 21.1% of the full sample.
Within-Survey Comparisons
Data from T2 surveys of the follow-up sample were used
to examine consistency between time- and partner-referent
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
Sieving, Hellerstedt, McNeely, Fee, Snyder, and Resnick163
Table 1. Demographic & Lifetime Sexual History Descriptors, Baseline (n = 196) and Follow-Up (n = 152) Samples
Baseline SampleFollow-up Sample
Current age
13-14 years old
15-16 years old
17-18 years old
School status1
In middle/high school
Completed high school
Dropped out of school
Unknown
Reasons for clinic visit
Pelvic exam
Contraception
STD test
Pregnancy test
Other/Unknown
Race/ethnic background2
Caucasian
African American
Hispanic/Latina
American Indian
Asian
Other
Age at first vaginal intercourse
< 14 years
15-16 years
17-18 years
Unknown
Contraceptive use at first sex
Yes
No
Lifetime vaginal sex partners
1
2
3
4 or more
Unknown
6
64
126
155
128
7
6
89
128
69
8
16
161
25
12
10
8
7
67
94
34
1
155
41
62
36
18
78
2
(3.1%)
(32.6%)
(64.3%)
(79.1%)
(14.3%)
(3.6%)
(3.1%)
(45.4%)
(65.3%)
(35.2%)
(4.1%)
(8.2%)
(82.1%)
(12.8%)
(6.1%)
(5.1%)
(4.1%)
(3.6%)
(34.2%)
(48.0%)
(17.3%)
(0.5%)
(79.1%)
(20.9%)
(31.6%)
(18.4%)
(9.2%)
(39.8%)
(1.0%)
6
47
99
123
22
2
5
69
94
54
7
13
122
23
11
7
7
5
51
74
27
0
123
28
53
26
14
57
2
(4.0%)
(30.9%)
(65.1%)
(80.9%)
(14.5%)
(1.3%)
(3.3%)
(45.3%)
(61.8%)
(35.5%)
(4.6%)
(8.6%)
(80.3%)
(15.1%)
(7.2%)
(4.6%)
(4.6%)
(3.3%)
(33.5%)
(48.7%)
(17.8%)
(0.0%)
(81.5%)
(18.5%)
(34.9%)
(17.1%)
(9.2%)
(37.5%)
(1.3%)
'Significantly different frequency distribution
(%23df
= 10.07; p = 0.02) between participants who completed baseline surveys only and follow-up
sample.
2Column totals > 100%, since participants were asked to mark all applicable race/ethnic groups.
measures of contraceptive use. Partner-referent measures
were compared to both 3-month and 6-month time-refer-
ent measures. As listed in Table 3, weighted kappa statis-
tics indicate substantial to high levels of agreement
between time- and partner-referent reports. Discrepancies
between time- and partner-referent measures are some-
what greater for questions about condom use than for
questions about hormonal contraceptive use.
For each method, the consistency of respondents' recall
was similar using 3- and 6-month reference points. For
example, the consistency between time- and partner-refer-
ent measures of OCP use over a 3-month recall period was
k = 0.86; consistency of these measures over a 6-month
period was k = 0.84. Thus, a 6-month recall period did not
appear to create more inconsistency between time- and
partner-referent reports than a 3-month recall period.
Test-Retest Comparisons
In a second set of reliability analyses, data from the fol-
low-up sample were used to examine consistency between
Tl and T2 survey responses to questions regarding recent
sexual activity and contraceptive use.
Consistency of Time-Referent Items. As listed in Table
4,
participants' responses to time-referent measures were
generally consistent across Tl and T2 surveys. High levels
of consistency were seen with both 3- and 6-month mea-
sures.
For example, kappa statistics associated with dura-
tion of OCP use were 0.85 and 0.88 for 3- and 6-month
questions, respectively, indicating that participants were
likely to report the same duration of OCP use when asked
about use on two occasions 1 to 2 weeks apart.
Two exceptions to the pattern of high-level consistency
across time are shown in Table 4. First, participants' esti-
mates of the frequency of their sexual intercourse over the
past 3 or 6 months were not consistent. Although the
Pearson's correlation coefficients indicate strong associa-
tions between baseline and follow-up reports (e.g., girls
who report having sex relatively frequently at baseline will
also report high frequency at follow-up), the low percent
matching suggests girls' frequency point estimate varied
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
164Reliability of Teens' Contraceptive Self-Reports
Table 2. Recent Patterns of Sexual Activity and
Contraceptive Use, Follow-Up Sample (« = 152)Table 3. Consistency between Partner-Referent and Time-
Referent Measures, Follow-Up Sample1
Number of vaginal sex partners, past six months
1
2
3
4 or more
Relationship with most recent sexual partner at the
time of first sex
Dating for > 1 month
Dating for 1 week - 1 month
Dating for < 1 week
Not in a dating relationship
Closest relationship with most recent sexual partner
Living together
Planning to get married
Monogamous dating relationship
Dating each other and other people
Not in a dating relationship
Other
Consistency of contraceptive use, past six months
Always used
More than half the time
Half the time or less
Never used
Consistency of condom use, past six months
Always used
More than half the time
Half the time or less
Never used
Duration of OCP use, past six months
4-6 months
3 months or less
Did not use OCPs
Number of times missed OCPs, past six months
(among n = 54 reporting OCP use)
Never
One time
More than one time
Combined condom and OCP use, past six months
(among n = 54 reporting OCP use)
Always
More than half the time
Half the time or less
Never
Duration of DMPA use, past six months
More than 3 months (2 shots)
3 months or less (1 shot)
Did not use DMPA
Combined condom and DMPA use, past six months
(among n = 18 reporting DMPA use)
Always
Half the time or less
Never
67.5%
17.2%
7.3%
8.0%
46.6%
28.7%
11.4%
13.3%
4.0%
17.2%
54.3%
12.6%
10.6%
1.3%
55.9%
17.8%
17.8%
8.5%
26.3%
18.4%
34.2%
21.1%
28.9%
6.6%
64.5%
24.1%
42.6%
33.3%
16.65%
16.65%
38.9%
27.8%
9.2%
2.6%
88.2%
16.7%
27.8%
55.5%
Duration (# months) OCP use
Duration (#months) DMPA use
Level of condom use4>5
Level of any birth control use4'6
Past 3 months
(n = 134)2
0.86
0.94
0.66
0.68
Past 6 months
(n = 137)3
0.84
0.93
0.74
0.69
'Consistency between measures estimated using weighted kappa statis-
tic (all measures were categorical).
2Follow-up sample (n = 152) minus participants who did not report sex
in past 3 months on partner-specific survey items (n = 7) and partici-
pants who reported having more than 3 sexual partners in past 6 months
(n = ll).
3Follow-up sample (n = 152) minus participants who did not report hav-
ing sex in past 6 months on partner-specific survey items (n = 4) and
participants who reported having more than 3 sexual partners in past 6
months (n = 11).
4Level of use classified into five categories ranging from 1 {Never) to 5
(Always).
5Partner-referent measure of condom use is a proportion based on cal-
endar counts of number of months using condoms divided by number
of months sexually active. When a participant reported 2 partners in a
month, condom use for that month was estimated by averaging reports
of condom use with each partner.
6Partner-referent measure a proportion based on calendar counts of num-
ber of months using birth control divided by number of months sexually
active. When a participant reported 2 partners in a month, birth control
use for that month was estimated using reported hormonal use in that
month and an average of reported condom use with each partner.
Table 4. Test-Retest Consistency of Time-Referent
Measures, Follow-Up Sample (n = 152)
Continuous Measures1
Number of sexual partners
Frequency of intercourse
Categorical Measures2
Duration (# months) OCP use
Number of times missed OCPs
Duration (# months) DMPA use
Delayed DMPA injection
Level of condom use3
Level of any birth control use3
'Consistency estimated using percent matching and Pearson's correla-
tion coefficient (r).
Consistency estimated using unweighted kappa statistic.
3Level of use classified into five categories ranging from 1 (Never) to
5 (Always).
Past
0.86
0.53
3 months
(r = 0.69)
(r = 0.96)
0.85
0.75
0.90
1.0
0.85
0.77
Past 6 months
0.82 (r = 0.82)
0.48 (r= 0.86)
0.88
0.60
0.87
1.0
0.80
0.82
between surveys. Second, reports of number of occasions
in which OCPs were missed over the past 6 months were
only moderately consistent over time. Thus, memory of
missing pills appears to be less consistent using a 6-month,
compared to a 3-month, retrospective reporting period.
Consistency of Partner-Referent Items. We conducted a
separate analysis to assess consistency of partner-referent
measures over time. This analysis used data from partici-
pants'
contraceptive calendars; it included 148 participants
from the follow-up sample who completed partner-specific
calendars on both Tl and T2 surveys. Results indicated sub-
stantial agreement between baseline and follow-up calendar
reports. A T1-T2 comparison of month-by-month calendar
reports revealed highly consistent reports of whether dyads
had sex, used condoms, and were protected by OCPs or
DMPA in each of six months included in the calendar.
Specifically, unweighted kappa statistics for monthly pro-
files,
including information on whether participant had sex,
used condoms, and used hormonal methods during that
month, ranged from 0.75 to 0.82. The unweighted kappa
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
Sieving,
Hellerstedt, McNeely, Fee, Snyder, and Resnick
165
statistic estimating consistency of overall calendar reports
between Tl and T2 was 0.89.
DISCUSSION
This study demonstrates the reliability of self-reported
measures of sexual behavior and contraceptive use among
adolescent girls who have had vaginal intercourse in the
past 6 months. Several findings are notable. Related to
Research Question 1, the consistency between time-refer-
ent and partner-referent methods of measuring contracep-
tive use was substantial. An interesting pattern emerged
from this comparison: the consistency between time-refer-
ent estimates and partner-referent estimates was lower
with condom use than with hormonal contraceptive use.
Thus,
time-referent reports were less likely to match part-
ner-referent reports when measuring contraceptive behav-
iors that are partner-dependent (e.g., condom use) than
when measuring behaviors that are partner-independent
(e.g., OCP use, DMPA). Although earlier research sug-
gests that partner-by-partner estimates may enhance
respondents' recall ability (Catania et al., 1990), findings
from this study suggest that partner-specific reporting
methods may be useful in assessing contraceptive use
behaviors that depend on a male partner (Grimley,
Prochaska, Velicer, & Prochaska, 1995).
Regarding Research Question 2, test-retest consistency
of contraceptive use and sexual behavior measures was
high. Test-retest results indicated that adolescent girls
respond to questions in these domains in a consistent fash-
ion over a 1- to 2-week interval. Both partner- and time-
referent measures produced stable estimates of behavior
over time. For time-referent measures, high levels of con-
sistency were evident with both 3- and 6-month measures.
One exception was participants' exact responses to ques-
tions about frequency of intercourse, which varied over
time.
It is not surprising that participants did not report the
exact same frequency of intercourse across surveys, nor is
it necessarily of programmatic or clinical significance.
Higher reliability may be obtained by asking participants
to choose from meaningful response categories (e.g., once,
less than once a month, 1-3 times per month, more than 3
times per month) rather than to provide an exact number.
A discrepancy occurred in one pair of temporal stabili-
ty estimates. Although the percent matching statistic asso-
ciated with the measure of sexual partners in the past 3
months (86%) indicated relatively high levels of test-retest
consistency, the Pearson's correlation coefficient is lower
(r = 0.69). Due to the restricted range of item responses in
this sample (e.g., at T2, 91% reported 1 or 2 sexual part-
ners in past 3 months), the correlation coefficient may
have underestimated the temporal stability of this measure
(Howell, 1987; Sieving et al., 2001).
Findings from this study expand our understanding of
the range of sexual behaviors that urban adolescent girls
are able to report with high reliability. Previous studies
with 12- to 14-year-old girls (Hearn et al., 2003) and 15-
to 18-year-old girls (Hornberger, Rosenthal, Biro, &
Stanberry, 1995) suggest that adolescent girls are highly
reliable in reporting age at first sexual and romantic mile-
stone experiences (e.g., age first in love, age first touched
a penis, age at first intercourse). Measuring recent sexual
experience, Shew et al. (1997) found that 13- to 21-year-
old girls' self-reports of condom use with up to 2 most
recent male sexual partners was highly consistent with
their verbal reports to clinicians. In addition to consistent
reports of recent condom use, our findings suggest that
sexually experienced 13- to 18-year-old girls can provide
reliable reports of number of sexual partners and duration
of hormonal contraceptive use in the past 6 months.
Our findings regarding consistency of participants'
recall over recent time periods extend those of McFarlane
and St. Lawrence (1999), who found that 12- to 19-year-
olds provided consistent estimates of sexual behavior over
time using a 2-month recall period. Comparing reliability
of 3- and 6-month reports, our findings suggest that 13- to
18-year-old girls report contraceptive use patterns with
equal consistency over these retrospective recall periods,
with the exception of higher consistency in reports of
missed OCPs in the past 3 months than in the past 6
months. Thus, a 6-month referent may be more likely than
a 3-month referent to capture behaviors that are episodic in
nature, and therefore may provide a more accurate picture
of sexually active adolescents' behavior over time
(McFarlane & St. Lawrence, 1999).
This study has several limitations. Although a 1- to 2-
week period between Tl and T2 surveys minimizes the
likelihood of retest response inconsistencies due to actual
changes in sexual and contraceptive behaviors, this retest
interval may inflate reliability estimates due to memory of
responses from Tl surveys. A 1- to 2-week interval has
been used with other psychometric studies of self-reported
behavior among adolescents (Brener et al., 2002; Romer et
al.,
1997; Williams, Toomey, McGovern, Wagenaar, &
Perry, 1995).
Second, although this study tests the reliability of
self-
reported contraceptive use and sexual behaviors among
sexually active adolescent girls, we do not examine the
validity of these self-reports. Without the availability of a
gold standard, it is impossible to conclude whether any of
the three measurement methods produces valid indicators
of sexual behavior or contraceptive use (Carmines &
Zeller, 1979). This study's measures should be used with
an awareness of previous research suggesting that adoles-
cent girls may over-report OCP use (Potter, Oakley, Leon-
Wong, & Canamar, 1996) as well as condom use with
casual sexual partners (Jeannin, Konings, Dubois-Arber,
Landert, & Van Melle, 1998). To establish validity of ado-
lescent girls' retrospective self-reports, further research is
needed comparing daily self-monitoring data to retrospec-
tive reports of sexual behaviors and contraceptive use.
Third, for practical reasons, response options to catego-
rize condom use differed in time-referent and
partner-ref-
erent sections of the survey. Although we created a mea-
sure to compare these items, the statistical tests of reliabil-
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
166Reliability of Teens' Contraceptive Self-Reports
ity between time- and partner-referent measures may have
been attenuated because of the differences in response
options.
Fourth, this study was conducted with a sample of
sexually active adolescent girls seeking reproductive care
in urban clinic settings. Thus, findings may not be gener-
alizable to other populations. Additional studies need to be
conducted to assess the reliability of these measures with
normative and high-risk populations living in rural, subur-
ban and urban settings.
In summary, survey items and data collection methods
used in this study appeared to be understandable and
acceptable to the target group of sexually active adolescent
girls.
With the exception of frequency of intercourse, girls
were able to recall their sexual and contraceptive behav-
iors over as long as a 6-month interval in a consistent fash-
ion. Although recall of condom use differs somewhat
using time- and partner-referent reporting methods, our
data suggest that sexually active adolescent girls respond
reliably to both referents.
REFERENCES
Brener, N., Kann, L., McManus, T., Kinchen, S., Sundberg, E., & Ross, J.
(2002). Reliability of the 1999 Youth Risk Behavior Survey
Questionnaire. Journal of Adolescent
Health,
31, 336-342.
Brindis, C., Peterson, J., Card, J. J., & Eisen, M. (1996). Prevention mini-
mum evaluation data set
(PMEDS):
A minimum data set for
evaluating
programs aimed at preventing adolescent pregnancy and
STD/HIV/AIDS.
Los Altos, CA: Sociometrics.
Carmines, E., & Zeller, R. (1979). Reliability and validity assessment.
Newbury Park, CA: Sage.
Catania, J. (1999). A framework for conceptualizing reporting bias and its
antecedents in interviews assessing human sexuality. The Journal of
Sex
Research, 36(1), 25-38.
Catania, J., Chitwood, D., Gibson, D., & Coates, T. (1990). Methodologic
problems in AIDS research: Influences on measurement error and par-
ticipation bias in studies of sexual behavior. Psychological Bulletin,
108(3), 339-362.
Grimley, D., Prochaska, J., Velicer, W., & Prochaska, G. (1995).
Contraceptive and condom use adoption and maintenance: A stage par-
adigm approach. Health Education
Quarterly,
22(1), 20-35.
Hearn, K., O'Sullivan, L., & Dudley, M. (2003). Assessing reliability of
early adolescent girls' reports of romantic and sexual behavior. Archives
of
Sexual
Behavior,
32(6),
513-521.
Hornberger, L., Rosenthal, S., Biro, F., & Stanberry, L. (1995). Sexual his-
tories of adolescent girls: Comparison between interview and chart.
Journal of
Adolescent
Health,
16, 235-239.
Howell, D. (1987). Statistical
methods
for
psychology.
Boston, MA: PWS-
Kent.
Jeannin, A., Konings, E., Dubois-Arber, F., Landert, C., & Van Melle, G.
(1998). Validity and reliability in reporting sexual partners and condom
use in a Swiss population survey. European Journal of
Epidemiology,
14, 139-146.
Kauth, M., St. Lawrence, I, & Kelly, J. (1991). Reliability of retrospective
assessments of sexual risk behavior: A comparison of biweekly, three-
month, and twelve-month self-reports. AIDS Education and Prevention,
3,
207-215.
Landis, J., & Koch, G. (1977). The measurement of observer agreement for
categorical data.
Biometrics,
33, 159-174.
McFarlane, M., & St. Lawrence, J. (1999). Adolescents' recall of sexual
behavior: Consistency of self-report and effect of variations in recall
duration. Journal of Adolescent
Health,
25(3), 199-206.
Potter, L., Oakley, D., Leon-Wong, E., & Canamar, R. (1996). Measuring
compliance among oral contraceptive users. Family Planning
Perspectives,
28, 154-158.
Romer, D., Hornik, R., Stanton, B., Black, M., Xiannina, L., Ricardo, I., et
al.
(1997). 'Talking" computers: A reliable and private method to con-
duct interviews on sensitive topics with children. The Journal of Sex
Research,
34(1), 3-9.
Rosner, B. (1990).
Fundamentals
ofbiostatistics. Boston, MA: PWS-Kent.
Shew, M., Remafedi, G., Bearinger, L., Faulkner, P., Taylor, B.,
Potthoff,
S.,
et al. (1997). The validity of self-reported condom use among adoles-
cents.
Sexually
Transmitted
Diseases,
24(9), 503-510.
Sieving, R., Beuhring, T., Resnick, M., Shew, M., Bearinger, L., Ireland, M.,
et al. (2001). Development of adolescent self-report measures from the
National Longitudinal Study of Adolescent Health. Journal of
Adolescent Health, 28(1),
73-81.
Sieving, R., Resnick, M., Bearinger, L., Remafedi, G., Harmon, B., &
Taylor, B. (1997). Cognitive and behavioral predictors of sexually trans-
mitted disease risk behavior among sexually active adolescents.
Archives of
Pediatrics
and Adolescent
Medicine,
151(3), 243-25.
Sieving, R., Resnick, M., Fee, R., Oliphant, J., Pettingell, S., Russ, P., et al.
(2003). Prime Time: A youth development approach to preventing neg-
ative sexual health outcomes. Journal of
Adolescent
Health,
32(2), 154.
Turner, C, Ku, L., Rogers, S., Lindberg, L., Pleck, J., & Sonenstein, F.
(1998). Adolescent sexual behavior, drug use, and violence: Increased
reporting with computer survey technology. Science, 280, 867-873.
Turner, C, Lessler, I., & Devore, J. (1992). Effects of mode of administra-
tion and wording on reporting of drug use. In C. Turner, J. Lessler J, &
J. Gfroerer (Eds.), Survey measurement of drug use: Methodological
issues (pp. 177-219). Washington, DC: U.S. Government Printing
Office.
Upchurch, D., Lillard, L., Aneshensel, C, & Fang, L. (2002).
Inconsistencies in reporting the occurrence and timing of first inter-
course among adolescents. The Journal of Sex Research, 39(3),
197-206.
Weinhardt, L., Forsyth, A., Carey, M., Jaworski, B., & Durant, L.(1998).
Reliability and validity of self-report measures of HIV-related sexual
behavior: Progress since 1990 and recommendations for research and
practice. Archives of
Sexual
Behavior, 27(2), 155-178.
Williams, C, Toomey, T., McGovern, P., Wagenaar, A., & Perry, C. (1995).
Development, reliability, and validity of self-report alcohol-use mea-
sures with young adolescents. Journal of Child and Adolescent
Substance
Abuse, 4(3), 19^0.
Manuscript accepted October 5, 2004
Downloaded by [University of Tennessee, Knoxville] at 07:55 20 May 2016
