Content uploaded by Wick Warren
Author content
All content in this area was uploaded by Wick Warren on Apr 18, 2014
Content may be subject to copyright.
JOURNAL OF ADOLESCENT HEALTH 2002;31:336–342
ORIGINAL ARTICLE
Reliability of the 1999 Youth Risk Behavior
Survey Questionnaire
NANCY D. BRENER, Ph.D., LAURA KANN, Ph.D., TIM McMANUS, M.S.,
STEVEN A. KINCHEN, B.S., ELIZABETH C. SUNDBERG, M.A., AND JAMES G. ROSS, M.S.
Purpose: To assess the test-retest reliability of the 1999
Youth Risk Behavior Survey (YRBS) questionnaire.
Methods: A sample of 4619 male and female high
school students from white, black, Hispanic, and other
racial/ethnic groups completed the YRBS questionnaire
on two occasions approximately two weeks apart. The
questionnaire assesses a broad range of health risk be-
haviors. This study used a protocol that maintained
anonymity yet allowed matching of Time-1 and Time-2
responses. The authors computed a kappa statistic for the
72 items measuring health risk behaviors, and compared
group prevalence estimates at the two testing occasions.
Results: Kappas ranged from 23.6% to 90.5%, with a
mean of 60.7% and a median of 60.0%. Kappas did not
differ by gender, grade, or race/ethnicity of the respon-
dent. About one in five items (22.2%) had significantly
different prevalence estimates at Time 1 vs. Time 2. Ten
items, or 13.9%, had both kappas below 61% and signif-
icantly different Time-1 and Time-2 prevalence esti-
mates.
Conclusions: Overall, students appeared to report
health risk behaviors reliably over time, but several
items need to be examined further to determine whether
they should be revised or deleted in future versions of
the YRBS.
From the Division of Adolescent and School Health, National Center
for Chronic Disease Prevention and Health Promotion, Centers for
Disease Control and Prevention, Atlanta, Georgia (N.D.B., L.K., T.M.,
S.A.K.); and Macro International Inc. (ORC Macro), Calverton, Mary-
land (E.C.S., J.G.R.).
Address correspondence and reprint requests to: Nancy D. Brener,
Ph.D., Division of Adolescent and School Health, CDC, Mailstop K-33,
4770 Buford Highway NE, Atlanta, Georgia 30341. E-mail: nad1@cdc.gov.
Manuscript accepted November 12, 2001.
1054-139X/02/$–see front matter
KEY WORDS:
Adolescence
Data collection
Health surveys
Psychometrics
The Youth Risk Behavior Surveillance System
(YRBSS) was developed in 1989 by the Centers for
Disease Control and Prevention (CDC) to monitor
health risk behaviors that contribute to the leading
causes of mortality, morbidity, and social problems
among youth and adults in the United States. The
YRBSS monitors six categories of behaviors: (a) those
that contribute to unintentional injuries and violence;
(b) tobacco use; (c) alcohol and other drug use; (d)
sexual behaviors that contribute to unintended preg-
nancy and sexually transmitted disease, including
human immunodeficiency virus infection; (e) dietary
behaviors; and (f) physical activity.
The YRBSS consists of national, state, and local
school-based surveys of representative samples of
students in grades 9 through 12, a national house-
hold-based survey of 12- through 21-year-olds, a
national survey of college students, a national survey
of alternative school students, and other surveys of
special populations of young people. These surveys
all use a similar instrument, the Youth Risk Behavior
Survey (YRBS) questionnaire, which was developed
with extensive research and testing [1].
Data from the YRBSS are used to develop policies
and programs to prevent health risk behaviors
among youth [2]. It is important, therefore, to have
confidence in the reliability of these data. Studies of
other measures have demonstrated the reliability of
adolescent self-report of tobacco, alcohol, and other
PII S1054-139X(02)00339-7 Published by Elsevier Science Inc., 360 Park Avenue South, New York, NY 10010
October 2002 RELIABILITY OF THE 1999 YRBS 337
drug use [3,4]; sexual behavior [5]; suicide attempts
[6]; dietary behaviors [7,8]; and physical activity
[8–10]. In addition, a recent study by Klein et al.
examined the reliability of many YRBS items [11].
These studies, however, were limited by small sam-
ple sizes and, in most cases, by a lack of diversity
within those samples. Further, with the exception of
the study by Klein et al., none of these studies
assessed the reliability of all categories of health risk
behavior. Such an assessment would allow for be-
tween-category comparisons.
In 1992, CDC conducted a test-retest reliability
study of the original YRBS questionnaire [12]. That
study was the first to demonstrate the test-retest
reliability of all categories of health risk behavior
among a diverse sample of adolescents. The study
found that nearly three-quarters of the questionnaire
items had “substantial” or higher reliability, accord-
ing to the categories suggested by Landis and Koch
[13]. In addition, the study found that the responses
of seventh-grade students were less consistent than
those of students in higher grades.
Over the past decade, CDC has modified the YRBS
questionnaire to meet federal needs and those of the
state and local health and education agencies that
conduct the surveys in their jurisdictions. These
modifications have included the addition of new
questions, as well as changes in the wording of
original questions. Because of these modifications, it
has become desirable to conduct a reliability study of
the updated questionnaire. The present study, there-
fore, assessed the test-retest reliability of that ques-
tionnaire on a large and diverse sample of high
school students.
Methods
Sample
A convenience sample of respondents was drawn
from 61 schools in 20 states plus the District of
Columbia. Because the goal of sampling was to
obtain a diverse group of respondents, the 20 states
were geographically dispersed. In addition, 48% of
the schools in the sample were in urban areas, 39%
were in suburban areas, and 13% were in rural areas
[14]. Selection of ninth- through 12th-grade classes
within each volunteer school varied according to the
school’s schedule. In about half of the schools, stu-
dents in health education or physical education
classes were eligible to participate. In about one-
quarter of schools, students in required academic
subjects (e.g., English) were eligible to participate. In
Table 1. Demographic Characteristics of Respondents
and of Students in Grades 9 Through 12 Nationwide
Sample Distribution National Distribution*
Characteristic (%) (%)
Gender
Male 46.6 51.0
Female 53.4 49.9
Grade
9 30.6 25.7
10 31.8 25.7
11 21.9 24.5
12 15.7 24.1
Race or ethnicity
White 52.2 64.8
Black 31.4 12.1
Hispanic 6.1 13.3
Other 10.3 N/A
Age (yrs)
13 0.1 1.4
14 12.4 17.4
15 28.9 24.0
16 28.5 24.5
17 21.2 22.3
18 8.9 6.7
*Source: U.S. Bureau of the Census [15].
other schools, all students were eligible to partici-
pate. In each school, local parental consent proce-
dures were followed. This study was approved by
CDC’s Institutional Review Board.
Of the 6802 students enrolled in the selected
classes, 5216 (77%) completed questionnaires during
the first survey administration. The remaining 23%
were absent on the day of the survey, failed to return
a parental consent form, or, to a much lesser extent,
refused to participate or had parents who refused to
have their child participate. Of those who completed
questionnaires in the first administration, 4628 (89%)
completed questionnaires during the second admin-
istration. Nine students did not have matching iden-
tification numbers on Time-1 and Time-2 question-
naires. The final sample, therefore, consisted of 4619
students.
As shown in Table 1, the demographic character-
istics of the sample were similar to the national
distribution of ninth- through 12th-grade students
[15]. For some groups, the sample percentage dif-
fered from the national percentage by more than five
percentage points. Specifically, 10th-grade students
were overrepresented and twelfth-grade students
were underrepresented. In addition, white students
and Hispanic students were underrepresented, but
black students were overrepresented.
338 BRENER ET AL. JOURNAL OF ADOLESCENT HEALTH Vol. 31, No. 4
Questionnaire
As part of a larger study designed to test the reliabil-
ity of all of the items and the effect of alternative
question wording for some items, eight very similar
forms of the questionnaire were developed. All ques-
tionnaires were self-administered and consisted of
between 97 and 100 multiple-choice questions. Five
questions measured demographic information, two
asked students to report their height and weight, and
the remaining items assessed health risk behaviors.
The questionnaires were identical to the instrument
used in the 1999 national YRBS, except for three
alternatively worded questions on certain forms, and
five questions not on previous versions of the YRBS.
The results related to the alternatively worded and
new questions are beyond the scope of this article.
Data collection began in February 2000 and was
completed in April 2000. The questionnaire was
administered in a regular classroom setting and took
students about 40 minutes to complete. A standard
computer-scannable questionnaire booklet contained
the questions and was used to record responses. As
with the standard YRBS, no skip patterns were
included in the questionnaire. This technique helps
to safeguard privacy because comparable amounts of
time are required to complete the questionnaire
regardless of risk behavior status, and because stu-
dents cannot detect the risk behaviors of other stu-
dents simply by looking at the pattern of responses.
Data Collection Procedures
Before the first survey administration, a unique num-
ber was assigned to two scannable questionnaire
booklets. Each set of two identically numbered book-
lets was placed in an envelope. During the adminis-
tration of the first survey, students removed and
used one booklet. The envelope, now containing only
the second booklet, was then sealed by the student,
who wrote his or her name across the seal. During
the second survey administration, each student re-
ceived the envelope with his or her name across the
seal. After removing and completing the second
booklet, the student destroyed the envelope. This
technique has been used successfully in previous
studies, and students perceive that it adequately
safeguards their privacy [12,16].
The survey was conducted by trained data collec-
tors from Macro International Inc. (ORC Macro). The
data collectors read aloud scripts that explained the
survey procedures. Students were informed during
the first survey administration that they would be
asked to complete a “very similar” questionnaire a
few weeks later. Other than that variation, the ad-
ministration procedures used in this study were the
same as those used for the standard YRBS.
For 57% of the schools, the first and second
administrations of the survey were exactly 14 days
apart. The first and second survey administrations in
the remaining schools ranged from 10 to 22 days,
with an average span of 15.6 days.
Data Analysis
Collapsing and editing procedures. After the ques-
tionnaire booklets were cut and scanned, the data
were edited for inconsistency according to standard
YRBSS procedures. These procedures exclude ques-
tionnaires that do not have more than 20 valid
responses or have the same response option 15 or
more times in a row. Although many questions
contain multiple response categories, standard
YRBSS reports dichotomize responses into “no risk”
vs. “at risk.” For example, students who responded
that they carried a weapon on 0 of the past 30 days
are classified as “no risk,” whereas those who re-
ported that they carried a weapon on 1 or more of the
past 30 days are classified as “at risk.” Because we
wanted to examine how the data perform in stan-
dard YRBSS reports, these same procedures were
followed for the analyses reported here.
Fourteen items using “the last time” and “in the
past 7 days” as the reference period could not be
expected to be consistent across a 2-week timeframe
and were eliminated from the analysis. In addition,
the five questions not on previous or current versions
of the YRBS questionnaire were eliminated from the
analyses for this study.
Kappa statistic and prevalence rates. A kappa statis-
tic, which provides a measure of agreement that
corrects for what would be expected by chance, was
computed for each of the 72 items. Prevalence rates
for each risk behavior at Time 1 and Time 2 also were
calculated. These rates were considered significantly
different if their 95% confidence intervals (CIs) did
not overlap. This is the same criterion used in assess-
ing the statistical significance of subgroup differ-
ences in reports of YRBSS data [17].
Results
Kappas ranged from 23.6% to 90.5%, with a mean of
60.7% and a median of 60.0% (Table 2). Using qual-
October 2002 RELIABILITY OF THE 1999 YRBS 339
Table 2. Kappa Statistics, Time-1, and Time-2 Prevalence Rates, by Questionnaire Item
Kappas Time 1 Time 2
Item (%) (%) (%)
Behaviors related to unintentional injuries and violence
Rarely or never wear helmet when riding a motorcycle 66.1 37.8 46.8*
Rarely or never wear helmet when riding a bicycle 75.8 84.6 83.8
Rarely or never wear seatbelt when riding in a car 61.6 15.7 19.6*
Rode with drinking driver during the past 30 days 60.3 30.3 29.6
Drove after drinking during the past 30 days 57.2 8.5 10.3*
Carried weapon 1 day during the past 30 days 65.7 15.0 13.3
Carried gun 1 day during the past 30 days 50.8 4.2 4.4
Carried weapon on school property 1 day during the past 30 days 57.7 5.1 5.7
Felt too unsafe to go to school 1 day during the past 30 days 42.0 5.5 5.0
Threatened or injured with weapon on school property 1 time in the past 12 months 40.6 7.3 5.9
In a physical fight 1 time during the past 12 months 67.8 34.6 30.3*
Injured in a physical fight 1 time during the past 12 months 47.0 2.9 4.4*
In a physical fight on school property 1 time in past 12 months 64.4 13.1 12.4
Physically hurt by boyfriend or girlfriend during the past 12 months 53.6 9.1 9.9
Ever forced to have sexual intercourse 65.8 9.1 10.3
Felt sad and hopeless during the past 12 months 56.4 28.2 24.1*
Considered suicide during the past 12 months 74.3 17.0 16.0
Planned suicide during the past 12 months 66.6 13.0 12.9
Had 1 suicide attempt during the past 12 months 72.7 8.4 8.5
Had injurious suicide attempt during the past 12 months 52.3 2.1 2.7
Tobacco use behaviors
Ever used cigarettes 85.7 65.8 63.9
Age first smoked whole cigarette 13 years 70.9 21.4 23.7
Smoked cigarettes 1 day during the past 30 days 81.9 27.2 27.5
Smoked 20 cigarettes per day on the days smoked during the past 30 days 83.5 17.5 17.1
Bought cigarettes in a store or gas station during the past 30 days 69.3 6.4 7.2
Asked to show ID when buying cigarettes during the past 30 days 52.8 6.8 8.2
Smoked cigarettes 1 day on school property during the past 30 days 71.4 9.7 9.1
Ever smoked cigarettes regularly 79.8 17.7 19.0
Tried to quit smoking cigarettes during the past 12 months 70.3 18.4 16.7
Used smokeless tobacco during 1 of the past 30 days 71.1 6.6 6.4
Used smokeless tobacco on school property during 1 of the past 30 days 60.4 3.9 3.9
Smoked cigars 1 day during the past 30 days 59.7 12.2 11.8
No usual brand of cigarettes during the past 30 days 37.3 1.6 1.5
Alcohol and other drug use behaviors
Ever used alcohol 81.9 76.1 72.5*
Age first drank alcohol 13 years 65.9 28.9 29.9
Drank alcohol 1 day during the past 30 days 70.9 41.1 39.9
Had 5 or more drinks in a row 1 day during the past 30 days 67.6 23.9 23.7
Drank alcohol on school property 1 day during the past 30 days 49.4 3.9 4.1
Ever used marijuana 89.8 42.8 41.7
Age first used marijuana 13 years 70.3 10.5 11.3
Used marijuana during the past 30 days 76.0 22.6 22.1
Used marijuana on school property during the past 30 days 59.1 5.5 5.3
Ever used cocaine 73.4 5.6 6.2
Used cocaine during the past 30 days 48.3 2.2 2.7
Ever used inhalants 67.0 11.3 10.6
Used inhalants during the past 30 days 42.2 2.9 3.5
Ever used heroin 57.4 1.9 3.0*
Ever used methamphetamines 70.7 6.3 6.9
Ever used steroids 45.1 4.0 4.1
Ever injected illegal drugs 53.9 1.4 2.0
Offered, sold, or given illegal drugs on school property during the past 12 months 52.2 23.0 21.9
Sexual behaviors
Ever had sexual intercourse 90.5 49.5 50.2
Age first had sexual intercourse 13 years 40.4 18.0 14.8*
Had 4 lifetime sex partners 57.9 19.1 17.6
(Continued)
340 BRENER ET AL. JOURNAL OF ADOLESCENT HEALTH Vol. 31, No. 4
Table 2. Continued
Kappas Time 1 Time 2
Item (%) (%) (%)
Had 1 sex partner during the past 3 months 72.7 32.9 35.0
Ever been pregnant or gotten someone pregnant 51.9 8.6 8.2
Dietary behaviors
Perceive self as overweight 58.6 22.7 26.1*
Trying to lose weight 58.2 33.8 37.2*
Exercised to lose or keep from gaining weight during the past 30 days 57.6 58.6 53.9*
Ate less food, calories, or fat to lose or keep from gaining weight during the past 30 days 53.2 43.1 40.4
Fasted to lose or keep from gaining weight during the past 30 days 40.1 18.4 15.3*
Took diet pills, powders, or liquids to lose or keep from gaining weight during the past 30 days 42.1 7.8 7.9
Vomited or took laxatives to lose or keep from gaining weight during the past 30 days 40.3 4.9 5.0
Physical activity behaviors
Watch 2 hours of television on an average school day 46.7 62.4 63.2
Attend physical education class 1 day a week 84.8 62.4 56.8*
Exercise 20 minutes during physical education class 41.1 72.3 69.0
Played on 1 sports team during the past 12 months 56.2 54.6 53.3
Injured during physical activity 1 time during the past 12 months 47.1 40.8 35.2*
Other health-related topics
Ever been taught about AIDS or HIV in school 23.6 85.0 86.2
Had physical examination when not sick during the past 12 months 50.5 58.9 58.1
Saw dentist during the past 12 months 63.8 66.5 63.4*
Rarely or never use sun screen when in the sun for 1 hour 61.1 66.6 66.7
*Time 1 prevalence significantly different from Time 2 prevalence based on
nonoverlapping 95% CIs.
CI confidence interval.
itative labels for values of kappa suggested by Lan-
dis and Koch [13], 47.2% of items had at least
“substantial” reliability (kappas 61%), and 93.1%
had at least “moderate” reliability (kappas 41%).
Based on nonoverlapping 95% CIs, 22.2% of items
had significantly different prevalence estimates at
Time 1 vs. Time 2. Ten items, or 13.9%, had both
kappas below 61% and significantly different Time-1
and Time-2 prevalence estimates.
Examination of reliability by respondent charac-
teristics revealed no significant differences in mean
values of kappa by gender, grade, or race/ethnicity
(Table 3). In addition, although mean kappas were
somewhat higher for questions that used lifetime as
a reference period than those that used the past 30
days and the past 12 months, these differences were
not statistically significant.
Examination of reliability by risk behavior cate-
gory, however, did reveal some significant differ-
ences. Specifically, items related to tobacco use dem-
onstrated significantly higher reliability (mean
kappa 68.8%) than items related to unintentional
injuries and violence (mean kappa 59.9%), dietary
behaviors (mean kappa 50.0%), physical activity
(mean kappa 55.2%), and other health-related
topics (mean kappa 49.7%). In addition, items
related to alcohol and other drug use (mean kappa
Table 3. Mean Kappa Statistics and 95% Confidence
Intervals by Demographic and Question Characteristics
Mean
Kappa
Characteristic (%) 95% CI
Gender
Male 57.1 51.3, 63.0
Female 64.3 58.1, 70.6
Grade
9 57.2 49.5, 64.9
10 61.0 53.9, 68.2
11 60.7 52.3, 69.1
12 63.7 53.8, 73.6
Race or ethnicity
White 62.5 57.3, 67.8
Black 51.4 42.0, 60.8
Hispanic 58.5 41.7, 75.3
Other 59.0 45.7, 72.2
Reference periods
Past 30 days 58.1 53.5, 62.6
Past 12 months 59.9 56.1, 63.7
Lifetime 66.3 62.2, 70.3
Risk behavior categories
Unintentional injuries and 59.9 55.7, 64.3
violence
Tobacco use 68.8 64.9, 72.7
Alcohol and other drug use 63.4 58.8, 68.0
Sexual behaviors 62.7 59.6, 65.7
Dietary behaviors 50.0 46.5, 53.5
Physical activity 55.2 52.3, 58.1
Other health-related topics 49.7 46.9, 52.5
October 2002 RELIABILITY OF THE 1999 YRBS 341
63.4%) and those related to sexual behavior (mean
kappa 62.7%) demonstrated significantly higher
reliability than items related to dietary behaviors,
physical activity, and other health-related topics.
Finally, items related to unintentional injuries and
violence demonstrated significantly higher reliability
than those related to dietary behaviors and other
health-related topics.
This pattern of results is parallel to that found
when examining which risk behavior categories
were more likely to have items with significantly
different Time-1 and Time-2 prevalence estimates.
For example, although none of the 13 items related to
tobacco use had significantly different Time-1 and
Time-2 prevalence estimates, four of seven dietary
behavior items, two of five physical activity items,
and six of twenty injury-related items did demon-
strate significant differences between Time 1 and
Time 2.
Discussion
Nearly all items on the YRBS questionnaire had at
least “moderate” reliability and nearly half had “sub-
stantial” reliability. Several items, however, had low
reliability and significantly different Time-1 vs.
Time-2 prevalence estimates. These items need to be
examined further to determine whether they should
be revised or deleted in future versions of the YRBS.
The overall findings can be compared with those
found in the reliability study of the original YRBS
questionnaire [12], as well as the recent study by
Klein et al. that included a subset of YRBS items [11].
Although the results of the earlier YRBS reliability
study and Klein’s study were quite similar, the
kappa values in the current study tended to be lower
than those in the previous studies, with a few excep-
tions. The results of all three studies were similar,
however, with respect to the relationship between
demographic variables and reliability. That is,
among high school students, values of kappa did not
differ by gender, grade, or race/ethnicity [11,12].
One reason that the mean kappa was lower in the
present study than in the previous YRBS reliability
study is that many of the items that have been added
to the YRBS questionnaire since the earlier study
showed kappas lower than 61%. Most of those items,
however, such as those measuring behaviors on
school property, were of low prevalence. Very low
and very high prevalences can adversely affect
kappa values because it only takes a few respondents
changing their responses between Time 1 and Time 2
to have a substantial effect on kappa [18]. For exam-
ple, the item assessing whether students smoked a
usual brand of cigarettes had a kappa of 37.3%, but
the prevalences at Time 1 and Time 2 were 1.6% and
1.5%, respectively.
This study showed that items related to tobacco
use, alcohol and other drug use, and sexual behavior
demonstrated significantly higher reliability than
items related to dietary behaviors, physical activity,
and other health-related topics. This is not surpris-
ing, given that behaviors related to substance use
and sexual activity are likely to be more salient to
adolescents, and therefore more reliably recalled,
than behaviors related to nutrition, physical activity,
and other health-related topics such as health care
[19]. Notably, the items related to health care tended
to be less reliable than most of the items related to
substance use and sexual activity, a finding similar to
that of Klein et al. [11].
Limitations
As in other test-retest reliability studies [11,12], any
inconsistent response in this study was considered to
be a response error when calculating kappa. It is
possible, however, that an inconsistent response be-
tween Time 1 and Time 2 could reflect an actual
behavior change. For example, a student could re-
port at Time 1 that he had not smoked cigarettes in
the past 30 days, then report at Time 2 that he had
smoked in the past 30 days. Such responses would be
inconsistent yet accurate if the student did indeed
smoke during the 2-week test-retest interval and not
before. The values of kappa computed for this study,
therefore, must be considered to be conservative
estimates.
Although reliability is a necessary characteristic of
a valid measure, the demonstration of the reliability
of items on the YRBS questionnaire does not ensure
the instrument’s validity. Although research with
adolescent populations has demonstrated the valid-
ity of self-reported alcohol and other drug use [3,20],
tobacco use [21–25], suicidal ideation [26], sexual
behavior [27, 28], dietary behaviors [29, 30], and
physical activity [31], much work remains to be done
in assessing the validity of self-report measures of all
types of health risk behaviors. This is a challenge,
given the lack of objective measures, or “gold stan-
dards,” for many behaviors of interest. Even when
objective measures exist, as is the case for tobacco use
and drug use, the use of these measures is not
without limitations, especially among adolescent
populations [32,33]. To address these issues, re-
342 BRENER ET AL. JOURNAL OF ADOLESCENT HEALTH Vol. 31, No. 4
searchers have used various techniques, such as
randomized response [34], bogus pipeline [35], and
computer-assisted data collection [36], with mixed
results. Future research should examine ways to
encourage even more reliable and valid self-reports
of health risk behaviors among adolescents.
References
1. Kolbe LJ, Kann L, Collins JL. Overview of the Youth Risk
Behavior Surveillance System. Public Health Rep 1993;
108(Suppl 1):2–10.
2. Everett SA, Kann L, McReynolds L. The Youth Risk Behavior
Surveillance System: Policy and program applications. J Sch
Health 1997;67:333–5.
3. Needle R, McCubbin H, Lorence J, et al. Reliability and
validity of adolescent self-reported drug use in a family-based
study: A methodological report. Int J Addict 1983;18:901–12.
4. O’Malley PM, Bachman JG, Johnston LD. Reliability and
consistency in self-reports of drug use. Int J Addict 1983;18:
805–24.
5. Davoli M, Perucci CA, Sangalli M, et al. Reliability of sexual
behavior data among high school students in Rome. Epidemi-
ology 1992;3:531–5.
6. Velting DM, Rathus JH, Asnis GM. Asking adolescents to
explain discrepancies in self-reported suicidality. Suicide Life
Threat Behav 1998;28:187–96.
7. French SA, Peterson CB, Story M, et al. Agreement between
survey and interview measures of weight control practices in
adolescents. Int J Eat Disord 1998;23:45–56.
8. Gilmer MJ, Speck BJ, Bradley C, et al. The youth health survey:
Reliability and validity of an instrument for assessing cardio-
vascular health habits in adolescents. J Sch Health 1996;66:
106–11.
9. Aaron DJ, Kriska AM, Dearwater SR, et al. Reproducibility
and validity of an epidemiologic questionnaire to assess past
year physical activity. Am J Epidemiol 1995;142:191–201.
10. Sallis JF, Buono MJ, Roby JJ, et al. Seven-day recall and other
physical activity self-reports in children and adolescents. Med
Sci Sports Exerc 1993;25:99–108.
11. Klein JD, Graff CA, Santelli JS, et al. Improving adolescent
health care surveillance. In: Cynamon MA, Kulka RA (eds).
Seventh Conference on Health Survey Research Methods.
DHHS Pub No. 01-1013. Hyattsville, MD: National Center for
Health Statistics, 2001:11–18.
12. Brener ND, Collins JL, Kann L, et al. Reliability of the Youth
Risk Behavior Survey questionnaire. Am J Epidemiol 1995;141:
575–80.
13. Landis JR, Koch GG. The measurement of observer agreement
for categorical data. Biometrics 1977;33:159 –74.
14. Quality Education Data, Inc. Quality Education Data National
Education Database. Denver, CO: Quality Education Data,
Inc., May 2000.
15. U.S. Bureau of the Census. School enrollment—social and
economic characteristics of students: October 1998. Current
Population Reports, Series P-20, no. 521. Washington, DC: U.S.
Bureau of the Census, 1998.
16. Popham WJ. Appraising two techniques for increasing the
honesty of students’ answers to self-report assessment de-
vices. J Personnel Eval Educ 1993;7:33–41.
17. Kann L, Kinchen SA, Williams BI, et al. Youth risk behavior
surveillance — United States, 1999. MMWR CDC Surveill
Summ 2000;49:1–96.
18. Maclure M, Willett WC. Misinterpretation and misuse of the
kappa statistic. Am J Epidemiol 1987;126:161–9.
19. Tourangeau R. Remembering what happened: Memory errors
and survey reports. In: Stone AA, Turkkan JS, Bachrach CA, et
al. (eds). The Science of Self-Report: Implications for Research
and Practice. Mahwah, NJ: Lawrence Erlbaum Associates,
2000:29–47.
20. Winters KC, Stinchfield RD, Henly GA, et al. Validity of
adolescent self-report of alcohol and other drug involvement.
Int J Addict 1990–1991;25:1379–95.
21. Akers RL, Massey J, Clarke W, et al. Are self-reports of
adolescent deviance valid? Biochemical measures, random-
ized response, and the bogus pipeline in smoking behavior.
Social Forces 1983;62:234–51.
22. Martin GL, Newman IM. Assessing the validity of self-
reported adolescent cigarette smoking. J Drug Educ 1988;18:
275–84.
23. Stacy AW, Flay BR, Sussman S, et al. Validity of alternative
self-report indices of smoking among adolescents. Psychol
Assess 1990;2:442–6.
24. Williams CL, Eng A, Botvin GJ, et al. Validation of students’
self-reported cigarette smoking status with plasma cotinine
levels. Am J Public Health 1979;69:1272–4.
25. Wills TA, Cleary SD. The validity of self-reports of smoking:
Analyses by race/ethnicity in a school sample of urban
adolescents. Am J Public Health 1997;87:56–61.
26. De Man AF, Leduc CP. Validity and reliability of a self-report
suicide ideation scale for use with adolescents. Soc Behav
Personality 1994;22:261–6.
27. Orr DP, Fortenberry JD, Blythe MJ. Validity of self-reported
sexual behaviors in adolescent women using biomarker out-
comes. Sex Transm Dis 1997;24:261–6.
28. Shew ML, Remafedi GJ, Bearinger LH, et al. The validity of
self-reported condom use among adolescents. Sex Transm Dis
1997;24:503–10.
29. Rockett HR, Breitenbach M, Frazier AL, et al. Validation of a
youth/adolescent food frequency questionnaire. Prev Med
1997;26:808–16.
30. Rosen JC, Poplawski D. The validity of self-reported weight
loss and weight gain efforts in adolescents. Int J Eat Disord
1987;6:515–23.
31. Weston AT, Petosa R, Pate RR. Validation of an instrument for
measurement of physical activity in youth. Med Sci Sports
Exerc 1997;29:138 –43.
32. Bauman KE, Koch GG, Bryan ES, et al. On the measurement of
tobacco use by adolescents: Validity of self-reports of smoke-
less tobacco use and validity of cotinine as an indicator of
cigarette smoking. Am J Epidemiol 1989;130:327–37.
33. Cone EJ. New developments in biological measures of drug
prevalence: The Validity of Self-Reported Drug Use: Improv-
ing the Accuracy of Survey Estimates. NIDA Res Monogr
1997;167:108–30.
34. Fisher M, Kupferman LB, Lesser M. Substance use in a
school-based clinic population: Use of the randomized re-
sponse technique to estimate prevalence. J Adolesc Health
1992;13:281–5.
35. Murray DM, O’Connell CM, Schmid LA, et al. The validity of
smoking self-reports by adolescents: A reexamination of the
bogus pipeline procedure. Addict Behav 1987;12:7–15.
36. Turner CF, Ku L, Rogers SM, et al. Adolescent sexual behav-
ior, drug use, and violence: Increased reporting with com-
puter survey technology. Science 1998;280:867–73.
