Diagnostic accuracy of brief PTSD screening instruments in military veterans

Abstract

Post-traumatic stress disorder (PTSD) is prevalent but is under-detected and under-treated, despite available efficacious treatments. To improve detection rates, screening instruments such as the PTSD Checklist (PCL) and the Primary Care-PTSD (PC-PTSD) screen have been widely used. However, validation of these screening instruments among patients seeking treatment in substance use disorder (SUD) specialty treatment clinics and general mental health (MH) treatment clinics is limited. Therefore, this study assessed the area under the ROC curve (AUC), sensitivity, specificity, efficiency, and positive and negative predictive values of the PCL, PC-PTSD, and five abbreviated versions of the PCL in detecting PTSD among samples of patients seeking treatment in SUD specialty treatment (n=158) and general MH treatment settings (n=242). A computer-assisted structured diagnostic interview (C-DIS-IV) was used to ascertain patient DSM-IV PTSD diagnostic status. Based on the C-DIS-IV, prevalence of PTSD was found to be 36.7 and 52.9% in the SUD and MH samples, respectively. The PCL, PC-PTSD, and five abbreviated versions of the PCL were found to have adequate psychometric properties for screening patients in SUD (AUC ranged from 0.80 to 0.86) and MH (AUC ranged from 0.77 to 0.80) outpatient treatment settings.

Full text

Diagnostic accuracy of brief PTSD screening instruments in military veterans
Quyen Q. Tiet, Ph.D.
a,b,c,d,
⁎, Kathleen K. Schutte, Ph.D.
b
, Yani E. Leyva, Ph.D.
a,b
a
National Center for PTSD, Dissemination and Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA
b
Center for Health Care Evaluation, VA Palo Alto Health Care System, Menlo Park, CA, USA
c
Department of Psychiatry, Stanford University School of Medicine, Stanford, CA, USA
d
California School of Professional Psychology at Alliant International University, San Francisco, CA, USA
abstractarticle info
Article history:
Received 9 March 2012
Received in revised form 20 December 2012
Accepted 28 January 2013
Keywords:
Substance use disorder
Post-traumatic stress disorder
Screening instrument
Co-occurring disorders
Dual diagnosis
Validation study
Sensitivity and specificity
ROC AUC
Post-traumatic stress disorder (PTSD) is prevalent but is under-detected and under-treated, despite available
efficacious treatments. To improve detection rates, screening instruments such as the PTSD Checklist (PCL)
and the Primary Care–PTSD (PC-PTSD) screen have been widely used. However, validation of these screening
instruments among patients seeking treatment in substance use disorder (SUD) specialty treatment clinics
and general mental health (MH) treatment clinics is limited. Therefore, this study assessed the area under the
ROC curve (AUC), sensitivity, specificity, efficiency, and positive and negative predictive values of the PCL, PC-
PTSD, and five abbreviated versions of the PCL in detecting PTSD among samples of patients seeking treatment
in SUD specialty treatment (n= 158) and general MH treatment settings (n= 242). A computer-assisted
structured diagnostic interview (C-DIS-IV) was used to ascertain patient DSM-IV PTSD diagnostic status. Based
on the C-DIS-IV, prevalence of PTSD was found to be 36.7 and 52.9% in the SUD and MH samples, respectively.
The PCL, PC-PTSD, and five abbreviated versions of the PCL were found to have adequate psychometric
properties for screening patients in SUD (AUC ranged from 0.80 to 0.86) and MH (AUC ranged from 0.77 to
0.80) outpatient treatment settings.
Published by Elsevier Inc.
1. Validation of brief PTSD screening instruments
PTSD is common in the general population and among military
veterans. In a national representative sample of 8098 Americans, the
National Comorbidity Survey (NCS) found that 7.8% of individuals (5%
of men and 10.4% of women) had a lifetime PTSD diagnosis (Kessler,
Sonnega, Bromet, Hughes, & Nelson, 1995). The National Comorbidity
Survey Replication Study found comparable rates (Kessler et al., 2005,
Kessler, Chiu, Demler, Merikangas, & Walters, 2005;http://www.hcp.
med.harvard.edu/ncs/publications.php). Higher rates have been
found among veterans. The National Vietnam Veterans Readjustment
Study (NVVRS) interviewed a representative sample of 3016 veterans
who served during the Vietnam War, and estimated a prevalence of
18.7% for a lifetime diagnosis and 9.1% for a current PTSD diagnosis
(Dohrenwend et al., 2006). A recent review study (Ramchand et al.,
2010) found estimates between 5 and 20% among non-treatment
seeking previously deployed personnel. Among VA SUD patients, a
prevalence between 20 and 35% have been reported (Dalton &
McKellar, 2007; McKellar & Saweikis, 2005; Tiet, Byrnes, Barnett, &
Finney, 2006), but prevalence of PTSD in VA mental health clinics
is lacking.
Despite its high prevalence and the existence of efficacious
treatments (Foa, Keane, Friedman, & Cohen, 2008; Institute of
Medicine, 2007), PTSD is under-detected (Liebschutz et al., 2007;
Magruder et al., 2005) and undertreated, which may lead to increased
health care cost (Davidson, Stein, Shalev, & Yehuda, 2004; Kessler et al.,
1995; Schnurr, Friedman, Sengupta, Jankowski, & Holmes, 2000). For
example, Kimerling, Trafton, and Nguyen (2006) found that 75% of
individuals in a sample of SUD patients identified as meeting
diagnostic criteria for PTSD did not have a PTSD diagnosis documented
in their patient record. The consequences of untreated PTSD can be
grave and can include medical morbidity (Beckham et al., 2003), worse
mental health, substance abuse, and decreased quality of life out-
comes, including marital problems and unemployment, as well as
increased risk for suicide (Tanielian & Jaycox, 2008), and premature
mortality (Johnson, Fontana, Lubin, Corn, & Rosenheck, 2004).
Efforts have been made to improve detection of PTSD through
the development of brief screening instruments that are practical
for clinical settings. However, although PTSD has been found to be
prevalent among patients seeking services at substance use
disorder (SUD) and general mental health (MH) specialty clinics
(e.g., Grubaugh, Elhai, Cusack, Wells, & Frueh, 2007; Kimerling
et al., 2006), no study has validated the PCL among patients
receiving outpatient services at SUD clinics and only one study has
evaluated it among patients at general MH treatment settings
(Grubaugh et al., 2007).
Journal of Substance Abuse Treatment 45 (2013) 134–142
⁎Corresponding author. NC-PTSD, VAPAHCS, Menlo Park, CA 94025, USA.
E-mail addresses: Quyen.Tiet@va.gov,TietQ2@yahoo.com (Q.Q. Tiet).
0740-5472/$ –see front matter. Published by Elsevier Inc.
http://dx.doi.org/10.1016/j.jsat.2013.01.010
Contents lists available at SciVerse ScienceDirect
Journal of Substance Abuse Treatment

Two of the most widely used measures are the PTSD Checklist
(PCL; Weathers, Litz, Herman, Huska, & Keane, 1993) and the Primary
Care–PTSD screen (PC-PTSD; Prins et al., 2003). The PCL assesses the
17 PTSD DSM-IV symptoms using 17 self-report items, in which each
item is scored on a five-point scale ranging from 0 (not at all) to 5
(extremely), yielding total scores between 0 and 68 (Weathers et al.,
1993). The PC-PTSD is a four-item screen that has been mandated by
the United States Department of Defense (DoD) to be used during
post-deployment health assessments (Hoge, Auchterlonie, & Milliken,
2005) and by the Veterans Health Administration within the
Department of Veterans Affairs (VA) to be used during routine
primary care visits (Department of Veterans Affairs and Administra-
tion, 2004). It comprises four dichotomous (yes/no) items assessing
the presence of nightmares, avoidance, being on guard, and feeling
numb with total scores ranging from zero to four.
Having brief, validated screening instruments reduces respondent
burden and may lead to reduced health care cost. In addition to the
four-item PC-PTSD, abbreviated two- to six-item versions of the PCL
have been developed. In addition to assessing the psychometric
properties of the PCL and PC-PTSD, the current study evaluated five
alternative, abbreviated versions of the PCL: Bliese et al. (2008) four-
item PCL as well as Lang and Stein's (2005) two-, three-, four- and six-
item PCLs.
The PCL was developed using a sample of military veterans and a
cut-point equal to or greater than (referred to from this point forward
as “cut-point”) 50 was recommended (Weathers et al., 1993).
However, with a cut-point of 50, Stein, McQuaid, Pedrelli, Lenox,
and McCahill (2000) and Widows, Jacobsen, and Fields (2000)
reported poor psychometric properties of the PCL, with a sensitivi-
ty = 0.32 and 0.20–0.40, respectively. For a review, see McDonald
and Calhoun (2010). As summarized in Table 1, different cut-points
have been suggested for different samples by subsequent validation
studies, including the highest recommended cut-point of 60 on a
sample of male veterans (Keen, Kutter, Niles, & Krinsley, 2008), to the
lowest recommended cut-points of between 28 and 30 for primary
care female veterans (Lang, Laffaye, Satz, Dresselhaus, & Stein, 2003).
When positive and negative predictive values and estimated
population prevalence were not reported in original study reports,
they were derived using the techniques specified by McDonald and
Calhoun. Psychometric properties of abbreviated versions of the PCL
are provided in Table 2.
As compared to the PCL, fewer validation studies have been
conducted on the PC-PTSD (see Table 2). The PC-PTSD was developed
on a primary care patient sample (Prins et al., 2003), and has been
validated on military primary care patients (Gore, Engel, Freed, Liu, &
Armstrong, 2008), substance use disorder patients (Kimerling et al.,
Table 1
Psychometrics properties and recommended cut-points of the PTSD Checklist (PCL) from previous studies.
Study Sample Sample size (n) PTSD BR (%) Criterion measure Cut-point SN SP PPV NPV Efficiency Prevalence
Weathers et al., 1993 Vietnam veterans 123 54 SCID ≥50 0.82 0.83 0.83 0.82 0.83 .52
Blanchard et al., 1996 MV and sexual abuse patients 40 68 CAPS ≥44 0.94 0.86 0.85 0.95 0.90 .50
≥50 0.78 0.86 0.82 0.83 0.83 .43
Manne et al., 1998 Mothers of cancer survivors 65 6 SCID ≥50 0.75 0.89 0.30 0.98 0.88 .15
≥45 0.75 0.82 0.21 0.98 0.82 .21
≥40 1.00 0.77 0.22 1.00 0.79 .28
Dobie et al., 2002 Women receiving VA services 282 36 CAPS ≥50 0.58 0.92 0.80 0.80 0.80 .26
≥38 0.79 0.79 0.68 0.87 0.79 .42
≥30 0.85 0.64 0.57 0.88 0.72 .54
Bliese et al., 2008 Combat returnees 352 b1 MINI ≥50 0.24 0.98 0.56 0.93 0.98 .02
≥34 0.71 0.91 0.43 0.97 0.91 .09
≥30 0.78 0.88 0.38 0.98 0.88 .12
Andrykowski et al., 1998 Breast cancer patients 82 6 SCID ≥50 0.60 0.99 0.79 0.97 0.96 .05
≥30 1.00 0.83 0.27 1.00 0.84 .22
Walker et al., 2002 Women in HMO 261 11 CAPS ≥45 0.36 0.95 0.46 0.93 0.89 .08
≥30 0.82 0.76 0.29 0.97 0.78 .30
Lang & Stein, 2005 Primary care female patients 221 31 CIDI ≥50 0.39 0.94 0.74 0.77 0.77 .16
≥30 0.78 0.71 0.55 0.88 0.73 .44
Primary care patients 154 16 CIDI ≥50 0.54 0.94 0.63 0.91 0.88 .14
≥30 0.96 0.59 0.31 0.99 0.65 .51
Stein et al., 2000 Primary care patients 132 12 CIDI ≥50 0.32 0.94 0.42 0.91 0.87 .09
Yeager et al., 2007 VA primary care patients 840 11 CAPS ≥50 0.53 0.95 0.57 0.94 0.90 .11
≥31 0.81 0.81 0.35 0.97 0.81 .26
Lang et al., 2003 Primary care female veterans 49 31 CIDI ≥50 0.39 0.94 0.74 0.77 0.77 .16
≥30 0.78 0.71 0.55 0.88 0.73 .44
≥28 0.94 0.68 0.57 0.96 0.76 .51
Hudson et al., 2008 Older adults in hospitals 100 10 SCID ≥50 0.40 0.97 0.60 0.94 0.91 .07
≥36 0.90 0.87 0.43 0.99 0.87 .21
Widows et al., 2000 BMT recipients 102 5 SCID ≥50 0.20 0.95 0.17 0.96 0.91 .06
SCS 0.40 0.93 0.22 0.97 0.90 .09
Harrington &
Newman, 2007
Female substance users 44 39 CAPS ≥44 0.76 0.79 0.69 0.84 0.78 .42
≥38 0.82 0.60 0.57 0.84 0.69 .56
≥35 1.00 0.52 0.57 1.00 0.71 .68
Keen et al., 2008 Male veterans 114 22 CAPS ≥60 0.56 0.92 0.66 0.88 0.84 .19
SCS 0.72 0.79 0.49 0.91 0.77 .32
Grubaugh et al., 2007 Mental health patients 44 59 CAPS ≥54 0.69 0.78 0.82 0.64 0.73 .50
Bollinger et al., 2008 HIV-seropositive patients 57 12 CAPS ≥50 0.86 0.79 0.36 0.98 0.80 .29
≥52 0.71 0.84 0.38 0.96 0.82 .23
Note: BR = base rate; PR = positive rate; SN = sensitivity; SP = specificity; PPV = positive predictive value; NPV = negative predictive value; Prevalence = estimated
population prevalence; BMT = bone marrow transplant; MV = motor vehicle; VA = Department of Veteran Affairs; HMO = Health Maintenance Organization; HIV = human
innunodeficiency virus; CAPS = Clinician-Administered PTSD Scale; SCID = Structured Clinical Interview for DSM; MINI = Mini International Neuropsychiatric Interview; SCS =
symptom cluster scoring.
135Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

2006; Van Dam, Ehring, Vedel, & Emmelkamp, 2010), soldiers
returning from combat (Bliese et al., 2008), and veterans serving in
the military subsequent to September 11, 2001 (Calhoun et al., 2010).
However, no information is available regarding how well this
instrument performs among patients seeking treatment at outpatient
MH clinics. Given the high rates of comorbidity of PTSD among
patients with substance use or other psychiatric disorders (e.g., Grant
et al., 2004; Regier et al., 1990), validation of the PC-PTSD and the PCL
in such settings is critically needed to enhance clinicians' and
managers' ability to evaluate the overall accuracy and psychometric
properties of PTSD screening information.
In this study, psychometric properties of the described instru-
ments were examined for patients seeking treatment in SUD versus
MH settings separately, because patient characteristics were expected
to be different. A number of factors may potentially influence the
psychometric properties (e.g., positive predictive value and negative
predictive value, and sensitivity and specificity) of an instrument. For
example, patients in the MH settings were expected to have higher
prevalence of PTSD than patients initiating treatment in SUD
treatment settings. Positive predictive value (PPV) and negative
predictive value (NPV) are directly and algebraically related to the
base rate of PTSD (prevalence) of the sample. Other indicators of
diagnostic accuracy, such as sensitivity and specificity are not
algebraically related to the base rate of the sample; however, they
are influenced by sample characteristics, including symptom severity,
comorbidity, and other study characteristics. For example, patients in
the MH settings were expected to have greater likelihood of having
other comorbid psychiatric disorders (e.g., other anxiety disorders or
Table 2
Psychometrics properties of the Primary Care–PTSD (PC-PTSD) screen and abbreviated variations of the PTSD Checklist (PCL) from previous studies.
Instrument/study Sample Sample size (n) PTSD BR (%) Criterion measure Cut-point SN SP PPV NPV Efficiency Prevalence
PC-PTSD
Prins et al. (2003) Primary care 188 25 CAPS ≥3 0.78 0.87 0.66 0.92 0.85 .29
Gore et al. (2008) Military primary care 213 21 PSS-I ≥3 0.70 0.92 0.70 0.92 0.87 .21
Kimerling et al. (2006) Veterans SUD patients 97 33 CAPS ≥3 0.91 0.80 0.69 0.95 0.84 .43
Bliese et al. (2008) Combat returnees 352 b1 MINI ≥3 0.76 0.88 0.46 0.96 0.88 .12
Calhoun et al. (2010) Veterans since 9/11 220 25 CAPS ≥3 0.83 0.85 0.64 0.94 0.85 .32
Bliese, Wright, Adler,
and Thomas (2004);
Bliese, Wright, Adler,
Thomas, and Hoge (2004)
Iraqi soldiers 356 10 MINI ≥3 0.46 0.97 0.63 0.94 0.92 .07
Van Dam et al., 2010 Civilian SUD patients 142 15 SCID 2 0.86 0.57 0.26 0.96 0.61 .49
3 0.67 0.72 0.29 0.93 0.71 .34
4 0.52 0.88 0.43 0.91 0.83 .18
PCL abbreviated versions
PCL-LS-2 (Lang & Stein, 2005) Primary care patients 154 16 CIDI ≥4 0.96 0.58 0.30 0.99 0.64 .50
PCL-LS-3 (Lang & Stein, 2005) Primary care patients 154 16 CIDI ≥5 1.00 0.51 0.28 1.00 0.58 .57
PCL-LS-4 (Lang & Stein, 2005) Primary care patients 154 16 CIDI ≥8 0.83 0.68 0.33 0.95 0.70 .40
PCL-LS-6 (Lang & Stein, 2005) Primary care patients 154 16 CIDI ≥14 0.92 0.72 0.38 0.98 0.75 .38
PCL-Bliese-4 (Bliese et al., 2008) Combat returnees 352 b1 MINI ≥6 0.88 0.68 0.26 0.98 0.68 .32
≥7 0.76 0.80 0.33 0.96 0.80 .20
≥8 0.66 0.85 0.37 0.95 0.85 .15
Note: BR = base rate; PR = positive rate; SN = sensitivity; SP = specificity; PPV = positive predictive value; NPV = negative predictive value; Prevalence = estimated
population prevalence; CAPS = Clinician-Administered PTSD Scale; PSS-I = PTSD Symptom Scale–Interview version; MINI = Mini International Neuropsychiatric Interview;
SCID = Structured Clinical Interview for DSM; SUD = substance use disorder.
C-DIS PTSD Diagnosis
PTSD Screen
Result
Sensitivity = a / (a + c)
Specificity = d / (b + d)
Positive predictive value = a / (a + b)
Negative predictive value = d / (c + d)
Efficiency = (a + d) / (a + b + c + d)
Area Under the ROC Curve (AUC) = Area under the curve formed by mapping the True Positive
Rate by False Positive Rate, using all the cut-off points of a measure.
Yes No
Yes a b
No c d
Fig. 1. Calculations made to compare PTSD diagnosis Obtained through Computerized Diagnostic Interview Schedule (C-DIS) with screening instrument results.
136 Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

depression) that are characterized by symptoms similar to those of
PTSD than patients in the SUD settings. Such overlapping symptoms
could inflate scores on screening tests, which may result in different
recommended cut points for patients treated in SUD versus MH
settings (McDonald & Calhoun, 2010). In addition to evaluating
psychometric properties specific to certain cut-off points (e.g.,
sensitivity, specificity, efficiency, positive and negative predictive
value), we also report the area under the curve (AUC), which is
independent of particular cut-off scores. AUC specifies the area under
the receiver operating curve, mapping the test's true positive rate to
its false positive rate. AUC values of 1.0 indicate a perfect fit; values of
.5 indicate the utility of a test not better than a 50–50 chance.
In summary, there is a lack of available information regarding the
psychometric properties of the PCL, PC-PTSD, and five abbreviated
versions of the PCL among patients receiving mental health or
substance use disorders services, despite the need of such screening
instruments for these settings. In this study, psychometric properties
were examined separately among patients seeking treatment in SUD
versus MH settings because patient characteristics were expected to
be different among patients treated at different types of treatment
settings. The current study considered the psychometric utility of a
variety of possible cut-points for each screening measure and at SUD
and MH settings separately to inform clinical use of these instruments.
In addition, the study compared the AUC for screening tests to
evaluate the tests' performance independent of the selected cut-off
points (see Fig. 1).
2. Method
2.1. Procedure
Data were collected from August 2003 to December 2004 at
treatment initiation (baseline) on patients entering treatment at one
of four outpatient treatment programs at two VA medical centers, one
on the West Coast and one in the Northeast region of the United
States. These included three SUD and one general MH treatment
programs. Patients were recruited at the clinics, provided informed
consent, and were paid for their participation. The study protocol was
approved by local institute review boards.
Intending to capture a representative treatment seeking patient
sample, this prospective study did not use any exclusion criteria.
Varying across the four treatment programs, between 50 and 75% of
all patients who initiated treatment at one of the four programs
during the study recruitment period were contacted. A total of 467
patients were contacted, and 411 patients (88%) agreed to participate.
Based on study protocol, participants at treatment entry completed a
computerized structured diagnostic interview administered by a
research assistant and then a self-report survey, during a single
session. Follow-up data were collected, but the current study utilized
only baseline data. Validation of the instruments was conducted
separately for patients receiving services within SUD (n= 158) and
MH (n= 242) treatment settings.
2.2. Measures
2.2.1. Demographic variables
The survey collected patients' demographic information, including
age (years), gender, marital status (never married; married; or
separated, divorced, or widowed), race/ethnicity (non-Hispanic
White, African-American, Hispanic/Latino, Native American, Asian/
Pacific islander, and other or multiple races), and education (high
school or less; some college; four or more years of college).
2.2.2. Diagnostic interview
The Computerized Diagnostic Interview Schedule for DSM-IV
(C-DIS-IV) was used as the criteria standard to ascertain whether a
current (last 12 months) PTSD diagnosis was present at treatment
initiation. The instrument showed a high concordance rate with a
PTSD diagnosis made by a clinician (Breslau, Kessler, & Peterson,
1998). The C-DIS-IV also provided information about participants'
comorbid psychiatric diagnoses, including other anxiety disorders,
major depressive disorder, bipolar disorder, schizophrenia, alcohol
use disorders, and drug use disorders. The C-DIS-IV is a structured
diagnostic interview that has been validated and is widely used
(Robins, Marcus, Reich, Cunningham, & Gallagher, 1996). Inter-
viewers were bachelor's degree-level research assistants who were
trained to administer the C-DIS-IV at a 3-day training seminar at the
Washington University St. Louis School of Medicine and by the first
and second authors. Inter-rater reliability was established at kappa of
.90 or higher for all diagnoses on 20 cases before the actual data
collection. Inter-rater reliability was established among all inter-
viewers and was periodically reassessed by the first and second
authors by reviewing tape-recorded participant interviews for
accuracy and reliability (recordings occurred with participants'
informed consent). Most participants completed the self-report,
paper-and-pencil PTSD questionnaires without assistance. Diagnostic
interviews were conducted before the self-report questionnaires to
prevent interviewer biases.
To assess PTSD, participants were administered the full PTSD C-
DIS-IV module. Participants were first asked if they had experienced
combat-related events (e.g., wounded, saw someone seriously injured
or killed, or unexpectedly discovered a dead body) and/or events not
related to military combat (e.g., threatened with a weapon, or
experienced a break-in or robbery, raped/sexually assaulted, experi-
enced an unexpected, sudden death of a close friend or relative, or
diagnosed with a life-threatening illness). Events encountered only
through reading or electronic media were not counted. Next,
participants' reactions to the event were assessed by asking: “After
a very frightening or horrible experience, some people can't get it out
of their minds. They may lose interest in other people or activities;
they may not sleep well; and they may become very jumpy and easily
startled or frightened. Did this experience have that effect on you?”
Then PTSD symptoms were assessed.
2.2.3. PTSD Checklist–Civilian version (PCL-C)
The 17-item posttraumatic stress disorder checklist (Forbes,
Creamer, & Biddle, 2001; Weathers et al., 1993) assessed 17 DSM-IV
PTSD symptoms during the last 30 days. Using a five-point scale from
not at all to extremely, patients were asked to rate how bothered they
had been by the problems, such as disturbing dreams, intrusive
thoughts or images, reminders of stressful experience, avoidance of
thinking or talking about stressful experience, avoidance of activities
or situations, feeling distant, feeling irritable, difficulty concentrating,
and feeling easily startled. The PCL exhibited high internal consistency
in our samples of patients initiating treatment at SUD and MH clinics,
with Cronbach's alpha of .96 for both samples.
2.2.4. Abbreviated versions of the PCL
Bliese et al. (2008) have suggested the use of a four-item,
abbreviated PCL scale consisting of item 1 (repeated disturbing
memories), 5 (physical reactions), 7 (avoiding activities), and 15
(difficulty concentrating) that has an item total range of 0 to 16. For
brevity, this scale will hereafter be referred as PCL-Bliese-4.Lang and
Stein (2005) have suggested using four other abbreviated variations
of the PCL. They included: (1) a two-item scale with total scores
ranging from 0 to 8 that comprises PCL items 1 and 4 (upset when
reminded); (2) a three-item scale with total score values ranging from
0 to 12 that comprises items 1, 4, and 7; (3) a four-item scale with
total scores ranging from 0 to 16 that comprises items 1, 4, 7, and 16
(being on guard); and (4) a six-item scale ranging in total scores from
0 to 24 and comprising items 1, 4, 7, 10 (feeling distant), 14 (irritable),
and 15. For brevity, these abbreviated PCL scales suggested by Lang
137Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

and Stein will hereafter be referred to as PCL-LS-2, PCL-LS-3, PCL-LS-4,
and PCL-LS-6, respectively.
2.2.5. Primary Care–PTSD screen (PC-PTSD)
The PC-PTSD is a four-item self-report measure suitable for
individuals at an eighth grade reading level. Participants responded
to yes/no questions related to nightmares, avoidance, being on guard,
and feeling numb in the past month. The measure has demonstrated
good test–retest reliability (Calhoun et al., 2010; Prins et al., 2003).
2.2.6. Diagnosis retrieved from VA medical record
We obtained information from nationwide VA medical electronic
databases on patients' psychiatric diagnoses and substance use
diagnoses for 9 months before to 9 months after the C-DIS-IV was
administered, with time-frame individualized to each participant's
interview date. We focused on whether the following current
diagnoses were present: PTSD, other anxiety disorders, major
depressive disorder, bipolar, schizophrenia, alcohol abuse, alcohol
dependence, drug abuse, and drug dependence. Diagnoses were made
by VA licensed clinicians during regular clinical intake interviews in
the usual process of care.
2.3. Data analyses
Analyses assessed concordance between C-DIS-IV and medical
record diagnoses separately among our samples of patients initiating
treatment at SUD and MH clinics to evaluate whether PTSD was
under-diagnosed in each of these settings. Signal detection analyses
were conducted to examine the psychometric properties of each
PTSD screen at all possible cut-points to determine the diagnostic
utility of the cutoff values (see Fig. 1). Evaluated psychometric
properties included: sensitivity (the proportion of those who have
the C-DIS-IV diagnosis and are correctly identified by the screening
test), specificity (the proportion of those who do not have the C-
DIS-IV diagnosis and are correctly identified as such by the test),
positive predictive value (PPV; the proportion of those who screen
positive that actually have the C-DIS-IV diagnosis), negative
predictive value (NPV; the proportion of those who screen negative
that actually do not have the C-DIS-IV diagnosis), and efficiency (the
proportion of those who are correctly classified by the test). In
addition, AUC was calculated for the PCL, PC-PTSD, and the five
abbreviated versions of the PCL across all cut-off points. Statistical
comparisons of the areas under two ROC curves using Wilcoxon
analyses (Hanley & McNeil, 1983) were conducted separately for
each of the two samples to compare AUCs between the PCL and the
PC-PTSD, and between the PCL as well as the PC-PTSD with the five
abbreviated versions of the PCL.
3. Results
3.1. Participant characteristics
Demographic characteristics of the 158 participants who received
services within an SUD specialty treatment program are provided in
Table 3. The sample was, on average, middle aged (M= 48.48; SD =
9.72) and male (96.8%), and few were currently married (10.8%). The
majority of participants was African American (74.8%), and had 12 or
fewer years of education (53.2%). About a quarter of the participants
(24.7%) were homeless at treatment initiation; the average income in
the past 30 days was $458.58 (SD = 969.68). Among the three SUD
clinics, there was no statistically significant difference on age, the
proportion of non-Hispanic White participants, homeless status, or
income. Based on the C-DIS-IV results, 37.2% (n= 58) of the
participants met current (12-month) DSM-IV criteria for PTSD, 64.7%
met criteria for other anxiety disorders (general anxiety disorder,
panic disorder, agoraphobia, social phobia, and obsessive compulsive
disorder), 68.4% met criteria for major depressive disorder, 35.5% for
bipolar, 14.7% for schizophrenia, 68.2% for alcohol abuse or depen-
dence, 87.5% for drug abuse or dependence, and 98% for alcohol or
drug abuse or dependence. Among the 58 individuals who met the
diagnostic criteria for PTSD based on the C-DIS-IV, only 28 individuals
(48%) also had a PTSD diagnosis in their medical records. Individuals
who met criteria for PTSD, based on the C-DIS-IV, had a mean PCL
score of 41.29 (SD = 15.90), and a mean PC-PTSD score of 3.35
(SD = 1.08). Participants who had a PTSD diagnosis in their medical
records had a mean PCL score of 36.98 (SD = 20.97), and a mean PC-
PTSD score of 2.96 (SD = 1.51).
Table 3 also shows that participants who received services from
the general mental health treatment program were older, less likely to
be men, more likely to be non-Hispanic White, less likely to be African
American, more likely to be Hispanic or “other”race, more likely to
have higher educational level, higher income, PTSD, other anxiety
Table 3
Characteristics of validation samples of patients receiving care at specialty substance
use disorders (SUD) clinics and general mental health (MH) clinics.
Variable Substance use
disorder clinics
(n= 158)
General mental
health clinic
(n= 242)
χ
2
/t-tests
Age in years, M(SD) 48.48 (9.7) 50.59 (9.0) t=−2.22⁎
Gender—male, n(%) 153 (96.8) 209 (86.4) χ
2
= 12.19⁎⁎⁎
Marital status, n(%) χ
2
= .11
Never married 43 (27.2) 63 (26.0)
Married 17 (10.8) 25 (10.3)
Separated/
divorced/widowed
98 (62.0) 154 (63.6)
Race/ethnicity, n(%)
White
(non-Hispanic)
36 (22.8) 121 (50.0) χ
2
= 29.69⁎⁎⁎
African American 116 (73.4) 77 (31.8) χ
2
= 66.25⁎⁎⁎
Hispanic/Latino 3 (1.9) 19 (7.9) χ
2
= 6.517⁎
Other race 3 (1.9) 24 (9.9) χ
2
= 9.77⁎⁎
Education, n(%) χ
2
= 19.54⁎⁎⁎
High school
or below
84 (53.2) 78 (32.2)
Some college 61 (38.6) 120 (49.6)
Four or more years
of college
13 (8.2) 44 (18.2)
Homeless
(shelter/street), n(%)
39 (24.7) 50 (20.7) χ
2
= .89
Income in last 30 days,
M(SD)
$458.58 (969.7) $946.60 (2250.1) t=−2.93⁎⁎
C-DIS-IV diagnoses, n(%)
PTSD 58 (37.2) 128 (53.1) χ
2
= 9.65⁎⁎
Other anxiety
disorders
101 (64.7) 190 (78.5) χ
2
= 9.15⁎⁎
Major depressive
disorder
106 (68.4) 194 (80.2) χ
2
= 7.10⁎⁎
Bipolar 54 (35.5) 68 (28.2) χ
2
= 2.33
Schizophrenia 23 (14.7) 51 (21.1) χ
2
= 2.51
Alcohol abuse/
dependence
103 (68.2) 165 (68.8) χ
2
= .01
Drug abuse/
dependence
133 (87.5) 155 (64.3) χ
2
= 25.59⁎⁎⁎
Alcohol or drug
disorders
149 (98.0) 200 (83.0) χ
2
= 21.20⁎⁎⁎
PCL, M (SD) 28.0 (18.5) 32.5 (19.6) t=−2.30⁎
PC-PTSD, n(%) χ
2
= 4.75
0“yes”responses 41 (26.1) 44 (18.2)
One or more “yes”19 (12.1) 29 (12.0)
Two or more “yes”18 (11.5) 24 (9.9)
Three or more “yes”21 (13.4) 35 (14.5)
Four or more “yes”58 (36.9) 110 (45.5)
Note: Other race = Asian/Pacific Islander, Native American, and multiracial; C-DIS-
IV = Computerized Diagnostic Interview Schedule for DSM-IV; PTSD = post traumatic
stress disorder; PCL = PTSD Checklist; PC-PTSD = Primary Care–PTSD screen; χ
2
=
Chi-square; t=t-tests.
⁎pb.05.
⁎⁎ pb.01.
⁎⁎⁎ pb.001.
138 Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

disorders, major depressive disorder, less likely to have drug use
disorders or alcohol and drug use disorders, and had higher scores on
the PCL than participants who received services within an SUD
specialty treatment program. The mental health sample had a mean
age of 50.59 (SD = 9.00), and 86.4% were male (Table 3). As among
the SUD clinic sample, few participants in this sample were married
(10.3%). Half of the participants were non-Hispanic White (50%), and
almost half of the participants (49.6%) had some college education.
Approximately one-fifth (20.7%) of the MH clinic participants were
homeless, and the average past-30-day income was $946.60 (SD =
2250.10). More than half of the MH clinic participants were found to
meet the diagnostic criteria of a current PTSD disorder (n= 128,
53.1%), 78.5% met criteria for other anxiety disorders (general anxiety
disorder, panic disorder, agoraphobia, social phobia, and obsessive
compulsive disorder), 80.2% for major depressive disorder, 28.2% for
bipolar, 21.1% for schizophrenia, 68.8% for alcohol abuse or depen-
dence, 64.3% for drug abuse or dependence, and 83% for alcohol or
drug abuse or dependence. Among the 128 individuals who met the
diagnostic criteria for PTSD based on the C-DIS-IV, only 77 individuals
(60%) also had a PTSD diagnosis in their medical records. Individuals
who met the diagnostic criteria for PTSD, based on the C-DIS-IV, had a
mean of PCL = 41.36 (SD = 16.18), and a mean PC-PTSD = 3.33
(SD = 1.20). The mean PCL and PC-PTSD scores were 42.23 (SD =
15.70) and 3.36 (SD = 1.16), respectively, among individuals who
had a PTSD diagnosis in their VA medical records.
3.2. Patients initiating treatment at substance use disorder (SUD) clinics
Table 4 summarizes and compares the psychometric properties of
the PC-PTSD, PCL, and its abbreviated versions. The PCL had an
efficiency of .79 at cut-points 33 and 34. Sensitivity and specificity
differed slightly for each of these cut-points. The PC-PTSD had a
sensitivity of .79 at a cut point of 3 (specificity = .67, PPV = .58,
NPV = .85, efficiency = .72). It had a better efficiency of .76 at a cut
point of 4, but a much lower sensitivity level of .67 (specificity = .82,
PPV = .68, and NPV = .81). The PCL-Bliese-4 (Bliese et al., 2008) had
the highest efficiency (E= .81) among the abbreviated variations of
the PCL examined in this study, at both cut-points 8 and 9. With a cut-
point of 8, the scale had a sensitivity of .71 and specificity of .88; with a
cut point of 9, the scale had a lower sensitivity of .65, and a higher
specificity of .90. The second highest efficiency (E= .80) among the
variations of the PCL examined was the PCL-LS-4 (Lang & Stein, 2005),
with a sensitivity of .71 and specificity of .86. The AUCs ranged from
.86 to .80. Differences in AUCs were calculated (Hanley & McNeil,
1983) between the PCL, PC-PTSD, and each of the abbreviated
variations of the PCL screening measures, and no significant
differences were found (pN.05).
3.3. Patients initiating treatment at the general mental health (MH) clinic
Table 5 summarizes the psychometric properties of the PCL, PC-
PTSD, and abbreviated variations of the PCL. Compared to psycho-
metric properties observed among participants treated in SUD clinics,
the sensitivity was similar or slightly higher but the overall specificity
was reduced among the MH clinic sample. The PCL exhibited its
highest efficiency of .72 at cut-points 30 and 32, with the former
having a slightly higher sensitivity (.79). The PC-PTSD exhibited its
highest efficiency (.76) among the scales examined in the mental
health clinic sample, with sensitivity = .70 and specificity = .82.
However, this instrument showed a better sensitivity (0.81) at a cut-
point of 3. Among the abbreviated variations of the PCL, the PCL-
Bliese-4 and the PCL-LS-2 had the same and highest efficiency of .74.
At a cut point of 7, the PCL-Bliese-4 had a sensitivity = .80 and
specificity = .67. The two-item PCL-LS-2 had a sensitivity = .82, and
specificity = .65. The AUCs ranged from .80 to .77. As was the case
among the SUD clinic sample, differences in AUCs among the
screening measures were non-significant (pN.05).
4. Discussion
PTSD is prevalent but under-detected in studied SUD and general
MH treatment settings, highlighting the need to consider use of a
PTSD screening instrument. Results of this study provide confirma-
tion of the validity of using the PCL, PC-PTSD, PC-Bliese-4, and the
PCL-LS-2 to help identify PTSD among patients seen in SUD and MH
treatment settings.
The four-item PCL-Bliese-4 was found to have the most potential
among abbreviated screeners for use in both the SUD and MH
treatment settings.
Consistent with results reported by previous studies (Andry-
kowski, Cordova, Studts, & Miller, 1998; Bliese et al., 2008; Lang &
Stein, 2005; Walker, Newman, Dobie, Ciechanowski, & Katon, 2002;
Yeager, Magruder, Knapp, Nicolas, & Frueh, 2007), current findings
showed that a cut-point between 30 and 34 provided optimal
Table 4
Psychometric properties of the PTSD Checklist (PCL), the Primary Care–PTSD (PC-PTSD) screen, and abbreviated variations of the PCL among patients treated in substance use
disorder (SUD) specialty treatment clinics (n= 158).
Measure AUC Cut-
point
Sensitivity
[95% CI]
Specificity
[95% CI]
PPV
[95% CI]
NPV
[95% CI]
Efficiency Test +
(%)
PCL 0.84 ≥30 0.77 [0.64–0.86] 0.76 [0.67–0.84] 0.65 [0.53–0.76] 0.85 [0.76–0.91] 0.76 44.4
≥31 0.75 [0.62–0.85] 0.78 [0.69–0.85] 0.67 [0.54–0.77] 0.84 [0.76–0.91] 0.77 43.1
≥32 0.73 [0.61–0.83] 0.78 [0.69–0.85] 0.66 [0.54–0.77] 0.84 [0.75–0.90] 0.76 41.2
≥33 0.73 [0.61–0.83] 0.82 [0.74–0.89] 0.71[0.58–0.81] 0.84 [0.76–0.90] 0.79 40.5
≥34 0.70 [0.58–0.80] 0.85 [0.76–0.90] 0.72 [0.59–0.82] 0.83 [0.74–0.89] 0.79 37.9
PC-PTSD 0.80 ≥3 0.79 [0.68–0.88] 0.67 [0.58–0.76] 0.58 [0.47–0.69] 0.85 [0.75–0.91] 0.72 49.7
≥4 0.67 [0.54–0.78] 0.82 [0.73–0.88] 0.68 [0.55–0.79] 0.81 [0.72–0.87] 0.76 36.1
PCL-Bliese-4 0.86 ≥7 0.76 [0.64–0.85] 0.78 [0.68–0.85] 0.67 [0.55–0.77] 0.84 [0.76–0.91] 0.77 42.3
(Bliese et al., 2008)≥8 0.71 [0.58–0.81] 0.88 [0.80–0.93] 0.77 [0.65–0.87] 0.83 [0.75–0.89] 0.81 34.0
≥9 0.65 [0.53–0.76] 0.90 [0.82–0.94] 0.79 [0.66–0.88] 0.81 [0.73–0.88] 0.81 30.8
PCL-LS-2 0.81 ≥4 0.72 [0.60–0.82] 0.73 [0.64–0.81] 0.62 [0.50–0.72] 0.82 [0.73–0.89] 0.73 43.6
(Lang & Stein, 2005)≥5 0.59 [0.46–0.70] 0.86 [0.77–0.91] 0.71 [0.57–0.82] 0.78 [0.69–0.85] 0.76 30.8
PCL-LS-3 0.84 ≥6 0.78 [0.65–0.86] 0.72 [0.63–0.80] 0.63 [0.51–0.73] 0.85 [0.75–0.91] 0.74 46.2
(Lang & Stein, 2005)≥7 0.67 [0.54–0.78] 0.81 [0.72–0.87] 0.67 [0.54–0.78] 0.81 [0.72–0.87] 0.76 37.2
PCL-LS-4 0.85 ≥8 0.74 [0.62–0.84] 0.77 [0.67–0.84] 0.65 [0.53–0.76] 0.83 [0.74–0.90] 0.76 42.3
(Lang & Stein, 2005)≥9 0.71 [0.58–0.81] 0.86 [0.77–0.91] 0.75 [0.62–0.84] 0.83 [0.75–0.89] 0.80 35.3
PCL-LS-6 0.84 ≥11 0.75 [0.63–0.85] 0.76 [0.67–0.84] 0.65 [0.53–0.76] 0.84 [0.75–0.90] 0.76 42.9
(Lang & Stein, 2005)≥12 0.75 [0.63–0.85] 0.78 [0.69–0.85] 0.67 [0.55–0.77] 0.84 [0.76–0.91] 0.77 41.6
AUC = area under the ROC curve; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value; Test + = test positive rate. Based on the C-DIS-IV,
36.8% of the SUD specialty treatment clinic sample met criteria for PTSD.
139Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

psychometric properties for the PCL in the studied samples. For
example, a cut-point at 33 provided the highest efficiency level for the
SUD sample (sensitivity = .73, specificity = .82, efficiency = .79,
positive test rate = 40.5%) and 32 as the best cut-point for the general
MH clinic sample (sensitivity = .75, specificity = .68, efficiency =
.72, positive test rate 57.3). Given the findings from previous studies
(e.g., Andrykowski et al., 1998; Blanchard, Jones-Alexander, Buckley,
& Forneris, 1996; Bliese et al., 2008; Dobie et al., 2002; Keen et al.,
2008; Lang & Stein, 2005; Manne, Du Hamel, Gallelli, Sorgen, & Redd,
1998; Terhakopian, Sianii, Engel, Schnurr, & Hoge, 2008; Walker et al.,
2002), the optimal cut-point for the PCL varies across study
populations. Consistent with the current findings, Terhakopian et al.,
(2008) found that in populations with high PTSD prevalence, cutoff
values of 44 or higher will likely provide too many false negatives.
However, some studies utilizing samples for which the PTSD
prevalence rate was lower also found lower optimal cutoff values
(e.g., Andrykowski et al., 1998; Bliese et al., 2008; Hudson, Beckford,
Jackson, & Philpot, 2008; Lang & Stein, 2005; Lang et al., 2003; Walker
et al., 2002; Yeager et al., 2007).
This study found that the PC-PTSD can be used as a screening
instrument for PTSD in patients receiving services at SUD specialty
clinics or general mental health clinics. The sensitivity levels were
found to be comparable to previous studies, but specificity levels were
lower. Specifically, a sensitivity levels of .79 (SUD sample) and .81
(MH sample) were higher than two previous studies (Bliese et al.,
2008; Prins et al., 2003) and lower than two others (Calhoun et al.,
2010; Kimerling et al., 2006), but specificity levels (.65 and .67) were
found to be lower than previous studies. In other words, this
instrument may provide higher rates of false positives (e.g., flagging
a patient who does not have a PTSD diagnosis for further diagnostic
evaluation) in SUD and general MH clinics than the rates identified by
previous studies. However, as noted by Calhoun et al. (2010) false
positives are clinically less costly than false negatives (e.g., failing to
identify a patient with PTSD), especially when the costs of follow-up
diagnostic evaluation are low. However, in settings where false
positives out-cost false negatives, clinicians and managers should
consider using a cut-point of 4 instead of 3 to raise the specificity level
and positive predictive value, and reduce the rate of positive test
(with a compromised sensitivity level and negative predictive value).
For example, the positive test rate will drop from 50 to 36% in the SUD
clinics sample, and from 60 to 45% in the MH clinic sample. A 15%
reduction in positive test rate means a reduction of follow-up
assessment, referral to other clinics, or providing intervention to
15% of the clinic population, which has important management and
financial implications. However, the considerable cost associated with
leaving PTSD undetected and untreated should also be considered.
Two abbreviated variations of the PCL should be noted. The PCL-
Bliese-4 exhibited psychometrics properties better than or compara-
ble to those identified for the PCL and PC-PTSD in both the SUD and
MH samples. This measure had the best AUC (0.86) among patients
initiating treatment in SUD settings, and it tied with the PC-PTSD for
the best AUC among patients initiating treatment in MH settings
(AUC = .80). In addition, it exhibited the highest efficiency among all
measures examined in this study, even higher than the 17-item PCL, in
the SUD clinics sample (see Tables 4 and 5). In the MH clinic sample
(see Table 5), its efficiency was second to the PC-PTSD when a PC-
PTSD cut-point of 4 was considered. However, such a PC-PTSD cut-
point is not recommended because of the associated compromised
sensitivity (0.7). After ruling out use of a PC-PTSD cut point of four, the
PCL-Bliese-4 exhibited the best efficiency without compromising
sensitivity. This finding is consistent with those suggested by Bliese et
al. (2008), who found an accuracy estimate for the PCL-Bliese-4
equivalent to the PC-PTSD and full PCL. Our data suggest that the PCL-
Bliese-4 could be used as an alternative for the PC-PTSD. Another brief
scale, the PCL-LS-2, despite having only two items, provided
comparable psychometric properties with other longer scales in
both the SUD and MH settings (SUD setting: AUC = .81; efficiency =
.73, sensitivity = .72; specificity = .73; MH setting: AUC = .77;
efficiency = .74, sensitivity = .82; specificity = .65). These results
are consistent with Lang and Stein (2005), which found the PC-LS-2 to
exhibit the best psychometric properties when at a cut-point of four
was used. The PCL-Bliese-4 and the PCL-LS-2 may potentially be used
as alternatives to the PC-PTSD, particularly in settings where a brief
measure is necessary.
The current results need to be interpreted with consideration of a
number of limitations. Although the prevalence of PTSD found in this
SUD sample (37.2%) was similar to over 350,000 SUD patients
receiving VA services in the fiscal year 2006 and 2008 (34.7 and
37.4%, respectively; Dalton & McKellar, 2007; McKellar, Dalton, &
Trafton, 2009), the findings may not be representative of all patients
in other SUD treatment programs. Second, the criterion of this study
relied on a structured diagnostic interview, and not a clinical
interview by a licensed psychologist or psychiatrist. Although
participants were informed that the study was independent from
Table 5
Psychometric properties of the PTSD Checklist (PCL), the Primary Care–PTSD (PC-PTSD) screen, and abbreviated variations of the PCL among patients treated at the general mental
health (MH) clinic (n= 241).
Measure AUC Cut-point Sensitivity
[95% CI]
Specificity
[95% CI]
PPV
[95% CI]
NPV
[95% CI]
Efficiency Test +
(%)
PCL 0.79 ≥30 0.79 [0.71–0.85] 0.63 [0.54–0.72] 0.71 [0.63–0.78] 0.72 [0.62–0.80] 0.72 58.5
≥31 0.77 [0.69–0.84] 0.63 [0.54–0.72] 0.71 [0.63–0.78] 0.70 [0.61–0.79] 0.71 58.1
≥32 0.75 [0.67–0.82] 0.68 [0.59–0.76] 0.73 [0.65–0.80] 0.70 [0.61–0.78] 0.72 57.3
≥33 0.72 [0.64–0.79] 0.70 [0.61–0.78] 0.73 [0.65–0.80] 0.69 [0.59–0.76] 0.71 54.4
≥34 0.69 [0.61–0.77] 0.72 [0.63–0.79] 0.74 [0.65–0.81] 0.67 [0.58–0.75] 0.70 51.9
PC-PTSD 0.80 ≥3 0.81 [0.74–0.87] 0.65 [0.55–0.73] 0.72 [0.64–0.79] 0.75 [0.66–0.83] 0.73 59.8
≥4 0.70 [0.61–0.77] 0.82 [0.74–0.88] 0.82 [0.73–0.88] 0.70 [0.62–0.78] 0.76 45.2
PCL-Bliese-4 0.80 ≥7 0.80 [0.72–0.86] 0.67 [0.58–0.75] 0.73 [0.66–0.80] 0.75 [0.65–0.82] 0.74 57.7
(Bliese et al., 2008)≥8 0.76 [0.68–0.82] 0.69 [0.60–0.77] 0.73 [0.65–0.80] 0.72 [0.63–0.79] 0.73 54.8
≥9 0.66 [0.57–0.73] 0.73 [0.65–0.81] 0.74 [0.65–0.81] 0.65 [0.57–0.73] 0.69 47.3
PCL-LS-2 0.77 ≥4 0.82 [0.75–0.88] 0.65 [0.55–0.73] 0.72 [0.65–0.80] 0.76 [0.67–0.84] 0.74 60.2
(Lang & Stein, 2005)≥5 0.67 [0.59–0.75] 0.71 [0.62–0.78] 0.72 [0.64–0.80] 0.66 [0.57–0.73] 0.69 49.4
PCL-LS-3 0.77 ≥6 0.80 [0.73–0.86] 0.65 [0.55–0.73] 0.72 [0.64–0.79] 0.74 [0.65–0.82] 0.73 59.3
(Lang & Stein, 2005)≥7 0.68 [0.60–0.75] 0.70 [0.61–0.78] 0.72 [0.63–0.79] 0.66 [0.57–0.74] 0.69 50.2
PCL - LS - 4 0.78 ≥7 0.80 [0.73–0.86] 0.63 [0.54–0.71] 0.71 [0.63–0.78] 0.74 [0.64–0.82] 0.72 60.2
(Lang & Stein, 2005)≥8 0.77 [0.69–0.84] 0.65 [0.55–0.73] 0.71 [0.63–0.78] 0.72 [0.62–0.79] 0.71 57.7
PCL-LS-6 0.77 ≥8 0.88 [0.82–0.93] 0.50 [0.41–0.60] 0.67 [0.60–0.74] 0.79 [0.68–0.87] 0.71 82.6
(Lang & Stein, 2005)≥9 0.82 [0.75–0.88] 0.53 [0.44–0.62] 0.66 [0.59–0.73] 0.72 [0.62–0.81] 0.68 78.0
AUC = area under the ROC curve; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value; Test + = test positive rate. Based on the C-DIS-IV,
53.1% of the mental health clinic sample met criteria for PTSD.
140 Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

and had no influence on their application to become service
connected for PTSD for VA benefits, potential over-reporting of PTSD
symptoms by some participants was not completely ruled out. In
addition, analyses comparing AUCs of PTSD screening measures
revealed no statistically significant differences; therefore, application
of findings should be considered in light of this finding. The ecological
validity of study findings could also be limited. Study participants
were informed that their symptom reports would have no influence
on VA benefits claims, whereas accounts of symptoms and impair-
ment among patients in clinical settings can be used to inform benefit-
related decisions. Furthermore, screening usually takes place before
diagnostic interview in clinical practice whereas in this study
screening instruments were administered after diagnostic interview
to prevent potential interviewer biases. Finally, both the PCL and PC-
PTSD were based on the past 30 days whereas the diagnostic
interview relied on a past 12-month period; therefore, these two
measures could not be compared with the diagnostic interview
directly in this study.
Given the existing low detection rate (e.g., Liebschutz et al., 2007;
Magruder et al., 2005) and under-treatment of PTSD (e.g., Davidson et
al., 2004; Kessler et al., 1995; Schnurr et al., 2000), combined with
validity information provided in this study regarding the use of brief
screening instruments (e.g., PC-PTSD, PCL-Bliese-4, PCL-LS-2), imple-
mentation of PTSD screens in SUD and MH treatment settings should
be considered. Future studies on the feasibility and cost-effectiveness
of the implementation of brief PTSD screening instruments in SUD
specialty and general MH treatment settings are recommended.
Routine PTSD screening and better identification of PTSD diagnosis in
mental health systems will provide opportunities to better serve
patients with PTSD, lead to better patient outcomes, and may contain
health care cost.
Acknowledgment
This work was partially supported by Department of Veterans
Affairs Program Evaluation and Resource Center/Mental Health
Strategic Healthcare Group (XVA 62-004). The opinions expressed
in this paper are those of the authors and do not necessarily reflect
those of the Department of Veterans Affairs. We wish to thank the
staff and patients of the VA treatment programs who participated in
this project and Valerie Jackson, Leah McKechnie, Dr. Michele
Stefan, Dr. Joseph Liberto, Dr. Mark Mann, and Allison Davis for
their contributions.
References
Andrykowski, M. A., Cordova, M. J., Studts, J. L., & Miller, T. W. (1998). Posttraumatic
stress disorder after treatment for breast cancer: Prevalence of diagnosis and use of
the PTSD Checklist–Civilian Version (PCL-C) as a screening instrument. Journal of
Consulting and Clinical Psychology,66, 586–590.
Beckham, J. C., Taft, C. T., Vrana, S. R., Feldman, M. E., Barefoot, J. C., Moore, S., et al.
(2003). Ambulatory monitoring and physical health report in Vietnam veterans
with and without chronic posttraumatic stress disorder. Journal of Trauma Stress,
16, 329–335, http://dx.doi.org/10.1023/A:1024457700599.
Blanchard, E. B., Jones-Alexander, J., Buckley, T. C., & Forneris, C. A. (1996). Psychometric
properties of the PTSD Checklist (PCL). Behavior Research and Therapy,34, 669–673.
Bliese, P. D., Wright, K. M., Adler, A. B., Cabrera, O., Castro, C. A., & Hoge, C. W. (2008).
Validating the primary care posttraumatic stress disorder screen and the
posttraumatic stress disorder checklist with soldiers returning from combat. Journal
of Consulting and Clinical Psychology,76, 272–281, http://dx.doi.org/10.1037/0022-
006X.76.2.272.
Bliese, P. D., Wright, K., Adler, A. B., & Thomas, J. (2004). Validation of the 90 and
120 day post-deployment psychological short screen (U.S. Army Medical Research
Unit—Europe Report No. 2004-002). Heidelberg, Germany: USAMRU-E.
Bliese, P. D., Wright, K., Adler, A. B., Thomas, J., & Hoge, C. (2004). Screening for
traumatic stress among re-deploying soldiers (No. USAMRU-E Research Report
#2004-001). Heidelberg, Germany: USAMRU-E.
Bollinger, A. R., Cuevas, C. A., Vielhauer, M. J., Morgan, E. E., & Keane, T. M. (2008). The
operating characteristics of the PTSD checklist in detecting PTSD in HIV+ sub-
stance abusers. Journal of Psychological Trauma,7, 213–234, http://dx.doi.org/
10.1080/19322880802384251.
Breslau, N., Kessler, R., & Peterson, E. (1998). Post-traumatic stress disorder assessment
with a structured interview: Reliability and concordance with a standardized
clinical interview. International Journal of Methods in Psychiatric Research,7,
121–127.
Calhoun, P. S., McDonald, S. D., Guerra, V. S., Eggleston, A. M., Beckham, J. C., & Straits-
Troster, K. (2010). Clinical utility of the primary care—PTSD screen among U.S.
veterans who served since September 11, 2001. Psychiatry Research,178, 330–335,
http://dx.doi.org/10.1016/j.psychres.2009.11.009.
Dalton, A., & McKellar, K. (2007). Health services for VA substance use disorder
patients: Comparison of utilization in fiscal years 2006, 2005, and 2002. Palo Alto,
CA: Department of Veterans Affairs Program Evaluation and Resource Center and
HSR&D Center for Health Care Evaluation.
Davidson, J. R., Stein, D. J., Shalev, A. Y., & Yehuda, R. (2004). Posttraumatic stress
disorder: Acquisition, recognition, course, and treatment. The Journal of Neuropsy-
chiatry and Clinical Neurosciences,16,135–147, http://dx.doi.org/10.1176/
appi.neuropsych.16.2.135.
Department of Veterans Affairs and Administration. (2004). Under secretary for
health's information letter: Clinical reminder regarding veterans of the recent
conflicts in Afghanistan and Iraq (no. IL 10-2004-01). U.S. Department of Veterans
Affairs and Administration.
Dobie, D. J., Kivlahan, D. R., Maynard, C., Bush, K. R., McFall, M., Epler, A. J., & Bradley,
K. A. (2002). Screening for post-traumatic stress disorder in female veterans
affairs patients: Validation of the PTSD Checklist. General Hospital Psychiatry,24,
367–374.
Dohrenwend, B. P., Turner, J. B., Turse, N. A., Adams, B. G., Koenen, K. C., Marshall, R.,
et al. (2006). The psychological risks of Vietnam for U.S. Veterans: A revisit with
new data and methods. Science,313,979–982, http://dx.doi.org/10.1126/
science.1128944.
Foa, E. B., Keane, T. M., Friedman, M. J., & Cohen, J. A. (Eds.). (2008). Effective treatments
for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies
(2nd ed.). New York: The Guilford Press.
Forbes, D., Creamer, M., & Biddle, D. (2001). The validity of the PTSD Checklist as a
measure of symptomatic change in combat-related PTSD. Behavior Therapy and
Research,39, 977–986, http://dx.doi.org/10.1016/S0005-7967(00)00084-X.
Gore, K. L., Engel, C. C., Freed, M. C., Liu, X., & Armstrong, D. W. (2008). Test of a single-item
posttraumatic stress disorder screener in a military primary care setting. General
Hospital Psychiatry,30, 391–397, http://dx.doi.org/10.1016/j.genhosppsych.2008.05.002.
Grant, B. F., Stinson, F. S., Dawson, D. A., Chou, S. P., Dufour, M. C., & Compton, W. (2004).
Prevalence and co-occurrence of substance use disorders and independent mood
and anxiety disorders: Results from the National Epidemiologic Survey on Alcohol
and Related Conditions. Archives of G eneral Psy chiatry,61,807–816, http:
//dx.doi.org/10.1001/archpsyc.61.8.807.
Grubaugh, A. L., Elhai, J. D., Cusack, K. J., Wells, C., & Frueh, B. C. (2007). Screening for
PTSD in public-sector mental health settings: The diagnostic utility of the PTSD
Checklist. Depression and Anxiety,24, 124–129, http://dx.doi.org/10.1002/da.20226.
Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under receiver
operating characteristic curves derived from the same cases. Radiology,148,
839–843.
Harrington, T., & Newman, E. (2007). The psychometric utility of two self-report
measures of PTSD among women substan ce users. Addictive Behaviors,32,
2788–2798, http://dx.doi.org/10.1016/j.addbeh.2007.04.016.
Hoge, C. W., Auchterlonie, J. L., & Milliken, C. S. (2005). Mental health problems, use of
mental health services, and attrition from military service after returning from
deployment to Iraq or Afghanistan. Journal of the American Medical Association,295,
1023–1032.
Hudson, S. A., Beckford, L. A., Jackson, S. D., & Philpot, M. P. (2008). Validation of a
screening instrument for post-traumatic stress disorder in a clinical sample of older
adults. Aging & Mental Health,12,670–673, http://dx.doi.org/10.1080/
13607860802343274.
Institute of Medicine. (2007). Treatment of PTSD: An assessment of the evidence.
Washington, DC: National Academies Press.
Johnson, D. R., Fontana, A., Lubin, H., Corn, B., & Rosenheck, R. (2004). Long-term course
of treatment-se eking Vietnam veterans with post traumatic stress disorder:
Mortality, clinical condition, and life satisfaction. The Journal of Nervous and Mental
Disease,192,35–41, http://dx.doi.org/10.1097/01.nmd.0000105998.90425.6a.
Kimerling, R., Trafton, J. A., & Nguyen, B. (2006). Validation of a brief screen for post-
traumatic stress disorder with substance use disorder patients. Addictive Behaviors,
31, 2074–2079, http://dx.doi.org/10.1016/j.addbeh.2006.02.008.
Keen, S. M., Kutter, C. J., Niles, B. L., & Krinsley, K. E. (2008). Psychometric properties of
PTSD Checklist in sample of male veterans. Journal of Rehabilitation Research and
Development,45, 465–474, http://dx.doi.org/10.1682/JRRD.2007.09.0138.
Kessler, R. C., Berglund, P., Delmer, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005a).
Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the
National Comorbidity Surv ey Replication. Archive s of General Psychiatry,62,
593–602, http://dx.doi.org/10.1001/archpsyc.62.6.593.
Kessler, R. C., Chiu, W. T., Demler, O., Merikangas, K. R., & Walters, E. E. (2005b).
Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the
National Comorbidity Surv ey Replication. Archive s of General Psychiatry,62,
617–627, http://dx.doi.org/10.1001/archpsyc.62.6.617.
Kessler, R. C., Sonnega, A., Bromet, E., Hughes, M., & Nelson, C. B. (1995). Posttraumatic
stress disorder in the National Comorbidity Survey. Archives of General Psychiatry,
52, 1048–1060.
Lang, A. J., Laffaye, C., Satz, L. E., Dresselhaus, T. R., & Stein, M. B. (2003). Sensitivity and
specificity of the PTSD Checklist in detecting PTSD in female veterans in primary
care. Journal of Traumatic Stress,16,257–264, http://dx.doi.org/10.1023/A:
1023796007788.
141Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142

Lang, A. J., & Stein, M. B. (2005). An abbreviated PTSD Checklist for use as a screening
instrument in primary care. Behaviour Research and Therapy,43, 585–594.
Liebschutz, J., Saitz, R., Brower, V., Keane, T. M., Lloyd-Travaglini, C., Averbuch, T., &
Samet, J. H. (2007). PTSD in urban primary care: High prevalence and low physician
recognition. Journal of General Internal Medicine,22, 719–726, http://dx.doi.org/
10.1007/s11606-007-0161-0.
Magruder, K. M., Frueh, B. C., Knapp, R. G., Davis, L., Hammer, M. B., Martin, R., et al.
(2005). Prevalence of posttraumatic stress disorder in Veterans Affairs primary care
clinics. GeneralHospitalPsychiatry,27,169–179, http://dx.doi.org/10.1016/
j.genhosppsych.2004.11.001.
Manne, S. L., Du Hamel, K., Gallelli, K., Sorgen, K., & Redd, W. H. (1998). Posttraumatic
stress disorder among mothers of pediatric cancer survivors: Diagnosis, comorbid-
ity, and utility of the PTSD Checklist as a screening instrument. Journal of Pediatric
Psychology,23, 357–366 Retrieved from , http://jpepsy.oxfordjournals.org/
McDonald, S. D., & Calhoun, P. S. (2010). The diagnostic accuracy of PTSD Checklist: A
critical review. Clinical Psychology Review,30, 976–987, http://dx.doi.org/10.1016/
j.cpr.2010.06.012.
McKellar, J. D., Dalton, A., & Trafton, J. (2009). Health services for VA substance use
disorder patients: Comparison of Utilization for Fiscal Years 2002 to 2007. Palo Alto,
CA: Department of Veterans Affairs Program Evaluation and Resource Center and
HSR&D Center for Health Care Evaluation.
McKellar, J. D., & Saweikis, M. (2005). Health services for VA substance use disorder
patients: Comparison of utilization in fiscal years 2004, 2003, and 1998. Palo Alto,
CA: Department of Veterans Affairs Program Evaluation and Resource Center and
HSR&D Center for Health Care Evaluation.
Prins, A., Ouimette, P., Kimerling, R., Cameron, R. P., Hugelshofer, D. S., Shaw-Hegwer, J.,
et al. (2003). The primary care PTSD screen (PC-PTSD): Development and operating
characteristics. Primary Care Psychiatry,9,9–14, http://dx.doi.org/10.1185/
135525703125002360.
Ramchand, R., Schell, T. L., Karney, B. R., Osilla, K. C., Burns, R. M., & Caldarone, L. B.
(2010). Disparate prevalence estimates of PTSD among service members who
served in Iraq and Afghanistan: Possible explanations. Journal of Traumatic Stress,
23,59–68, http://dx.doi.org/10.1002/jts.20486.
Regier, D. A., Farmer, M. E., Rae, D. S., Locke, B. Z., Keith, S. J., Judd, L. L., & Goodwin, F. K.
(1990). Comorbidity of mental disorders with alcohol and other drug abuse:
Results from the Epidemiologic Catchment Area (ECA) study. Journal of the
American Medical Association,264, 2511–2518.
Robins, L., Marcus, L., Reich, W., Cunningham, R., & Gallagher, T. (1996). NIMH diagnostic
interview schedule: Version IV (DIS-IV). St. Louis: Washington University.
Schnurr, P. P., Friedman, M. J., Sengupta, A., Jankowski, M. K., & Holmes, T. (2000). PTSD
and utilization of medical treatment services among male Vietnam veterans. The
Journal of Nervous and Mental Disease,188, 496–504.
Stein, M. B., McQuaid, J. R., Pedrelli, P., Lenox, R., & McCahill, M. E. (2000). Posttraumatic stress
disorder in the primary care medical setting. GeneralHospitalPsychiatry,22,261–269.
Tanielian, T., & Jaycox, L. (Eds.). (2008). Invisible wounds of war: Psychological and
cognitive injuries, their consequences, and services to assist recovery (pp. 124–148).
Santa Monica, CA: RAND Corporation.
Terhakopian, A., Sianii, N., Engel, C. C., Schnurr, P. P., & Hoge, C. W. (2008). Estimating
population prevalence of posttraumatic stress disorder: An example using the PTSD
Checklist. Journal of Traumatic Stress,21,290 –300, http://dx.doi.org/10.1002/jts.20341.
Tiet, Q. Q., Byrnes, H. F., Barnett, P., & Finney, J. W. (2006). A practical system for
monitoring the outcomes of substance use disorder patients. Journal of Substance
Abuse Treatment,30, 337–347.
Van Dam, D., Ehring, T., Vedel, E., & Emmelkamp, P. M. G. (2010). Validation of the
Primary Care Posttraumatic Stress Disorder screening questionnaire (PC-PTSD) in
civilian substance use disorder patients. Journal of Substance Abuse Treatment,39,
105–113, http://dx.doi.org/10.1016/j.jsat.2010.05.005.
Walker, E. A., Newman, E., Dobie, D. J., Ciechanowski, P., & Katon, W. (2002). Validation
of the PTSD checklist in an HMO sample of women. General Hospital Psychiatry,24,
375–380.
Weathers, F., Litz, B., Herman, D., Huska, J., & Keane, T. (1993). The PTSD checklist (PCL):
Reliability, validity, and diagnostic utility. Paper presented at the Annual Convention
of the International Society for Traumatic Stress Studies, San Antonio, TX.
Widows, M. R., Jacobsen, P. B., & Fields, K. K. (2000). Relation of psychological
vulnerability factors to posttraumatic stress disorder symptomatology in bone
marrow transplant recipients. Psychosomatic Medicine,62, 873–882.
Yeager, D. E., Magruder, K. M., Knapp, R. G., Nicolas, J. S., & Frueh, B. C. (2007).
Performance characteristics of the posttraumatic stress disorder checklist and
SPAN in veterans affairs primary care settings. General Hospital Psychiatry,29,
294–301, http://dx.doi.org/10.1016/j.genhosppsych.2007.03.004.
142 Q.Q. Tiet et al. / Journal of Substance Abuse Treatment 45 (2013) 134–142