Content uploaded by Natoshia Raishevich Cunningham
Author content
All content in this area was uploaded by Natoshia Raishevich Cunningham on Sep 04, 2014
Content may be subject to copyright.
Phenomenology of Clinic-Referred Children and Adolescents
with Oppositional Defiant Disorder and Comorbid Anxiety
Natoshia Raishevich Cunningham &
Thomas H. Ollendick &James L. Peugh
Published online: 29 December 2012
#Springer Science+Business Media New York 2012
Abstract This study examined profiles of clinic-referred
youth with co-morbid oppositional defiant disorder (ODD)
and anxiety disorders (ADs) compared to youth with ODD
without ADs. One hundred and twenty seven clinic-referred
youth with ODD (ages 7–14, 85.6 % Caucasian) were assessed
through a multi-method, multi-informant approach. Global
functioning, ODD symptom impairment, child internalizing
symptoms, caregiver distress, and parent-child relationship
quality were explored to test group differences based on AD
diagnosis. Youth with ODD and comorbid ADs generally had
higher levels of global impairment, internalizing symptoms,
caregiver distress, and parent-child relationship problems as
compared to youth with ODD only. These findings, which
generally suggest greater impairment in the group of youth
with ODD/AD, offer support for the presence of distinct clin-
ical features in youth with ODD/AD compared to youth with
ODD alone. Such findings may have important implications
for assessment and treatment of ODD in youth. For example,
interventions to target broader child internalizing symptoms,
caregiver distress, and parent-child relationships may be par-
ticularly important in youth with ODD/AD profiles.
Keywords Comorbidity .Anxiety .Oppositional defiant
disorder .Impairment
The co-occurrence of anxiety disorders (ADs) and opposition-
al defiant disorder (ODD) affects a substantial proportion of
children and may cause significant impairment in functioning.
Epidemiological data indicate 11.3 % of children meet criteria
for ODD and 9.9 % of children meet criteria for an AD
(Costello et al. 2003); thus, the occurrence of both disorders
in youth is common. Interestingly, the comorbidity rates of
ADs in youth with ODD are three to seven times more likely
to occur than what might be expected by chance (Angold et al.
1999). In their review of youth with ODD in epidemiological
samples, Angold and colleagues reported that the rates of co-
occurring ADs ranged from 9.6 % to 49.86 %. Similarly
Greene and colleagues (2002) noted high rates of comorbidity
in clinical samples, where approximately 40 % of clinic-
referred youth with ODD met criteria for an AD. These dis-
orders may be more likely to co-occur for both methodolog-
ical (e.g., referral bias) and substantive (e.g., shared risk
factors) reasons (e.g., Angold et al. 1999;Cunninghamand
Ollendick 2010). Although it is known that ADs are signifi-
cantly more likely to co-occur in youth with ODD, the impact
of ADs on functioning in youth with ODD has not been well
established. Thus, the purpose of the current study was to
examine whether the clinical profile of youth with ODD/AD
is distinct and whether this profile signals greater impairment
compared to youth with ODD in the absence of an AD.
The role of anxiety in predicting impairment in youth with
ODD is not well understood though conceptual models suggest
comorbid anxiety generally serves as either a risk factor or a
protective factor in predicting impairment in youth with ODD
(e.g., Cunningham and Ollendick 2010; Drabick et al. 2010).
This study was based on Natoshia Raishevich Cunningham’s
dissertation completed at Virginia Polytechnic Institute and State
University and through an NIMH Grant awarded to Thomas H.
Ollendick (R01 MH076141).
N. R. Cunningham (*):J. L. Peugh
Behavioral Medicine and Clinical Psychology,
Cincinnati Children’s Hospital Medical Center,
MLC 3014, 333 Burnet Avenue,
Cincinnati, OH 45229-3026, USA
e-mail: natoshia.cunningham@cchmc.org
J. L. Peugh
e-mail: james.peugh@cchmc.org
N. R. Cunningham :T. H. Ollendick
Child Study Center, Department of Psychology, Virginia
Polytechnic Institute and State University, 460 Turner Street,
Blacksburg, VA 24061-0355, USA
T. H. Ollendick
e-mail: tho@vt.edu
J Psychopathol Behav Assess (2013) 35:133–141
DOI 10.1007/s10862-012-9335-0
For those who conceptualize anxiety as a risk factor, it is
purported that the effect of anxiety is additive and increases
impairment in the presence of other psychopathology, such as
ODD. On the other hand, anxiety may also serve as a protective
factor in youth with ODD, depending on contextual factors
such as emotionality, family functioning, and age of the child.
Forexample,achildwithODDwhoisalsosociallyanxious
may be less likely to engage in disruptive behavior due to fear
of negative evaluation from parents, teachers, or peers; more-
over, this effect may be stronger for older children who have
greater insight into social norms. In fact, based on a systematic
review, Cunningham and Ollendick (2010) concluded that
youth with ODD and ADs may be at increased risk for both
global impairment, elevated emotional and behavioral symp-
tomatology, and increased family dysfunction, but that this risk
might diminish as youth age. However, these conclusions were
based on research examining youth with diverse anxiety dis-
orders and a broad array of behavioral disorders, including
attention deficit hyperactivity disorder (ADHD), ODD, and
conduct disorder, instead of ODD alone. To our knowledge,
no study to date has specifically compared general functioning,
symptom levels, or family stressors in a diverse age-range of
youth with diagnoses of ODD/AD to youth with ODD in the
absence of AD. Given our limited understanding of the impact
of ADs in youth with ODD, further examination of the clinical
profile of such youth is of importance.
Although the clinical features of comorbid ODD/AD are not
well understood, the clinical correlates and family environment
characteristics of ADs and ODD have been well substantiated
when examined as singular disorders (American Psychiatric
Association 2000). Of course, true to their respective diagnoses,
elevated levels of anxiety have been found in youth with ADs
whereas elevated levels of oppositional, aggressive, and delin-
quent behavior have characterized youth with ODD
(Achenbach 1990; American Psychiatric Association 2000;
March et al. 1997). These relationships have generally been
obtained across clinician, parent, and child informants. Further-
more, there is an extensive literature that links these disorders to
specific forms of parent psychopathology and family environ-
ment functioning. Angold and colleagues (1999)revieweda
series of papers that linked maternal internalizing symptoms to
ADs and conduct problems, such as ODD, in children (Beidel
and Turner 1997;Hammen1992;Lastetal.1987). Lansford
and colleagues (2002,2005) suggested higher levels of family
conflict in youth with internalizing and externalizing behaviors.
These findings suggest similar correlates of these quite phe-
nomenologically different disorders. However, parental and
familial differences in youth with ODD/AD versus ODD alone
have not been well examined to date. It is important, therefore,
to examine if the phenomenology of youth with comorbid ODD
and AD captures the features of both disorders, or if the comor-
bid profile is clinically distinct and characteristic of greater
impairment than a diagnosis of either ODD or AD alone.
Although there has been limited research examining clinical
profiles of youth with co-morbid ODD/AD, at least one inves-
tigation has examined the characteristics of children with
comorbid ADs and one of several disruptive behavior disorders
(e.g., ODD, CD, ADHD) in relation to children with pure AD,
co-morbid ADs, and comorbid depressive disorders (Franco et
al. 2006). Results indicated the group with comorbid ADs and
disruptive behavior disorders had significantly higher child-
reported anxiety and parent-reported internalizing symptoms
than youth with anxiety alone, but not higher ratings than the
other two comorbid groups. Additionally, ratings of parent-
reported externalizing symptoms were significantly higher in
the anxious and disruptive behavior group as compared to the
other groups. This study did not compare youth with comorbid
AD/disruptive behavior disorders to youth with disruptive be-
havior disorders only. Further, this investigation used a broad
disruptive behavior disorders group (ADHD, ODD, or CD)
rather than a single disruptive behavior disorder such as ODD.
Recently, Humphreys et al. (2012) reported similar findings in
that ODD symptoms were more pronounced in youth with co-
occurring ADs and ADHD as compared to youth with ADHD
only, AD only, ADHD/anxiety, and controls (non-ADHD); how-
ever ODD diagnosis was not examined across groups in this
investigation. Thus, the study of youth with comorbid AD/ODD
as compared to youth with ODD alone would yield a unique
contribution towards addressing current gaps in the literature.
The assessment of parent and family functioning may also
serve to highlight differences across groups. In terms of comor-
bidity investigations, Franco and colleagues (2006) found
parents of children with ADs/disruptive behavior disorders
endorsed more parent psychopathology than parents of children
with AD alone. On the other hand, Greene and colleagues
(2002) found that youth with ODD had significantly greater
rates of family impairment relative to other psychiatric com-
parison groups, even when ADs and other psychiatric condi-
tions were controlled for. A recent investigation by Drabick and
colleagues (2007) reported higher levels of family conflict in
younger boys with co-occurring anxiety symptoms and ODD
symptoms as compared to groups with either condition alone.
However, it is unclear whether these same results would be
obtained with a sample that included females and adolescents.
Also, there is little known about family variables and whether
they differ across groups based on mother or father informant.
Given the large number of youth with ODD and comorbid
ADs and the paucity of research examining the clinical char-
acteristics of this comorbid group, the current study examined
clinic-referred youth with comorbid ODD/AD in comparison
to clinic-referred youth with ODD alone (no ADs). Group
differences in ODD symptom impairment, global functioning,
presence of internalizing symptoms, and parent and family
environment factors (with a focus on caregiver distress and
parent-child relationship) were examined via a multi-method,
multi-informant approach.
134 J Psychopathol Behav Assess (2013) 35:133–141
As there is a lack of previous research examining the
clinical profiles of youth with ODD/AD as compared to youth
with ODD alone, the investigation was largely exploratory.
However, several hypotheses were examined: 1 ) In line with
conceptual evidence and research suggesting increased im-
pairment in youth with ODD/AD, we hypothesized children
and adolescents with comorbid ODD/AD would have higher
levels of ODD-symptom impairment, and greater impairment
in global functioning compared to youth with ODD alone; 2)
youth with comorbid ODD/AD would have more pronounced
internalizing symptoms (both broader internalizing symptoms
and symptom-specific anxiety); 3) parents of youth with
ODD/AD would have higher levels of family stress as com-
pared to youth with ODD alone, and 4) youth with ODD/AD
would have more dysfunctional parent-child relationships as
compared to youth with ODD alone. Group differences in
child externalizing symptoms were also examined. Additional
analyses were conducted to explore the possibility that the
presence of ADHD in this clinical sample moderated the
relationships between AD and one or more outcome variables.
Method
Participants
One hundred and twenty seven clinic-referred youth who were
taking part in an ongoing NIMH study comparing a cognitive
intervention (collaborative problem solving) to a behavioral
intervention (parent management training) for the treatment of
ODD in youth were enlisted. Participants were recruited from a
rural area in southwestern Virginia. For purposes of this study,
pre-treatment assessment data were used. To participate, par-
ticipants had to receive a diagnosis of ODD in the initial
assessment. Children with pervasive developmental delays or
serious cognitive impairments were excluded. Table 1reports
sociodemographic characteristics for the 127 youth as a whole,
for youth with ODD alone (no ADs, n= 47), and for youth
with ODD/AD (n= 80). A number of the youth in both groups
were also diagnosed with Attention-Deficit Hyperactivity
Disorder (see below). Independent sample t- tests were com-
puted for group differences among continuous variables (e.g.,
age, income) and chi-square values were calculated for cate-
gorical variables (e.g., gender, race, family structure). There
were no significant differences in sex, age, race, family struc-
ture, or family income between youth with ODD alone and
youth with ODD/AD (see Table 1).
Participants ranged in age from 7 to 14 years. A broad age
range of youth from childhood to adolescence was used to
determine whether comorbid anxiety differentially impacted
youth with ODD during both childhood and early adoles-
cence. Males comprised approximately two thirds of the sam-
ple, and the majority of the participants were Caucasian (n=
109, 85.8 %). The remaining 18 (14.2 %) participants self-
identified as African American (n=10),Hispanic(n=5),
Asian American (n= 2), and bi-racial (n= 1). Approximately
three quarters of the sample resided in two-parent households.
Of the youth with comorbid ODD/AD (n= 80), the most
commonly occurring anxiety disorders were generalized anx-
iety disorder (n= 28), specific phobia (n= 28), social phobia
(n= 17), separation anxiety disorder (n= 14), posttraumatic
stress disorder (n= 3), and obsessive compulsive disorder (n=
2). Other psychological disorders diagnosed in this subset
included dysthymia (n=2), and major depressive disorder
(n= 2). Twelve of the 80 youth with ODD/AD were
diagnosed with at least two clinically significant ADs. In
addition, 48 of these 80 youth were also characterized as
having ADHD. In the group of youth with ODD without
an AD (n= 47), the most common co-occurring diagnoses
were ADHD (n=30),enuresis(n= 5), dysthymia (n=3),
and major depressive disorder (n= 1). As both groups
contained a large number of youth with ADHD, the percent
of youth with ADHD was examined across the two groups.
Presence of ADHD did not differ in youth with ODD alone
as compared to youth with ODD/AD, χ
2
=0.18,p=.66).
Procedure
Data were collected by trained-to-criterion graduate clini-
cians in an APA-approved clinical psychology program
Table 1 Socio-demographic
characteristics of total sample,
ODD group, and ODD/AD group
ODD Oppositional defiant
disorder; AD anxiety disorder;
Mmean; SD standard deviation;
Pparent; Cchild; Tvalues
reported for continuous varia-
bles; chi2 values are reported for
categorical variables
Total (n=127) ODD (n=47) ODD/AD (n=80) t/chi
2
P
Age M (SD) 9.64 (1.76) 9.85 (1.98) 9.51 (1.62) 1.05 0.30
Gender n(%male) 83 (65.4) 31 (66) 52 (65) 0.01 0.91
Race n(%)
Caucasian 109 (85.8) 40 (85.1) 69 (86.3)
Other 18 (14.2) 7 (14.9) 11 (13.8) 0.05 0.82
Family structure n(%)
Dual parent 85 (66.9) 35 (74.5) 50 (62.5)
Single parent 22 (17.3) 7 (14.9) 15 (18.8) 0.03 0.86
Family income M (SD) 63564 (39956) 60514 (44547) 65705 (36654) -0.63 0.53
J Psychopathol Behav Assess (2013) 35:133–141 135
during two assessment sessions lasting a total of 4 h. After
obtaining parent consent/permission and child assent, children
completed self-report questionnaires and a diagnostic inter-
view with one clinician, while mothers completed several
questionnaires about themselves and their child in addition to
a diagnostic interview regarding their child with a second
clinician. Teacher consent was then obtained in accordance
with our IRB procedures. When available, multi-informant
data were obtained and analyzed. Specifically, father data were
available for BASC reports (n=89),BSIreports(n= 66),
and PSI reports (n= 76) and teacher data were available
for BASC reports (n= 65). Missing data were largely due
to the absence of fathers in some families and to the fact
that many youth were seen over the summer months when
teacher ratings were not available. Presence of ODD and other
psychiatric disorders were determined during a clinical con-
sensus meeting attended by the parent clinician, child clini-
cian, and principal investigator (a licensed clinical
psychologist). Parent and child clinicians independently pre-
sented diagnoses based on the parent and child diagnostic
interviews. Based on these reports a consensus diagnosis
was arrived at by the team. In addition to establishing clinical
diagnoses, clinician severity ratings of diagnoses and global
impairment for each participant were obtained during the
consensus process.
Measures
Anxiety Disorders Interview Schedule for DSM-IV, Child
and Parent Versions (ADIS-C/P; Silverman and Albano
1996)The ADIS-C/P versions are semi-structured interviews
designed for the diagnosis of most psychiatric disorders seen
in childhood and adolescence. During the interview, the clini-
cian assessed symptoms of various psychiatric conditions.
These symptoms were used by the clinician to identify diag-
nostic criteria and to develop a clinician severity rating (CSR),
which reflects the degree of clinical interference in function-
ing associated with each psychological disorder assessed. A
CSR of 4 or above (0–8) indicates a diagnosable condition.
Generally, parent and child report were used though it should
be noted that the ADIS-P alone was used to assess for ODD
(as the ADIS-C does not include ODD symptoms). The
ADIS-C/P (for DSM-IV) has yielded acceptable to excellent
7 to 14-day test-retest reliability estimates regarding child
(ages 7–16; κ=.61–.80) and parent (κ=.65–1.00) diagnoses
(Silverman et al. 2001). Inter-rater agreement analyses of
earlier versions of the ADIS-C/P have shown some variability
in video (κ=.45–.82; Rapee et al. 1994) and live observer
paradigms (κ=.35–1.00; Silverman and Nelles 1998), but in
general, acceptable interrater agreement has been established
for all specific diagnoses assessed bythe ADIS. The inter-rater
agreement (independent of the consensus process) for the
ODD diagnoses obtained on approximately 30 % of the
participants in the current study was excellent (κ= 1.00).
CSR ratings of ODD were also found to be reliable (0.84).
Children’s Global Assessment Scale (CGAS; Green et al.
1994; Shaffer et al. 1983)The CGAS is a 100-point rating
scale measuring psychological, social, and school functioning
in children ages 6 –17 (where a low score indicates greater
impairment in functioning). This measure was adapted from the
Adult Global Assessment Scale and has been found to be a
reliable and valid tool in rating a child’s general functioning on
a health-illness continuum. This measure was used as a depen-
dent variable in assessing group differences in functioning of
youth with ODD alone versus youth with ODD/AD. It was
determined by the consensus team based on information
obtained in the clinical interviews and the psychiatric histories
of the participants.
Behavior Assessment System for Children (BASC; Reynolds
and Kamphaus 1992)The BASC is a system of instruments
that evaluates behaviors, thoughts, and emotions of children
and adolescents. The measures vary between 100 and 160
items. Parent and teacher reports were used. Subscales used
for this study included internalizing symptoms and exter-
nalizing problems. Internal consistency for the Teacher
Ratings Scales is .80, and median test-retest reliability is
.91 (Kamphaus and Frick 2005). The Parent Rating Scales
also have good to excellent internal consistency (.70s–.90s;
Kamphaus and Frick 2005). Research also provides evi-
dence that the BASC demonstrates good convergent and
discriminant validity (Merrell et al. 2003), as well as accept-
able criterion validity (Reynolds and Kamphaus 1992). In the
current investigation, the internal consistencies were as fol-
lows: Mother Internalizing = 0.92, Mother Externalizing =
0.90, Father Internalizing = 0.94, Father Externalizing = 0.92,
Teacher Internalizing = 0.88, Teacher Externalizing = 0.97.
Beck Youth Inventories of Emotional and Social Impairment
(BYI; Beck et al. 2001)The Beck Youth Inventories (BYI)
measure internalizing and externalizing symptoms. The present
study used the subscales of anxiety (Beck Anxiety Inventory
for Youth: BAI-Y) and disruptive behavior (Beck Disruptive
Behaviors Inventory for Youth: BDBI-Y). Each subscale has
20 items, developed for use in children ages 7 to 14 with
responses on a 4 point Likert scale. Past research suggests
test-retest reliability coefficients ranged from .74 to .90 for 7–
10 year olds and .84 to .93 for 11–14 year olds (Beck et al.
2001). Convergent validity has also been demonstrated. For the
current investigation, the internal consistencies of the BYI
subscales were as follows: BAI-Y = .92 and BDBD-Y = .85.
Brief Symptom Inventory (BSI; Derogatis and Melisaratos
1983)The BSI has been used to identify self-reported clini-
cally relevant psychological symptoms in adolescents and
136 J Psychopathol Behav Assess (2013) 35:133–141
adults. The BSI consists of 53 items covering various symptom
dimensions. The subscales of focus for the current investiga-
tion included three internalizing subscales: interpersonal sen-
sitivity, depression, and anxiety. Both test-retest and internal
consistency reliabilities have been shown to be high (Boulet
and Boss 1991). Additionally, high convergence between BSI
scales and the MMPI provide good evidence of convergent
validity (Boulet and Boss 1991). For the current study, the
internal consistencies for the maternal scales were as follows:
interpersonal sensitivity = 0.79. depression = .87, and anxiety =
.82. The consistencies of the paternal scales were: interpersonal
sensitivity = .81, depression = .85, and anxiety = .65.
Parenting Stress Index Short Form (PSI; Abidin 1995;Abidin
and Brunner 1995)The PSI-SF is a 36-item questionnaire
that assesses parenting stress. The items are rated on a 5-
point scale ranging from “strongly agree”to “strongly dis-
agree.”The PSI subscales for this investigation included:
parental distress, and parent-child (PC) dysfunctional inter-
actions. Test-retest and internal consistency studies have
yielded good reliability estimates (6 month test-retest reli-
ability = .70 to .80; Cronbach’s alpha = .80) (Abidin 1995).
For the current investigation, the internal consistencies were
as follows: Maternal distress: 0.82, Maternal PC interaction:
0.80, Father distress: 0.84, and Father PC interaction: 0.85.
Analytic Plan
Several Multivariate Analyses of Covariance (MANCOVAs)
1
were conducted to examine differences in youth with ODD
alone versus youth with ODD/AD using MLR estimation in
Mplus 6.12. For these analyses, missing data were handled
using maximum likelihood parameter estimation with a ‘sat-
urated correlates’model to increase the likelihood of the
missing at random (MAR) assumption being met, which is
inherent in maximum likelihood missing data handling (e.g.,
Enders 2010). Since age and ADHD as covariates did not
change results, we regressed the predictor variable only (ODD
alone versus ODD/AD) for all analyses onto correlated re-
sponse variables. To control for experiment-wise Type-1 error
inflation while simultaneously preserving the statistical power
of the investigation under consideration, we used the False
Discovery Rate (FDR; Benjamini and Hochberg 1995).
The first MANCOVA, which assessed impairment, in-
volved the combination of clinician ratings of general impair-
ment (CGAS) and ODD-specific symptom impairment
(CSR), both of which were clinician rated. The second MAN-
COVA assessed child internalizing symptoms via a multi-
informant approach consisting of child, caregiver, and teacher
reports. Both MANCOVAs 3 and 4 examined the impact of
anxiety on internalizing symptoms separately for mothers and
fathers. These models were run separately because there are
no investigations to our knowledge that examine the role of
child anxiety in predicting mother and father internalizing
symptoms independently in youth with ODD. Finally, MAN-
COVA 5 assessed broader measures of caregiver-child dys-
function; thus we included parent-child dysfunction
ratings from both mothers and fathers.
Results
Means and standard deviations of study variables com-
prising each MANCOVA are listed in Table 2.Both
groups were characterized by poor global functioning
(CGAS) and moderate/high ODD symptoms (CSR).
Children with ODD/AD generally had elevated rates of
child internalizing symptoms, caregiver distress, and
parent-child dysfunction as compared to youth with
ODD only.
Although not a primary focus of the current investigation,
it should be noted that the results did not differ when re-run
controlling for age and ADHD. Further, presence of ADHD
did not moderate the relationship between anxiety and any
of the dependent variables.
MANCOVA 1: Measures of Global Functioning (CGAS)
and ODD Symptom Impairment (CSR)
Consistent with the first study hypothesis, youth with ODD/
AD had significantly poorer global functioning (CGAS) as
compared to youth with ODD alone (See Table 3). However,
there were no significant differences among ODD CSR
1
All MANCOVAs were performed in a structural equation modeling
(SEM) statistical software program (Mplus Version 6.12) and are often
referred to as generalized estimating equation (GEE) analyses. GEE
analyses were performed instead of the more traditional MANCOVA
for four reasons. First, GEE was used to handle missing data with
maximum likelihood estimation via the inclusion of additional variables
to form a ‘saturated correlates’data analysis model that increases the
likelihood that the implicit missing data assumption of missing at random
(MAR) is met without biasing the parameter estimates of theoretical
interest. Specifically, in our analyses, missing data for each MANOVA
was handled by including all other response variables not being analyzed
(e.g., for MANOVA #1, we included the response variables for MAN-
OVAs 2-5) as missing data correlates in a ‘saturated correlates’missing
data handling model. Second, GEE was used to minimize Type-1 errors
that could result from non-normally distributed data by using a maximum
likelihood parameter estimation algorithm robust to non-normally distrib-
uted data (e.g., MLR). Third, GEE ensures that the traditional MAN-
COVA assumption of ‘homogeneity of covariate regression slopes’was
met through parameter estimate constraint procedures available in SEM
software packages. Lastly, GEE allows for proceeding directly to the
between-group tests of mean differences among correlated response
variables without first conducting the MANCOVA omnibus F.Forthese
reasons, the analyses performed in this paper are more accurately de-
scribed as GEE analyses. The analysis logic employed is consistent with
traditional MANCOVAs, but many of the expected statistics associated
with MANCOVAs (i.e., an omnibus Fstatistic, such as Wilk’s Lambda)
are neither needed nor computed.
J Psychopathol Behav Assess (2013) 35:133–141 137
scores in youth with ODD alone versus those with ODD/
AD. These findings suggest that both groups were char-
acterized by moderate to severe ODD symptom impair-
ment, with the ODD/AD group evidencing greater overall
impairment in global functioning, but not greater ODD
symptom severity.
MANCOVA 2: Measures of Child Internalizing Symptoms
Table 3presents significant group differences in child internal-
izing symptoms. As expected, child internalizing symptoms
as reported by mother, father, and child were significantly
higher for youth with ODD/AD as compared to youth with
ODD alone. This was true for both broader internalizing
symptoms, as reported by the parents and anxiety-specific
symptoms, as reported by the children. Although teachers
did not report significantly higher internalizing symp-
toms in youth with ODD/AD, the results trended in the
hypothesized direction.
MANCOVAS 3 and 4: Mother Distress and Father Distress
Group differences in caregiver distress are reported in the
lower half of Table 3. Both mothers and fathers reported
higher parental distress in families of youth with comorbid
ODD/AD as compared with youth with ODD alone. Further,
on the BSI, mothers of children with ODD/AD reported
significantly higher levels of interpersonal sensitivity, and
higher levels of other internalizing symptoms (depression,
anxiety) that approached significance as compared to moth-
ers of children with ODD alone. However, there were no
significant differences across the groups in paternal reports
of psychopathology on the BSI.
MANCOVA 5: Parent-Child Dysfunctional Interaction
Both mothers and father reported higher levels of parent-
child dysfunction in youth with ODD/AD as compared to
youth with ODD alone.
Table 2 Means and standard
deviations of study variables
ODD oppositional defiant disor-
der; AD anxiety disorder; CGAS
Children’s Global Assessment
Scale; CSR Clinician Severity
Rating of ODD Symptoms;
BASC Behavior Assessment
System for Children; BAI Beck
Anxiety Inventory; BSI Brief
Symptom Inventory; PSI
Parenting Stress Index; int sens
interpersonal sensitivity
Total ODD ODD/AD
MANCOVA 1: Impairment
MSDMSDMSD
CGAS 58.91 5.96 60.77 6.02 57.81 5.67
CSR 5.91 1.08 5.98 1.07 5.87 1.08
MANCOVA 2: Child Internalizing Symptoms
BASC TSDTSDTSD
mother 61.08 13.39 57.81 12.71 63.18 13.48
father 57.49 13.66 51.94 13.35 60.93 12.79
teacher 54.28 11.21 50.36 8.48 56.28 11.98
BAI
child 45.7 9.5 43.18 7.58 47.38 10.31
MANCOVA 3: Mother Distress
BSI TSDTSDTSD
int sens 42.77 26.16 36.73 27.65 46.8 24.53
depress 39.2 27.92 30.6 29.8 44.84 25.3
anxiety 45.91 22.69 43.42 23.46 47.5 22.24
PSI M (%) SD M (%) SD M(%) SD
distress 27.62(56) 7.05 25.21(49) 5.76 28.97(61) 7.38
MANCOVA 4: Father Distress
BSI T(SD) TSDTSDTSD
int sens 36.77 31.3 34.04 31.22 38.44 31.62
depress 34.68 30.4 33.12 30.34 35.63 30.77
anxiety 42.17 25.74 36.12 28.18 45.85 23.74
PSI M (%) SD M (%) SD M(%) SD
distress 27.74(58) 7.07 25.32(49) 6.98 29.15(63) 6.8
MANCOVA 5: Parent-Child Dysfunctional Interaction
PSI M (%) SD M (%) SD M(%) SD
mother 28.53(82) 7.36 26.03(75) 6.94 29.97(87) 14.08
father 27.23(79) 6.99 24.28(70) 6.51 29.02(85) 6.72
138 J Psychopathol Behav Assess (2013) 35:133–141
Secondary Analyses: Child Externalizing Symptoms
No significant group differences between youth with ODD
and youth with ODD/AD were found in parent, teacher, or
child reported externalizing symptoms. Thus, both groups
showed comparable levels of externalizing difficulties.
Discussion
To our knowledge, this is the first investigation to compare a
clinic-referred sample of children and adolescents with
comorbid AD/ODD diagnoses to youth with ODD in the
absence of ADs. Specifically, this study is the first to examine
comorbid AD in an ODD sample in terms of ODD symptom
impairment, global functioning, emotional/behavioral func-
tioning, and family environmental functioning. This research
builds upon other investigators (e.g., Franco et al. 2006) which
have examined clinical profiles of youth with disruptive be-
havior disorders/ADs as compared to youth with other forms
of psychopathology. Further, this investigation expands the
current literature by focusing on a sample of youth with ODD
specifically versus those with broad disruptive behavior dis-
orders. Additionally, this study utilized a “gold-standard”
multi-informant, multi-method approach (e.g., De Los Reyes
and Kazdin 2005; Grills and Ollendick 2002) in examining
group differences. Specifically, the current study utilized
clinician-, parent-, teacher-, and youth- reports via interviews
and questionnaires. The addition of mother and father reports
of caregiver distress and parent-child relationship is a partic-
ularly important contribution to the literature in this area.
The findings suggest some support for a unique clinical
profile for youth with ODD/AD as compared to youth with
ODD alone. For example, youth with ODD/AD have lower
levels of global functioning as compared to youth with ODD
in the absence of an AD diagnosis, which is in line
with both theoretical models and empirical research
(e.g., see Cunningham and Ollendick 2010, for review). There
are several possible explanations for this finding. For exam-
ple, comorbid psychopathology may have an additive effect,
resulting in greater impairment of global functioning, partic-
ularly so in the case of a diverse comorbid profile that includes
both internalizing and externalizing disorders (see Drabick et
al. 2010). Another possible explanation is that youth with
ODD/AD are more likely to show greater impairment in
global functioning under certain conditions, such as in the
presence of greater family dysfunction. Further, the potential
buffering effect of anxiety on ODD may not develop until
the transition from adolescence to adulthood. Future studies
to test moderators and mediators of functioning in youth,
adolescents, and young adults with disruptive behavior
disorders and comorbid ADs are needed and would help
explicate these findings.
It is interesting to note that no differences were noted in
severity of ODD symptoms (nor in levels of other external-
izing symptoms). These findings suggest that AD in the
presence of ODD may not impact the presentation of the
severity of the ODD diagnosis, but the ODD/AD presenta-
tion may lead to greater impairment that is captured in
broader domains of functioning.
As expected, youth with ODD/AD had higher levels of
internalizing symptoms across mother, father, and child
reports, demonstrating higher levels of both both broad inter-
nalizing symptoms (mother and father report) and specific
increases in anxiety symptoms (via child reports), consistent
with study hypotheses. These findings provide validity to our
group assignments based on clinical diagnoses (Achenbach
Table 3 Differences in youth with ODD/AD versus ODD
Est. SE Z pCohen’sd
MANCOVA 1: Impairment
CGAS -3.15 1.12 -2.81 0.02 0.53
CSR -0.10 0.21 -0.50 0.63 0.10
MANCOVA 2: Child Internalizing Symptoms
BASC
mother 14.89 6.26 2.38 0.03 0.46
father 19.82 6.50 3.05 0.02 0.62
teacher 12.97 7.19 1.80 0.10 0.47
BAI
child 4.66 1.90 2.46 0.03 0.48
MANCOVA 3: Mother Distress
BSI
int sens 0.34 0.15 2.27 0.04 0.45
depress 0.23 0.13 1.72 0.11 0.34
anxiety 0.24 0.13 1.86 0.10 0.37
PSI
distress 4.03 1.45 2.78 0.02 0.59
MANCOVA 4: Father Distress
BSI
int sens 0.20 0.16 1.24 0.26 0.30
depress 0.13 0.13 0.95 0.37 0.23
anxiety 0.12 0.10 1.14 0.29 0.28
PSI
distress 4.23 1.67 2.54 0.03 0.59
MANOVA 5: P-C Dysfunctional Interaction
PSI
mother 4.06 1.51 2.69 0.02 0.57
father 4.47 1.54 2.91 0.02 0.68
ODD Oppositional defiant disorder; AD anxiety disorder; CGAS Child-
ren’s Global Assessment Scale; CSR Clinician Severity Rating of ODD
Symptoms; BASC Behavior Assessment System for Children; BAI
Beck Anxiety Inventory; BSI Brief Symptom Inventory; PSI Parenting
Stress Index; int sens interpersonal sensitivity; depress depressive symp-
toms; P-C –parent-child
J Psychopathol Behav Assess (2013) 35:133–141 139
1990; Seligman et al. 2004) and support the notion that youth
with co-occurring anxiety may express global internalizing
symptoms. Although teachers did not report greater levels of
internalizing symptoms in youth with ODD/AD, these find-
ings may be due to teachers being less attuned to the internal-
izing symptoms these youth experience in light of the more
obvious ODD symptoms they express in the classroom. Of
note, neither of our groups had significantly higher levels of
externalizing symptoms across study informants. Thus, both
groups displayed comparable amounts of these symptoms as
determined by multiple informants.
This study also suggests unique family environmental risk
factors may be associated with comorbid diagnoses of ODD/
AD compared to those with ODD alone. Specifically, the
current investigation found that both mothers and fathers of
children with ODD/AD reported higher levels of parent stress
and poorer parent-child relations compared to parents of chil-
dren with ODD alone. Thus, parents may perceive more
strained relationships in youth with comorbid conditions, as
the comorbidity may pose an added burden to the family
system. Although the findings regarding caregiver anxiety
symptoms and depressive symptoms were not statistically sig-
nificant, mothers of children with ODD/AD did report more
interpersonal sensitivity compared to mothers of youth with
ODD alone. This finding supports previous research (e.g.,
Franco et al. 2006) and suggests that mothers may be more
likely to exhibit internalizing symptoms when their children are
diagnosed with an internalizing disorder, which may be due to
a combination of shared environmental and/or genetic factors.
In interpreting these findings, several limitations should be
noted. For example, youth with ODD have a tendency to
underreport symptoms (e.g., Kamphaus and Frick 2005); thus,
caution should be exercised when interpreting results from the
youth themselves. Also, there was more missing data for
fathers and teachers, thereby limiting the potential interpret-
ability of the findings. In addition, the findings as a whole are
limited in generalizability as they were derived from a largely
Caucasian, middle childhood/early adolescence, clinic-
referred sample of youth. Testing these research questions
with epidemiological and more representative samples of
youth would facilitate generalizability of the findings.
Yet another limitation of the current study is that we were
unable to group youth with ODD based on their specific
AD; however, given the current study sample size limita-
tions, as well as sample size limitations of previous studies
(e.g. Franco et al. 2006), such analyses have generally not
been undertaken. Future studies examining clinical profiles
of youth with ODD/AD should examine youth with ODD
and individual types of ADs as compared to youth with
ODD alone. It is also important to consider the role of other
disorders in terms of impact on clinical profiles and impair-
ment levels. In addition, it would be interesting to examine
clinical profiles of youth with ODD/ADs longitudinally. In
the context of a longitudinal investigation, it would be possi-
ble to determine if the presence of AD changes from a risk
factor to a protective factor over time. Finally, it would be
beneficial to examine contextual factors (e.g., gender, family
environment) that might lead to lower global functioning in
youth with ODD/AD. In particular, it would be interesting the
examine the role of family environment in predicting func-
tioning in youth with ODD/AD as compared to youth with
ODD alone in subsequent investigations.
These results may have important implications for both
assessment and treatment. Our findings underscore the im-
portance of a multi-informant multi-method approach to
assessment in children with ODD. A comprehensive ap-
proach to assessment, with a special focus on the assessment
of ADs is of particular importance as the presence of an AD
may be associated with lower global functioning and related
complications for a child with ODD. These findings may also
have important ramifications for treatment planning of youth
with comorbid ODD/AD. The use of evidenced-based treat-
ments to target the child’s comorbidity may be particularly
beneficial (e.g., Chorpita et al. 2004;Levyetal.2007). Based
on the findings from this investigation, youth with ODD/AD
may benefit from family-focused interventions given that
increased caregiver distress and increased difficulties in
parent-child relationship were more evident in families of
youth with comorbid ODD/AD. Given that both mothers
and fathers reported increased distress and increased parent-
child dysfunction, it may be important that both caregivers
are involved in the treatment process to optimize outcomes
for such youth. Interventions that target caregiver psycho-
logical functioning may also increase the efficacy of treat-
ment interventions for youth with comorbid ODD/AD.
References
Abidin, R. R. (1995). Parenting stress index: Professional manual (3rd
ed.). Odessa: Psychological Assessment Resources, Inc.
Abidin, R. R., & Brunner, J. F. (1995). Development of a parenting
alliance inventory. Journal of Clinical Child Psychology, 24,31–
40. doi:10.1207/s15374424jccp2401_4.
Achenbach, T. (1990). “Comorbidity”in child and adolescent psychiatry:
categorical and quantitative perspectives. Journal of Child and Ado-
lescent Psychopharmacology, 1,1–8. doi:10.1089/cap.1990.1.271.
American Psychiatric Association. (2000). Diagnostic and statistical
manual of mental disorders IV-TR. DC: Washington.
Angold, A., Costello, E. J., & Erkanli, A. (1999). Comorbidity. Journal
of Child Psychology and Psychiatry, 40, 57–87. doi:10.1111/
1469-7610.00424.
Beck, J. S., Beck, A., & Jolly, J. (2001). Manual for the Beck Youth
Inventories of Emotional and Social Adjustment. San Antonio.
Beidel, D., & Turner, S. M. (1997). At risk for anxiety: psychopathol-
ogy in the offspring of anxious parents. Journal of the American
Academy of Child and Adolescent Psychiatry, 36,918–924.
doi:10.1097/00004583-199707000-00013.
140 J Psychopathol Behav Assess (2013) 35:133–141
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery
rate: a practical and powerful approach to multiple testing. Jour-
nal of the Royal Statistical Society Series B, 57, 289–300.
Boulet, J., & Boss, M. W. (1991). Reliability and validity of the brief
symptom inventory. Psychological Assessment: A Journal of
Consulting and Clinical Psychology, 3, 433–437. doi:10.1037/
1040-3590.3.3.433.
Chorpita, B. F., Taylor, A. A., Francis, S. E., Moffitt, C. E., & Austin,
A. A. (2004). Efficacy of modular cognitive behavior therapy for
childhood anxiety disorders. Behavior Therapy, 35, 263–287.
doi:10.1016/S0005-7894(04)80039-X.
Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A.
(2003). Prevalence and development of psychiatric disorders in
childhood and adolescence. Archives of General Psychiatry, 60,
837–844. doi:10.1001/archpsyc.60.8.837.
Cunningham, N. R., & Ollendick, T. H. (2010). Comorbidity of anx-
iety and conduct problems in children: implications for clinical
research and practice. Clinical Child and Family Psychology
Review, 13, 333–347. doi:10.1007/s10567-010-0077-9.
De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the
assessment of childhood psychopathology: a critical review, theo-
retical framework, and recommendations for further study. Psycho-
logical Bulletin, 131,483–509. doi:10.1037/0033-2909.131.4.483.
Derogatis, L., & Melisaratos, N. (1983). The brief symptom inventory:
an introductory report. Psychological Medicine, 13, 595–605.
doi:10.1017/S0033291700048017.
Drabick, D. A. G., Gadow, K., & Loney, J. (2007). Source-specific
oppositional defiant disorder: comorbidity and risk factors in
referred elementary schoolboys. Journal of the American Acade-
my of Child and Adolescent Psychiatry, 46,92–101. doi:10.1097/
01.chi.0000242245.00174.90.
Drabick, D. A. G., Ollendick, T. H., & Bubier, J. L. (2010). Co-
occurrence of ODD and anxiety: shared risk processes and evidence
for a dual-pathway model. Clinical Psychology: Science and Prac-
tice, 17,307–318. doi:10.1111/j.1468-2850.2010.01222.x.
Enders, C. K. (2010). Applied missing data analysis. New York: Guilford.
Franco, X., Saavedra, L. M., & Silverman, W. K. (2006). External
validation of comorbid patterns of anxiety disorders in children
and adolescents. Journal of Anxiety Disorder, 21,717–729.
doi:10.1016/j.janxdis.2006.10.002.
Green, B., Shirk, S., Hanze, D., & Wanstrath, J. (1994). The Children’
s
global assessment scale in clinical practice: an empirical evaluation.
Journal of the American Academy of Child and Adolescent Psychi-
atry, 33, 1158–1164. doi:10.1097/00004583-199410000-00011.
Greene, R. G., Biederman, J., Zerwas, S., Monuteaux, M., Goring, J.,
& Faraone, S. (2002). Psychiatric comorbidity, family dysfunc-
tion, and social impairment in referred youth with oppositional
defiant disorder. The American Journal of Psychiatry, 159, 1214–
1224. doi:10.1176/appi.ajp.159.7.1214.
Grills, A. E., & Ollendick, T. H. (2002). Issues in parent-child agreement:
the case of structured diagnostic interviews. Clinical Child and Fam-
ily Psychology Review, 5,57–83. doi:10.1023/A:1014573708569.
Hammen, C. (1992). The family-environmental context of depression:
A perspective on children’s risk. In D. Cicchetti & S. L. Toth
(Eds.), Developmental perspectives on depression- Rochester
Symposium of Developmental Psychopathology (pp. 251–281).
Rochester: University of Rochester Press.
Humphreys, K. L., Aguirre, V. P., & Lee, S. S. (2012). Association of
anxiety and ODD/CD in children with and without ADHD.
Journal of Clinical Child and Adolescent Psychology.
doi:10.1080/15374416.2012.656557.
Kamphaus, R., & Frick, P. (2005). Clinical assessment of child and
adolescent personality and behavior (2nd ed.). New York: Springer.
Lansford, J. E., Dodge, K. A., Pettit, G. S., Bates, J. E., Crozier, J., &
Kaplow, J. (2002). A 12-year prospective study of the long-term
effects of early child physical maltreatment on psychological,
behavioral, and academic problems in adolescence. Archives of
Pediatrics & Adolescent Medicine, 156, 824–830. Retrieved from
www.archpediatrics.com.
Lansford, J. E., Dodge, K. A., Malone, P. S., Bacchini, D., Zelli, A.,
Chaudhary, N., et al. (2005). Physical discipline and children’s
adjustment: cultural normativeness as a moderator. Child Devel-
opment, 76, 1234–1246. doi:10.1111/j.1467-8624.2005.00847.x.
Last, C. G., Hersen, M., Kazdin, A. E., Francis, G., & Grubb, H. J.
(1987). Psychiatric illness in the mothers of anxious children. The
American Journal of Psychiatry, 144, 1580–1583.
Levy, K., Hunt, C., & Heriot, S. (2007). Treating comorbid anxiety and
aggression in children. Journal of the American Academy of Child
and Adolescent Psychiatry, 46, 1111–1118. doi:10.1097/
chi.0b013e318074eb32.
March, J. S., Parker, J. D. A., Sullivan, K., Stallings, P., & Conners, C.
K. (1997). The multidimensional anxiety scales for children
(MASC): factor structure, reliability, and validity. Journal of the
American Academy of Child and Adolescent Psychiatry, 36, 554–
565. doi:10.1097/00004583-199704000-00019.
Merrell, K., Blade, R., Lund, J., & Kemp, K. (2003). Convergent and
discriminant construct validity of the internalizing symptoms
scale for children with the BASC-SRP-C. Psychology in the
Schools, 40, 139–144. doi:10.1002/pits.10076.
Rapee, R., Barrett, P., Dadds, M., & Evans, L. (1994). Reliability of the
DSM-III-R childhood anxiety disorders using structured inter-
view: Interrater and parent-child agreement. Journal of the Amer-
ican Academy of Child and Adolescent Psychiatry, 33, 984–992.
doi:10.1097/00004583-199409000-00008.
Reynolds, C., & Kamphaus, R. (1992). Behavior assessment system for
children (BASC). Circle Pines: American Guidance Service.
Seligman, L. D., Ollendick, T. H., Langley, A. K., & Baldacci, H. B.
(2004). The utility of measures of child and adolescent anxiety: a
meta-analytic review of the revised children’s manifest anxiety
scale, the state-trait anxiety inventory for children, and the child
behavior checklist. Journal of Clinical Child and Adolescent
Psychology, 33, 557–565. doi:10.1207/s15374424jccp3303_13.
Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H.
R., et al. (1983). A Children’s Global Assessment Scale (CGAS).
Archives of General Psychiatry, 40, 1228–1231. doi:10.1001/
archpsyc.1983.01790100074010.
Silverman, W., & Albano, A. (1996). Anxiety Disorders Interview
Schedule for DSM-IV: Child Version: Psychological Corp.
Silverman, W. K., & Nelles, W. B. (1998). The anxiety disorders inter-
view schedule for children. Journal of the American Academy of
Child and Adolescent Psychiatry, 27,779–784. doi:10.1097/
00004583-198811000-00020.
Silverman, W. K., Saavedra, L. M., & Pina, A. A. (2001). Test-retest
reliability of anxiety symptoms and diagnoses using the Anxiety
Disorders Interview Schedule for DSM-IV: Child and Parent
Versions (ADIS for DSM-IV: C/P). JournaloftheAmerican
Academy of Child and Adolescent Psychiatry, 40,937–944.
doi:10.1097/00004583-200108000-00016.
J Psychopathol Behav Assess (2013) 35:133–141 141
Copyright of Journal of Psychopathology & Behavioral Assessment is the property of Springer Science &
Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv
without the copyright holder's express written permission. However, users may print, download, or email
articles for individual use.
