Content uploaded by Florian Naudet
Author content
All content in this area was uploaded by Florian Naudet on Jan 03, 2017
Content may be subject to copyright.
Content uploaded by Florian Naudet
Author content
All content in this area was uploaded by Florian Naudet on Jan 03, 2017
Content may be subject to copyright.
Specific and non-specific effects of psychotherapeutic interventions
for depression: Results from a meta-analysis of 84 studies
Cl
ement Palpacuer
a
,
*
,
1
, Laurent Gallet
b
,
1
, Dominique Drapier
b
,
c
, Jean-Michel Reymann
a
,
Bruno Falissard
d
, Florian Naudet
a
,
c
a
INSERM Centre d'Investigation Clinique 1414, Centre Hospitalier Universitaire de Rennes, Rennes, France
b
Centre Hospitalier Guillaume R
egnier, Rennes, France
c
EA 4712 Comportement et Noyaux Gris Centraux, Centre Hospitalier Universitaire de Rennes et Universit
e de Rennes 1, Rennes, France
d
INSERM UMR 1178, Facult
edeM
edecine Paris Sud, D
epartement de Biostatistiques, H^
opital Paul Brousse, Assistance Publique H^
opitaux de Paris, Villejuif,
France
article info
Article history:
Received 4 July 2016
Received in revised form
20 December 2016
Accepted 20 December 2016
Keywords:
Psychotherapy
Depression
Meta-analysis
abstract
There is a long-standing and very active debate regarding which psychotherapeutic intervention should
be used in depressive disorders. However, the effects of psychotherapies may result majorly from non-
specific factors rather than from specific factors related to the type of psychotherapeutic intervention.
We performed a systematic review and meta-analysis on aggregated data to understand how the effects
of different psychotherapies are impacted by non-specific factors. We included randomized controlled
trials that assessed the efficacy of psychotherapeutic interventions in the treatment of adult depressive
disorders. The primary outcome was the change in depression score from baseline to the latest follow-up
visit (i.e. response). A meta-regression was performed to predict response according to the type of
intervention and non-specific factors (e.g. number of treatment sessions, length of follow-up, therapeutic
allegiance of the investigator). The main analysis included 214 study arms from 84 trials. The effects of
psychotherapies compared to the waiting list control condition failed to remain significant after
adjusting for non-specific factors. Response increased with the number of treatment sessions (
b
¼0.03,
95% CI [0.01; 0.04]) and the length of follow-up (
b
¼0.01, 95% CI [0.00; 0.02]). Response also improved in
case of presumed therapeutic allegiances among investigators (
b
¼0.29, 95% CI [0.07; 0.52]). Response to
psychotherapies seems to be closely related to non-specific effects. The development of a well-designed
trial that controls for non-specific factors might help disentangle the effects of psychotherapies.
©2016 Elsevier Ltd. All rights reserved.
1. Introduction
There is a long-standing and very active debate concerning
which treatment should be used in major depressive disorder
(MDD) (Gøtzsche, 2014; Nutt et al., 2014). This debate goes beyond
a mere opposition between pharmacological and psychological
interventions, and there are many controversies regarding the type
of psychotherapy that would be the best for most patients (Cuijpers
et al., 2008a; Ekers et al., 2008; Gloaguen et al., 1998; Leichsenring
et al., 2004; Wampold et al., 2002). Despite considerable evidence
in favor of an equivalency of psychotherapeutic interventions in
depression, in the sense of the so-called “Dodo Bird Verdict”, this
topic is still controversially discussed among clinicians and scien-
tists (Budd and Hughes, 2009). All these controversies arise from
divergent interpretations of the evidence derived from the
numerous randomized controlled trials (RCTs) that are available
(Cuijpers et al., 2008b), against a backdrop of ideological conflicts of
interest. But one must bear in mind that while this debate is
focused on hypothetical “specific effects”(the treatment/control
difference) of these distinct interventions (for example arising from
a theoretical framework underpinning a given psychotherapeutic
intervention), it fails to consider the complexity of the phenome-
non resulting in the benefit experienced by a given patient (i.e. the
pre/post treatment difference, also called response). Some de-
terminants of response, such as empathy shown by caregivers who
are actively involved in an intervention in which they trust, are
considered as experimental noise and as non-specific effects. These
*Corresponding author. Centre d’Investigation Clinique INSERM1414, H^
opital de
Pontchaillou, 2 rue Henri le Guilloux, 35033 Rennes cedex 9. France.
E-mail address: clement.palpacuer@gmail.com (C. Palpacuer).
1
These authors contributed equally to the work.
Contents lists available at ScienceDirect
Journal of Psychiatric Research
journal homepage: www.elsevier.com/locate/psychires
http://dx.doi.org/10.1016/j.jpsychires.2016.12.015
0022-3956/©2016 Elsevier Ltd. All rights reserved.
Journal of Psychiatric Research 87 (2017) 95e104
non-specific effects have been shown to be important determinants
of antidepressant response (Naudet et al., 2011), and may be even
more crucial in the response observed after a psychotherapeutic
intervention (Ahn and Wampold, 2001; Chatoor and Krupnick,
2001; Parker et al., 2003). For example, it has been suggested that
the improvement in cognitive psychotherapy may occur before the
introduction of cognitive restructuring (the specific part of the
therapy) (Ilardi and Craighead, 1994). One can hypothesize that this
early effect may result from the behavioral techniques that are
typically applied early in cognitive therapy (Jacobson et al., 1996),
but it might also be hypothesized that the initiation of a rather
“unspecific”empathic attitude by the clinician could trigger this
improvement. An earlier meta-analysis (Barth et al., 2013) found no
differences between psychotherapies, but did between different
types of control condition (usual care and placebo groups evi-
denced better responses than waiting list groups). Although mostly
based on indirect evidence, this result suggests that the differences
that were evidenced resulted from non-specific factors rather than
from specific factors related to the type of psychotherapeutic
intervention. Since these non-specific effects are obviously not well
balanced across control and experimental groups, an analysis of
response determinants in psychotherapeutic interventions for
MDD adjusting for these potential confounders is thus a crucial
issue 1/to understand how psychotherapies work, and 2/to find
ways of maximizing these factors in care that is delivered to pa-
tients. It is also a methodological challenge in the promotion of
alternative ways of evaluating psychotherapies. We therefore per-
formed a meta-analysis on aggregated data to understand how the
effects of different psychotherapies and control conditions are
impacted by non-specific factors in MDD psychotherapeutic
research.
2. Methods
The protocol of the study was notified prior to implementation
on an international register for systematic reviews (Registration-
PROSPERO, 2014:CRD42014009522).
2.1. Eligibility criteria
2.1.1. Type of included studies
RCTs involving adults (aged 18 and over) suffering from
depression (according to international classifications such as DSM-
IV or according to the severity of symptoms measured on depres-
sion scales), with or without concomitant anxiety disorder, were
reviewed. In order to have a homogenous sample, studies including
patients with comorbidity were taken into account only if the
comorbidities were not an explicit inclusion criterion. Studies
involving more than 20% bipolar disorder were also excluded, as
well as studies focusing on elderly people (>65 years), postnatal
depression, seasonal depression, postmenopausal depression or
atypical depression.
We included studies that assessed the efficacy of psychothera-
peutic interventions using a depression rating scale, regardless of
treatment modalities. Psychotherapeutic intervention is defined as
an intervention primarily based on communication between a pa-
tient and a therapist, or as bibliotherapy (the use of books or
internet as therapy in the treatment of mental disorders) supported
by a therapist.
In this review, we only considered RCTs published between
1966, corresponding to the date used in the VU University database
(Cuijpers et al., 2008b), and 2013. Only studies in English, French
and Spanish were included.
2.1.2. Type of outcome
The primary outcome was the change in depression score from
baseline to the latest available follow-up, measured by a clinician-
rated depression scale [such as the Hamilton Depression Rating
Scale (HDRS) or the Montgomery Asberg Depression Rating Scale
(MADRS)] or a self-report scale [such as the Beck Depression In-
ventory (BDI) or the Center for Epidemiologic Studies Depression
Scale (CES-D)]. When both clinician-rated and self-report depres-
sion scale scores were reported in a study, priority was given to the
clinician scores. Secondary outcomes were 1/the change in
depression score measured exclusively on a clinician-rated scale,
and 2/the change in depression score measured exclusively on a
self-report scale.
2.2. Search strategy
The literature search was based on the recent meta-analysis
published by Barth et al. (2013), and targeted RCTs evaluating
psychotherapeutic interventions on the Psychotherapy Random-
ized Controlled Trials for meta-analysis database (VU University).
The methods used for the development of the VU University
database have been described elsewhere (Cuijpers et al., 2008b). In
brief, the authors conducted a comprehensive literature search of
the major bibliographical databases (Pubmed; Psycinfo; Embase;
Cochrane Central Register of Controlled Trials) and examined the
references of 22 meta-analyses of psychological treatment for
depression.
2.3. Study selection
Titles and abstracts of studies were screened independently by
two reviewers (PC and GL). When needed, full-texts were exam-
ined. Any disagreement was resolved by consensus or in consul-
tation with a third reviewer (NF). A comparison across studies was
performed, checking authors, treatment comparisons, sample sizes
and outcomes, to avoid duplicates and compilations of data from
several reports on the same study.
2.4. Assessment of methodological quality
Each paper was assessed for methodological quality prior to
inclusion in the review. We adapted the current Cochrane risk of
bias assessment tool (Higgins et al., 2011) and examined 1/the
method used to generate the allocation sequence, 2/the strategy for
data analysis [Intention-to-treat (ITT) or not], and 3/when a
clinician-rated depression scale was used, the blinding of assessors.
2.5. Data collection
A data extraction sheet based on the Cochrane Handbook for
Systematic Reviews of Interventions guidelines (Higgins and Green,
2011) was used. Two reviewers (PC and GL) extracted data from 1/
full-text articles, and 2/the VU University database.
For each arm of each study included, we extracted information
on: 1/characteristics of the study (year, continent, number of pa-
tients randomized, number of patients analyzed); 2/characteristics
of participants including patient recruitment methods, diagnosis,
mean age, proportion of women; 3/type of intervention including
the type of psychotherapy (cognitive behavioral, psychodynamic,
interpersonal, problem-solving, behavioral activation, non-
directive supportive or other therapy) and the type of control
group (waiting list, care-as-usual, placebo pill, ‘fake’psychotherapy
or other control); 4/non-specific therapeutic components of the
experimental or control groups including the format of interven-
tion (individual, group, guided self-help, other), the frequency of
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e10496
contacts between the patient and the therapist during the inter-
vention (number of treatment sessions and number of assessment
sessions), the length of follow-up, and the presence or not of a
particular therapeutic allegiance on the part of the investigators; 5/
outcome measures as stated above. Data from combined treat-
ments (psychological plus pharmacological) were not collected, as
this could have introduced heterogeneity. Researcher allegiance
has been defined as a researcher's belief in the superiority of a
treatment and the superior validity of the theory of change that is
associated with the treatment implemented (Leykin and DeRubeis,
2009). Since there is no consensual way to measure researcher
allegiance, we considered that there was allegiance towards a given
therapy when: 1/the first and/or last named author of a publication
had advocated the superiority of this therapy in a previous or
subsequent study to that included in the current meta-analysis, 2/
the authorship of the intervention tested in the RCT was attributed
to one of the authors, or 3/the superiority of the intervention was
hypothesized a priori.
2.6. Data analysis
Analyses were performed using R (R Development Core Team,
2008) and the meta (Schwarzer G), lme4 (Maechler D), and MICE
(Van Buuren S, Groothuis-Oudshoorn K) libraries. Results are pre-
sented according to PRISMA statements (Preferred Reporting Items
for Systematic Reviews and Meta-Analyses) (Liberati et al., 2009).
2.6.1. Main analysis
To quantify how the effects of different psychotherapies and
control conditions are impacted by non-specific factors, a multi-
variate meta-regression was performed using a mixed-effect
model. The dependent variable was the change in depression
score from baseline to the latest available follow-up (pre/post dif-
ference), measured in each arm of each study by a clinician-rated or
a self-report depression scale. When both clinician-rated and self-
report depression scale scores were reported in a study, we used
the scores provided by the clinician-rated scale. Since the scales
used to assess depression differed across studies, we calculated a
standardized mean difference (SMD) by dividing the pre/post dif-
ference by its standard deviation. In cases where the standard de-
viation (SD) of the pre/post difference was not reported, it was
calculated using 1/the SDs of the depression scores at baseline and
at the end of the studies, or 2/a correlation coefficient value of 0.40,
as found in a previous meta-analysis (Naudet et al., 2011). As a first
step, we ran univariate analysis with the type of intervention (i.e.
type of psychotherapy including cognitive behavioral, psychody-
namic, interpersonal, problem-solving, behavioral activation, non-
directive supportive and other therapy, type of control condition
including waiting list, care-as-usual, placebo pill and other control)
as the only covariate. As a second step, we performed a multivariate
analysis with the following explanatory variables (covariates): Type
of intervention; Continent; Intervention format (Individual, Group,
Guided self-help, Other); Frequency of contacts between the pa-
tient and the therapist during the therapy; Length of follow-up;
Age; Gender; Patient recruitment methods (Community, Clinical,
Other); Blinding of outcome assessment; Type of depression scale
used (i.e. self-rating scale or clinician-rated scale); Processing of
missing outcome data (i.e. ITT analysis or not); Researchers' alle-
giance (i.e. presence or absence of a therapeutic allegiance). These
covariates are described in detail in the Web-appendix.
When data were missing, multiple imputation was performed
using a Gibbs sampler (Little and Rubin, 2014), except for data
involved in the calculation of the dependent variable (i.e. stan-
dardized mean change in depression scores before and after
treatment).
We did not investigate publication bias (funnel plots, rank cor-
relation test) because of the considerable heterogeneity across
studies, which limits the interest of this investigation (Sterne et al.,
2011).
To assess the robustness of our results, sensitivity analyses were
performed 1/using a wide range of pre/post correlation coefficients,
2/by removing each study in turn. Multicollinearity between vari-
ables was investigated using variance inflation factors (VIF).
2.6.2. Secondary analyses
We performed secondary analyses based on 1/the change in
depression scores measured exclusively on a clinician-rated
depression scale, and 2/the change in depression scores
measured exclusively on a self-report scale.
2.6.3. Minor changes to the initial protocol
We chose to adjust our mixed-effect model to the type of patient
recruitment instead of patient hospital status (i.e. inpatient or
outpatient), as it appeared that in the articles included in the pre-
sent study, there were only outpatients. We also decided to adjust
the model on the continent, as it was identified as another potential
confounding factor.
3. Results
3.1. Study selection
A total of 198 studies were identified. After reviewing titles and
abstracts, 82 articles were discarded because they did not meet the
eligibility criteria. Of the remaining 116 studies, 84 were included in
the meta-analysis (one article reported results from two distinct
trials), corresponding to 217 study arms for which data were
collected. Three arms were then excluded (two arms presented
missing data on depression scores, and one arm corresponded to
pooled data from two distinct interventions), and the remaining
214 arms were analyzed. A flow chart detailing the study selection
process is given in Fig. 1.
3.2. Study characteristics and risk of bias within studies
The main characteristics of the different studies and study
quality are given in Table 1. The description of study arms is detailed
in Table 2. There were 141 psychotherapeutic intervention arms
and 73 control arms included in the analysis, involving respectively
4213 and 2617 patients. The RCTs predominantly recruited women
(72%), and mean patient age was 39.2 years. Data were analyzed on
an intention-to-treat basis for 26 (31%) RCTs: 10 (38%) used the last
observation carried forward approach, 5 (19%) used a mixed-effect
model, and 2 (8%) used other methods. For the remaining nine
studies, the method was not clearly identified.
3.3. Results from individual studies and synthesis of results
3.3.1. Main analysis
After adjusting the model on the type of intervention only
(univariate analysis), we found a superiority of all psychotherapies
and control conditions compared to the waiting list reference
condition (Fig. 2). After adjusting the model on non-specific factors
(multivariate analysis), estimates of treatment effects were
considerably reduced in comparison with those obtained in uni-
variate analysis and failed to remain significant (all confidence in-
tervals for these estimates included zero, see Fig. 2 and Table 3). The
effect of interpersonal psychotherapy was the most robust with an
increase in response of around 0.40 (95% CI [-0.01; 0.81]) compared
to the waiting list condition. Interestingly, certain factors were
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104 97
Fig. 1. Prisma flow chart.
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e10498
associated with improvement in response. The increase in treat-
ment response was 0.55 (95% CI [0.20; 0.90]) for North-American
patients, 0.03 (95% CI [0.01; 0.04]) per treatment session and 0.01
(95% CI [0.00; 0.02]) per week of follow-up. Response was also
better in case of presumed researcher allegiance to the method
implemented (
b
¼0.29, 95% CI [0.07; 0.52]).
Regarding methodological quality criteria, we did not evidence
any improvement in response when data were not analyzed on an
intention-to-treat-basis (
b
¼0.02, 95% CI [-0.31; 0.35]) nor when
patients were assessed by a blinded (
b
¼0.00, 95% CI [-0.37; 0.38])
or unblinded clinician (
b
¼0.31, 95% CI [-0.84; 0.23]) compared to
self-report evaluation. All the VIFs (see Web-appendix) were under
ten, which could be regarded as acceptable. However, the vari-
ability of the random effect was 0.29, reflecting substantial het-
erogeneity across studies.
3.3.2. Additional analyses
Results from meta-regression analyses performed on secondary
outcomes are presented in detail in the Web-appendix. Overall,
when response was measured bya self-report depression scale only
(n ¼193 arms), findings from the main analysis were replicated,
Table 1
Study characteristics.
Studies (n ¼84)
Year of publication 2002 (1976; 1991; 2008; 2012)
Continent
Europe 26 (31.0)
North America 47 (56.0)
Central America and South America 1 (1.2)
Asia 5 (6.0)
Africa 1 (1.2)
Australia 4 (4.8)
Depression assessment
Self-report scale 45 (53.6)
Clinician-rated scale:
Blinded assessment 31 (36.9)
Unblinded assessment 8 (9.5)
Intent-to-treat analysis
Yes 26 (31.0)
No 58 (69.0)
Randomisation bias
Low Risk 28 (33.3)
High Risk 8 (9.5)
Unclear Risk 48 (57.1)
Quantitative variables: median (Min; Q1; Q3; Max); Qualitative variables: N (%).
Table 2
Arm characteristics.
Psychotherapy
(n ¼141)
Control condition
(n ¼73)
Total
(n ¼214)
Characteristics of patients
Mean age 39.4
(19.1; 36.0; 43.0; 55.9)
(n ¼132)
39.1
(19.5; 33.5; 42.5; 55.9)
(n ¼70)
39.2
(19.1; 35.9; 42.7; 55.9)
(n ¼202)
Proportion of women 72.0
(0.0; 66.6; 82.8; 100)
73.3
(9.4; 65.0; 84.0; 100)
72.0
(0.0; 66.0; 83.1; 100)
Patient recruitment method
Community 89 (63.1) 38 (52.1) 127 (59.3)
Clinical 35 (24.8) 24 (32.9) 59 (27.6)
Other 17 (12.1) 11 (15.1) 28 (13.1)
Characteristics of therapies and control groups
Type of psychotherapy
Cognitive behavioral 86 (61.0) e86 (61.0)
Psychodynamic 4 (2.8) e4 (2.8)
Interpersonal 12 (8.5) e12 (8.5)
Problem-solving 11 (7.8) e11 (7.8)
Behavioral activation 8 (5.7) e8 (5.7)
Non-directive supportive 10 (7.1) e10 (7.1)
Other psychotherapy 10 (7.1) e10 (7.1)
Type of control group
Waiting list e38 (52.1) 38 (52.1)
Care-as-usual e22 (30.1) 22 (30.1)
Placebo pill e6 (8.2) 6 (8.2)
'Fake' psychotherapy e0 (0.0) 0 (0.0)
Other control e7 (9.6) 7 (9.6)
Treatment format
Individual 69 (48.9) 68 (93.2) 137 (64.0)
Group 38 (27.0) 2 (2.7) 40 (18.7)
Guided self-help 27 (19.1) 1 (1.4) 28 (13.1)
Other 7 (5.0) 2 (2.7) 9 (4.2)
Number of treatment sessions 10.0
(0.0; 8.0; 16.0; 24.0)
0.0
(0.0; 0.0; 1.3; 20.0)
(n ¼51)
8.0
(0.0; 4.0; 12.0; 24.0)
(n ¼192)
Number of assessment sessions 2.0
(1.0; 2.0; 3.0; 11.0)
(n ¼136)
2.0
(1.0; 1.0; 3.0; 10.0)
(n ¼69)
2.0
(1.0; 2.0; 3.0; 11.0)
(n ¼205)
Length of follow-up (weeks) 20.0
(1.7; 12.0; 34.0; 116)
(n ¼136)
12.0
(1.7; 8.0; 22.5; 116)
(n ¼72)
16.0
(1.7; 10.0; 32.0; 116)
(n ¼208)
Researcher allegiance
Yes 108 (76.6) 0 (0.0) 108 (50.5)
No 33 (23.4) 73 (100) 106 (49.5)
Quantitative variables: median (Min; Q1; Q3; Max); Qualitative variables: N (%).
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104 99
except for behavioral activation therapy which showed a difference
with the waiting list control condition. The confidence intervals of
the estimates regarding researcher allegiance and the number of
treatment sessions included zero. The heterogeneity was found to
be small (see Web-appendix).
In contrast, when response was measured by a clinician-rated
scale exclusively (n ¼101 arms), all psychotherapies and control
conditions showed superiority to the waiting list condition, except
for controls classified as “other”. The VIF associated with cognitive-
behavioral therapy was above ten, indicating a multicollinearity
issue. In addition, we found substantial heterogeneity across
studies (see Web-appendix).
Finally, results were robust when the analyses were performed
using different pre/post correlation coefficients and when each
Fig. 2. Forest plots presenting estimates of treatment effects before and after adjusting on non-specific factors.
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104100
study was removed in turn.
4. Discussion
4.1. Summary of evidence
Without adjusting on non-specific factors (univariate analysis),
we found a superiority of all psychotherapies and control condi-
tions compared to the waiting list condition. These results are in
accordance with those obtained in the Barth et al. meta-analysis
(2013). The effects associated with psychotherapies and control
conditions compared to the waiting list reference condition were
considerably reduced after adjusting on non-specific factors
(multivariate analysis), and all the confidence intervals of the es-
timates included zero. Conversely, response improved with the
number of encounters between the patient and the therapist. In the
case of antidepressants, previous studies have already shown the
importance of non-specific components in response (Naudet et al.,
2011). This meta-analysis suggests that these results are also
applicable to psychotherapeutic interventions, and underlines the
importance of a regular follow-up of depressed patients.
We adjusted our model on researcher allegiance, which is
known to be a considerable source of bias in psychotherapy eval-
uation (Munder et al., 2012, 2013). We observed that researcher
allegiance was associated with an increase in response, suggesting
two nonexclusive interpretations: 1/a selective publication bias (or
selective outcome reporting bias) could be involved here or, more
interestingly, 2/the degree of belief by the therapist in the efficiency
of a treatment directly impacts patient response. This is why
Munder et al. recommend 1/that comparative studies should be
conducted collaboratively by teams with mixed allegiances, 2/that
therapists engaging in research on different treatment conditions
should be motivated to learn and deliver the treatments, 3/that
meta-analyses on the efficacy of treatments should include a
consideration of researcher allegiance as a potential rival expla-
nation for their findings.
We expected an increase in treatment response when depres-
sion scores were not reported on an intention-to-treat basis, but
the confidence interval of the estimate included zero (Altman,
2009; Nüesch et al., 2009). This could be explained by the use of
a very strict definition of ITT, so that certain RCTs that used the
modified intention-to-treat approach were considered as not hav-
ing been performed on an ITT basis. We also observed better
treatment response for North-American patients compared to Eu-
ropeans. This might be explained by 1/different personal concep-
tions of psychotherapy depending on the country, and 2/patients'
expectations towards psychotherapies varying according to the
country.
Table 3
Meta-regression analysis, change measured using both clinician-rated and self-report depression scale (n ¼214 arms).
Coefficient (
b
) Confidence interval (95%)
Characteristics of patients
Mean age 0.02 [-0.04; 0.00]
Proportion of women 0.00 [-0.01; 0.01]
Patient recruitment method (Ref ¼Community)
Clinical 0.28 [-0.08; 0.63]
Other ¡0.62 [-1.09; -0.15]
Characteristics of therapies and control groups
Continent (Ref ¼Europe)
North America 0.55 [0.20; 0.90]
Central America and South America 0.45 [-0.80; 1.69]
Asia 0.15 [-0.49; 0.79]
Africa 0.60 [-0.68; 1.88]
Australia 0.08 [-0.60; 0.77]
No Intent-to-treat analysis (Ref ¼ITT) 0.02 [-0.31; 0.35]
Depression assessment (Ref ¼Self-report scale)
Clinician-rated scale, unblinded assessment 0.31 [-0.84; 0.23]
Clinician-rated scale, blinded assessment 0.00 [-0.37; 0.38]
Type of intervention (Ref ¼Waiting list)
Cognitive behavioral therapy 0.11 [-0.23; 0.46]
Psychodynamic therapy 0.02 [-0.36; 0.40]
Interpersonal therapy 0.40 [-0.01; 0.81]
Problem-solving therapy 0.27 [-0.10; 0.63]
Behavioral activation therapy 0.38 [-0.24; 1.00]
Non-directive supportive therapy 0.08 [-0.30; 0.46]
Other psychotherapy 0.10 [-0.26; 0.46]
Care-as-usual 0.20 [-0.48; 0.08]
Placebo pill 0.16 [-0.11; 0.44]
Other control 0.18 [-0.55; 0.18]
Treatment format (Ref ¼Individual)
Group 0.08 [-0.27; 0.11]
Guided Self-help 0.07 [-0.15; 0.30]
Other 0.31 [-0.01; 0.64]
Number of treatment sessions 0.03 [0.01; 0.04]
Number of assessment sessions 0.00 [-0.04; 0.05]
Length of follow-up (weeks) 0.01 [0.00: 0.02]
Researcher allegiance (Ref ¼No) 0.29 [0.07: 0.52]
This analysis included 214 arms involving 6830 participants.
The variance of the random effect was 0.29 reflecting substantial heterogeneity across studies.
For each qualitative covariate implemented in the multivariate analysis, a level has been chosen as reference (Ref).
Each covariate increases the response (i.e. the standardized mean change in depression score before and after treatment) by a value equal to its
corresponding beta coefficient.
Estimates for which the confidence interval does not include zero are in bold.
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104 101
4.2. Limitations
Statistical modelling is an art rather than hard science, and
certain limitations need to be taken into account. First, the asso-
ciations derived from meta-regressions are observational, and have
a weaker interpretation than the causal relationships derived from
randomized comparisons. Second, we chose to describe treatment
response from aggregated data. This is more likely to detect effects
at study level, but it can lead to misinterpretations at patient level,
where an aggregation bias can occur. This cannot be investigated
without individual patient data (Thompson and Higgins, 2002).
Thus results relating to gender and age should be interpreted
cautiously. We also performed a multiple imputation of missing
data. This implies that data is missing at random (White et al.,
2011). There is however no way to ascertain this assumption. In
the multivariate analysis, all of the psychotherapies still had effect
estimates in line with a superiority over waiting list condition
(although zero was included in the confidence intervals), particu-
larly interpersonal therapy. In addition, the confidence intervals of
the estimates for the type of intervention were larger in the
multivariate analysis than in the univariate analysis. This can be
due to 1/a lack of power, or 2/the multiple imputation process,
which could have introduced variability. Furthermore, in the
multivariate model we considered only covariates defined a priori
in the protocol, whereas other unknown relevant covariates could
have confounded our results, such as the presence in the studies of
a risk of bias in the concealment of allocation, which is under-
reported in the studies included (this parameter was mostly
quoted as “not reported or inadequate”in the meta-analysis by
Barth et al. (2013)). This restricts interpretation of our results, as
this methodological issue could have biased the estimates of the
meta-regression. Further to this, we cannot exclude a publication
bias, which has been well documented for studies on psychother-
apies (Cuijpers et al., 2010), thought to give an overestimation of
the results of the intervention under study. For these reasons, our
results should be taken cautiously and are liable to suggest hy-
potheses rather than provide a definite answer.
Additionally, the results observed in the main analysis were not
robust when we considered changes in depression scores as
measured exclusively on a clinician-rated depression scale. This
result could be explained 1/by the very small number of studies
reporting clinician-rated scale scores, making this analysis under-
powered, 2/by the great heterogeneity observed across studies, 3/
because double-blind condition is somewhat difficult to achieve in
these studies, 4/because of potential problems in numerical esti-
mations (VIF >10 for cognitive-behavioral therapy), and 5/by a
possible performance bias on the part of the clinicians, particularly
for the waiting list control condition where it is impossible to blind.
More generally, because of the very nature of the question
tackled in the paper, it is noticeable that several variables included
in the model are correlated. For instance, some non-specific factors
can be closely associated with a particular type of therapy. This has
an important consequence: the estimations of the parameters
related to the therapy variable do not reflect the actual effect of
these therapies (this effect is adjusted on a series of variables that
are associated with them).
Regarding the quality of the studies, we observed that for
approximately one third of the RCTs, an ITT analysis was performed.
Nevertheless, ITT analysis can also introduce significant bias if there
is 1/high or unbalanced drop-out, 2/missing data is not missing at
random, and 3/the imputation is via the last observation carried
forward method. Unfortunately, the paucity of the reporting
regarding this information did not enable us to investigate this
source of bias.
All in all, these limitations should preclude any over-
interpretation of our findings, which provide fresh insight into
psychotherapeutic interventions in MDD rather than a firm and
definite answer.
4.3. Perspectives
Our objective was not to compare psychotherapies, but to
examine the impact of non-specific elements on the treatment ef-
fect of a range of psychotherapies. From another perspective, it calls
the question of whether or not psychotherapies rely on something
specific to succeed. It is nonetheless very difficult to define what is
specific in such complex interventions. This notion makes sense in
the context of a robust mechanical/theoretical framework, which is
probably not the case for psychotherapeutic interventions. For
example, one can have a psychodynamic explanation for the
change induced by a cognitive therapy, and a cognitive interpre-
tation of the change occurring in psychoanalysis. In addition, most
therapies can include a range of techniques, sometimes without
clear boundaries. A meta-analysis taking into account this range of
possible techniques might be a next step to explore the efficacy of
psychotherapeutic interventions.
Typically, RCTs are designed to assess the difference between an
intervention group and a placebo group or a reference control
group. RCTs rely on such an apparently simple approach that their
results are thought to be very credible when they claim that a given
treatment works. Nonetheless, while these studies are modeled on
the prototypical “pharmacological versus placebo RCT”, it is not
possible to obtain a control condition for a complex intervention
that is as “pure”as a placebo (Wampold et al., 2005). In the field of
psychotherapeutic intervention, RCTs make it possible to say that
an intervention works in comparison to a given control. But it does
not say why it works or whether this effect is specific or not,
because no ideal condition can control for non-specific effects (Ahn
and Wampold, 2001). This confusion should be avoided when
interpreting result of such studies. Inferring causality from RCTs
without bearing this perspective in mind could sterilize the debate.
Indeed, our results suggest that the theoretical framework is not
the major determinant of successful care.
On the other hand, our results do not mean that psychothera-
peutic interventions are ineffective, but rather suggest that their
efficacy is closely related to their non-specific effects. Additionally,
researcher allegiances can be seen here as reflecting the therapist's
implication in the care that is provided. It is well known that
therapeutic allegiance, and the empathy or motivation of the
caregiver to deliver the particular therapy is crucial in psycho-
therapies for MDD (Leykin and DeRubeis, 2009). In fact this review
of RCT evidence mainly supports this idea.
5. Conclusions
5.1. Implications for research
It is clear that psychotherapeutic interventions are useful, but
more research is needed to understand how they work. The
development of a well-designed trial that controls for non-specific
factors might help disentangle the effects of psychotherapies. In
addition, we believe that it would be interesting to explore non-
specific effects in future prospective studies, which could provide
stronger evidence than meta-regressions and/or indirect
comparisons.
5.2. Implications for practice
In psychotherapies, the most important ingredient is probably
the help given by the clinician to patients in coping with
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104102
depression, rather than strong views on his or her part on the way
to do it. Engaging in dialogue with the patient and establishing
regular follow-up appointments remain key points for good patient
care.
Contributors
Conceived and designed the experiments: NF, PC, GL.
Performed the experiments: PC, GL, NF.
Analyzed the data: PC, GL, NF.
Contributed reagents/materials/analysis tools: PC, GL, NF.
Wrote the paper: PC, GL, NF.
Revised the paper critically for important intellectual content:
DD, FB, RJM.
Final approval of the version to be published: PC, GL, DD, RJM,
FB, NF.
Role of the funding source
The funding source had no role in the design and conduct of the
study; collection, management, analysis, and interpretation of the
data; preparation, review, or approval of the manuscript; and de-
cision to submit the manuscript for publication.
Conflicts of interest
There are no conflicts of interest regarding this paper. All au-
thors have completed the Unified Competing Interest form at
http://www.icmje.org/coi_disclosure.pdf (available on request
from the corresponding author) and declare that (1) No authors
have support from any company for the submitted work; (2) P.C.
was a trainee in Servier (pharmacokinetics department) for 6
months in 2013; G.L. is a trainee in cognitive behavioral therapy; DD
has relationships (board membership or consultancy or travel/ac-
commodation expenses covered/reimbursed) with Servier, Lilly,
Janssen-Cilag, Otsuka, Lundbeck, Astra zeneca; R.J.M. have no re-
lationships with any company that might have an interest in the
submitted work in the previous 3 years; F.B has relationships
(board membership or consultancy or payment for manuscript
preparation or travel/accommodation expenses covered/reim-
bursed) with SanofiAventis, Servier, Pierre-Fabre, MSD, Lilly,
Janssen-Cilag, Otsuka, Lundbeck, Genzime, Roche, BMS who might
have an interest in the work submitted in the previous 3 years; N.F.
has relationships (travel/accommodation expenses covered/reim-
bursed) with Servier, BMS, Lundbeck and Janssen who might have
an interest in the work submitted in the previous 3 years; (3) No
author's spouse, partner, or children have any financial relation-
ships that could be relevant to the submitted work; and (4) none of
the authors has any non-financial interests that could be relevant to
the submitted work.
Acknowledgement
This work was supported by a local grant from Rennes CHU (D-
PSY-D-META) (CORECT: COmit
e de la Recherche Clinique et
Translationelle). We would like to thank Jean-Yves Lannou for his
help on bibliographic research, Cl
emence Pontoizeau for her help in
formatting manuscript, and Angela Swaine Verdier for revising the
English.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://
dx.doi.org/10.1016/j.jpsychires.2016.12.015.
References
Ahn, H., Wampold, B.E., 2001. Where oh where are the specific ingredients? A meta-
analysis of component studies in counseling and psychotherapy. J. Couns.
Psychol. 48, 251.
Altman, D.G., 2009. Missing outcomes in randomized trials: addressing the
dilemma. Open Med. 3, e51ee53.
Barth, J., Munder, T., Gerger, H., Nüesch, E., Trelle, S., Znoj, H., Jüni, P., Cuijpers, P.,
2013. Comparative efficacy of seven psychotherapeutic interventions for pa-
tients with depression: a network meta-analysis. PLoS Med. 10, e1001454.
http://dx.doi.org/10.1371/journal.pmed.1001454.
Budd, R., Hughes, I., 2009. The Dodo Bird Verdictdcontroversial, inevitable and
important: a commentary on 30 years of meta-analyses. Clin. Psychol. Psy-
chother. 16, 510e522. http://dx.doi.org/10.1002/cpp.648.
Chatoor, I., Krupnick, J., 2001. The role of non-specific factors in treatment outcome
of psychotherapy studies. Eur. Child. Adolesc. Psychiatry 10 (Suppl. 1), I19eI25.
Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S.D., Andersson, G., 2010. Efficacy of
cognitiveebehavioural therapy and other psychological treatments for adult
depression: meta-analytic study of publication bias. Br. J. Psychiatry 196,
173 e178. http://dx.doi.org/10.1192/bjp.bp.109.066001.
Cuijpers, P., van Straten, A., Andersson, G., van Oppen, P., 2008a. Psychotherapy for
depression in adults: a meta-analysis of comparative outcome studies.
J. Consult. Clin. Psychol. 76, 909e922. http://dx.doi.org/10.1037/a0013075.
Cuijpers, P., van Straten, A., Warmerdam, L., Andersson, G., 2008b. Psychological
treatment of depression: a meta-analytic database of randomized studies. BMC
Psychiatry 8, 36. http://dx.doi.org/10.1186/1471-244X-8-36.
Ekers, D., Richards, D., Gilbody, S., 2008. A meta-analysis of randomized trials of
behavioural treatment of depression. Psychol. Med. 38, 611e623. http://
dx.doi.org/10.1017/S0 033291707001614.
Gloaguen, V., Cottraux, J., Cucherat, M., Blackburn, I.M., 1998. A meta-analysis of the
effects of cognitive therapy in depressed patients. J. Affect. Disord. 49, 59e72.
Gøtzsche, P.C., 2014. Why I think antidepressants cause more harm than good.
Lancet Psychiatry 1, 104e106. http://dx.doi.org/10.1016/S2215-0366(14)70280-
9.
Higgins, J.P.T., Altman, D.G., Gøtzsche, P.C., Jüni, P., Moher, D., Oxman, A.D.,
Savovi
c, J., Schulz, K.F., Weeks, L., Sterne, J.A.C., 2011. The Cochrane Collabora-
tion's tool for assessing risk of bias in randomised trials. BMJ 343, d5928. http://
dx.doi.org/10.1136/bmj.d5928.
Higgins, J.P.T., Green, S. (Eds.), 2011. Cochrane Handbook for Systematic Reviews of
Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration.
Available from. www.cochrane-handbook.org (n.d).
Ilardi, S.S., Craighead, W.E., 1994. The role of nonspecific factors in cognitive-
behavior therapy for depression. Clin. Psychol. Sci. Pract. 1, 138e155. http://
dx.doi.org/10.1111/j.1468-2850.1994.tb00016.x.
Jacobson, N.S., Dobson, K.S., Truax, P.A., Addis, M.E., Koerner, K., Gollan, J.K.,
Gortner, E., Prince, S.E., 1996. A component analysis of cognitive-behavioral
treatment for depression. J. Consult. Clin. Psychol. 64, 295e304.
Leichsenring, F., Rabung, S., Leibing, E., 2004. The efficacy of short-term psycho-
dynamic psychotherapy in specific psychiatric disorders: a meta-analysis. Arch.
Gen. Psychiatry 61, 1208e1216. http://dx.doi.org/10.1001/archpsyc.61.12.1208.
Leykin, Y., DeRubeis, R.J., 2009. Allegiance in psychotherapy outcome research:
separating association from bias. Clin. Psychol. Sci. Pract. 16, 54e65. http://
dx.doi.org/10.1111/j.1468-2850.20 09.01143.x.
Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A.,
Clarke, M., Devereaux, P.J., Kleijnen, J., Moher, D., 2009. The PRISMA statement
for reporting systematic reviews and meta-analyses of studies that evaluate
health care interventions: explanation and elaboration. PLoS Med. 6, e1000100.
http://dx.doi.org/10.1371/journal.pmed.1000100.
Little, R.J., Rubin, D.B., 2014. Statistical Analysis with Missing Data. John Wiley &
Sons.
Munder, T., Brütsch, O., Leonhart, R., Gerger, H., Barth, J., 2013. Researcher allegiance
in psychotherapy outcome research: an overview of reviews. Clin. Psychol. Rev.
33, 501e511.
Munder, T., Flückiger, C., Gerger, H., Wampold, B.E., Barth, J., 2012. Is the allegiance
effect an epiphenomenon of true efficacy differences between treatments? a
meta-analysis. J. Couns. Psychol. 59, 631e637. http://dx.doi.org/10.1037/
a0029571.
Naudet, F., Maria, A.S., Falissard, B., 2011. Antidepressant response in major
depressive disorder: a meta-regression comparison of randomized controlled
trials and observational studies. PloS One 6, e20811. http://dx.doi.org/10.1371/
journal.pone.0020811.
Nüesch, E., Trelle, S., Reichenbach, S., Rutjes, A.W.S., Bürgi, E., Scherer, M.,
Altman, D.G., Jüni, P., 2009. The effects of excluding patients from the analysis in
randomised controlled trials: meta-epidemiological study. BMJ 339. http://
dx.doi.org/10.1136/bmj.b3244.
Nutt, D.J., Goodwin, G.M., Bhugra, D., Fazel, S., Lawrie, S., 2014. Attacks on antide-
pressants: signs of deep-seated stigma? Lancet Psychiatry 1, 102e104. http://
dx.doi.org/10.1016/S2215-0366(14)70232-9.
Parker, G., Roy, K., Eyers, K., 2003. Cognitive behavior therapy for depression?
Choose horses for courses. Am. J. Psychiatry 160, 825e834. http://dx.doi.org/
10.1176/appi.ajp.160.5.825.
R Development Core Team, 2008. R: a Language and Environment for Statistical
Computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-
900051-07-0. http://www.R-project.org.
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104 10 3
Sterne, J.A.C., Sutton, A.J., Ioannidis, J.P.A., Terrin, N., Jones, D.R., Lau, J., Carpenter, J.,
Rücker, G., Harbord, R.M., Schmid, C.H., Tetzlaff, J., Deeks, J.J., Peters, J.,
Macaskill, P., Schwarzer, G., Duval, S., Altman, D.G., Moher, D., Higgins, J.P.T.,
2011. Recommendations for examining and interpreting funnel plot asymmetry
in meta-analyses of randomised controlled trials. BMJ 343. http://dx.doi.org/
10.1136/bmj.d40 02.
Thompson, S.G., Higgins, J.P.T., 2002. How should meta-regression analyses be
undertaken and interpreted? Stat. Med. 21, 1559e1573. http://dx.doi.org/
10.1002/sim.1187.
Wampold, B.E., Minami, T., Baskin, T.W., Callen Tierney, S., 2002. A meta-(re)analysis
of the effects of cognitive therapy versus “other therapies”for depression.
J. Affect. Disord. 68, 159e165.
Wampold, B.E., Minami, T., Tierney, S.C., Baskin, T.W., Bhati, K.S., 2005. The placebo
is powerful: estimating placebo effects in medicine and psychotherapy from
randomized clinical trials. J. Clin. Psychol. 61, 835e854.
White, I.R., Royston, P., Wood, A.M., 2011. Multiple imputation using chained
equations: issues and guidance for practice. Stat. Med. 30, 377e399. http://
dx.doi.org/10.1002/sim.4067.
C. Palpacuer et al. / Journal of Psychiatric Research 87 (2017) 95e104104
