ArticlePDF Available

A Short Guide and a Forest Plot Command (Ipdforest) for One-Stage Meta-Analysis

Authors:
Evangelos Kontopantelis at The University of Manchester
Evangelos Kontopantelis
David Reeves at The University of Manchester
David Reeves
Download full-text PDF
Read full-text
Download citation

Abstract

This article describes a new individual patient data (IPD) meta-analysis post-estimation command, ipdforest. The command produces a forest plot, following an one-stage meta-analysis with xtmixed or xtmelogit (renamed in Stata 13 to mixed and meqrlogit respectively; ipdforest is currently not compatible with the new names). The overall effect is obtained from the preceding mixed-effects regression and the study effects from linear or logistic regressions on each study which are executed within ipdforest. IPD meta-analysis models with Stata are discussed.
Content uploaded by Evangelos Kontopantelis
Author content
All content in this area was uploaded by Evangelos Kontopantelis on Jun 02, 2015
Content may be subject to copyright.
The Stata Journal (yyyy)vv, Number ii, pp. 1 –13
A short guide and a forest plot command
(ipdforest) for one-stage meta-analysis
Evangelos Kontopantelis
NIHR School for Primary Care Research
Institute of Population Health
University of Manchester
Manchester, UK
e.kontopantelis@manchester.ac.uk
David Reeves
Institute of Population Health
University of Manchester
Manchester, UK
david.reeves@manchester.ac.uk
Abstract. This article describes a new individual patient data (IPD) meta-
analysis post-estimation command, ipdforest. The command produces a forest
plot, following an one-stage meta-analysis with xtmixed or xtmelogit (renamed
in Stata 13 to mixed and meqrlogit respectively; ipdforest is currently not com-
patible with the new names). The overall effect is obtained from the preceding
mixed-effects regression and the study effects from linear or logistic regressions on
each study which are executed within ipdforest. IPD meta-analysis models with
Stata are discussed.
Keywords: st0001, ipdforest, meta-analysis, forest plot, individual patient data,
IPD, one-stage.
1 Introduction
Meta-analysis, the methodology that allows results from independent studies to be com-
bined, is usually a two-stage process. First, the relevant summary effect statistics are
extracted from published papers on the included studies and these are then combined
into an overall effect estimate using a suitable meta-analysis model (Harris et al. 2008;
Kontopantelis and Reeves 2010). However, problems often arise when papers do not
report all the statistical information required as input for the meta-analysis (e.g. fail
to provide a variance estimate for the outcome measure), report a statistic other than
the effect size (such as a t-value or p-value) which needs to be transformed with a loss
of precision, or the sample is too clinically heterogeneous for the study to be included
in the meta-analysis (Kontopantelis and Reeves 2009). When individual patient data
(IPD) from each study are available, meta-analysts can avoid these problems when esti-
mating study effects; outcomes can be easily standardized, while clinical heterogeneity
can be, at least, partially addressed with subgroup analyses and patient-level covari-
ate controlling. Furthermore, when IPD data are available, meta-analysts can use a
mixed-effects regression model to combine information across studies in a single stage.
This is recognized as the best approach for performing an IPD meta-analysis, with the
two-stage method being at best equivalent in certain scenarios (Mathew and Nordstrom
2010).
Notwithstanding these advantages of the one-stage approach, one obvious advantage
c
yyyy StataCorp LP st0001
2ipdforest
of two-stage meta-analysis is the ability to convey information graphically through a
forest plot. Since study effects have been calculated or extracted in the first stage of
the process, they, and their respective confidence intervals, can be used to demonstrate
the relative strength of the intervention in each study and across all. Forest plots are
informative, easy to follow and particularly useful for readers with little or no experience
in meta-analysis methods. It is not surprising then that they have become a key feature
of meta-analysis and are always presented when two-stage meta-analyses are performed.
However, under a one-stage meta-analysis model only the overall effect is calculated,
not individual study effects, and thus creating a forest-plot is not straightforward. A
search by the authors failed to identify one-stage meta-analysis forest-plot modules, in
any general or meta-analysis specialist statistical packages. We attempt to address this
gap in Stata with the ipdforest command.
This paper is divided into two sections. In the first section we describe IPD meta-
analysis models and their implementation in Stata with available mixed-effects models.
The second section describes the ipdforest command in detail and provides an exam-
ple.
2 Individual patient data meta-analysis
A description of IPD meta-analysis methods for continuous and binary outcomes has
been provided by Higgins et al. (2001) and Turner et al. (2000) respectively. Although
we will only explore a representative selection of linear random-effects models in Stata,
using the xtmixed command, application to the logistic case using xtmelogit should
be straightforward. Let us assume individual patient data from a group of studies. For
each trial, we have the exposure variable (continuous or binary e.g. control/intervention
membership), baseline and follow-up data for the continuous outcome and covariates.
We will also assume that both the outcome measure and any covariates have been
measured in the same way across studies and therefore standardization is not required.
In the models that follow, in general, we denote fixed effects with ‘γ’s and random effects
with ‘β’s.
Possibly the simplest approach is to assume a common intercept across studies and
that baseline is a fixed effect but allow the treatment effect to vary at random across
studies:
´
Yij=γ0+β1jgroupij+γ2Yij+ij
β1j=γ1+u1j
(1a)
and
ij∼N(0, σ2
j)
u1j∼N(0, τ1
2)(1b)
where
ithe patient
E. Kontopantelis and D. Reeves 3
jthe study
´
Yijthe outcome for patient iin study j
γ0the fixed common intercept
β1jthe random treatment effect for study j
γ1the mean treatment effect
groupijexposure for patient iin study j
γ2the fixed baseline effect
Yijthe baseline score for patient iin study j
u1jthe random treatment effect for study j(shifting the regression line up or down by
study)
τ1
2the between study variance
ijthe error term for patient iin study j
σ2
jthe within study variance for study j
However, the common intercept and fixed baseline assumptions are difficult to justify
and such a model should be approached with caution - if at all. A more accepted model
allows for different fixed intercepts and fixed baseline effects for each study:
´
Yij=γ0j+β1jgroupij+γ2jYij+ij
β1j=γ1+u1j
(2)
where
γ0jthe fixed intercept for study j
γ2jthe fixed baseline effect for study j
Another possibility, although contentious (Whitehead 2002), is to assume study
intercepts are random, in a multi-centre study for example:
´
Yij=β0j+β1jgroupij+γ2jYij+ij
β0j=γ0+u0j
β1j=γ1+u1j
(3a)
In this case it is probably wiser to assume a non-zero correlation ρbetween the random
effects:
ij∼N(0, σ2
j)
u0j∼N(0, τ 2
0)
u1j∼N(0, τ 2
1)
cov(u0j, u1j) = ρτ0τ1
(3b)
The baseline could also have been modeled as a random-effect and allowing for non-
4ipdforest
zero correlations between the three random effects, thus complicating (3) further:
´
Yij=β0j+β1jgroupij+β2jYij+ij
β0j=γ0+u0j
β1j=γ1+u1j
β2j=γ2+u2j
(4a)
with effects
ij∼N(0, σ2
j)
u0j∼N(0, τ 2
0)
u1j∼N(0, τ 2
1)
u2j∼N(0, τ 2
2)
cov(u0j, u1j) = ρ1τ0τ1
cov(u0j, u2j) = ρ2τ0τ2
cov(u1j, u2j) = ρ3τ1τ2
(4b)
In some cases the focus might be on interactions. For example, if we assume a
continuous and standardized variable Xwe can expand (2) to include fixed effects, in
this instance, for both Xand its interaction with the treatment:
´
Yij=γ0j+β1jgroupij+γ2jYij+γ3Xij+γ4groupijXij+ij
β1j=γ1+u1j
(5)
If we assume Yfin and Ybas the outcome and baseline respectively, exposure variable
group and study identifier studyid for 4 studies, we can implement the models described
above using xtmixed.
Model 1: Fixed common intercept; random treatment effect; fixed effect for baseline.
. xtmixed Yfin i.group Ybas || studyid:group, nocons
Note that the nocons option suppresses estimation of the intercept as a random effect.
Model 2: Fixed study-specific intercepts; random treatment effect; fixed study-
specific effects for baseline (where Ybas‘i’=Ybas if studyid=‘i’ and zero otherwise).
. xtmixed Yfin i.group i.studyid Ybas1 Ybas2 Ybas3 Ybas4 || studyid:group
> , nocons
Model 3: Random study intercept; random treatment effect; fixed study-specific
effects for baseline.
. xtmixed Yfin i.group Ybas1 Ybas2 Ybas3 Ybas4 || studyid:group, cov(uns)
Model 4: Random study intercept; random treatment effect; random effect for base-
line.
E. Kontopantelis and D. Reeves 5
. xtmixed Yfin i.group Ybas || studyid:group Ybas, cov(uns)
In general, a covariate (or an interaction term) can be modeled as a fixed- or random-
effect but in the latter case the complexity of the model increases and non-convergence
issues are more likely to be encountered. If we also assume patient covariate age and
its interaction with the treatment effect, model 5 will be:
. xtmixed Yfin i.group i.studyid Ybas1 Ybas2 Ybas3 Ybas4 age i.group#c.age
> || studyid: group, nocons
Or alternatively, age can be modeled as a a random effect:
. xtmixed Yfin i.group i.studyid Ybas1 Ybas2 Ybas3 Ybas4 age i.group#c.age
> || studyid: group age, nocons
3 The ipdforest command
3.1 Syntax
ipdforest varname , re(varlist) fe(varlist) fets(namelist ) ia(varname)
auto label(varlist) or gsavedir(string ) gsavename(string ) eps gph
export(string )
where
varname the exposure variable (continuous or binary, e.g. intervention/control).
3.2 Options
re(varlist) Covariates to be included as random factors. For each covariate specified, a
different regression coefficient is estimated for each study.
fe(varlist) Covariates to be included as fixed factors. For each covariate specified, the
respective coefficient in the study-specific regressions is fixed to the value returned
by the multi-level regression.
fets(namelist) Covariates to be included as study-specific fixed factors (i.e. using
the estimated study fixed effects from the main regression in all individual study
regressions). Only baseline scores and/or study identifiers can be included. For each
covariate specified, the respective coefficient in the study-specific regressions is fixed
to the value returned by the multi-level regression, for the specific study. For study-
specific intercepts the study identifier, not in factor variable format (e.g. studyid),
or the stub of the dummy variables whould be included (e.g. studyid when dummy
study identifiers are studyid 1 studyid 3 etc). For study-specific baseline scores
only the stub of the dummy variables is accepted (e.g. dept0s when dummy study
baseline scores are dept0s 1 dept0s 3 etc)
ia(varname) Covariate for which the interaction with the exposure variable will be
calculated and displayed. The covariate should also be specified as a fixed, random
6ipdforest
or study-specific fixed effect. If binary, the command will provide two sets of results,
one for each group. If categorical, it will provide as many sets of results as there
are categories. If continuous, it will provide one set of results for the main effect
and one for the interaction. Although the command will accept a variable to be
interacted with the exposure variable as a fixed or study-specific fixed effect, the
variable necessarily will be included as a random effect in the individual regressions
(will not run a regression with the interaction term only, the main effects must be
included as well). Therefore, although the overall effect will differ between a model
with a fixed effect interacted variable and a random effect one, the individual study
effects will be identical across the two approaches.
auto Allows ipdforest to automatically detect the specification of the preceding model.
This option cannot be issued along with options re(),fe(),fets() or ia(). The
auto option will work in most situations but it comes with certain limitations. It
uses the returned command string of the preceding command which is effectively
constrained to 244 characters and therefore the auto option will return an error if
ipdforest follows a very wide regression model - in such a situation only the man-
ual specification can be used. In addition, the variable names used in the preceding
model must follow certain rules: i) fixed-effect covariates (manually with option
fe()) must not contain underscores, ii) for study-specific intercepts (manually with
option fets()) factor variable format is allowed or a varlist (e.g. cons 2-cons 16)
but each variable must contain a single underscore followed by the study num-
ber (not necessarily continuous) and iii) for study-specific baseline scores (manually
with option fets()) each variable must contain a single underscore followed by the
study number (again, not necessarily continuous). Note that there are no restric-
tions for random-effects covariates (manually with option re()). For interactions
(manually with option ia()) the factor variable notation should be preferred (e.g.
i.group#c.age) and alternatively the older xi notation. Interactions expanded to
dummy variables cannot be identified with the auto option and only the manual
specification should be used in this case. Variables whose names start with a ‘ I’
and contain a capital ‘X’ will be assumed to be expanded interaction terms and, if
detected in last model, ipdforest will terminate with a syntax error.
label(varlist) Selects labels for the studies. Up to two variables can be selected and
converted to strings. If two variables are selected they will be separated by a comma.
Usually, the author names and the year of study are selected as labels. If this option is
not selected the command automatically uses the value labels of the numeric cluster
variable, if any, to label the forest plot. Either way, the final string is truncated to
20 characters.
or Reporting odds ratios instead of coefficients. Can only be used following execution
of xtmelogit.
gsavedir(string) The directory where to save the graph(s), if different from the active
directory.
gsavename(string) Optional name prefix for the graph(s). Graphs are saved as ‘gsave-
name’ ‘graphname’.gph or ‘gsavename’ ‘graphname’.eps where ‘graphname’ includes
E. Kontopantelis and D. Reeves 7
a description of the summary effect (e.g. ”main group” for the main effect, if group
is the exposure variable)
eps Save the graph(s) in eps format, instead of the default gph.
gph Save the graph(s) in gph format - the default. Use to save in both formats, since
inlcluding only the eps option will save the graph(s) in eps format only.
export(string) Export the study identifiers, weights, effects and standard errors in a
Stata dataset (named after string). Provided for users wishing to use other com-
mands or software to draw the forest plots.
3.3 Saved results
ipdforest saves the following scalar results in r():
r(Isq) Heterogeneity measure I2r(Hsq) Heterogeneity measure H2
M
r(tausq) ˆτ2, between study variance esti-
mate
r(tausqlo) ˆτ2, lower 95% CI r(tausqup) ˆτ2, upper 95% CI
r(eff1pe ov) Overall effect estimate r(eff1se ov) Standard error of the overall ef-
fect
r(eff1pe st’i’) Effect estimate for study ’i’ r(eff1se st’i’) Standard error of the effect for
study ’i’
If an interaction with a continuous variable is included in the model, it also returns:
r(eff2pe ov) Overall interaction effect esti-
mate
r(eff2se ov) Standard error of the overall in-
teraction effect
r(eff2pe st’i’) Interaction effect estimate for
study ’i’
r(eff2se st’i’) Interaction effect standard error
for study ’i’
If the interaction variable is binary, the first set of effect results corresponds to the
effects for the first category of the binary (e.g. sex=0) and the second set for the second
category (e.g. sex=1). If the variable is categorical the command returns as many sets
of effect results as there are categories (with each set corresponding to one category).
Estimation results from xtmixed or xtmelogit in e() are restored after the execution
of ipdforest.
3.4 Methods
The ipdforest command is issued following a random-effects IPD meta-analysis con-
ducted using a linear (xtmixed) or logistic (xtmelogit) two-level regression, with pa-
tients nested within studies, and provides a meta-analysis summary table and a forest
plot. Study effects are calculated within ipdforest while the overall effect and variance
estimates are extracted from the preceding regression. The default estimation methods
for xtmixed and xtmelogit are restricted maximum likelihood (REML) and maximum
likelihood (ML) respectively. A description of these methods is beyond the scope of this
8ipdforest
paper.
The command estimates individual study effects and their standard errors using
one-level linear or logistic regression analyses. Following xtmixed,regress is used
and following xtmelogit,logit is used, for each study in the meta-analysis. The
ipdforest command controls these regressions for fixed- or random-effects covariates
that were specified in the preceding two-level regression. The user has full control over
the covariates to be included in the ipdforest command, including their specification
as fixed- or random-effects, but we strongly recommend using the same specification as
in the preceding xtmixed/xtmelogit command, as the reported overall effect and its
confidence interval is taken from that model.
In the estimation of individual study effects, ipdforest controls for a random-
effects covariate (i.e. allowing the regression coefficient to vary by study) by including
the covariate as an independent variable in each regression. Control for a fixed-effect
covariate (where the regression coefficient is assumed constant across studies and is
given by the coefficient estimated under xtmixed/xtmelogit) is a little more complex.
Since it is not possible to specify a fixed value for a regression coefficient under regress,
the continuous outcome variable is adjusted by subtracting the contribution of the fixed
covariates to its values in a first step prior to analysis. For a binary outcome the
equivalent is achieved through use of the offset option in logit. Patient weights are
uniform and therefore each study’s weight is the ratio of its participants over the total
number of participants across all studies.
Between-study variability in the treatment effect, known as heterogeneity, arises
from differences in study design, quality, outcomes or populations and needs to be ac-
counted for in the meta-analysis model when present. Heterogeneity is usually reported
in the form of measures or tests which compare the between- and within-study variance
estimates. For continuous outcomes, ipdforest reports two heterogeneity measures,
I2and H2
M, based on the xtmixed output. I2values of 25%, 50% and 75% correspond
to low, moderate and high heterogeneity respectively (Higgins et al. 2003), while H2
M
takes values in the [0,+∞) range with 0 indicating perfect homogeneity (Mittlbock and
Heinzl 2006). We have not attempted to calculate an IPD version of Cochran’s Q, the
orthodox χ2
k−1homogeneity test, considering its poor performance when the number of
studies kis small(Hardy and Thompson 1998). For binary outcomes, an estimate of the
within-study variance is not reported under xtmelogit and hence heterogeneity mea-
sures cannot be computed. The between-study variance estimate ˆτ2and its confidence
interval is reported under both models.
Fixed-effect meta-analysis models are widely used when heterogeneity is very low
or zero. However, a more conservative approach is to take account of even low levels
of between-study variability by adopting a random-effects model(Hunter and Schmidt
2000). When between-study variance is estimated to be close to zero, results with the
two approaches converge. Therefore, although ipdforest is a post-estimation command
for random-effects IPD meta-analysis output is close to that for a fixed-effect model
when ˆτ2≈0.
E. Kontopantelis and D. Reeves 9
3.5 Example
As an example, we apply the ipdforest command to a dataset of 4 depression interven-
tion studies. Data were provided by the authors of the and we had complete information
in terms of age, gender, exposure (control/intervention group membership), continuous
outcome baseline and endpoint values for 518 patients. Since the findings of the IPD
meta-analysis have not been published yet, we used fake author names and generated
random continuous and binary outcome variables for the purposes of this example, while
keeping the covariates at their actual values. We introduced correlation between base-
line and endpoint scores and between-study variability, although the exact specification
of the data generation is unimportant. Using the semi-artificial dataset we perform a
logistic IPD meta-analysis, followed by the ipdforest command.
. use ipdforest_example.dta,
. describe
Contains data from ipdforest_example.dta
obs: 518
vars: 17 6 Feb 2012 11:14
size: 20,202
storage display value
variable name type format label variable label
studyid byte %22.0g stid Study identifier
patid int %8.0g Patient identifier
group byte %20.0g grplbl Intervention/control group
sex byte %10.0g sexlbl Gender
age float %10.0g Age in years
depB byte %9.0g Binary outcome, endpoint
depBbas byte %9.0g Binary outcome, baseline
depBbas1 byte %9.0g Bin outcome baseline, trial 1
depBbas2 byte %9.0g Bin outcome baseline, trial 2
depBbas5 byte %9.0g Bin outcome baseline, trial 5
depBbas9 byte %9.0g Bin outcome baseline, trial 9
depC float %9.0g Continuous outcome, endpoint
depCbas float %9.0g Continuous outcome, baseline
depCbas1 float %9.0g Cont outcome baseline, trial 1
depCbas2 float %9.0g Cont outcome baseline, trial 2
depCbas5 float %9.0g Cont outcome baseline, trial 5
depCbas9 float %9.0g Cont outcome baseline, trial 9
Sorted by: studyid patid
We generate a centered age variable, interacted with the exposure variable in a mixed-
effects logistic regression model. The model includes fixed study-specific intercepts,
fixed study specific effects for baseline and random treatment and age effects. The
ipdforest command follows the regression model, requesting outcomes for both the
main effect and the interaction.
. qui sum age
. qui gen agec = age-r(mean)
(Continued on next page)
10 ipdforest
. xtmelogit depB group agec sex i.studyid depBbas1 depBbas2 depBbas5 depBbas9 i
> .group#c.agec || studyid:group agec, var nocons or
Refining starting values:
Iteration 0: log likelihood = -347.40378 (not concave)
Iteration 1: log likelihood = -336.07882 (not concave)
Iteration 2: log likelihood = -329.28297
Performing gradient-based optimization:
Iteration 0: log likelihood = -329.28297 (not concave)
Iteration 1: log likelihood = -326.79725
Iteration 2: log likelihood = -326.56885
Iteration 3: log likelihood = -326.55747
Iteration 4: log likelihood = -326.55747
Mixed-effects logistic regression Number of obs = 518
Group variable: studyid Number of groups = 4
Obs per group: min = 42
avg = 129.5
max = 214
Integration points = 7 Wald chi2(11) = 42.06
Log likelihood = -326.55747 Prob > chi2 = 0.0000
depB Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
group 1.840804 .3666167 3.06 0.002 1.245894 2.71978
agec .9867902 .0119059 -1.10 0.270 .9637288 1.010403
sex .7117592 .1540753 -1.57 0.116 .4656639 1.087912
studyid
2 1.050007 .5725516 0.09 0.929 .3606166 3.057303
5 .8014551 .5894511 -0.30 0.763 .189601 3.387799
9 1.281413 .6886057 0.46 0.644 .4469619 3.673735
depBbas1 3.152908 1.495281 2.42 0.015 1.244587 7.987253
depBbas2 4.480302 1.863908 3.60 0.000 1.982385 10.12574
depBbas5 2.387336 1.722993 1.21 0.228 .5802064 9.823007
depBbas9 1.881203 .7086507 1.68 0.093 .8990569 3.936262
group#c.agec
1 1.011776 .0163748 0.72 0.469 .9801858 1.044385
_cons .5533714 .2398342 -1.37 0.172 .2366472 1.293993
Random-effects Parameters Estimate Std. Err. [95% Conf. Interval]
studyid: Independent
var(group) 8.86e-21 2.43e-11 0 .
var(agec) 5.99e-18 4.40e-11 0 .
LR test vs. logistic regression: chi2(2) = 0.00 Prob > chi2 = 1.0000
Note: LR test is conservative and provided only for reference.
(Continued on next page)
E. Kontopantelis and D. Reeves 11
. ipdforest group, fe(sex) re(agec) ia(agec) or
One-stage meta-analysis results using xtmelogit (ML method) and ipdforest
Main effect (group)
Study Effect [95% Conf. Interval] % Weight
Hart 2005 2.118 0.942 4.765 19.88
Richards 2004 2.722 1.336 5.545 30.69
Silva 2008 2.690 0.748 9.676 8.11
Kompany 2009 1.895 0.969 3.707 41.31
Overall effect 1.841 1.246 2.720 100.00
One-stage meta-analysis results using xtmelogit (ML method) and ipdforest
Interaction effect (group x agec)
Study Effect [95% Conf. Interval] % Weight
Hart 2005 0.972 0.901 1.049 19.88
Richards 2004 0.995 0.937 1.055 30.69
Silva 2008 0.987 0.888 1.098 8.11
Kompany 2009 1.077 1.015 1.144 41.31
Overall effect 1.012 0.980 1.044 100.00
Heterogeneity Measures
value [95% Conf. Interval]
I^2 (%) .
H^2 .
tau^2 est 0.000 0.000 .
Maximum Likelihood converged succesfully in this example and the between-study
variance estimate ˆτ2was practically zero. Note that the intercept for the reference study
(studyid=1) was estimated in cons. The reported coefficients under studyid are the
differences in intercept, compared to the first study. I2and H2
Mcould not be estimated
since residual variability is not reported under xtmelogit. The overall treatment effect
was significant but the overall effect for treatment and age interaction was not, at the
95% level. The forest plots created by ipdforest are displayed in Figures 1 and 2.
12 ipdforest
Overall effect
Kompany 2009
Silva 2008
Richards 2004
Hart 2005
Studies
0 2 3 4 5 6 7 8 9 101 Effect sizes and CIs (ORs)
Main effect (group)
Figure 1: Main effect IPD forest plot, reporting Odds Ratios.
Overall effect
Kompany 2009
Silva 2008
Richards 2004
Hart 2005
Studies
0 .2 .4 .6 .8 1.2 1.41
Effect sizes and CIs (ORs)
Interaction effect (group x agec)
Figure 2: Interaction effect IPD forest plot, reporting Odds Ratios.
E. Kontopantelis and D. Reeves 13
4 Discussion
The aim of this article was to provide a practical guide for conducting one-stage IPD
meta-analysis and to present ipdforest, a new forest-plot command. The command
aims to help meta-analysts in better communicating their results through the familiar
and distinctive forest-plot, which cannot be obtained as standard in one-stage IPD
meta-analysis.
Although only binary or continuous exposure variables can be modeled, categorical
exposures can also be investigated with the use of dummy variables and a focus on the
comparison of interest through one of these. It should also be noted that the command is
fully compatible with the estimates produced by the mi multiple imputation estimation
commands, for xtmixed or xtmelogit.
Note that these commands were renamed in Stata 13; xtmixed to mixed, and
xtmelogit to meqrlogit.ipdforest is not yet compatible with the new commands but
users of version 13 or later can still use the older commands, before calling ipdforest.
5 Acknowledgments
We would like to thank Isabel Canette, Senior Statistician in StataCorp, for her help
with advanced aspects of xtmixed and xtmelogit and the anonymous reviewer whose
comments and suggestions improved the command significantly. Evan Kontopantelis is
on an NIHR School for Primary Care (NSPCR) Fellowship.
6 References
Hardy, R. J., and S. G. Thompson. 1998. Detecting and describing heterogeneity in
meta-analysis. Stat Med 17(8): 841–856.
Harris, R. J., M. J. Bradburn, J. J. Deeks, R. M. Harbord, D. G. Altman, and J. A. C.
Sterne. 2008. metan: fixed- and random-effects meta-analysis. Stata Journal 8(1):
3–28.
Higgins, J. P., S. G. Thompson, J. J. Deeks, and D. G. Altman. 2003. Measuring
Inconsistency in Meta-Analyses. British Medical Journal 327(7414): 557–560.
Higgins, J. P., A. Whitehead, R. M. Turner, R. Z. Omar, and S. G. Thompson. 2001.
Meta-analysis of continuous outcome data from individual patients. Stat Med 20(15):
2219–2241. http://dx.doi.org/10.1002/sim.918.
Hunter, J. E., and F. L. Schmidt. 2000. Fixed Effects vs. Random Effects Meta-Analysis
Models: Implications for Cumulative Research Knowledge. International Journal of
Selection and Assessment 8(4): 275–292. http://dx.doi.org/10.1111/1468-2389.00156.
Kontopantelis, E., and D. Reeves. 2009. MetaEasy: A Meta-Analysis Add-In for Mi-
crosoft Excel. Journal of Statistical Software 30(7): 1–25.
14 ipdforest
———. 2010. metaan: Random-effects meta-analysis. Stata Journal 10(3): 395–407.
Mathew, T., and K. Nordstrom. 2010. Comparison of one-step and two-step
meta-analysis models using individual patient data. Biom J 52(2): 271–287.
http://dx.doi.org/10.1002/bimj.200900143.
Mittlbock, M., and H. Heinzl. 2006. A Simulation Study Comparing Properties of
Heterogeneity Measures in Meta-Analyses. Statistics in Medicine 25(24): 4321–4333.
Turner, R. M., R. Z. Omar, M. Yang, H. Goldstein, and S. G. Thompson. 2000. A
multilevel model framework for meta-analysis of clinical trials with binary outcomes.
Stat Med 19(24): 3417–3432.
Whitehead, A. 2002. Meta-Analysis of Controlled Clinical Trials. Statistics in Practice,
Wiley.
About the authors
Evangelos (Evan) Kontopantelis is a research fellow in statistics at the Centre for Biostatistics,
Institute of Population Health, University of Manchester, England. His research interests
include statistical methods in health sciences with a focus on meta-analysis, longitudinal data
modeling and large clinical databases. For this piece of work he was partly supported by a
NIHR School for Primary Care Research (NSPCR) fellowship in primary health care.
David Reeves is a senior research fellow in statistics at the Centre for Biostatistics, Institute
of Population Health, University of Manchester, England. His methodological research inter-
ests include the robustness of statistical methods, the analysis of observational studies, and
applications of social network analysis methods to health systems.
... It is known that the correct specification of the 1-stage model is critical, 7,8 and guides and software to aid researchers in this are available. [9][10][11] Fewer modelling options are available for two-stage analysis, and they are mainly around the choice of the second step model, fixed effect, or one of the numerous random-effects options. 12,13 A well-developed implementation of a two-stage analysis is available in Stata, 14 with the less flexible ipdmeta command, which can only analyse a continuous outcome, available in R software. ...
... 15 One-stage analyses rely on widely used mixedeffects models and are considered more flexible, 16 but they are more challenging in both conducting and communicating the findings, especially regarding visualisation with the hallmark forest plot, although software solutions are available. 10 These challenges with the one-stage approach drive meta-analysts to the two-stage approach, which is more prevalent, 17 although other challenges like power calculations, 18 or multiple imputation, [19][20][21] are common across both approaches. ...
... Additional random effects can be added, in this context for the covariate, although convergence issues may limit the practical usefulness of such models. 10 Finally, in some cases, the focus may be on interactions, and in such a scenario, we would expand (6) to ...
A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study
Article
Full-text available
  • Jun 2018
Background Individual patient data (IPD) meta‐analysis allows for the exploration of heterogeneity and can identify subgroups that most benefit from an intervention (or exposure), much more successfully than meta‐analysis of aggregate data. One‐stage or two‐stage IPD meta‐analysis is possible, with the former using mixed‐effects regression models and the latter obtaining study‐estimates through simpler regression models before aggregating using standard meta‐analysis methodology. However, a comprehensive comparison of the two methods, in practice, is lacking. Methods We generated 1,000 datasets for each of many simulation scenarios covering different IPD sizes and different between‐study variance (heterogeneity) assumptions at various levels (intercept and exposure). Numerous simulation settings of different assumptions were also used, while we evaluated performance both on main effects and interaction effects. Performance was assessed on mean bias, mean error, coverage and power. Results Fully specified one‐stage models (random study intercept or fixed study‐specific intercept; random exposure effect; fixed study‐specific effects for covariate) were the best performers overall, especially when investigating interactions. For main effects, performance was almost identical across models unless intercept heterogeneity was present, in which case the fully specified one‐stage and the two‐stage models performed better. For interaction effects, differences across models were greater with the two‐stage model consistently outperformed by the two fully specified one‐stage models. Conclusions A fully specified one‐stage model should be preferred (accounting for potential exposure, intercept and, possibly, interaction heterogeneity), especially when investigating interactions. If non‐convergence is encountered with a random study intercept, the fixed study specific intercept one‐stage model should be used instead.
View
... A p-value < 0.05 was considered statistically significant. I 2 statistics could not be estimated in the one-stage approach because residual variability is not reported under mixed-effects regression with binary outcomes [44]. Hence, the parameters were fitted as random effects in the model to account for between-study variations [44]. ...
... I 2 statistics could not be estimated in the one-stage approach because residual variability is not reported under mixed-effects regression with binary outcomes [44]. Hence, the parameters were fitted as random effects in the model to account for between-study variations [44]. Furthermore, a two-stage IPD meta-analysis was performed to assess heterogeneity between studies using I 2 statistics and to assess the robustness of the analyses. ...
The choice of reference chart affects the strength of the association between malaria in pregnancy and small for gestational age: an individual participant data meta-analysis comparing the Intergrowth-21 with a Tanzanian birthweight chart
Article
Full-text available
  • Oct 2022
  • MALARIA J
Background The prevalence of small for gestational age (SGA) may vary depending on the chosen weight-for-gestational-age reference chart. An individual participant data meta-analysis was conducted to assess the implications of using a local reference (STOPPAM) instead of a universal reference (Intergrowth-21) on the association between malaria in pregnancy and SGA. Methods Individual participant data of 6,236 newborns were pooled from seven conveniently identified studies conducted in Tanzania and Malawi from 2003–2018 with data on malaria in pregnancy, birthweight, and ultrasound estimated gestational age. Mixed-effects regression models were used to compare the association between malaria in pregnancy and SGA when using the STOPPAM and the Intergrowth-21 references, respectively. Results The 10th percentile for birthweights-for-gestational age was lower for STOPPAM than for Intergrowth-21, leading to a prevalence of SGASTOPPAM of 14.2% and SGAIG21 of 18.0%, p < 0.001. The association between malaria in pregnancy and SGA was stronger for STOPPAM (adjusted odds ratio (aOR) 1.30 [1.09–1.56], p < 0.01) than for Intergrowth-21 (aOR 1.19 [1.00–1.40], p = 0.04), particularly among paucigravidae (SGASTOPPAM aOR 1.36 [1.09–1.71], p < 0.01 vs SGAIG21 aOR 1.21 [0.97–1.50], p = 0.08). Conclusions The prevalence of SGA may be overestimated and the impact of malaria in pregnancy underestimated when using Intergrowth-21. Comparing local reference charts to global references when assessing and interpreting the impact of malaria in pregnancy may be appropriate.
View
... Primary analyses used a 1-stage linear mixed effect model that incorporated random effects to allow for heterogeneity across trials [20,26,27], fitted using the Stata (version 16.1; StataCorp LLC) commands mixed and ipdforest to summarize the evidence by study and obtain forest plots [28]. Restricted maximum likelihood was used for model estimation, and centering of covariates by study-specific means was performed to avoid aggregation bias [29]. ...
... Visual and statistical evidence (P=.002; Egger test) of a small study effect was found in the funnel plots (Multimedia Appendix 1, pages [24][25][26][27][28][29][30]. After removing the studies with high or unclear RoB from the analysis, the symmetry of the funnel plot improved somewhat (P=.06). ...
Association Between Patient Factors and the Effectiveness of Wearable Trackers at Increasing the Number of Steps per Day Among Adults With Cardiometabolic Conditions: Meta-analysis of Individual Patient Data From Randomized Controlled Trials
Article
Full-text available
  • Apr 2022
  • J MED INTERNET RES
Background: Current evidence supports the use of wearable trackers in by people with cardiometabolic conditions. However, as the health benefits are small and conflicted confounded by heterogeneity, there remains uncertainty of as to which patient groups are most helped by wearable trackers are most helpful. Objective: This study examined the effects of wearable trackers in patients with cardiometabolic conditions to identify subgroups of patients who benefit most benefited and to understand interventional differences. Methods: We obtained individual participant data (IPD) from randomized controlled trials of wearable trackers performed RCTs before December 2020 on wearable trackers, measuring the primary outcome of ‘“steps/day’ steps per day” in participants with cardiometabolic conditions, including diabetes, overweight/ or obesity, and cardiovascular disease. We used statistical models to accounting for clustering of participants within trials and heterogeneity across trials to estimate mean differences with the 95% confidence intervals (CIs).confidence interval (CI). Results: Individual participant data IPD were obtained from nine 9 of the 25 eligible randomized controlled trials, RCTs comprisingwhich included 1,481/3,178 (47%) of total participants. Wearable trackers showed revealed that over the median duration of 12 weeks, significant increases in steps/daysteps per day over median 12 weeks by increased by 1,656 (95% CI 918, -2,395), a significant change steps/day. Greater increases in steps per day from interventions using wearable trackers Men were observed in men (coefficient –668, 95% CI –1,157 to –180), those patients in aged categories over 50 years (50-59 years=, 1,175 /3178 , 95% CI 377, -1973; 60-69 years, =981/3178, 95% CI 222, -1740; 70-90 years, =1060/3178, 95% CI 200, -1920), white White patients (coefficient= 995, 95% CI% 360, -1631), and patients with fewer comorbidities (coefficient –517, 95% CI –1188 to –11) achieved greater increases in steps/day from interventions using wearable trackers compared to women, those aged below 50, non--wWhite patients, and patients with multimorbidity. In terms of interventional differences, only face-to-face delivery of the tracker impacted the effectiveness of the interventions by increasing steps/daysteps per day. Conclusions: In patients with cardiometabolic conditions, interventions using wearable trackers to improve steps/daysteps per day mostly benefited older white White men without multimorbidity.
View
... IPD meta-analyses were performed in Stata 16.0. Along with a two-stage approach 32 specified in the published proposal, one-stage approach was also used, 33 as there remain debates regarding which approach is optimal. 34 Using the two-stage approach mean differences and 95% confidence intervals (CI) for outcomes between the VP/VLBW and control groups were derived using linear regression in each study separately. ...
... This approach also accounted for the clustering effect of participants in studies. 33,34 The above procedure was repeated to estimate effect sizes after adjusting for covariates (sex, age at assessment, and SES) which were added as fixed effects. Betweenstudy heterogeneity between the two-stage and one-stage approaches were compared using τ 2 . ...
Social Functioning in Adults Born Very Preterm: Individual Participant Meta-analysis
Article
  • Oct 2021
Context: There is a lack of research on individual perceptions of social experiences and social relationships among very preterm (VP) adults compared with term-born peers. Objective: To investigate self-perceived social functioning in adults born VP (<32 weeks' gestation) and/or with very low birth weight (VLBW) (<1500g) compared with term-born adults (≥37 weeks' gestation) using an individual participant data (IPD) meta-analysis. Data sources: Two international consortia: Research on European Children and Adults born Preterm and Adults Born Preterm International Collaboration. Study selection: Cohorts with outcomes assessed by using the Adult Self-Report Adaptive Functioning scales (friends, spouse/partner, family, job, and education) in both groups. Data extraction: IPD from 5 eligible cohorts were collected. Raw-sum scores for each scale were standardized as z scores by using mean and SD of controls for each cohort. Pooled effect size was measured by difference (Δ) in means between groups. Results: One-stage analyses (1285 participants) revealed significantly lower scores for relationships with friends in VP/VLBW adults compared with controls (Δ -0.37, 95% confidence interval [CI]: -0.61 to -0.13). Differences were similar after adjusting for sex, age, and socioeconomic status (Δ -0.39, 95% CI: -0.63 to -0.15) and after excluding participants with neurosensory impairment (Δ -0.34, 95% CI: -0.61 to -0.07). No significant differences were found in other domains. Limitations: Generalizability of research findings to VP survivors born in recent decades. Conclusions: VP/VLBW adults scored their relationship with friends lower but perceived their family and partner relationships, as well as work and educational experiences, as comparable to those of controls.
View
... The analyses used 'one-stage' linear mixed effect models which incorporated random-effects to allow for heterogeneity across trials [50,53,54], fitted using Stata (software version 16.1) [55] command mixed and the ipdforest command to summarise the evidence by study and obtain forest plots [56]. Restricted maximum likelihood was used for model estimation and centring of covariates by study-specific means was performed to avoid aggregation bias [57]. ...
Benefits and harms of Risperidone and Paliperidone for treatment of patients with schizophrenia or bipolar disorder: a meta-analysis involving individual participant data and clinical study reports
Article
Full-text available
  • Aug 2021
  • BMC MED
Background Schizophrenia and bipolar disorder are severe mental illnesses which are highly prevalent worldwide. Risperidone and Paliperidone are treatments for either illnesses, but their efficacy compared to other antipsychotics and growing reports of hormonal imbalances continue to raise concerns . As existing evidence on both antipsychotics are solely based on aggregate data, we aimed to assess the benefits and harms of Risperidone and Paliperidone in the treatment of patients with schizophrenia or bipolar disorder, using individual participant data (IPD), clinical study reports (CSRs) and publicly available sources (journal publications and trial registries). Methods We searched MEDLINE, Central, EMBASE and PsycINFO until December 2020 for randomised placebo-controlled trials of Risperidone, Paliperidone or Paliperidone palmitate in patients with schizophrenia or bipolar disorder. We obtained IPD and CSRs from the Yale University Open Data Access project. The primary outcome Positive and Negative Syndrome Scale (PANSS) score was analysed using one-stage IPD meta-analysis. Random-effect meta-analysis of harm outcomes involved methods for coping with rare events. Effect-sizes were compared across all available data sources using the ratio of means or relative risk. We registered our review on PROSPERO, CRD42019140556. Results Of the 35 studies, IPD meta-analysis involving 22 (63%) studies showed a significant clinical reduction in the PANSS in patients receiving Risperidone (mean difference − 5.83, 95% CI − 10.79 to − 0.87, I ² = 8.5%, n = 4 studies, 1131 participants), Paliperidone (− 6.01, 95% CI − 8.7 to − 3.32, I ² = 4.3%, n = 13, 3821) and Paliperidone palmitate (− 7.89, 95% CI − 12.1 to − 3.69, I ² = 2.9%, n = 5, 2209). CSRs reported nearly two times more adverse events (4434 vs. 2296 publication, relative difference (RD) = 1.93, 95% CI 1.86 to 2.00) and almost 8 times more serious adverse events (650 vs. 82; RD = 7.93, 95% CI 6.32 to 9.95) than the journal publications. Meta-analyses of individual harms from CSRs revealed a significant increased risk among several outcomes including extrapyramidal disorder, tardive dyskinesia and increased weight. But the ratio of relative risk between the different data sources was not significant. Three treatment-related gynecomastia events occurred, and these were considered mild to moderate in severity. Conclusion IPD meta-analysis conclude that Risperidone and Paliperidone antipsychotics had a small beneficial effect on reducing PANSS score over 9 weeks, which is more conservative than estimates from reviews based on journal publications. CSRs also contained significantly more data on harms that were unavailable in journal publications or trial registries. Sharing of IPD and CSRs are necessary when performing meta-analysis on the efficacy and safety of antipsychotics.
View
... [38] Stata's (v16.1) mixed and ipdforest [39,40] commands were used to pool evidence from studies and to obtain forest plots, respectively. Restricted maximum likelihood was used for model estimation. ...
Effectiveness of collaborative care in reducing suicidal ideation: An individual participant data meta-analysis
Article
Full-text available
  • Jul 2021
  • GEN HOSP PSYCHIAT
Objective To assess whether CC is more effective at reducing suicidal ideation in people with depression compared with usual care, and whether study and patient factors moderate treatment effects. Method We searched Medline, Embase, PubMed, PsycINFO, CINAHL, CENTRAL from inception to March 2020 for Randomised Controlled Trials (RCTs) that compared the effectiveness of CC with usual care in depressed adults, and reported changes in suicidal ideation at 4 to 6 months post-randomisation. Mixed-effects models accounted for clustering of participants within trials and heterogeneity across trials. This study is registered with PROSPERO, CRD42020201747. Results We extracted data from 28 RCTs (11,165 patients) of 83 eligible studies. We observed a small significant clinical improvement of CC on suicidal ideation, compared with usual care (SMD, −0.11 [95%CI, −0.15 to −0.08]; I², 0·47% [95%CI 0.04% to 4.90%]). CC interventions with a recognised psychological treatment were associated with small reductions in suicidal ideation (SMD, −0.15 [95%CI -0.19 to −0.11]). CC was more effective for reducing suicidal ideation among patients aged over 65 years (SMD, − 0.18 [95%CI -0.25 to −0.11]). Conclusion Primary care based CC with an embedded psychological intervention is the most effective CC framework for reducing suicidal ideation and older patients may benefit the most.
View
... Allocation concealment was selected as an indicator of risk of bias, as it is sensitive to changes in the treatment effect, especially for self-reported outcomes. ( (36,37) commands were used to pool evidence from studies and to obtain forest plots, respectively. Restricted maximum likelihood was used for model estimation. ...
Effectiveness of Collaborative Care in Reducing Suicidal Ideation: An Individual Participant Data Meta-Analysis
Article
  • Apr 2021
To assess whether CC is more effective at reducing suicidal ideation in people with depression compared with usual care, and whether study and patient factors moderate treatment effects. Method: We searched Medline, Embase, PubMed, PsycINFO, CINAHL, CENTRAL from inception to March 2020 for Randomised Controlled Trials (RCTs) that compared the effectiveness of CC with usual care in depressed adults, and reported changes in suicidal ideation at 4 to 6 months post-randomisation. Mixed-effects models accounted for clustering of participants within trials and heterogeneity across trials. This study is registered with PROSPERO, CRD42020201747. Results: We extracted data from 28 RCTs (11,165 patients) of 83 eligible studies. We observed a small significant clinical improvement of CC on suicidal ideation, compared with usual care (SMD, -0.11 [95%CI, -0.15 to -0.08]; I2, 0·47% [95%CI 0.04% to 4.90%]). CC interventions with a recognised psychological treatment were associated with small reductions in suicidal ideation (SMD, -0.15 [95%CI -0.19 to -0.11]). CC was more effective for reducing suicidal ideation among patients aged over 65 years (SMD, - 0.18 [95%CI -0.25 to -0.11]). Conclusion: Primary care based CC with an embedded psychological intervention is the most effective CC framework for reducing suicidal ideation and older patients may benefit the most.
View
In dividual participant data meta-analysis of trials comparing f r o zen versus f r esh e m bryo transfer strategy (INFORM): a protocol
Article
Full-text available
  • Jul 2022
Introduction Existing randomised controlled trials (RCTs) comparing a freeze-all embryo transfer strategy and a fresh embryo transfer strategy have shown conflicting results. A freeze-all or a fresh transfer policy may be preferable for some couples undergoing in-vitro fertilisation (IVF), but it is unclear which couples would benefit most from each policy, how and under which protocols. Therefore, we plan a systematic review and individual participant data meta-analysis of RCTs comparing a freeze-all and a fresh transfer policy. Methods and analysis We will search electronic databases (Medline, Embase, PsycINFO and CENTRAL) and trial registries (ClinicalTrials.gov and the International Clinical Trials Registry Platform) from their inception to present to identify eligible RCTs. We will also check reference lists of relevant papers. The search was performed on 23 September 2020 and will be updated. We will include RCTs comparing a freeze-all embryo transfer strategy and a fresh embryo transfer strategy in couples undergoing IVF. The primary outcome will be live birth resulting from the first embryo transfer. All outcomes listed in the core outcome set for infertility research will be reported. We will invite the lead investigators of eligible trials to join the In dividual participant data meta-analysis of trials comparing f r o zen versus f r esh e m bryo transfer strategy (INFORM) collaboration and share the deidentified individual participant data (IPD) of their trials. We will harmonise the IPD and perform a two-stage meta-analysis and examine treatment–covariate interactions for important baseline characteristics. Ethics and dissemination The study ethics have been granted by the Monash University Human Research Ethics Committee (Project ID: 30391). The findings will be disseminated via presentations at international conferences and publication in peer-reviewed journals. PROSPERO registration number CRD42021296566.
View
View
Psychological interventions to improve self-management of type 1 and type 2 diabetes: a systematic review
Article
  • Jun 2020
Background For people with diabetes mellitus to achieve optimal glycaemic control, motivation to perform self-management is important. The research team wanted to determine whether or not psychological interventions are clinically effective and cost-effective in increasing self-management and improving glycaemic control. Objectives The first objective was to determine the clinical effectiveness of psychological interventions for people with type 1 diabetes mellitus and people with type 2 diabetes mellitus so that they have improved (1) glycated haemoglobin levels, (2) diabetes self-management and (3) quality of life, and fewer depressive symptoms. The second objective was to determine the cost-effectiveness of psychological interventions. Data sources The following databases were accessed (searches took place between 2003 and 2016): MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Library, PsycINFO, EMBASE, Cochrane Controlled Trials Register, Web of Science, and Dissertation Abstracts International. Diabetes conference abstracts, reference lists of included studies and Clinicaltrials.gov trial registry were also searched. Review methods Systematic review, aggregate meta-analysis, network meta-analysis, individual patient data meta-analysis and cost-effectiveness modelling were all used. Risk of bias of randomised and non-randomised controlled trials was assessed using the Cochrane Handbook (Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343 :d5928). Design Systematic review, meta-analysis, cost-effectiveness analysis and patient and public consultation were all used. Setting Settings in primary or secondary care were included. Participants Adolescents and children with type 1 diabetes mellitus and adults with types 1 and 2 diabetes mellitus were included. Interventions The interventions used were psychological treatments, including and not restricted to cognitive–behavioural therapy, counselling, family therapy and psychotherapy. Main outcome measures Glycated haemoglobin levels, self-management behaviours, body mass index, blood pressure levels, depressive symptoms and quality of life were all used as outcome measures. Results A total of 96 studies were included in the systematic review ( n = 18,659 participants). In random-effects meta-analysis, data on glycated haemoglobin levels were available for seven studies conducted in adults with type 1 diabetes mellitus ( n = 851 participants) that demonstrated a pooled mean difference of –0.13 (95% confidence interval –0.33 to 0.07), a non-significant decrease in favour of psychological treatment; 18 studies conducted in adolescents/children with type 1 diabetes mellitus ( n = 2583 participants) that demonstrated a pooled mean difference of 0.00 (95% confidence interval –0.18 to 0.18), indicating no change; and 49 studies conducted in adults with type 2 diabetes mellitus ( n = 12,009 participants) that demonstrated a pooled mean difference of –0.21 (95% confidence interval –0.31 to –0.10), equivalent to reduction in glycated haemoglobin levels of –0.33% or ≈3.5 mmol/mol. For type 2 diabetes mellitus, there was evidence that psychological interventions improved dietary behaviour and quality of life but not blood pressure, body mass index or depressive symptoms. The results of the network meta-analysis, which considers direct and indirect effects of multiple treatment comparisons, suggest that, for adults with type 1 diabetes mellitus (7 studies; 968 participants), attention control and cognitive–behavioural therapy are clinically effective and cognitive–behavioural therapy is cost-effective. For adults with type 2 diabetes mellitus (49 studies; 12,409 participants), cognitive–behavioural therapy and counselling are effective and cognitive–behavioural therapy is potentially cost-effective. The results of the individual patient data meta-analysis for adolescents/children with type 1 diabetes mellitus (9 studies; 1392 participants) suggest that there were main effects for age and diabetes duration. For adults with type 2 diabetes mellitus (19 studies; 3639 participants), baseline glycated haemoglobin levels moderated treatment outcome. Limitations Aggregate meta-analysis was limited to glycaemic control for type 1 diabetes mellitus. It was not possible to model cost-effectiveness for adolescents/children with type 1 diabetes mellitus and modelling for type 2 diabetes mellitus involved substantial uncertainty. The individual patient data meta-analysis included only 40–50% of studies. Conclusions This review suggests that psychological treatments offer minimal clinical benefit in improving glycated haemoglobin levels for adults with type 2 diabetes mellitus. However, there was no evidence of benefit compared with control interventions in improving glycated haemoglobin levels for people with type 1 diabetes mellitus. Future work Future work should consider the competency of the interventionists delivering a therapy and psychological approaches that are matched to a person and their life course. Study registration This study is registered as PROSPERO CRD42016033619. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment ; Vol. 24, No. 28. See the NIHR Journals Library website for further project information.
View
Fixed Effects vs. Random Effects Meta-Analysis Models: Implications for Cumulative Research Knowledge
Article
Full-text available
  • Dec 2000
Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.
View
Fixed Effects vs. Random Effects Meta‐Analysis Models: Implications for Cumulative Research Knowledge
Article
Full-text available
  • Dec 2002
  • INT J SELECT ASSESS
Research conclusions in the social sciences are increasingly based on meta-analysis, making questions of the accuracy of meta-analysis critical to the integrity of the base of cumulative knowledge. Both fixed effects (FE) and random effects (RE) meta-analysis models have been used widely in published meta-analyses. This article shows that FE models typically manifest a substantial Type I bias in significance tests for mean effect sizes and for moderator variables (interactions), while RE models do not. Likewise, FE models, but not RE models, yield confidence intervals for mean effect sizes that are narrower than their nominal width, thereby overstating the degree of precision in meta-analysis findings. This article demonstrates analytically that these biases in FE procedures are large enough to create serious distortions in conclusions about cumulative knowledge in the research literature. We therefore recommend that RE methods routinely be employed in meta-analysis in preference to FE methods.
View
Metaan: Random-effects Meta-analysis
Article
Full-text available
  • Aug 2010
This article describes the new meta-analysis command metaan, which can be used to perform fixed- or random-effects meta-analysis. Besides the stan- dard DerSimonian and Laird approach, metaan offers a wide choice of available models: maximum likelihood, profile likelihood, restricted maximum likelihood, and a permutation model. The command reports a variety of heterogeneity mea- sures, including Cochran’s Q, I2, HM2 , and the between-studies variance estimate τb2. A forest plot and a graph of the maximum likelihood function can also be generated.
View
MetaEasy: A Meta-Analysis Add-In for Microsoft Excel
Article
Full-text available
  • Apr 2009
  • J STAT SOFTW
Meta-analysis is a statistical methodology that combines or integrates the results of several independent clinical trials considered by the analyst to be 'combinable' (Huque'88). However, completeness and user-friendliness are uncommon both in specialised meta-analysis software packages and in mainstream statistical packages that have to rely on user-written commands. We implemented the meta-analysis methodology in an Microsoft (Excel) add-in which is freely available and incorporates more meta-analysis models (including the iterative maximum likelihood and profile likelihood) than are usually available, while paying particular attention to the user-friendliness of the package.
View
A multilevel model framework for meta-analysis of clinical trials with binary outcomes
Article
  • Dec 2000
In this paper we explore the potential of multilevel models for meta-analysis of trials with binary outcomes for both summary data, such as log-odds ratios, and individual patient data. Conventional fixed effect and random effects models are put into a multilevel model framework, which provides maximum likelihood or restricted maximum likelihood estimation. To exemplify the methods, we use the results from 22 trials to prevent respiratory tract infections; we also make comparisons with a second example data set comprising fewer trials. Within summary data methods, confidence intervals for the overall treatment effect and for the between-trial variance may be derived from likelihood based methods or a parametric bootstrap as well as from Wald methods; the bootstrap intervals are preferred because they relax the assumptions required by the other two methods. When modelling individual patient data, a bias corrected bootstrap may be used to provide unbiased estimation and correctly located confidence intervals; this method is particularly valuable for the between-trial variance. The trial effects may be modelled as either fixed or random within individual data models, and we discuss the corresponding assumptions and implications. If random trial effects are used, the covariance between these and the random treatment effects should be included; the resulting model is equivalent to a bivariate approach to meta-analysis. Having implemented these techniques, the flexibility of multilevel modelling may be exploited in facilitating extensions to standard meta-analysis methods. Copyright © 2000 John Wiley & Sons, Ltd.
View
Metan: Fixed- and Random-Effects Meta-Analysis
Article
  • Apr 2008
This article describes updates of the meta-analysis command metan and options that have been added since the command's original publication (Bradburn, Deeks, and Altman, metan – an alternative meta-analysis command, Stata Technical Bulletin Reprints, vol. 8, pp. 86–100). These include version 9 graphics with flexible display options, the ability to meta-analyze precalculated effect estimates, and the ability to analyze subgroups by using the by() option. Changes to the output, saved variables, and saved results are also described.
View
A multilevel model framework for meta-analysis of clinical trials with binary outcomes†
Article
  • Dec 2000
  • STAT MED
In this paper we explore the potential of multilevel models for meta-analysis of trials with binary outcomes for both summary data, such as log-odds ratios, and individual patient data. Conventional fixed effect and random effects models are put into a multilevel model framework, which provides maximum likelihood or restricted maximum likelihood estimation. To exemplify the methods, we use the results from 22 trials to prevent respiratory tract infections; we also make comparisons with a second example data set comprising fewer trials. Within summary data methods, confidence intervals for the overall treatment effect and for the between-trial variance may be derived from likelihood based methods or a parametric bootstrap as well as from Wald methods; the bootstrap intervals are preferred because they relax the assumptions required by the other two methods. When modelling individual patient data, a bias corrected bootstrap may be used to provide unbiased estimation and correctly located confidence intervals; this method is particularly valuable for the between-trial variance. The trial effects may be modelled as either fixed or random within individual data models, and we discuss the corresponding assumptions and implications. If random trial effects are used, the covariance between these and the random treatment effects should be included; the resulting model is equivalent to a bivariate approach to meta-analysis. Having implemented these techniques, the flexibility of multilevel modelling may be exploited in facilitating extensions to standard meta-analysis methods. Copyright © 2000 John Wiley & Sons, Ltd.
View
View
Detecting and describing heterogeneity in mat-analysis
Article
  • Apr 1998
  • STAT MED
The investigation of heterogeneity is a crucial part of any meta-analysis. While it has been stated that the test for heterogeneity has low power, this has not been well quantified. Moreover the assumptions of normality implicit in the standard methods of meta-analysis are often not scrutinized in practice. Here we simulate how the power of the test for heterogeneity depends on the number of studies included, the total information (that is total weight or inverse variance) available and the distribution of weights among the different studies. We show that the power increases with the total information available rather than simply the number of studies, and that it is substantially lowered if, as is quite common in practice, one study comprises a large proportion of the total information. We also describe normal plots that are useful in assessing whether the data conform to a fixed effect or random effects model, together with appropriate tests, and give an application to the analysis of a multi-centre trial of blood pressure reduction. We conclude that the test of heterogeneity should not be the sole determinant of model choice in meta-analysis, and inspection of relevant normal plots, as well as clinical insight, may be more relevant to both the investigation and modelling of heterogeneity. © 1998 John Wiley & Sons, Ltd.
View
Comparison of One-Step and Two-Step Meta-Analysis Models Using Individual Patient Data
Article
  • Mar 2010
  • BIOMETRICAL J
The problem of combining information from separate trials is a key consideration when performing a meta-analysis or planning a multicentre trial. Although there is a considerable journal literature on meta-analysis based on individual patient data (IPD), i.e. a one-step IPD meta-analysis, versus analysis based on summary data, i.e. a two-step IPD meta-analysis, recent articles in the medical literature indicate that there is still confusion and uncertainty as to the validity of an analysis based on aggregate data. In this study, we address one of the central statistical issues by considering the estimation of a linear function of the mean, based on linear models for summary data and for IPD. The summary data from a trial is assumed to comprise the best linear unbiased estimator, or maximum likelihood estimator of the parameter, along with its covariance matrix. The setup, which allows for the presence of random effects and covariates in the model, is quite general and includes many of the commonly employed models, for example, linear models with fixed treatment effects and fixed or random trial effects. For this general model, we derive a condition under which the one-step and two-step IPD meta-analysis estimators coincide, extending earlier work considerably. The implications of this result for the specific models mentioned above are illustrated in detail, both theoretically and in terms of two real data sets, and the roles of balance and heterogeneity are highlighted. Our analysis also shows that when covariates are present, which is typically the case, the two estimators coincide only under extra simplifying assumptions, which are somewhat unrealistic in practice.
View