Access to this full-text is provided by Springer Nature.
Content available from BMC Health Services Research
This content is subject to copyright. Terms and conditions apply.
R E S E A R C H A R T I C L E Open Access
E-IMR: e-health added to face-to-face
delivery of Illness Management & Recovery
programme for people with severe mental
illness, an exploratory clustered
randomized controlled trial
Titus A. A. Beentjes
1,2,3*
, Peter J. J. Goossens
3,4
, Hester Vermeulen
1
, Steven Teerenstra
5
,
Maria W. G. Nijhuis-van der Sanden
1
and Betsie G. I. van Gaal
1,6
Abstract
Background: E-mental health holds promise for people with severe mental illness, but has a limited evidence base.
This study explored the effect of e-health added to face-to-face delivery of the Illness Management and Recovery
Programme (e-IMR).
Method: In this multi-centre exploratory cluster randomized controlled trial, seven clusters (n= 60; 41 in intervention
group and 19 in control group) were randomly assigned to e-IMR + IMR or IMR only. Outcomes of illness management,
self-management, recovery, symptoms, quality of life, and general health were measured at baseline (T
0
), halfway (T
1
),
and at twelve months (T
2
). The data were analysed using mixed model for repeated measurements in four models: in 1)
we included fixed main effects for time trend and group, in 2) we controlled for confounding effects, in 3) we controlled
for interaction effects, and in 4) we performed sub-group analyses within the intervention group.
Results: Notwithstanding low activity on e-IMR, significant effects were present in model 1 analyses for self-management
(p= .01) and recovery (p= .02) at T
1
, and for general health perception (p = .02) at T
2
, all in favour of the intervention
group. In model 2, the confounding covariate gender explained the effects at T
1
and T
2
, except for self-management. In
model 3, the interacting covariate non-completer explained the effects for self-management (p=.03)atT
1
.Inmodel4,
the sub-group analyses of e-IMR-users versus non-users showed no differences in effect.
Conclusion: Because of confounding and interaction modifications, effectiveness of e-IMR cannot be concluded. Low
use of e-health precludes definite conclusions on its potential efficacy. Low use of e-IMR calls for a thorough process
evaluation of the intervention.
Trial registration: The Dutch Trial Register (NTR4772)
Keywords: Severe mental illness, E-mental health, Illness management and recovery
* Correspondence: titus.beentjes@radboudumc.nl
1
Titus Beentjes, IQ Healthcare, Radboud University Medical Center, Radboud
Institute for Health Sciences, PO Box 9101, 6500, HB, Nijmegen, the
Netherlands
2
Center for Nursing Research, Saxion University of Applied Science,
Deventer/Enschede, the Netherlands
Full list of author information is available at the end of the article
© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Beentjes et al. BMC Health Services Research (2018) 18:962
https://doi.org/10.1186/s12913-018-3767-5
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Background
In spite of the growing interest in e-mental health, evi-
dence for the effectiveness of e-health for people with a
severe or serious mental illness (SMI) is limited [1,2].
Naslund et al. [2] found that e-health interventions for
people with SMI have high feasibility and acceptability.
Van der Krieke et al. [3] found that people with psych-
otic disorders were able and willing to engage in
e-health, and found larger effects for medication man-
agement [3]. However, one should be cautious about
drawing conclusions regarding the effectiveness [2,3].
E-health is used in a wide range of interventions for
people with SMI on (1) illness self-management and re-
lapse prevention, (2) promoting adherence to medica-
tions and/or treatment, (3) psycho-education, supporting
recovery, and promoting health and wellness, and (4)
symptom monitoring [2]. E-health interventions make
use of personal digital assistance, medication tracking
devices, home monitoring systems, smartphone applica-
tions, SMS, and web-based interventions [2].
Also in general mental health, e-health approaches show
great potential and offer the possibility of expanding ac-
cess to care while being economically and socially efficient
[4]. But e-health interventions in mental health have high
attrition rates [5]. The addition of face-to-face contact to
e-health is supposed to increase the therapeutic relation
and prevent attrition [6]..In the case of people with SMI,
e-health components could be added to an
evidence-based face-to-face recovery-oriented interven-
tion. Such an intervention is the Illness Management &
Recovery programme (IMR) [7]. The IMR is a standard-
ized curriculum-based approach designed to provide
people with SMI the information and skills necessary for
managing their illnesses effectively and working towards
achieving personal recovery goals. In addition to the
standard face-to-face delivery of the IMR, an e-health
intervention (e-IMR) was designed which follows the
IMR-curriculum, and was further developed with the
end-users of the intervention [8]. The aim of this study
was to explore the effect of the e-IMR for people with
SMI who were referred to the Illness Management & Re-
covery programme.
Methods
The e-IMR was tested in an exploratory multi-centre
cluster randomized controlled trial. According to the
Medical Research Council guidance [9], an exploratory
trial evaluated an intervention before testing it in a con-
firmative trial. In this study, a cluster was a subdivision
of a mental health institute. The cluster randomization
prevented contamination between the intervention and
control group participants. Data were collected at base-
line, halfway and endpoint. The inclusion period was
between January and October 2015. Data collecting
lasted until October 2016.
Eligible clusters delivered the IMR-programme as a
whole package with an experienced trainer-couple
meaning that at least one trainer completed the
IMR-total-training organized by the Dutch IMR-network
and executed at least the first five modules of the
IMR-programme before starting the IMR-programme in
the trial.
Trial monitoring
An employee of the ‘Radboudumc Technology Center –
Clinical Studies’monitored the process of trial adminis-
tration. The administration of Trial Master Files, both
paper as well as computerized files, was independently
checked for completeness and accuracy.
Randomization
A statistician generated a randomization schedule using
Statistical Analysis System®, version 9.4. The allocation
to the intervention or control group was communicated
after the participating institutional board provided their
consent to participation. Because of the nature of the
intervention, blinding was not possible.
Sample size
Because of the exploratory character of this study, a
power calculation was considered unnecessary.
Participants
Eligible participants met the following criteria: above 18
years of age; capable of giving informed consent; and
meeting the Dutch SMI criteria according to Delespaul
[10] (being diagnosed with a psychiatric disorder that
causes, and is due to, serious impairments in social and/
or occupational functioning which lasts longer than at
least a couple of years and necessitates coordinated
multidisciplinary care. Persons who were overwhelmed
by disability, including dependence, denial, confusion,
anger or despair, were excluded from participating.
Care as usual
All participants, in both the intervention and control
group, received care consisting of extensive inpatient
and/or outpatient psychiatric treatment including case
management. They also received the IMR-programme,
which was provided in weekly, 2-h, face-to-face group
sessions according to the Dutch version of the IMR 3.0
programme [11] using the hard-copy version of 11
modules.
Intervention
On top of this care as usual, participants in the intervention
group had the opportunity to use the e-IMR intervention
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 2 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
[8]. The e-IMR intervention started with a ‘welcome page’
explaining the use of e-IMR and leading participants to the
11 modules. The e-IMR intervention included the same
fill-in forms as in the hard-copy version of the
IMR-programme. E-IMR added illustrative videos showing
peer testimonials to encourage participants to talk more
freely about themselves and to take steps in their recovery
process. E-IMR also added problem-solving forms at the
end of each module, registration of successful coping strat-
egies, and a symptom-monitoring page.
The e-IMR was introduced to the trainers and partici-
pants of the intervention group by the first researcher in
the second group session. Individuals who did not pro-
vide informed consent were allowed to join the e-IMR
without participating in the research. The trainer-cou-
ples were supported in learning how to support partici-
pants in the use of e-IMR; how to install e-IMR on a
computer in the session room and how to use e-IMR
during the sessions.
In e-IMR, the registration forms of successful coping
strategies and the symptom-monitoring page were intro-
duced after the second module ‘practical facts about men-
tal illnesses’. Weekly emails with a link to the e-IMR
platform led the participants to the symptom-monitoring
page. After closing each module, one of the trainers gave
feedback to the participants via the platform and guided
the participants to the next module.
Data collection
Data were collected in face-to-face interviews by the re-
searcher or a researcher assistant at three time points: at
baseline, a week before starting the IMR-programme
(T
0
); halfway, after completing the 5th module (T
1
); and
endpoint, at least a week after finishing the IMR-
programme (T
2
). The data were recorded on paper and
later transferred into a LimeSurvey® [12] database. The
original recorded data as well as the transferred were
double-checked for accuracy and completeness.
Outcome measures
At baseline, independent demographic and clinical char-
acteristics were recorded. At all three time points, six
dependent outcome domains were gathered.
At T
0
the following participant characteristics were
collected: age, gender, physical comorbidities, treatment
history, cultural background, social economic status,
education level, computer/Internet availability and use.
At T
0
, the participant’s case manager provided their
diagnostic classification according to the Diagnostic and
Statistical Manual of Mental Disorders, 4th edition.
The participant’s ability to manage their illness was
measured with the consumer version of the Illness
Management & Recovery Scales (IMRS), consisting of
15 items [13]. The response anchors, on a five-point
Likert scale (1–5) vary depending on the item. The
IMRS total-up score ranged between 15 and 75. The
IMRS’Cronbach’s alpha is .55–.83 [14–17].
The participants’self-management ability, which refers
to the individual’s knowledge, skill and confidence for
managing his/her own health and healthcare, was mea-
sured with the Patients Activation Measure (PAM-13)
[18], consisting of 13 items. The response anchors on a
five-point scale, vary from not applicable (0), ‘strongly
disagree’(1) to ‘strongly agree’(4). The term ‘doctor’in
the items five and six was explained as their mental
health clinician, which includes a nurse and/or case
manager. Raw scores were transformed into standard-
ized activation scores ranging between 0 and 100. The
PAM-13’s Cronbach’s alpha is .84–.88 [19–22].
The Mental Health Recovery Measure (MHRM)
assessed the participants’progress in their recovery
process. The MHRM consists of 30 items with response
anchors, on a five-point scale, varying from ‘strongly dis-
agree’(0) to ‘strongly agree’(4), and ‘neutral’(2) in be-
tween [23]. The MHRM total-up scores ranged between
0 and 120. The MHRM’s Cronbach’s alpha is .93 [24].
The participants estimated the level of burden of
symptoms they experienced using the Brief Symptom
Inventory(BSI), consisting of 53 items [25]. The re-
sponse anchors, on a five-point scale, vary from ‘not at
all’(0) to ‘extremely’(4). The mean BSI scores ranged
between 0 and 4. A negative time trend for the BSI
means a reduced level of burden. The BSI’s Cronbach’s
alpha is .96 [26].
The participants’subjective satisfaction with life was
measured with the Manchester Short Assessment of
quality of life (MANSA), consisting of 12 items [27].
The response anchors on a seven-point scale vary from
‘couldn’t be worse’(1) to ‘couldn’t be better’(7). The
mean MANSA score ranged between 1 and 7. The
MANSA’s Cronbach’s alpha is .81 [28].
The participants’general health status was measured
with the Rand 36-item Health Survey (Rand-36), con-
sisting of eight subscales: physical functioning (Rand-
PF), social functioning (Rand-SF), role limitations due to
a physical (Rand-RLPP) and an emotional problem
(Rand-RLEP), mental health (Rand-MH), vitality
(Rand-V), pain (Rand-P), and general health perception
(Rand-GHP) [29]. The response anchors vary between
yes/no to Likert scales with three, five, and six options.
Raw scores of all the concepts were transformed into
scores ranging between 0 and 100. The Cronbach’s alpha
of Rand-36’s eight concepts are .71 and .92 [30].
The extent of participants’activity on the e-IMR plat-
form was determined by counting the number of com-
pleted modules and number of log-ins. An e-IMR user is
identified by having completed at least module one or
having logged in at least five times. Users were regarded
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 3 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
as having had the opportunity to benefit from the
e-IMR.
As in other studies on IMR [31], participants who
attended the face-to-face IMR programme sessions less
than 50% were considered to be non-completers. In our
study, this resembles stopping the IMR programme be-
fore T
1
.
Statistical methods
The Statistical Package for the Social Sciences®.23 [32]
was used to carry out the analyses. Mixed model multi-
level regression analyses were used to examine the main
effects on the outcome measures, taking into account
clustering of participants and repeated measures. This
method automatically uses the ‘missing at random’as-
sumption to handle missing data. Random effects on
cluster, trainer-couple, and individual participants nested
within the cluster were included in the model. Model 1
included fixed main effects for time trend and group.
The analyses were executed according to the
intention-to-treat principle to prevent bias caused by the
loss of participants [33] and to reflect the normal prac-
tice [34] of high attrition rates in treatments of people
with SMI [7] and e-health [5].
Post hoc analyses of effect differences were performed
to control for covariates. We considered the covariate
gender to be a potential confounder because of its
known differences in exposure and reactions to stress
and health [35]. The covariate was included in model 2,
controlling for confounding time trend effects.
In model 3, covariates were included that were ex-
pected to interact with the effect differences.
Fig. 1 Participants flow diagram through the study
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 4 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Non-completion of the face-to-face IMR-programme
sessions was expected to interact with the effects be-
cause being a non-completer is correlated to lower func-
tioning; for instance, lower social functioning [36] and
higher emergency room visits and hospitalization [37].
In addition, we searched for correlations in T
0
scores be-
tween the groups of completers and non-completers.
Because of the known low adherence-rate to
Web-based interventions [5], additional subgroup ana-
lyses were performed within the intervention group to
investigate whether actual use compared to non-use of
the e-IMR leads to outcome differences. Thus in model
4, two groups of e-IMR users and non-users were in-
cluded according to the aforementioned adherence
measurement.
Results
Participant flow
Nine institutions with potentially 15 clusters were
screened for eligibility. Two clusters were not eligible
because they did not deliver the IMR-programme as a
whole. Two clusters did not start an IMR-programme
Table 1 Demographic and clinical characteristics at baseline per group
Variables Intervention group Control group
n (% within group) n (% within group)
Mean (SD) Mean (SD)
Participants 41 19
Age 46.9 (11.6) 40.7 (10.6)
Gender
**
Female 30 (73.2) 6 (31.6)
Male 11 (26.8) 13 (68.4)
Diagnoses
Psychotic disorders 14 (34.1) 6 (31.6)
Mood/anxiety disorders 15 (36.6) 10 (52.6)
Other disorders 12 (29.3) 3 (15.8)
Global Assessment of Functioning 50.86 (8.2) 49.8 (10)
Having a somatic comorbidity 23 (56.1) 7 (36.8)
Having a psychiatric comorbidity 27 (65.9) 11 (57.9)
Treatment history
Years ago since first treatment 17.15 (12.3) 16.17 (9.9)
Number of admissions 4.15 (3.9) 3.94 (3.3)
Never admitted 7 (17) 2 (10.5)
Cultural Background
Dutch 37 (90.2) 19 (100)
Turkish, Maroc, Surinam, or English 4 (9.8) 0 (0)
In/outpatients
*
Independent living 30 (73.2) 8 (42.1)
Supported housing 11 (26.8) 11 (57.9)
Netto income
≤Minimal income 31 (75.6) 16 (84.2)
> Minimal income 10 (24.4) 3 (15.8)
Highest graduated education
≤Middle school 26 (63.4) 12 (63.2)
≥High school 15 (36.6) 7 (36.8)
Computer availability / usage
I don’t have a computer/laptop 8 (19.5) 3 (15.8)
I never use a computer/laptop 6 (14.6) 2 (10.5)
Abbreviations:nnumber; SD Standard Deviation;
*
&
**
: significant between group differences
*
p< .05;
**
p< .01 (2-tailed)
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 5 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
group. Four clusters declined because of organizational
problems. Seven clusters were included: four were allo-
cated to the intervention group and three to the control
group. In three intervention clusters, a second
trainer-couple started a second IMR group. So in total,
ten IMR-programme groups (seven in the intervention
and three in the control group) trained 60 participants:
41 in the intervention and 19 in the control group (see
Fig. 1).
Table 1shows baseline characteristics and distribution
over the two groups. The characteristics ‘gender’and ‘in-
patients/outpatients’were unequally distributed over the
groups, p= .002 and p= .02 respectively.
All ten IMR-programme groups completed the trial. In
the intervention group, 12 out of 41 participants were
lost in the follow-up measures in the study. We lost five
at T
1
and another seven at T
2
. In the control group, four
out of 19 participants were lost in the follow-up at T
1
and T
2
. We have missing data at T
1
for one participant
in the control group. Participants either refused to be
interviewed because of being too burdened by the inter-
views, or they did not respond to attempts to get in
touch with them. Out of the 60 participants, 51 (36 and
15) participants were interviewed at T
1
, and 45 at T
2
(29
and 16) (See Fig. 1).
Out of the total of 60 participants, eighteen (30%)
were identified as a non-completer: participants who
attended the face-to-face IMR programme sessions less
than 50%. Eight participants (20%) in the intervention
group and ten participants (58%) in the control group
were non-completers, which differed significantly (p
= .01). Of these non-completers, 14 participants entered
the intention-to-treat analyses, eight in the intervention
group and six in the control group at T
1
, and seven in
both groups at T
2
.
Out of the 41 participants in the intervention group,
23 (56.1%) logged in on the e-IMR platform, twelve of
whom completed the first online module and eight of
whom visited the symptom-monitoring page (See Fig. 2).
In total, 14 (34.1%) participants were identified as e-IMR
users.
Outcomes and estimation
The mean scores and standard deviations of the out-
comes in both groups are presented in the Additional
file 1. Since the random effect of cluster was zero in
nearly all the analyses, this factor was excluded from the
analyses models. The relevant results of the mixed
model analyses are shown in Table 2. In model 1, the
participants in the intervention group scored signifi-
cantly higher compared to the control group for the
measures PAM-13 (p= .01), MHRM (p= .02), and
Rand-RLEP (p= .03) at T
1
, which faded at T
2
.AtT
2
, the
effect on the Rand-GHP was significant (p = .02) in
favour of the intervention group.
Post hoc analyses
In model 2, the analyses accounting for the covariate
gender showed that the significant effects above could
be explained by confounding except for the remaining
effect for PAM-13 (p = .01) at T
1
.AtT
0
, male partici-
pants scored significantly higher on nearly all the mea-
sures except for the PAM-13. The same exception
occurred in the time trends, but contrarily in favour of
female participants.
In model 3, the analyses showed that the interaction of
the covariate non-completer was significant for the mea-
sures: PAM-13, (p = .03) and Rand-V (p = .03) at T
1
,
which faded at T
2
. As an illustration of the interaction,
the graphic in Fig. 3shows the scores for the PAM-13,
Fig. 2 Number of participants active on the e-IMR platform within the intervention group
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 6 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Table 2 Mixed Model analyses, effect differences for outcome domains
Model 1 Model 2 Model 3 Model 4
Main group effects Confounder analyses of covariate gender Interaction analyses of covariate non-completer Sub-group analyses
within intervention
group
Outcome domains Parameter T
1
*Group T
2
*Group T
0
*Male T
1
*Male T
2
*Male T
1
*Group T
2
*Group T
0
*
completer
T
1
*Group T
1
*Group*
completer
T
2
*Group T
2
*Group*
completer
T
1
*e-IMR-
users
T
2
*e-IMR-
users
Illness management:
IMRS
effect 2.55 3.06 4.89 −2.31 −3.26 2.26 2.42 1.57 1.12 .62 2.38 .74 1.89 1.71
p .13 .07 .00
**
.19 .08 .19 .17 .38 .70 .86 .41 .84 .32 .44
Self-management: PAM-
13
effect 7.95 3.90 4.40 .75 1.76 8.71 5.04 1.52 17.72 −15.13 6.80 −4.81 −.73 3.03
p .01
*
.22 .19 .82 .62 .01
*
.14 .67 .00
*
.03
*
.21 .48 .81 .40
Recovery: MHRM effect 7.23 5.06 12.69 −7.02 −3.55 5.35 4.42 6.89 13.36 −10.90 8.09 −3.85 3.27 .84
p .02
*
.11 .00
**
.03
*
.28 .10 .19 .13 .01
*
.10 .14 .57 .32 .82
Symptoms: BSI effect −.07 −.13 −.60 .28 .31 −.02 −.07 −.13 .08 −.12 .06 −.24 −.18 −.22
p .58 .30 .00
**
.03
*
.02
*
.90 .62 .47 .72 .66 .77 .39 .19 .17
Quality of Life: MANSA effect .15 .11 .37 −.22 −.36 .10 .00 .10 .49 −.58 −.04 .24 .10 .12
p .35 .52 .07 .18 .04
*
.57 .99 .65 .08 .09 .90 .50 .57 .55
General Health
Status:
Rand-
PF
effect 6.61 5.21 19.87 3.57 −6.59 8.73 3.63 −3.92 13.2 −13.01 6.1 −3.17 −.48 6.78
p .16 .27 .00
**
.45 .18 .07 .46 .58 .11 .20 .46 .76 .93 .26
Rand-
SF
effect −.02 .93 15.91 −1.24 −14.26 −1.67 −2.24 1355 2.04 −7.55 −3.39 5.04 .35 9.07
p 1.00 .88 .02
*
.13 .04
*
.80 .74 .04
*
.85 .58 .76 .71 .96 .24
Rand-
RLPP
effect 9.47 7.88 29.85 −3.43 −13.61 9.98 4.68 −11.85 29.73 −32.31 11.28 −6.33 −6.56 4.99
p .39 .48 .01
*
.76 .26 .39 .69 .36 .12 .17 .55 .79 .59 .73
Rand-
RLEP
effect −25.58 9.47 29.18 −9.99 −17.38 −21.33 11.43 8.18 −28.77 5.24 −21.27 44.85 −18.90 23.40
p .03
*
.43 .01
**
.46 .22 .09 .37 .47 .16 .84 .31 .08
*
.15 .13
Rand-
MH
effect −2.40 −1.09 15.11 −6.73 −5.80 −3.85 −2.17 8.07 −.50 −5.33 −4.94 5.65 2.24 −.83
p .57 .80 .00
**
.13 .21 .39 .64 .13 .95 .56 .50 .54 .61 .87
Rand-V effect −.26 .91 15.11 −3.18 −7.04 .55 .35 8.76 14.76 −22.9 8.6 −8.36 −7.62 −9.42
p .96 .85 .00
**
.53 .19 .92 .95 .11 .07 .03
*
.30 .42 .11 .09
Rand-P effect 1.63 −3.46 21.37 2.74 −1.11 6.01 −3.74 −10.34 −4.51 2.84 −6.01 .68 2.14 −1.73
p .82 .63 .00
**
.72 .21 .42 .62 .19 .72 .85 .63 .97 .80 .28
Rand-
GHP
effect 7.84 1.10 15.21 −2.91 −13.97 7.13 5.31 1.65 9.28 −3.13 9.44 3.43 −3.01 5.26
p .07 .02
*
.00
**
.50 .00
**
.10 .23 .76 .22 .73 .21 .71 .53 .35
BSI Brief Symptom Inventory, e-IMR e-health application to Illness Management & Recovery programme, IMRS Illness Management & Recovery Scales, MANSA Manchester Short Assessment of Quality of Life, MHRM
Mental Health Recovery Measure, pp-value, PAM Patient Activation Measure, Rand-GHP Rand General Health Perception, Rand-MH Rand Mental Health, Rand-P Rand Pain, Rand-PF Rand Physical Functioning, Rand-SF
Rand Social Functioning, Rand-RLEP Rand Role Limitation due to Emotional Problems, Rand-RLPP Rand Role Limitation due to Physical Problems, Rand-V Rand Vitality;
*
p-value < .05;
**
p-value < .01
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 7 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
which resembles the scores of the Rand-V. We did not
find significant correlations in PAM-13 scores at T
0
be-
tween the completers and non-completers (p= .77).
In model 4, the subgroup analyses within the interven-
tion group between the groups of e-IMR users and
non-users showed no significant effect differences at T
1
and T
2
.
Harm
No serious adverse events were reported during the trial.
Discussion
This study shows significant differences in main effects
for the parameters self-management (PAM-13), recovery
(MHRM), and role limitation due to emotional problems
(Rand-RLEP) in favour of the intervention group at T
1
,
which faded at T
2
.AtT
2
, a significant effect for general
health perception (Rand-GHP) occurred, also in favour
of the intervention group.
Post hoc analyses showed that the confounder gender
explained the effects for recovery and role limitation due
to emotional problems at T
1,
and for general health per-
ception at T
2
. The confounding effects of gender were
based on three types of differences: first, the baseline
distribution showed significantly more females in the
intervention group; second, at T
0
males scored signifi-
cantly higher on most of the measures; and third, time
trends were in favour of female participants. In general,
women do differ from men in a number of ways; for in-
stance, exposure and reactions to stress [35], needs and
care [38,39], and coping styles [40]. With regard to cop-
ing styles, women could benefit more from a
problem-solving-focused intervention and men from an
emotion-focused one [41]. IMR, with its emphasis on
learning how to manage an illness in a context of pursu-
ing recovery goals [42], has a greater focus on problem-
solving- than on emotional strategies. Therefore, women
could have benefitted more from the IMR-programme
than men.
Post-hoc analyses showed that the confounder gender
did not explain the effects for the parameter
self-management. Also in studies with people with dia-
betes II [43] and other chronic illnesses [44], no rela-
tions were found between gender and self-management,
measured by the PAM-13.
The interaction covariate non-completer significantly
modified the effect for the parameter self-management
(PAM) and vitality (Rand-V) such that a large interven-
tion effect was seen in the non-completers and a small
effect in the completers. Apparently, stopping the
IMR-programme was based on differences in their im-
provements. In this study, improvements in conditions
of people who dropped out of the IRM-programme were
unequally distributed over the groups, which modified
the effects. The unlikeliness of the effects is confirmed
by the subgroup analyses within the intervention group
which showed no significant effect differences between
the groups of e-IMR users and non-users.
A last issue to discuss is the low use of the e-IMR plat-
form by the participants in the intervention group which
resulted in a minor contrast in the treatments provided
to the participants in the intervention and control group
and further calls into question the validity of ascribing
the effects observed to the e-IMR. The modest use of
the e-IMR matches with 6% of consumers using e-health
in general mental health in the year of this study [45].
Fig. 3 The course of PAM-13 scores in analysis with covariate non-completer
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 8 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
A number of limitations should be noted. Unfortunately,
the planned sample size was not achieved and a lower
number of participants entered the control group. This
might have caused the unequal distribution of some co-
variates. Due to the small sample, we could not control
for more than one covariate in the mixed models without
risking overfitting. Notwithstanding the small sample, a
number of non-completers did not withdraw from the
study. The overall non-completer rate of 30% is similar to
other IMR studies [7]. Therefore, the intention-to-treat
analyses resemble normal practice.
Conclusion
Finally, this study precludes definite conclusions on the
potential efficacy of e-health for people with SMI. This
leaves us with many questions about the barriers and facil-
itators of the e-IMR intervention and its implementation.
Against the backdrop of the great promise of e-mental
health [46], the modest use of the e-IMR platform might
be an interesting outcome which needs to be further in-
vestigated. Before deciding how to continue studying the
effectiveness of e-IMR, we will investigate barriers and fa-
cilitators of the e-IMR and its implementation.
Additional files
Additional file 1: Mean scores and standard deviation of the outcome
domains per group at baseline (T
0
), halfway (T
1
) and post treatment (T
2
)
(DOCX 29 kb)
Abbreviations
BSI: Brief Symptom Inventory; e-IMR: e-health application to Illness Management
& Recovery programme; IMR: Illness Management & Recovery programme;
IMRS: Illness Management & Recovery Scales; MANSA: Manchester Short
Assessment of Quality of Life; MHRM: Mental Health Recovery Measure; PAM-
13: Patient Activation Measure; Rand-GHP: Rand General Health Perception;
Rand-MH: Rand Mental Health; Rand-P: Rand Pain; Rand-PF: Rand Physical
Functioning; Rand-RLEP: Rand Role Limitation due to Emotional Problems;
Rand-RLPP: Rand Role Limitation due to Physical Problems; Rand-SF: Rand
Social Functioning; Rand-V: Rand Vitality; SMI: Severe (or Serious) Mental Illness;
T
0
: Time point 0, baseline;; T
1
: Time point 1, halfway, after completing the 5th
module; T
2
: Time point 2, endpoint, at least a week after finishing the IMR-
programme
Acknowledgements
Not applicable.
Funding
This study was funded by the ZonMW (the Netherlands Organisation for
Health Care Research and Development) programme ‘Tussen Weten en
Doen’(Grant 520001001). The funder had no influence on study design, the
collection, analysis and interpretation of the data, the writing of the report,
and the decision to submit the article for publication.
Availability of data and materials
Data and materials will be made available after a request to the
corresponding author.
Authors’contributions
TB., PG, ST, MN and BvG contributed to the conception and design of the
study. TB contributed to the data collection. All authors contributed to the
analysis and interpretation, and provided drafting of the article. All authors
contributed to the critical revision of the article for important intellectual
content and final approval of the article.
Ethics approval and consent to participate
Individuals who were referred to the IMR-programme were informed about
the trial by their case manager. The researcher contacted those who
expressed an interest individually and explained the trial and research activ-
ities. If they were still interested and eligible, the written consent to partici-
pate was signed. No incentives were provided. The ethical approval for
conducting the e-IMR trial was provided by the Committee on Research In-
volving Human Subjects, Arnhem-Nijmegen (NL49693.091.14).
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’sNote
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
Author details
1
Titus Beentjes, IQ Healthcare, Radboud University Medical Center, Radboud
Institute for Health Sciences, PO Box 9101, 6500, HB, Nijmegen, the
Netherlands.
2
Center for Nursing Research, Saxion University of Applied
Science, Deventer/Enschede, the Netherlands.
3
Dimence Group Mental
Health Care Centre, Deventer, the Netherlands.
4
Department of Public
Health, Faculty of Medicine and Health Sciences, University Centre for
Nursing and Midwifery, Ghent University, Ghent, Belgium.
5
Department for
Health Evidence, Radboud University Medical Center, Radboud Institute for
Health Sciences, Group Biostatistics, Nijmegen, the Netherlands.
6
Faculty of
Health and Social Studies, HAN University of Applied Sciences, Nijmegen, the
Netherlands.
Received: 7 June 2018 Accepted: 23 November 2018
References
1. Ben-Zeev D, Drake RE, Corrigan PW, Rotondi AJ, Nilsen W, Depp C. Using
contemporary Technologies in the Assessment and Treatment of serious
mental illness. Am J Psychiatr Rehabil. 2012;15:357–76.
2. Naslund JA, Marsch LA, McHugo GJ, Bartels SJ. Emerging mHealth and
eHealth interventions for serious mental illness: a review of the literature. J
Ment Health. 2015;24:320–31.
3. van der Krieke L, Wunderink L, Emerencia AC, de Jonge P, Sytema S. E-
mental health self-management for psychotic disorders: state of the art and
future perspectives. Psychiatr Serv. 2014;65:33–49.
4. Bremmer F, van Es M. Een analyse van de Verwachte Kosten En Baten van
EHealth, blended Behandelen En Begeleiden [an analyses of the expected
costs and benefits of e-health, blended treatment and care]. Amersfoort:
GGZ Nederland; 2013.
5. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system
design does matter: a systematic review of adherence to web-based
interventions. J Med Internet Res. 2012;14:e152.
6. Postel M, Witting M, van Gemert-Pijnen L. Blended behandeling in de
geestelijke gezondheidszorg [Blended Treatment in Mental Health]. Dir Ther.
2013;33:210–21.
7. McGuire AB, Kukla M, Green A, Gilbride D, Mueser KT, Salyers MP. Illness
management and recovery: a review of the literature. Psychiatr Serv. 2014;
65:171–9.
8. Beentjes TAA, van Gaal BGI, Goossens PJJ, Schoonhoven L. Development of
an e-supported illness management and recovery programme for
consumers with severe mental illness using intervention mapping, and
design of an early cluster randomized controlled trial. BMC Health Serv Res.
2016;16:20.
9. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M.
Developing and evaluating complex interventions: the new Medical
Research Council guidance. Int J Nurs Stud. 2013;50:585–92.
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 9 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
10. Delespaul PH, EPA d c. Consensus regarding the definition of persons with
severe mental illness and the number of such persons in the Netherlands.
Tijdschr Psychiatr. 2013;55:427–38.
11. Brugman M, Stalenhoef L, Kamman R, Meinen G. Illness Management &
Recovery 3.0. Enschede: Saxion; 2014.
12. Limesurvey. An Open Source survey tool. Germany: Limesurvey GmbH,
Hamburg; 2012.
13. Mueser KT, Gingerich S, Salyers MP, McGuire AB, Reyes RU, Cunningham H.
Illness Management and Recovery (IMR) Scales. In: Campbell-Orde T,
Chamberlin J, Carpenter J, Leff HS, editors. Measuring the Promise: A
Compendium of Recovery Measures Volume II: Cambridge, Human Services
Research Institute; 2005. p. 32–5. & 124–132.
14. Goossens PJJ, Beentjes TAA, Knol S, Salyers MP, de Vries SJ. Investigating the
reliability and validity of the Dutch versions of the illness management and
recovery scales among clients with mental disorders. J Ment Health. 2017;
26:489–95.
15. Salyers MP, Godfrey JL, Mueser KT, Labriola S. Measuring illness
management outcomes: a psychometric study of clinician and consumer
rating scales for illness self management and recovery. Community Ment
Health J. 2007;43:459–80.
16. Hasson-Ohayon I, Roe D, Kravetz S. The psychometric properties of the
illness management and recovery scale: client and clinician versions.
Psychiatry Res. 2008;160:228–35.
17. Färdig R, Lewander T, Fredriksson A, Melin L. Evaluation of the illness
management and recovery scale in schizophrenia and schizoaffective
disorder. Schizophr Res. 2011;132:157–64.
18. Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and
testing of a short form of the patient activation measure. Health Serv
Res. 2005;40(6 Pt 1):1918–30.
19. Zill JM, Dwinger S, Kriston L, Rohenkohl A, Härter M, Dirmaier J.
Psychometric evaluation of the German version of the patient activation
measure (PAM13). BMC Public Health. 2013;13:1027.
20. Rademakers J, Nijman J, van der Hoek L, Heijmans M, Rijken M. Measuring
patient activation in the Netherlands: translation and validation of the
American short form patient activation measure (PAM13). BMC Public
Health. 2012;12:577.
21. Moljord IEO, Lara-Cabrera ML, Perestelo-Pérez L, Rivero-Santana A, Eriksen L,
Linaker OM. Psychometric properties of the patient activation Measure-13
among out-patients waiting for mental health treatment: a validation study
in Norway. Patient Educ Couns. 2015;98:1410–7.
22. Green CA, Perrin NA, Polen MR, Leo MC, Hibbard JH, Tusler M. Development
of the patient activation measure for mental health. Admin Pol Ment
Health. 2010;37:327–33.
23. van Nieuwenhuizen C, Wilrycx G, Moradi M, Brouwers E. Psychometric
evaluation of the Dutch version of the mental health recovery measure
(MHRM). Int J Soc Psychiatry. 2014;60:162–8.
24. Bullock W. The mental health recovery measure (MHRM): Updated
Normative Data and Psychometric Properties. Toledo: Department of
Psychology, University of Toledo; 2009.
25. Derogatis LR, Melisaratos N. The brief symptom inventory-an introductory
report. Psychol Med. 1983;13:595–605.
26. Beurs E, de ZF. De brief symptom inventory (BSI). De betrouwbaarheid en
validiteit van een handzaam alternatief voor de SCL-90. [the brief symptom
inventory, reliability and validity of a conveniant alternative]. Maandbl
Geestelijke Volksgezond. 2006;61:120–41.
27. Priebe S, Huxley P, Knight S, Evans S. Application and results of the
Manchester short assessment of quality of life (Mansa). Int J Soc Psychiatry.
1999;45:7–12.
28. Björkman T, Svensson B. Quality of life in people with severe mental illness.
Reliability and validity of the Manchester short assessment of quality of life
(MANSA). Nord J Psychiatry. 2005;59:302–6.
29. Hays RD, Sherbourne CD, Mazel RM. The RAND 36-item health survey 1.0.
Health Econ. 1993;2:217–27.
30. van der Zee KI, Sanderman R, Heyink JW, de Haes H. Psychometric qualities
of the RAND 36-item health survey 1.0: a of multidimensional measure of
general health status. Int J Behav Med. 1996;3:104–22.
31. McGuire AB, Bonfils KA, Kukla M, Myers L, Salyers MP. Measuring
participation in an evidence-based practice: illness management and
recovery group attendance. Psychiatry Res. 2013;210:684–9.
32. IBM C. IBM SPSS statistics for Macintosh, version 23; 2015.
33. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ,
Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and
elaboration: updated guidelines for reporting parallel group randomised
trials. BMJ. 2010;340(mar23 1):c869.
34. Portney L, Watkins M. Foundation of Clinical Research, application to practice.
3rd ed. Upper Sadle River, New Jersey: Pearson education, Inc.; 2009.
35. Mayor E. Gender roles and traits in stress and health. Front Psychol. 2015;6:1–7.
36. Lincoln TM, Rief W, Westermann S, Ziegler M, Kesting ML, Heibach E, Mehl
S. Who stays, who benefits? Predicting dropout and change in cognitive
behaviour therapy for psychosis. Psychiatry Res. 2014;216:198–205.
37. Salyers MP, Rollins AL, Clendenning D, McGuire AB, Kim E. Impact of illness
management and recovery programs on hospital and emergency room use
by Medicaid enrollees. Psychiatr Serv. 2011;62:509–15.
38. Arvidsson H. Gender differences in needs and care of severely mentally ill
persons: findings from a Swedish cross-sectional and longitudinal study. Int
J Soc Psychiatry. 2010;56:424–35.
39. Manuel JI, Stebbins MB, Wu E. Gender differences in perceived unmet
treatment needs among persons with and without co-occurring disorders. J
Behav Health Serv Res. 2018;45:1–12.
40. Turner H, Bryant-Waugh R, Peveler R, Bucks RS. A psychometric evaluation of an
English version of the Utrecht coping list. Eur Eat Disord Rev. 2012;20:339–42.
41. Schut HA, Stroebe MS, van den Bout J, de Keijser J. Intervention for the
bereaved: gender differences in the efficacy of two counselling
programmes. Br J Clin Psychol. 1997;36(Pt 1):63–72.
42. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The illness
management and recovery program: rationale, development, and
preliminary findings. Schizophr Bull. 2006;32(Suppl 1):S32–43.
43. Hendriks SH, Hartog LC, Groenier KH, Maas AHEM, van Hateren KJJ, Kleefstra
N, Bilo HJG. Patient activation in type 2 diabetes: does it differ between
men and women? J Diabetes Res. 2016;2016:1–8.
44. Rijken M, Heijmans M, Jansen D, Rademakers J. Developments in patient
activation of people with chronic illness and the impact of changes in self-
reported health: results of a nationwide longitudinal study in the
Netherlands. Patient Educ Couns. 2014;97:383–90.
45. Krijgsman J, Peeters J, Burghouts A, Brabers A, De Jong J, Moll T, Friele R,
Van Gennip L. Tussen vonk en vlam e-health-monitor 2015 [Dutch E-health
monitor 2015]. Den Haag: Nictiz & NIVEL; 2015.
46. WHO. The European Mental Health Action Plan 2013-2020. Copenhagen:
WHO; 2013.
Beentjes et al. BMC Health Services Research (2018) 18:962 Page 10 of 10
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
Available via license: CC BY 4.0
Content may be subject to copyright.
