Available via license: CC BY-NC 4.0
Content may be subject to copyright.
1
CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
Improving neurodevelopmental
outcomes in children with congenital
heart disease: protocol for a randomised
controlled trial of working
memory training
Johanna Calderon,1,2 David C Bellinger,2 Catherine Hartigan,3 Alison Lord,3
Christian Stopp,3 David Wypij,3,4 Jane W Newburger3,5
To cite: CalderonJ,
BellingerDC, HartiganC, etal.
Improving neurodevelopmental
outcomes in children with
congenital heart disease:
protocol for a randomised
controlled trial of working
memory training. BMJ Open
2019;9:e023304. doi:10.1136/
bmjopen-2018-023304
►Prepublication history and
additional material for this
paper are available online. To
view these les, please visit
the journal online (http:// dx. doi.
org/ 10. 1136/ bmjopen- 2018-
023304).
Received 30 March 2018
Revised 22 October 2018
Accepted 20 December 2018
For numbered afliations see
end of article.
Correspondence to
DrJohanna Calderon;
johanna. calderon@ childrens.
harvard. edu
Protocol
© Author(s) (or their
employer(s)) 2019. Re-use
permitted under CC BY-NC. No
commercial re-use. See rights
and permissions. Published by
BMJ.
ABSTRACT
Introduction Executive function (EF) impairments are
among the most prevalent neurodevelopmental morbidities
in youth with congenital heart disease (CHD). To date,
no studies have investigated the efcacy of cognitive
interventions to improve EF outcomes in children with CHD.
Methods and analysis This is a single-centre, single-
blinded, two-arm randomised controlled trial to test the
efcacy of Cogmed Working Memory Training (Cogmed)
versus standard of care in children with CHD after open-
heart surgery in infancy. Participants will consist of 100
children with CHD aged 7–12 years who underwent open-
heart surgery before the age of 12 months. Participants are
randomly allocated to either an intervention group including
training on the home-based Cogmed intervention for a
duration of approximately 5 weeks or a control group who
receive the standard of care. We will evaluate the efcacy of
Cogmed at post-treatment and 3 months after completion
of the intervention. Baseline, post-treatment and 3-month
follow-up assessments will include specic measures of
EF, cognitive and social functioning, and attention decit
hyperactivity disorder (ADHD) symptoms. The primary
outcome of this study is the change in standardised mean
score on the List Sorting Working Memory test from the
National Institutes of Health Toolbox for the Assessment of
Neurological and Behavioral Function. Secondary outcomes
include measures of social skills, inhibitory control,
cognitive exibility and behavioural EF as well as ADHD
symptoms as measured by the Behavior Rating Inventory of
Executive Function, Second Edition, and the Conners Third
Edition. The efcacy of the intervention will be evaluated
by comparing within-subject differences (baseline to post-
treatment, baseline to 3-month follow-up) between the two
groups using an intention-to-treat analysis.
Ethics and dissemination This study has received
Institutional Review Board (IRB) approval from Boston’s
Children’s Hospital IRB (P00022440) and the Human
Protection Agency from the US Department of Defense.
Trial registration number NCT03023644; Pre-results.
INTRODUCTION
Congenital heart lesions are among the most
common birth defects,1 2 as approximately
1% of infants are born with congenital heart
disease (CHD). Of these, more than one-third
will present with critical CHD, most broadly
defined as forms of CHD requiring surgical
or catheter interventions or resulting in death
in the first year of life.1 3 Advances in prenatal
diagnosis as well as medical and surgical care
have reduced mortality rates for all forms
Strengths and limitations of this study
►To our knowledge, this is the rst randomised con-
trolled trial (RCT) investigating the efcacy of an ex-
ecutive function intervention in improving outcomes
for children with congenital heart disease (CHD).
►The home-based Cogmed Working Memory Training
(Cogmed) is among the most widely used evi-
dence-based programmes targeting core executive
function skills and will directly address the most fre-
quent neurodevelopmental impairment for children
with critical CHD that strongly impacts their ability to
succeed in academic and social environments. This
intervention is individually adapted to each child’s
own executive function level, which ensures an op-
timal level of performance throughout the sessions.
►As a home-based intervention, Cogmed reduces the
need for hospital-based treatment visits, potential-
ly reducing the burden for families of children with
chronic health conditions such as critical CHD.
►This RCT includes computerised individual mea-
sures of neurodevelopment and parent-rating and
teacher-rating scales of behavioural and social out-
comes as well as collection of patient-specic fac-
tors to investigate their potential relationship with
response to treatment
►This is a phase II RCT with the goal of providing the
rst proof of concept that a cognitive intervention
can improve outcomes in children with CHD. It is sin-
gle blinded (investigators are blinded to intervention
status and patient characteristics but participants
know their treatment group) and has a relatively
short duration of follow-up (3 months).
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
2CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
of CHD. However, evidence of central nervous system
damage, including delayed brain maturation in utero and
abnormal brain metabolism and microstructure associ-
ated with hypoxic-ischaemic injury, has been reported by
a wealth of studies of critical CHD.4–7 A dramatic increase
in the population of survivors of infant heart surgery
has been accompanied by the increased recognition of
their long-term postoperative morbidities. Neurodevelop-
mental disabilities, particularly executive function (EF)
impairments, are currently the most prevalent long-term
morbidity in the population with CHD.4 EF refers to a
set of higher order neurocognitive abilities that serve to
coordinate and organise actions towards a goal, allowing
the individual to adapt to new or complex situations.8
Impairments in EF manifest as behavioural dysregulation
and attention problems, impaired working memory (ie,
the ability to keep information in mind and manipulate it
over a short period of time) and problems with organisa-
tion and planning abilities. EF is more strongly associated
with school readiness than is IQ, predicts both mathe-
matics and reading competence throughout the school
years8–10 and is strongly associated with social cognition
(ie, decoding other people’s mental and emotional states
and responding to rapid-paced social interactions).9
Executive dysfunction can profoundly impact all dimen-
sions of a child’s development11–14 and is a core feature
of attention deficit hyperactivity disorder (ADHD)15 and
autism spectrum symptoms.16 17 If untreated, deficits in EF
may also predispose individuals to later addiction,18 eating
disorders and obesity,19 and risk-taking behaviours.20
These adverse sequelae may carry profound implications
for the educational achievement, future employment and
quality of life of individuals with CHD.4
EF in critical CHD
Impairments in EF are at the heart of the neurodevel-
opmental phenotype associated with critical CHD after
open-heart surgery.21–34 EF deficits in children with
CHD were first reported in school-aged children with
dextro-transposition of the great arteries (d-TGAs).25
Standardised neuropsychological testing showed that
patients with d-TGA had substantial difficulty planning
and alternating between tasks, which suggested impair-
ments in cognitive flexibility and working memory as
well as deficits in planning and sustained attention. On
the Behavior Rating Inventory of Executive Function
(BRIEF), parents and teachers of adolescents with CHD
reported significant difficulties, particularly with regard
to working memory.22 Compared with normative values,
parents’ ratings were worse by ~0.5 SD and those of
teachers by ~1 SD, suggesting statistically significant and
clinically meaningful impairments. More recent find-
ings also reported specific EF impairments in preschool
and school-aged children with d-TGA.21 23 24 In partic-
ular, children had important difficulties in behavioural
regulation and cognitive control of attention, and they
had worse performances on verbal and visual working
memory tasks. Consistent findings have been reported
by studies that included children with other types of crit-
ical CHD such as tetralogy of Fallot28 or single ventricle
physiology requiring the Fontan operation.29 Finally, EF
impairments have been associated with worse psychoso-
cial health status and worse quality of life in youth with
critical CHD,31 highlighting the potential impact of long-
term executive dysfunction on mental health in CHD.32
Working memory intervention for children with CHD
The American Heart Association (AHA) recommends
routine neurodevelopmental screening of all CHD survi-
vors.4 A burgeoning literature documents the prevalence
and importance of impaired EF and ADHD in CHD
survivors,21–40 and brain imaging studies have provided
key information on the underlying disturbances in brain
structure and microstructure in patients with CHD.5–7
Yet, to date, no trials have been undertaken to test inter-
ventions targeting EF and attention deficits in the CHD
population.34
Cogmed Working Memory Training (Cogmed) is the
most widely used computerised evidence-based interven-
tion that targets EF, specifically providing intensive struc-
tured training of working memory.41–54 It has been shown
to improve executive performance in several clinical
and non-clinical paediatric populations, including chil-
dren with ADHD,41 42 46 47 low working memory and low
achievement43 44 and children who were born preterm or
extremely low birth weight.53 54 Unlike hospital-based or
laboratory-based interventions, Cogmed can be imple-
mented as a home-based intervention for children.
Studies using Cogmed have shown that subjects demon-
strate the ability to transfer skills to non-trained tests of
working memory as well as to tasks that involve similar
processes, including attention, inhibition and non-verbal
reasoning.51–53 The positive effect of training has been
observed on parental ratings of inattention, including the
Diagnostic and Statistical Manual of Mental Disorders,
Fourth Edition (DSM-IV) Parent Rating Scale, ADHD
Rating Scale, Fourth Edition, BRIEF and Conners’ Parent
Rating Scale. Sustained improvements in behaviour as
measured by rating scales have also been observed in
ADHD,48 brain injury49 50 and non-clinical groups.51 52
In summary, several randomised controlled trials (RCTs)
evaluating the use of Cogmed in healthy children44 51 52
and in children with various conditions41 42 47 53 54 have
demonstrated that this neurocognitive intervention
produces significant generalised and sustained enhance-
ment on measures of EF and also on everyday life
learning and behavioural skills. It is proposed that
training working memory using Cogmed is a promising
intervention for school-aged children with critical CHD
because: (1) it addresses the most frequent neurodevel-
opmental morbidities that strongly impact the ability
to succeed in academic and social environments; (2) it
allows for intensive and structured practice of targeted
skills, with possible transfer to other neurodevelopmental
domains; (3) it is individually adapted to each child’s own
EF levels, which ensures an optimal level of performance
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
3
CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
throughout the sessions; (4) it is closely monitored,
and various parameters of the child’s performance are
systematically recorded (eg, correct answers and speed
at which tasks are completed); (5) it is child-friendly and
rewarding, which facilitates children’s compliance; and,
finally, but importantly (6) as a home-based intervention,
it reduces the need for hospital/clinic-based visits and
multiple costs of individual therapy, potentially reducing
the burden for families of children with chronic health
conditions such as critical CHD.
In this project, we conduct a RCT to provide the first
proof of concept that Cogmed intervention improves
neurodevelopmental outcomes in children with CHD and
that the improvements persist to 3 months. We will enrol
children with CHD who underwent infant open-heart
surgery as this population corresponds to the highest
risk category for developmental disorders and disabili-
ties as per the AHA guidelines (Class I; Level of Evidence
A).4 We propose to determine immediate and 3-month
post-treatment effects on both laboratory-based tests and
ecological measures of children’s EF, ADHD and social
difficulties in everyday life. Our study will also provide
insight into factors that are associated with response to
treatment, identifying children who may be most likely to
benefit from the intervention.
Aims and hypotheses
Specific aim 1: to evaluate the immediate efficacy of
home-based Cogmed intervention for neurodevelop-
mental outcomes in children with CHD. We hypothe-
sise that children who receive the Cogmed intervention,
compared with controls receiving standard of care, will
display greater improvement from baseline to post-treat-
ment assessment in EF and social development, and
greater reduction in symptoms of ADHD.
Our primary outcome measure will be the change
in standardised mean score on the working memory
test from the National Institutes of Health Toolbox for
the Assessment of Neurological and Behavioral Func-
tion (NIH Toolbox)55 from baseline to post-treatment.
Secondary outcomes include changes in standardised
mean scores on tests of cognitive flexibility, attention,
inhibitory control and speed of processing from the
NIH Toolbox; the Global Executive Composite from
the Behavior Rating Inventory of Executive Function,
Second Edition (BRIEF-2),56 the Global Index and the
ADHD Index from the Conners Third Edition (Conners-
3)57 and the Social Responsiveness Scale, Second Edition
(SRS-2).58
Specific aim 2: to assess the effects of the Cogmed inter-
vention at 3-month follow-up. We predict that significant
gains in neurodevelopmental and behavioural outcomes
will persist 3 months after cessation of intervention
for children who received Cogmed as compared with
controls.
The primary and secondary outcomes will be the same
as those in specific aim 1, except that the change in scores
will be from baseline to 3-month follow-up (ie, approxi-
mately 3 months after the last Cogmed session).
Specific Aim 3: to explore cognitive, medical and socio-
demographic factors associated with changes in neuro-
developmental and behavioural scores for children who
received Cogmed intervention.
METHODS AND ANALYSIS
Study design
This is a single-centre, single-blinded, two-arm RCT to
test the efficacy of Cogmed intervention versus stan-
dard of care in children with CHD after neonatal and/
or infant open-heart surgery (n=50 in each group). All
eligible subjects undergo a baseline neurodevelopmental
assessment and then are randomly assigned to either the
home-based Cogmed intervention or to a control group
receiving the standard of care for children with CHD. All
participants will undergo a post-treatment and a 3-month
follow-up assessment. All investigators collecting outcome
data are blinded to patients’ group assignment (Cogmed
intervention versus standard of care) and to medical and
surgical histories. Participants and their parents know
their group assignment and thus are not blinded. For
children assigned to the Cogmed group, post-treatment
assessments are performed 1–2 weeks after the end of the
intervention (ie, approximately 7–8 weeks after baseline
assessment) and follow-up will be performed 3 months
after the end of the intervention (ie, approximately 5
months after baseline assessment). For children in the
control group, post-treatment and 3-month follow-up
assessments are performed approximately 7–8 weeks and
5 months after baseline assessment, respectively.
Participants and recruitment
Participants are included if they meet the following
criteria: (1) diagnosis of CHD requiring at least one
open-heart surgery before 1 year of age; (2) age between
7 years and 12 years at baseline assessment; (3) ≥6
months postcardiac surgery; (4) had received cardio-
vascular care at Boston Children’s Hospital; (5) English
or Spanish speaking; and (6) informed consent from
parent/guardian as well as assent of the child. Exclusion
criteria will be: (1) diagnosed chromosomal anomalies
and/or genetic syndromes; (2) severe physical and/or
sensory impairments (hearing, visual or psychomotor)
that would prevent the use of the computerised program;
(3) confirmed diagnosis of an autism spectrum disorder
and/or severe developmental or intellectual disorder
that would prevent successful completion of the planned
study testing; (4) placement in a separate classroom for
severe sensory, motor, language or other developmental
disability receiving individual support; (5) scheduled to
undergo major cardiac interventions in the 6 months
following enrolment; and (6) received, receiving or
scheduled to receive Cogmed or any other computerised
behavioural training programme targeting EF or ADHD.
We will not exclude children who underwent multiple
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
4CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
heart or other surgeries, children with a pre-existing
neurological history (eg, epilepsy and stroke) or with a
history of a concurrent diagnosis of ADHD (treated or
untreated). Rather, we will account for these factors in
the data analysis.
Eligible children living in the USA are recruited
through patient databases of Boston Children’s Hospital
Cardiology Clinic and affiliated New England medical
centres. Families are invited to participate in the study via
a mail packet and follow-up phone call. Flyers and study
brochures are displayed in Boston Children’s Hospital
Cardiology Clinic and affiliated medical centres as well
as in some local advocacy parent organisations. Partici-
pants are assessed for eligibility and enrolled by a study
coordinator and a research nurse. Informed consent
and assent from the child are obtained by a study coordi-
nator or a research neuropsychologist before the baseline
assessment at the hospital. Parents and children receive
monetary compensation for participation in the study.
These incentives are given at the second and third visits.
Additionally, in order to further facilitate participants’
compliance and reduce dropouts, the second and third
study assessments may be completed at a child’s home.
The study start date (ie, start of active enrolment) was 27
February 2017, and it is anticipated that enrolment will be
completed in September 2019.
Patient and public involvement
Patients, patient/family advocacy groups or the public
were not involved in the design, recruitment and conduct
of this study. Participants are informed of the burden of
the intervention and are given the option to stop at any
time point. All eligible patients completing the study will
receive an individual report of the results of his or her
baseline assessment as well as a general report on study
results for the group with CHD when data analyses are
completed.
Randomisation and stratication
All eligible subjects undergo a baseline neurodevelop-
mental assessment (figure 1) and then are randomly
assigned to either the standard home-based Cogmed inter-
vention group or to a control group (standard of care).
Allocations are assigned using a computerised system only
seen by the research assistant or study coordinator after
confirming all eligibility criteria and consent. Subjects are
Figure 1 Flow chart of trial design. ND, neurodevelopmental.
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
5
CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
assigned in the order in which they are enrolled into the
study. Randomisation is done by computerised permuted
blocks design with blocks of varying sizes. Once a subject
has been assigned to a group, he or she will remain in the
same trial arm for the duration of the study. The rando-
misation scheme involves two stratification factors: type of
CHD (univentricular or biventricular) and baseline level
of EF (a score <85 or ≥85 on the working memory test
from the NIH Toolbox). Figure 1 shows the flow chart of
the trial design.
Intervention group: home-based Cogmed Working Memory
Training
Children randomly assigned to receive the Cogmed inter-
vention will complete the standard home-based format of
the programme, Cogmed RM, for children aged 7 years
and older. The training programme contains 12 child-
friendly visual-spatial and verbal working memory tasks
(eg, remembering the order in which lamps light up on
a 4×4 grid; recalling a series of numbers of increasing
length on the screen). All tasks are adaptive, that is, task
complexity levels are automatically adjusted to match
each child’s working memory capacity, to improve perfor-
mance and to limit non-compliance to the intervention
due to lack of motivation. Tasks become more difficult,
on a session-by-session basis, as a child’s performance
improves. Each training session lasts approximately
40–50 min, and the child is instructed to complete one
session per day 5 days each week for 5 weeks, for a total
of 25 sessions. The programme yields individual session-
by-session and task-by-task training results, including the
children’s responses, time spent on each task and evolu-
tion curves. Cogmed is not Food and Drug Administration
(FDA)-regulated. Based on our specific aims, Cogmed is
considered a non-significant risk device.
Study tablets (ie, iPads) are provided to families
randomised to the Cogmed group in order to standardise
the method of delivery. Families receive a link for down-
loading a web-based software program to the tablet. The
program is installed on the tablet by a study coordinator
who explains how the training program works and how to
log into the system and complete training. The training
session and installation of the program are completed
after baseline assessment and randomisation. Parents and
children will be actively involved, and during the installa-
tion session, children will complete several practice trials
under the supervision of the study coordinator. The 25
sessions will be completed by the child, supervised by a
parent. For the first five sessions, the child trains on the
same set of games; on the sixth session and every fifth
session thereafter, a new task is introduced and replaces
one of the initial tasks. At the end of each session, the
child can play an age-appropriate tablet game as a reward.
After each session, a parent will upload the results to a
secure website. Families are contacted weekly to check
program function and discuss concerns. Compliance is
automatically registered by the computerised program
and is defined as completing at least 20 sessions, the
criterion by which children are categorised as compliant
or non-compliant to treatment.41 42
To implement this intervention, each investigator and
study coordinator involved in coaching is certified as a
‘Cogmed Coach’. The Cogmed Coaches will monitor chil-
dren’s performance on mycogmed. com secured website
every week during the intervention to permit continuous
evaluation of treatment compliance and fidelity. The
Cogmed coach specifically monitors performance of each
child and contacts the parents and the child by phone
on a weekly basis to discuss progression and any issues
arising during the training week. A designated Cogmed
coach will be available during the trial to respond to any
questions or help with any difficulties during the training.
Families and children are encouraged to continue the
training for at least 20 sessions. If parents or children
struggle with some aspects of the intervention such as
the time commitment or a lack of motivation to persist
with the training, the Cogmed coach discusses alternative
options for accommodating each individual child’s needs
(ie, rewards systems available, best time of the day to prac-
tice, number of breaks necessary during each session and
so on) We provide weekly feedback sessions and close
monitoring in order to discourage dropouts and increase
compliance with the intervention. Parents are asked to
complete a training evaluation scale following comple-
tion of Cogmed; this scale is an integrated component of
Cogmed that gathers information regarding the child’s
motivation and attention during the training as well as
parents’ feedback. As soon as a child finishes the inter-
vention, a blinded post-treatment assessment will be
scheduled to occur within the following weeks.
Control group: standard of care
Children randomly assigned to the control group will
receive the standard of care recommended for patients
with critical CHD. This includes cardiac surveillance
and, if needed, neurodevelopmental counselling and
screening at Boston Children’s Hospital Cardiac Neuro-
developmental Program. Once enrolled in the study,
a child in the control group will not receive Cogmed
intervention or any other cognitive intervention that
targets EFs or ADHD symptoms until after the 3-month
follow-up assessment is completed, that is, 5–6 months
after initial enrolment. Like children assigned to the
intervention group, children in the control group can
continue treatments that are already in place for other
neurodevelopmental disabilities (eg, speech therapy and
occupational services). For children in the control group,
post-treatment and 3-month follow-up assessments will
be performed 6–7 weeks and 4–5 months after baseline
assessment, respectively. After the study is completed,
children in the control group will be offered the possi-
bility of completing the Cogmed intervention at no cost.
Primary outcome measure
The NIH Toolbox55 is a set of computerised assess-
ments designed to measure outcomes in longitudinal or
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
6CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
intervention trials. This battery is particularly appropriate
for our study because it is presented in a computerised
child-friendly version, paralleling that of the Cogmed
intervention. The List Sorting Working Memory Test from
the NIH Toolbox is the primary outcome of the trial.
This standardised measure assesses the ability to process
information across a series of modalities (visual-spatial
and verbal), to hold this information in a short-term
buffer and to actively manipulate it mentally. It is consid-
ered an excellent composite indicator of children’s EF
skills, as it requires the simultaneous implementation
of control of attention and working memory abilities on
tasks of increasing complexity. Mean scores are automat-
ically computed and are compared with a standardisa-
tion sample of US children of the same age. Scores are
normally distributed (mean=100, SD=15) in the standard-
isation sample. The construct validity of the NIH Toolbox
working memory tasks is 0.58 for convergent validity and
0.30 for divergent validity. This test has a test–retest reli-
ability of 0.89 (95% CI 0.85 to 0.92).
Secondary outcome measures
NIH toolbox cognition battery
We include tests that measure cognitive flexibility, atten-
tion and inhibitory control, episodic memory, language
and processing speed. Mean scores on the following
tests will be our secondary outcomes: (1) Flanker Inhib-
itory Control and Attention Test, which measures a child’s
ability to control automatic response tendencies that may
interfere with achieving a goal; (2) Dimensional Change
Card Sort Test, which assesses a child’s capacity to switch
among multiple aspects of a task; (3) Picture Sequence
Memory Test, which measures a child’s ability to remember
the sequence of pictures shown on the screen; (4) Picture
Vocabulary Test and Oral Reading Recognition, which assess
receptive vocabulary and reading decoding skills; and (5)
Pattern Comparison Processing Speed Test, which assesses the
amount of time it takes a child to process a set amount
of information. All scores are standardised and normally
distributed (mean=100, SD=15) in the standardisation
sample. The test–retest reliability of these tests varies
between 0.82 and 0.96.55
Behaviour Rating Inventory of Executive Function, Second Edition
The BRIEF-2 is a standardised questionnaire that
measures children’s executive functioning in everyday
life. It includes nine scales: inhibit, self-monitor, shift,
emotional control, initiate, working memory, plan/orga-
nise, task-monitor and organisation of materials. Parent
and teacher versions of the BRIEF-2 will be included. We
will analyse the General Executive Composite T score
(mean=50, SD=10 for the standardisation sample) for
each version (Parent and Teacher), which integrates a
child’s scores on all of the clinical scales. The composite
indices of the BRIEF-2 have high internal consistency
(0.94–0.98 in the normative sample) and high test–retest
reliability (0.84–0.88 for parents over a 2-week interval;
0.90–0.92 for teachers over a 3.5-week interval).56
Conners, Third Edition
The Conners-3 is a questionnaire that assesses ADHD-re-
lated behaviours in children aged 3–17 years. We will
analyse mean T scores (mean=50, SD=10 in the stan-
dardisation sample) for the ADHD Inattentive and the
ADHD Hyperactive-Impulsive DSM-5 Symptom Scales as
well as the ADHD Index for both the Parent and Teacher
versions. For children aged 6–11 years, the Cronbach’s
alpha coefficients for scores on the scales range from 0.87
to 0.95 for both parent and teacher ratings, indicating
satisfactory internal consistency. Test–retest reliability for
the scales ranges from 0.67 to 0.72 for parents and 0.47–
0.80 for teachers.57
Social Responsiveness Scale, Second Edition
The SRS-2 questionnaire evaluates autism spectrum symp-
toms, including those relating to social awareness, social
cognition, communication, social motivation and autistic
traits, in individuals older than 2.5 years. We will analyse
T scores (mean=50, SD=10 in the standardisation sample)
from both versions (Parent and Teacher). Ratings show
good internal consistency and inter-rater reliability.58
The schedule of neurodevelopmental data collection is
presented in table 1.
Covariate measures
We will investigate cognitive, medical and sociodemo-
graphic patient-specific factors as predictors of response
to the intervention, at both post-treatment and 3-month
follow-up assessments. The following variables will be
investigated: baseline Full-Scale IQ scores and all subscales
on the Wechsler Intelligence Scale for Children, Fifth
Edition,59 and perinatal medical history, including birth
weight, gestational age, type of CHD, history of neuro-
logical abnormalities, number of open-heart surgeries,
intensive care unit length of stay and total number of
hospitalisations.
Data management and safety monitoring
Overall integration of the statistics, data management
and administrative functions of this trial occur in the
Department of Cardiology’s Research Support and Statis-
tics Core (RSSC) led by Drs Jane Newburger and David
Wypij. The RSSC provides the infrastructure necessary
to facilitate the conduct of this clinical trial including
biostatistical analysis, computerised data entry, data
base programming and development, data manage-
ment, quality control, assistance with manuscript prepa-
ration and administrative functions. The RSSC provides
a centralised resource for maintaining database. Study
documents are being stored in individual subject folders,
each folder containing a tracking page. All study mate-
rials are stored in a locked file cabinet accessible only to
authorised study staff. All study data are recorded on case
report forms and entered into a Research Electronic Data
Capture database.
A Data and Safety Monitoring Board (DSMB)
is composed of expert members in cardiology,
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
7
CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
neuropsychology and biostatistics. Members will be inde-
pendent of the study investigators and their Departments
at Boston Children’s Hospital as well as from the spon-
sors of this study. The function of the DSMB will be to
advise the funding sources, Boston Children’s Hospital
and the study investigators on: (1) final study designs
and protocols prior to the beginning of data collection,
(2) problems encountered protocol implementation,
(3) frequency of occurrence of adverse events and their
relation to study protocols, (4) withdrawals and losses to
follow-up, (5) data interpretation and ethical issues and
(6) recommendations arising from the study. The DSMB
chair will receive reports of any serious events that occur
in the conduct of the study. This trial has been consid-
ered as a non-significant risk device study and reviewed
accordingly by the Boston Children’s Hospital Institu-
tional Review Board and the Human Research Protection
Office, US Department of Defense.
A complete description of this trial’s data management
plan, safety monitoring board and risk/benefits assess-
ment is presented in online appendix 1.
Sample size and power considerations
Our specific aims are to determine whether there are
significant differences between the intervention and
control groups in the change in scores on the List Sorting
Working Memory Test between measurements at baseline
and post-treatment (specific aim 1) and between baseline
and 3-month follow-up (specific aim 2). Although this test
has a good test–retest reliability of ρ=0.87, to be conser-
vative, we will assume a value of ρ=0.70 between baseline
and post-treatment and between baseline and 3-month
follow-up on the same subject. Given a sample size of
100 subjects, ρ=0.70 for within-subject correlations, and
a two-sided type I error rate of 5%, we have 81.4% power
to detect a mean difference of 0.5 SD between treatment
groups, with a conservative 20% attrition rate (hence,
analysing a minimum of 80 subjects) in our primary
outcome measure. This corresponds to a mean differ-
ence of 7.5 units, given an expected SD of 15 for the List
Sorting Working Memory Test of the NIH Toolbox.
Among children who receive the Cogmed intervention,
we also seek to assess associations of cognitive, sociode-
mographic and medical factors with changes in the scores
for our primary outcome measure (specific aim 3). Given
a sample size of 50 children in the Cogmed group and a
two-sided type I error rate of 5%, we have 79.9% power
to detect a correlation of 0.43 (or R2=0.432 = 0.185 from
a linear regression) between a patient-specific factor and
the primary outcome variable even with a conservative
20% attrition rate (analysing a minimum of 40 subjects).
Data analysis plan
For specific aims 1 and 2, the efficacy of the intervention
will be evaluated by comparing within-subject differences
(baseline to post-treatment, baseline to 3-month follow-up
and, in secondary analyses, post-treatment to 3-month
follow-up) across treatment groups using an intention-to-
treat analysis.
Descriptive statistics will be calculated, including
means, SD, medians and IQRs for continuous variables
and frequency counts and percentages for categor-
ical variables. The primary outcome measure, the List
Sorting Working Memory Test of the NIH Toolbox and
most other study outcomes are continuous variables.
T-tests and linear regression will be used to assess differ-
ences between the intervention and control groups
for continuous outcomes (ie, differences in means,
95% CI). Proportions and logistic regression will be used
to examine group differences in binary outcomes (ie,
χ2 tests, ORs and 95% CI). We expect that randomisa-
tion will produce balance between treatment groups in
terms of demographic and baseline factors, but we will
use regression methods to adjust for any factors that
may be unbalanced. All analyses will be accompanied
by graphical exploration of the data to identify outlying
Table 1 Schedule of neurodevelopmental assessment data collection
Assessment Informant Baseline Post-treatment* Follow-up†
Primary outcome
NIH Toolbox List Sorting Working Memory Test Child X X X
Secondary outcomes
NIH Toolbox Cognition Battery Child X X X
Behavior Rating Inventory of Executive Function,
Second Edition
Parent X X X
Teacher X X X
Conners, Third Edition Parent X X X
Teacher X X X
Social Responsiveness Scale, Second Edition Parent X X X
Teacher X X X
*Post-treatment (one to 2 weeks after cessation of intervention and/or 6–7 weeks after baseline).
†Three-month follow-up (3 months after completion of the intervention and/or 4–5 months after baseline).
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
8CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
and influential observations. Data transformations and
non-parametric methods (eg, Wilcoxon rank-sum tests)
will be used as appropriate when parametric assumptions
are violated. Primary analyses of treatment group differ-
ences will focus on complete cases. In secondary analyses,
we will assume no change over time for subjects who do
not return for their post-treatment assessment (ie, last
value carried forward approach), but we will also carry
out other sensitivity analyses to assess the strength of our
findings based on other missing data assumptions.
For specific aim 3, we will explore the associa-
tions between patient factors and within-subject
differences (baseline to post-treatment, baseline
to 3-month follow-up) using correlation and linear
regression methods, including consideration of
possible confounding or effect modification. Specific
attention will be given to certain patient-specific risk
factors including age at first heart surgery (neonatal
vs non-neonatal), number of surgeries and neuro-
logical complications. Because we will be conducting
multiple analyses with several predictors and primary
and secondary outcomes in an exploratory fashion,
we will interpret results cautiously, based on signifi-
cance levels (p<0.05, two tailed) and on the magni-
tude of differences, correlations or regression effects.
As appropriate, we will also consider the use of other
statistical methods, such as generalised additive
models, partial and sparse partial least squares and
family-wise error rates, in our approach. Analyses will
be conducted primarily using SAS V.9.4, Stata V.15.1,
SPSS V.25 and R V.3.5.2.
ETHICS AND DISSEMINATION
Protocol modifications and amendments will be
submitted to the ethical committees for approval.
Amendments to the study protocol will be added to
publications reporting the study outcomes. This trial
has been registered with the American Clinical Trials
Registry. Prior to entering into the trial, all parents or
legal guardians and children will give written informed
consent or assent to participate. Online appendix
2 presents the study consent form. All information
will follow IRB and Human Protection guidelines for
confidentiality and data protection. The study results
will be disseminated through publications in scientific
journals, presentations at scientific conferences and
directly to the families who participated in the study.
Coinvestigators will be coauthors of the publications
describing trial outcomes, without the use of profes-
sional writers. Data will be provided on request.
Trial progress
The trial is currently in the active recruitment phase (first
baseline assessment February 2017). This is Protocol V.4,
10 July 2018. Substantial protocol amendments will be
communicated to investigators via email and to other
parties as required. All changes are submitted to Boston
Children’s Hospital’s IRB, to the Sponsor of this trial (US
Department of Defense) and updated in clinicaltrials. gov.
Discussion
This article presents the background and design of
an RCT investigating the efficacy of a 5-week working
memory intervention for children with CHD who under-
went open-heart surgery in infancy. This is the first study
to investigate the effects of a neurocognitive interven-
tion targeting EF in school-aged children with CHD. We
will evaluate children’s cognitive and social outcomes
including autism spectrum and ADHD symptoms.
Furthermore, the results from this trial will provide infor-
mation on potential patient-specific factors associated
with response to the intervention. As a first clinical trial,
we will test the efficacy of the intervention at 3 months
after the cessation of training. If the intervention is
proven effective at this time, longer term effects should
be investigated (eg, at 6-month or 12-month postinter-
vention). Assessment of longer term effects of working
memory training will provide key information about the
cost-efficacy of Cogmed in patients with CHD, the like-
lihood that lasting benefits generalise to other areas of
development and the duration of its benefits.
Executive dysfunction may have cascading adverse
effects on a myriad of domains ranging from specific
neurocognitive abilities to school achievement, social
adaptation and, ultimately, quality of life. Timely preven-
tion and treatment of these issues is a priority in the
care of patients with CHD. If proven effective, this type
of neurocognitive intervention could be implemented
in a clinical outpatient practice for patients at increased
neurodevelopmental risk.
Author afliations
1Boston Children’s Hospital, Department of Psychiatry, Boston, Massachusetts, USA
2Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
3Department of Cardiology, Boston Children’s Hospital, Boston, Massachusetts, USA
4Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston,
Massachusetts, USA
5Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA
Contributors JC, DCB and JWN are the primary investigators and, together with
DW, designed and established this research study. JC and JWN were responsible
for ethics applications and reporting. JC, DCB, CH and AL are responsible for
data collection and implementation of the intervention. JC will take lead roles on
preparation for publication of the clinical outcomes of the study. JWN, DCB, DW, CS,
CH and AL will contribute to the preparation of publications within their respective
elds of expertise. DW and CS will take on a lead role of the statistical analysis
for the study. JC drafted the nal version of this manuscript. All authors critically
reviewed and approved the nal version. All data from this study will be submitted
to peer-review journals and for presentation at national and international scientic
conferences.
Funding This research was funded by the US Department of Defense, Clinical
Trials awards (grant number W81XWH-16-1-0741).
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Full ethical approval for this study has been obtained by
the Boston Children’s Hospital’s Institutional Review Board (IRB) (IRB number
P00022440) and has also been reviewed and approved by the Human Protection
Agency from the US Department of Defense. All parent/guardians and children will
give written informed consent or assent to participate prior to entering into the trial.
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
9
CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
Provenance and peer review Not commissioned; externally peer reviewed.
Open access This is an open access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which
permits others to distribute, remix, adapt, build upon this work non-commercially,
and license their derivative works on different terms, provided the original work is
properly cited, appropriate credit is given, any changes made indicated, and the use
is non-commercial. See: http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.
REFERENCES
1. Fyler DC. Report of the New England Regional Infant Cardiac
Program. Pediatrics 1980;65:S375–61.
2. Loffredo CA. Epidemiology of cardiovascular malformations:
prevalence and risk factors. Am J Med Genet 2000;97:319–25.
3. Ferry PC. Neurologic sequelae of cardiac surgery in children. Am J
Dis Child 1987;141:309–12.
4. Marino BS, Lipkin PH, Newburger JW, et al. Neurodevelopmental
outcomes in children with congenital heart disease: evaluation
and management: a scientic statement from the American Heart
Association. Circulation 2012;126:1143–72.
5. Mahle WT, Tavani F, Zimmerman RA, et al. An MRI study of
neurological injury before and after congenital heart surgery.
Circulation 2002;106(Suppl 1):109–14.
6. Miller SP, McQuillen PS, Hamrick S, et al. Abnormal brain
development in newborns with congenital heart disease. N Engl J
Med 2007;357:1928–38.
7. Owen M, Shevell M, Majnemer A, et al. Abnormal brain structure
and function in newborns with complex congenital heart defects
before open heart surgery: a review of the evidence. J Child Neurol
2011;26:743–55.
8. Diamond A. Executive functions. Annu Rev Psychol 2013;64:135–68.
9. Blair C, Razza RP. Relating effortful control, executive function, and
false belief understanding to emerging math and literacy ability in
kindergarten. Child Dev 2007;78:647–63.
10. Diamond A, Barnett WS, Thomas J, et al. Preschool program
improves cognitive control. Science 2007;318:1387–8.
11. Bull R, Scerif G. Executive functioning as a predictor of children’s
mathematics ability: inhibition, switching, and working memory. Dev
Neuropsychol 2001;19:273–93.
12. Biederman J, Monuteaux MC, Doyle AE, et al. Impact of executive
function decits and attention-decit/hyperactivity disorder
(ADHD) on academic outcomes in children. J Consult Clin Psychol
2004;72:757–66.
13. Espy KA, McDiarmid MM, Cwik MF, et al. The contribution of
executive functions to emergent mathematic skills in preschool
children. Dev Neuropsychol 2004;26:465–86.
14. Diamond A. Normal development of prefrontal cortex from birth to
young adulthood: cognitive functions, anatomy, and biochemistry.
In: Stuss D, Knight RT, eds. Principles of Frontal Lobe Function. New
Yord, NY: Oxford University Press, 2002:466–503.
15. Barkley RA. Distinguishing sluggish cognitive tempo from ADHD in
children and adolescents: executive functioning, impairment, and
comorbidity. J Clin Child Adolesc Psychol 2013;42:161–73.
16. Lai M-C, Lombardo MV, Baron-Cohen S. Autism. The Lancet
2014;383:896–910.
17. Pellicano E. Individual differences in executive function and central
coherence predict developmental changes in theory of mind in
autism. Dev Psychol 2010;46:530–44.
18. Baler RD, Volkow ND. Drug addiction: the neurobiology of disrupted
self-control. Trends Mol Med 2006;12:559–66.
19. Miller AL, Lee HJ, Lumeng JC. Obesity-associated biomarkers and
executive function in children. Pediatr Res 2015;77(1-2):143–7.
20. Levy S, Katusic SK, Colligan RC, et al. Childhood ADHD and risk
for substance dependence in adulthood: a longitudinal, population-
based study. PLoS One 2014;9:e105640.
21. Calderon J, Jambaqué I, Bonnet D, et al. Executive functions
development in 5- to 7-year-old children with transposition
of the great arteries: a longitudinal study. Dev Neuropsychol
2014;39:365–84.
22. Bellinger DC, Wypij D, Rivkin MJ, et al. Adolescents with
d-transposition of the great arteries corrected with the arterial switch
procedure: neuropsychological assessment and structural brain
imaging. Circulation 2011;124:1361–9.
23. Calderon J, Bonnet D, Courtin C, et al. Executive function and theory
of mind in school-aged children after neonatal corrective cardiac
surgery for transposition of the great arteries. Dev Med Child Neurol
2010;52:1139–44.
24. Calderon J, Angeard N, Moutier S, et al. Impact of prenatal diagnosis
on neurocognitive outcomes in children with transposition of the
great arteries. J Pediatr 2012;161:94–8.
25. Bellinger DC, Wypij D, duPlessis AJ, et al. Neurodevelopmental
status at eight years in children with dextro-transposition of the great
arteries: the Boston Circulatory Arrest Trial. J Thorac Cardiovasc
Surg 2003;126:1385–96.
26. Bellinger DC, Rivkin MJ, DeMaso D, et al. Adolescents with tetralogy
of Fallot: neuropsychological assessment and structural brain
imaging. Cardiol Young 2015;25:338–47.
27. Miatton M, De Wolf D, François K, et al. Neuropsychological
performance in school-aged children with surgically corrected
congenital heart disease. J Pediatr 2007;151:73–8.
28. Hövels-Gürich HH, Konrad K, Skorzenski D, et al. Attentional
dysfunction in children after corrective cardiac surgery in infancy.
Ann Thorac Surg 2007;83:1425–30.
29. Gaynor JW, Gerdes M, Nord AS, et al. Is cardiac diagnosis a
predictor of neurodevelopmental outcome after cardiac surgery in
infancy? J Thorac Cardiovasc Surg 2010;140:1230–7.
30. Cassidy AR, White MT, DeMaso DR, et al. Executive function in
children and adolescents with critical cyanotic congenital heart
disease. J Int Neuropsychol Soc 2015;21:34–49.
31. Neal AE, Stopp C, Wypij D, et al. Predictors of health-related
quality of life in adolescents with tetralogy of Fallot. J Pediatr
2015;166:132–8.
32. Calderon J, Bellinger DC. Executive function decits in congenital
heart disease: why is intervention important? Cardiol Young
2015;25:1238–46.
33. Shillingford AJ, Glanzman MM, Ittenbach RF, et al. Inattention,
hyperactivity, and school performance in a population of school-
age children with complex congenital heart disease. Pediatrics
2008;121:e759–67.
34. Hansen E, Poole TA, Nguyen V, et al. Prevalence of ADHD symptoms
in patients with congenital heart disease. Pediatr Int 2012;54:838–43.
35. DeMaso DR, Labella M, Taylor GA, et al. Psychiatric disorders and
function in adolescents with d-transposition of the great arteries. J
Pediatr 2014;165:760–6.
36. Newburger JW, Sleeper LA, Bellinger DC, et al. Early developmental
outcome in children with hypoplastic left heart syndrome and related
anomalies: the single ventricle reconstruction trial. Circulation
2012;125:2081–91.
37. Yamada DC, Porter AA, Conway JL, et al. Early repair of congenital
heart disease associated with increased rate of attention decit
hyperactivity disorder symptoms. Can J Cardiol 2013;29:1623–8.
38. Sistino JJ, Atz AM, Simpson KN, et al. The prevalence of attention-
decit/hyperactivity disorder following neonatal aortic arch repair.
Cardiol Young 2015;25:663–9.
39. von Rhein M, Buchmann A, Hagmann C, et al. Brain volumes predict
neurodevelopment in adolescents after surgery for congenital heart
disease. Brain 2014;137(Pt 1):268–76.
40. Schaefer C, von Rhein M, Knirsch W, et al. Neurodevelopmental
outcome, psychological adjustment, and quality of life in
adolescents with congenital heart disease. Dev Med Child Neurol
2013;55:1143–9.
41. Klingberg T, Fernell E, Olesen PJ, et al. Computerized training of
working memory in children with ADHD–a randomized, controlled
trial. J Am Acad Child Adolesc Psychiatry 2005;44:177–86.
42. Klingberg T, Forssberg H, Westerberg H. Training of working memory
in children with ADHD. J Clin Exp Neuropsychol 2002;24:781–91.
43. Holmes J, Gathercole SE, Dunning DL. Adaptive training leads to
sustained enhancement of poor working memory in children. Dev Sci
2009;12:F9–F15.
44. Holmes J, Gathercole SE. Taking working memory training from the
laboratory into schools. Educ Psychol 2014;34:440–50.
45. Westerberg H, Klingberg T. Changes in cortical activity after training
of working memory–a single-subject analysis. Physiol Behav
2007;92(1-2):186–92.
46. Brehmer Y, Westerberg H, Bäckman L. Working-memory training in
younger and older adults: training gains, transfer, and maintenance.
Front Hum Neurosci 2012;6:63.
47. Beck SJ, Hanson CA, Puffenberger SS, et al. A controlled trial of
working memory training for children and adolescents with ADHD. J
Clin Child Adolesc Psychol 2010;39:825–36.
48. Gropper RJ, Gotlieb H, Kronitz R, et al. Working memory training in
college students with ADHD or LD. J Atten Disord 2014;18:331–45.
49. Johansson B, Tornmalm M. Working memory training for patients
with acquired brain injury: effects in daily life. Scand J Occup Ther
2012;19:176–83.
50. Lundqvist A, Grundström K, Samuelsson K, et al. Computerized
training of working memory in a group of patients suffering from
acquired brain injury. Brain Inj 2010;24:1173–83.
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
10 CalderonJ, etal. BMJ Open 2019;9:e023304. doi:10.1136/bmjopen-2018-023304
Open access
51. Bergman Nutley S, Söderqvist S, Bryde S, et al. Gains
in uid intelligence after training non-verbal reasoning in
4-year-old children: a controlled, randomized study. Dev Sci
2011;14:591–601.
52. Thorell LB, Lindqvist S, Bergman Nutley S, et al. Training and
transfer effects of executive functions in preschool children. Dev Sci
2009;12:106–13.
53. Grunewaldt KH, Løhaugen GC, Austeng D, et al. Working memory
training improves cognitive function in VLBW preschoolers.
Pediatrics 2013;131:e747–54.
54. Løhaugen GC, Antonsen I, Håberg A, et al. Computerized working
memory training improves function in adolescents born at extremely
low birth weight. J Pediatr 2011;158:555–61.
55. Zelazo PD, Anderson JE, Richler J, et al. II. NIH Toolbox Cognition
Battery (CB): measuring executive function and attention. Monogr
Soc Res Child Dev 2013;78:16–33.
56. Baron IS. Behavior rating inventory of executive function. Child
Neuropsychol 2000;6:235–8.
57. Conners CK. Manual for the Conners Rating Scales - Revised. North
Tonawanda, NY: MHS Assessments, 1997.
58. Constantino JN, Davis SA, Todd RD, et al. Validation of a brief
quantitative measure of autistic traits: comparison of the social
responsiveness scale with the autism diagnostic interview-revised. J
Autism Dev Disord 2003;33:427–33.
59. Wechsler D. Wechlser Intelligence Scale for Children. 5th edn:
Psychological Corporation, Inc, 2014.
on 6 February 2019 by guest. Protected by copyright.http://bmjopen.bmj.com/BMJ Open: first published as 10.1136/bmjopen-2018-023304 on 5 February 2019. Downloaded from
