Content uploaded by Lyane Trépanier
Author content
All content in this area was uploaded by Lyane Trépanier on Aug 17, 2017
Content may be subject to copyright.
Assessing the Quality of the OPQ’s Guidelines for the Evaluation of
Dyslexia in Children Using the Appraisal of Guidelines for Research and
Evaluation (AGREE) II: A Brief Report
Lyane Trepanier, Constantina Stamoulos, and Andrea Reyes
McGill University
Dyslexia is a complex neurologically based learning disorder requiring the skillful intervention of a
licensed psychologist for assessment, diagnosis, and intervention. As for any complex activity that may
involve risks for the patient, there is value in having guidelines to inform practitioners about the best
possible procedures when offering services to individuals with this condition. Practice guidelines are
indeed a useful resource for mental health professionals but only to the degree that they are developed
with the most rigorous standards. The Order of Psychologists of Quebec published the Guidelines for the
Evaluation of Dyslexia in Children (Ordre des psychologues du Quebec, 2014) to provide guidance for
psychologists in the assessment of dyslexia. The aim of this paper was to evaluate the quality of this
guideline using the Appraisal of Guidelines for Research and Evaluation II guideline evaluation
instrument (AGREE, 2009). Results show that this guideline was developed with important method-
ological flaws. Suggested improvements in the guideline development methodology using the Appraisal
of Guidelines for Research and Evaluation II approach are discussed.
Keywords: guidelines, guideline development, dyslexia
Children with dyslexia have difficulties processing language in
the areas of reading, writing, and spelling (Lyon, Shaywitz, &
Shaywitz, 2003). The Diagnostic and Statistical Manual of Mental
Disorders (5th ed.; American Psychiatric Association, 2013) clas-
sifies dyslexia as a specific learning disability and considers it to
be a type of neurodevelopmental disorder. The Diagnostic and
Statistical Manual of Mental Disorders, fifth edition, reports that
the prevalence of dyslexia in school-age children is between 3%
and 5%, whereas other studies found the prevalence to be between
5% and 15% (Daniel et al., 2006). It is estimated that 80 –90% of
learning difficulties present themselves in the school system
(Leach, Scarborough, & Rescorla, 2003;Lerner, 1989;Lyon et al.,
2001). Psychologists in both school and clinical settings may
benefit from high-quality professional practice guidelines because
they are expected to provide scientifically based recommendations
to guide service delivery.
The Quebec Order of Psychologists (OPQ), a prolific producer
of practice guidelines, published the Guidelines for the evaluation
of dyslexia in children (Lignes directrices pour l’évaluation de la
dyslexie chez les enfants) in 2014. The production of this guideline
began in 2009 with the participation of multiple experts. This
48-page document provides background information on dyslexia
and describes so-called best practices for the evaluation of the
condition, including the administration of tests. The guideline’s
stated objectives are to do the following: (a) to encourage rigorous
methods that reflect the most up-to-date research; (b) to propose a
standardized methodology based on diagnostic consensus; and (c)
to identify the specific needs of children affected by dyslexia for
support and accommodation (Ordre des psychologues du Quebec
[OPQ], 2014).
Although such objectives are both important and laudable, the
availability of practice guidelines alone does not automatically
lead to improved practices or health outcomes (Bergman, 1999;
Cabana et al., 1999;Sackett, Rosenberg, Gray, Haynes, &
Richardson, 1996;Ward & Grieco, 1996). The potential benefit of
a guideline is only as good as the quality of the guideline itself
(Burgers, Cluzeau, Hanna, Hunt, & Grol, 2003;Gordon & Cooper,
2010), and appropriate methodologies and rigorous strategies in
the guideline development process are important for the successful
dissemination and implementation of the clinical recommenda-
tions contained in the guideline (Alonso-Coello et al., 2010;Ansari
& Rashidian, 2012;Blozik et al., 2012;Cahill & Heyland, 2010;
Davis & Taylor-Vaisey, 1997;Grol, 2001;Norris, Holmer, Ogden,
& Burda, 2011;Steinert, Richter, & Bergk, 2010). Unfortunately,
in recent years, there has been a large volume of guidelines
published in which the methodologies used to design the guide-
lines were inadequately described within the guideline or in related
documents made available to professionals and service users (e.g.,
Rosenfeld, Shiffman, & Robertson, 2013;Stamoulos, Reyes,
Trepanier, & Drapeau, 2014). For example, a description of the
methods used to gather evidence and formulate recommendations,
and whether the guideline was externally reviewed by experts prior
to its publication, are considered minimal standards for the pro-
Lyane Trepanier, Constantina Stamoulos, and Andrea Reyes, Depart-
ment of Counseling Psychology, McGill University.
This project was supported by Social Sciences and Humanities Research
Council doctoral awards to Lyane Trepanier, Constantina Stamoulos, and
Andrea Reyes.
Correspondence concerning this article should be addressed to Lyane
Trepanier, Department of Counseling Psychology, McGill Psychotherapy
Process Research Group, McGill University, 3700 McTavish, Montreal,
Quebec H3A 1Y2, Canada. E-mail: lyane.trepanier@mail.mcgill.ca
Canadian Psychology / Psychologie canadienne © 2017 Canadian Psychological Association
2017, Vol. 58, No. 3, 211–217 0708-5591/17/$12.00 http://dx.doi.org/10.1037/cap0000121
211
duction of quality guidelines (Brouwers et al., 2010;Burgers,
Cluzeau, Hanna, Hunt, & Grol, 2003).
The quality of practice guidelines has indeed been closely
scrutinized over the past 15 years, especially in the United States
and Europe, with reports of high variability in quality, many of
which were cited as poor (Graham, Beardall, Carter, Tetroe, &
Davies, 2003;Ruszczy´
nski, Horvath, Dziechciarz, & Szajewska,
2016;Shaneyfelt, Mayo-Smith, & Rothwangl, 1999;Stamoulos,
Reyes, Trepanier, & Drapeau, 2014). These studies raise concerns
about the quality of the guidelines that are available to practitio-
ners, about the value of the advice and recommendations they
contain, and ultimately about the quality of the services received
by patients and clients. In light of this, studies that assess the
quality and usability of guidelines available to Canadian psychol-
ogists are needed. This brief report presents the findings from our
evaluation of the Guidelines for the evaluation of dyslexia in
children (OPQ, 2014).
Method
Although different scales exist to assess the quality of a guide-
line, the most widely used is the Appraisal of Guidelines for
Research and Evaluation (AGREE) instrument. This scale was
developed by scientists and clinicians from diverse health disci-
plines and geographical regions who formed an official indepen-
dent body, named the AGREE Research Trust. The first version of
the AGREE instrument was published in 2003 (AGREE Collabo-
ration, 2003). Thereafter the AGREE items were further refined,
and a new item was added to the newer AGREE II in 2009
(AGREE, 2009; see www.agreetrust.org). The AGREE II evalu-
ates the process of guideline development. Because it focuses
solely on the method used to develop a guideline, it can be applied
by trained raters to any guideline, regardless of the topic of that
guideline and without the need for raters to have expertise in the
area covered by the guideline. It contains 23 items grouped into six
quality domains (see Table 1 for all items): (a) scope and purpose
of the guideline, which refers to the overall aim of the guideline,
the specific health questions it addresses, and the target population
to whom it is dedicated; (b) stakeholder involvement, which fo-
cuses on the extent to which the guideline was developed by the
appropriate stakeholders and represents the views of the intended
users; (c) rigor of development, which refers to the process of
development of the guideline and to the methodology used; (d)
clarity of presentation, which includes items that evaluate the
language, structure, and format of the guideline, different manage-
ment options, and clearly identifiable recommendations; (e) appli-
cability, which generally refers to ease of use of the guideline; and
(f) editorial independence of the authors, which is concerned that
the formulation of the recommendation is not influenced by com-
peting interests. The AGREE II is widely used internationally and
is considered to be the gold standard for quality assessment of
practice guidelines in the medical and human services professions
(Alonso-Coello et al., 2010;Brouwers et al., 2010).
For this study, three trained raters independently applied the
AGREE II instrument to the Guidelines for the evaluation of
dyslexia in children (OPQ, 2014). The raters (L. T., C. S., A. R.)
had all received intensive training in using the AGREE Scale as
part of another large-scale study of guidelines; all raters in the
present study had demonstrated reliability in using the scale. The
raters scored the guideline against the 23 AGREE II items on a
7-point Likert scale from 1 (strongly disagree)to7(strongly
agree). Scores were entered directly into the rater’s account on the
agreetrust website (www.agreetrust.org/agree-ii), a section in-
tended to manage guideline evaluation projects. A space for com-
ments is available for each item, allowing the rater to provide a
rationale for assigning a particular score. Using this web platform,
the scores were also automatically totaled and grouped into the six
quality domains described above.
The three raters rated the guideline independently and then met
for a consensus session to agree on a score for each item. In what
follows we report the domain scores generated by the agreetrust
website, which is given as a percentage score; these domain scores
are calculated by summing up all the scores of the individual items
in a domain and by scaling the total as a percentage of the
maximum possible for that domain (see AGREE, 2009). Whereas
the AGREE handbook does not indicate what the minimum score
on a given domain should be to determine whether a guideline is
of sufficient quality, some authors have proposed using a cutoff
score of 60% (e.g., Barriocanal, López, Monreal, & Montané,
2016;Chang et al., 2016). In addition to this domain score, for
reasons of transparency and to facilitate comparisons with other
studies, we also report mean scores for each item across the three
raters as well as the rating that was obtained for each item follow-
ing the consensus sessions. A mean consensus score was also
calculated for each domain using the consensus scores for each
item.
Results
Interrater reliability on the individual AGREE items was excel-
lent with a mean intraclass coefficient of .92 (SD ⫽.025; with a
range, for each pair of raters between .90 and .95).
For Domain 1 (scope and purpose), which assesses the overall
aim of the guideline (see Table 1), all raters agreed that the scope
and purpose of the guideline were well explained, as reflected in
all scores: the mean consensus score for this domain (7 on 7) and
the AGREE II Domain score (100%).
Domain 2 (stakeholder involvement) is concerned with the
extent to which the guideline was developed by stakeholders and
represents the views of the target population. The target population
is comprised of the intended users of the guideline such as prac-
titioners and patient/client population. The mean group consensus
score for this domain was 4.7 of 7; the domain quality score was
59%. Although the target groups were clearly defined and relevant
professional groups were consulted, points were removed for not
having sought the views and preferences of the services users or of
other professionals involved in this area of practice.
Domain 3 (rigor of development) focuses on the processes used
to gather and synthesize the evidence and the methods to formulate
and update the recommendations. Major weaknesses were identi-
fied for this domain. Whereas the guideline states that systematic
methods were used to search for the evidence, it does not detail the
criteria for selecting the evidence, the strengths and limitations of
the body of evidence, the methods used for formulating the rec-
ommendations, or a procedure for updating the guideline. This
domain received a mean AGREE quality score of 51% and a mean
consensus score of 3.4 of 7. The consensus discussion resulted in
lowered scores for items 8, 12, and 13. Item 8 was lowered from
212 TREPANIER, STAMOULOS, AND REYES
a mean of 4 to a consensus score of 2. It was agreed that although
the method used for the literature search was indicated, the criteria
for selecting the final recommendations upon completion of the
research were completely absent. The consensus discussion re-
sulted in the lowering of item 12 from a mean of 7 to a consensus
score of 6; it was noted that although most recommendations in the
guideline were presented as supported by scientific studies, many
were not. Item 13 was also lowered from a mean of 6 to a score of
4 because the external guideline reviewers (experts) were not truly
external and were instead part of another committee associated
with the guideline.
Domain 4 (clarity of presentation) relates to language, struc-
ture, and format of the guideline. Domain 4 received one of the
higher ratings, although item scores for this domain remain
moderate, with a quality AGREE Domain score of 65% and a
mean consensus score of 5 of 7. This score would have been
higher if the recommendations for the management of the
condition had been further elaborated in the guideline by giv-
ing, for example, professionals various options or strategies to
manage or treat the condition. Specifically, item 16, “the dif-
ferent options for management of the condition are clearly
presented,” received a very low score, with a consensus score
among raters of 1 of 7.
Domain 5 (applicability) examines the guideline’s treatment of
possible barriers and facilitators to implementation, strategies to
improve uptake, and resource implications of applying the guide-
line. The applicability domain scored the lowest among all the
domains with a quality domain score of 28% and a mean consen-
sus core of 2.3 of 7. The consensus meeting led to further down-
grading for this domain because more discussions revealed that
few of the criteria were met. Raters indicated that the barriers and
facilitators to implementing this guideline were not addressed. The
Table 1
Mean and Consensus Scores for Dyslexia Guideline Using AGREE II
Domain groups Items AGREE II description
Mean
score
a
Group
consensus
b
Mean consensus
by domain
AGREE II
domain
score
1 The overall objectives are specifically described. 7 7 7 100%
1. Scope and purpose 2 The health questions covered are specifically
described.
77
3 The population to whom the guideline is meant to
apply is specifically described.
77
4 The development group includes individuals from all
relevant professional groups.
6 6 4.7 59%
2. Stakeholder
involvement
5 The views and preferences of the target population
were sought.
11
6 The target users are clearly defined. 7 7
7 Systematic methods were used to search for
evidence.
6 6 3.4 51%
3. Rigor of development 8 The criteria for selecting the evidence are clearly
described.
42
9 The strengths and limitations of the body of
evidence are clearly described.
11
10 The method for formulating the recommendations
are clearly described.
33
11 The health benefits, side effects, and risks have been
considered in formulating the recommendations.
34
12 The is an explicit link between the recommendations
and the supporting evidence.
76
13 The guideline has been externally reviewed by
experts prior to its publication.
64
14 A procedure for updating the guideline is provided. 1 1
15 The recommendations are specific and unambiguous. 6 7 5 65%
4. Clarity of presentation 16 The different options for management of the
condition or health issue are clearly presented.
21
17 Key recommendations are easily identifiable. 7 7
5. Applicability 18 The guideline describes facilitators and barriers to its
application.
1 1 2.3 28%
19 The guideline provides advice and/or tools on how
the recommendations can be put into practice.
66
20 The potential resource implications of applying the
recommendations have been considered.
11
21 The guideline presents monitoring and/or auditing
criteria.
21
6. Editorial
independence
22 The views of the funding body have not influenced
the content of the guideline.
2 2 4.5 56%
23 Competing interests of the development group
members have been recorded and addressed.
67
a
Average of independently scored items.
b
Single consensus score from discussion between raters.
213
EVALUATION OF GUIDELINE FOR DYSLEXIA
potential costs of implementation and for whom were also not
explained.
Domain 6 (editorial independence) is concerned with the for-
mulation of recommendations not being unduly biased with com-
peting interests. This domain was rated relatively low (56% and
4.5 of 7). The first of the two items in this domain aims to
determine whether the funding body influenced the recommenda-
tions of the guideline. This item received a score of 2 of 7 for both
the mean and the consensus score. It was concluded that although
one of the professionals involved in developing the guideline
received funding from a particular external organization, it was not
explicitly stated that the funding body did not have an influence on
the guideline’s recommendations, which would have been neces-
sary for full points. In addition, and perhaps more importantly, the
guideline was also funded by the OPQ, the OPQ oversaw the
guideline project, and the guideline committee included OPQ
employees, which raises some concern about editorial indepen-
dence. The second item in this domain received full points because
it stated that competing interests of the development group mem-
bers had been recorded.
Discussion
The Guidelines for the evaluation of dyslexia in children scored
the highest in Domain 1, scope and purpose, followed by Domain
4, clarity of presentation. Lower scores, all below the 60% cutoff,
were obtained for the other domains, in particular for domains 3,
rigor of development, and 5, applicability. In what follows, the
domains are discussed in the order of highest to lowest rating score
received.
Domain 1 examines whether the overall rationale of the guide-
line is well described, including the health questions, expected
benefits, and the target population. Guidelines typically score high
in this domain (Berrigan, Marshall, McCullagh, Velikonja, &
Bayley, 2011;MacQueen et al., 2017;Ye, Liu, Cui, & Liu, 2016),
so it is no surprise that this guideline on dyslexia received a high
score for the description of its scope and purpose. This domain
represents one of the most significant strengths of many guide-
lines, and this guideline on dyslexia is no exception to that.
Domain 4, clarity of presentation, is also one of the highest
scoring domains in the literature on guideline quality (Berrigan et
al., 2011;MacQueen et al., 2017;Ye et al., 2016), and this
guideline on dyslexia somewhat mirrors these findings. This do-
main had strong consensus among the raters for the scores regard-
ing the clarity and ease of use of the guideline. The appendices
were also noted as an added benefit to the guideline. With scores
of 65% and of 5 of 7, there is, however, room for improvement in
this area.
The two domains discussed above represent the guideline’s
stronger areas, although improvements would be needed in
some of these. The remaining domains, however, fall short of
meeting guideline standards. Domain 2, stakeholder involve-
ment, received a score of 59%. Each member of the guideline
development group stated their name, professional designation,
and their affiliation. This is important because it ensures greater
transparency about those involved in making the recommenda-
tions that professionals are encouraged to follow. Domain 2
also requires that the target population be consulted. In this
context the target population includes the practitioners who are
expected to use the guidelines and the patient/client population.
Although the development group did include at least one school
psychologist, clinical psychologist, and neuropsychologist, it
did not include speech and language pathologists, educators, or
school administrators or other professionals at any time during
the development process. In light of prevailing tendency toward
multidisciplinary work in mental health, the absence of these
additional contributions may have negative implications for
practice. Such a homogeneous group of professionals may
indeed lead to certain oversights. Although it is unclear whether
the absence of the other professionals was inadvertent or inten-
tional, the focus in this guideline is exclusively on the services
provided by psychologists, rather than all relevant stakeholders.
Additionally, the fact that service users were not involved in the
guideline development process suggests that their contribution
may be undervalued despite their hands-on experience of the
condition. With the increased recognition of patient-centered
care, methods that actively involve the patient in his or her own
treatment, and participatory health care (e.g., Del Campo, Gra-
cia, Blasco, & Andrasdas, 2011;Montori, Brito, & Murad,
2013;Rashid, Thomas, Shaw, & Leng, 2016), it is surprising
that service users (or their parents or guardians) were excluded
from these guidelines on the assessment of dyslexia.
Domain 6 refers to editorial independence. Whereas the guide-
line indicated that the competing interests of the professionals
involved in developing the document had been recorded (item 23),
the other item in this domain (item 22) received a very low score.
This was due to the fact that the OPQ, which had more than one
of its own representatives on the guideline committee, funded the
guideline, oversaw its development, and then approved and dis-
seminated the final product, which raises some concern about
editorial independence. It is not stated whether the committee
members received an honorarium from the OPQ or from another
source. It is also unclear whether the consultants and experts
involved in developing the guideline had sufficient independence
to express diverging opinions; indeed, the final document does not
include or make reference to a minority report (i.e., documentation
of the perspectives of all experts involved in developing the
guideline, including those that were not retained in the final
guideline), which is often recommended as a means of increasing
transparency and independence (e.g., Beauchamp, Drapeau, &
Dionne, 2015).
Domain 3, which refers to rigor of development of a guide-
line, received particularly low scores. This domain includes
eight items (see Table 1). The raters gave a high score for the
systematic search for evidence (item 7); however, the criteria
for selecting the evidence (item 8) were not clearly indicated,
and the strengths and limitations of the body of evidence (item
9) were completely unaddressed. Although the guideline may
lead readers to believe that a solid search and analysis of the
literature was conducted, the procedures that were used here
remain very unclear, and the literature search and process for
assessing the literature cannot be replicated; greater transpar-
ency should normally be expected in the reporting of guideline
development procedures. It was also noted that the criteria for
updating the guideline were not well defined. Our results sug-
gest that the OPQ will need to improve the quality of its
guideline development procedures with regard to Domain 3. Of
greater concern for practitioners and service users is that this
214 TREPANIER, STAMOULOS, AND REYES
guideline is not the exception. Indeed, the rigor of development
domain is frequently one of the lower scoring AGREE II
domains in the literature (Berrigan et al., 2011;MacQueen et
al., 2017;Ye et al., 2016).
Domain 5 received the lowest scores in this guideline. Ap-
plicability is rated on whether the guideline describes the fa-
cilitators and barriers to its application, whether it provides
advice or tools on how the recommendations can be put into
practice, and whether the guideline offers monitoring or audit-
ing criteria. Many of these issues were not addressed in the
guideline. There was no mention of facilitators and barriers
related to the application of the recommendations and resources
it offered. The guideline did, however, provide relevant back-
ground information on the etiology of dyslexia, tools, and
recommended steps and assessment considerations for the eval-
uation of the condition. Although it may be challenging and
costly to develop a guideline with an exhaustive list of potential
diagnostic profiles, to improve this guideline, practitioners may
find it helpful to have examples of the most typical profiles of
children with dyslexia. Perhaps the greatest shortcoming here is
with regard to auditing. The development of practice guidelines
requires important investments in time and money; as such, it
would be important to put means in place to assess the impact
that a guideline has on practices.
Conclusion
Guidelines are an important tool to educate practitioners and
service users and to improve practices; they are without doubt one
of the best ways of bridging the gap between science and practice.
A well-designed guideline will involve a systematic search and
synthesis of the scientific literature and an appropriate translation
of that science into clinical recommendations. The findings of this
study show that there is room for improvement with regard to the
rigor and transparency with which the scientific evidence is as-
sessed and the need for recommendations to be accompanied by
supporting references.
More research is needed, especially in psychology, mental
health, and social care, to evaluate the guidelines that are available
to professionals and to ascertain the quality of the recommenda-
tions that they are expected to follow. Until that is done, it is
unclear to what extent organizations that develop guidelines for
psychologists are promoting practices that are optimal and sup-
ported by science. Psychologists often claim to be evidence based
and well versed in science and methodology (Drapeau & Hunsley,
2014;Murdoch, Gregory, & Eggleton, 2015;Spilka & Dobson,
2015); this needs to be reflected in both our clinical work and the
methods we use to develop the guidelines that aim to inform
colleagues, other professionals, and service users about best clin-
ical practices.
It is commendable that the OPQ has made significant efforts
to provide its practitioners with guidelines. It is without doubt
the psychology organization that has produced the most guide-
lines in Canada, and it would be important for other provincial
or national bodies in Canada to embark on similar initiatives.
More guidelines that are solidly grounded in science and are
developed using rigorous methodologies are indeed needed to
bridge the gap between science and practice and to improve
practices and outcomes. Quality guidelines produced by psy-
chologists would also serve to maintain the credibility of psy-
chologists as scientist-practitioners who have an expertise in
professional practice as well as in science. In fact, given their
training, doctoral-level psychologists are in an ideal position to
both design high-quality guidelines for themselves and to con-
tribute to the development of guidelines for other professionals
as part of multidisciplinary groups. Such expertise would be
ideally suited not only to design good guidelines but also to
monitor and assess the effects in service delivery.
Résumé
La dyslexie est un trouble d’apprentissage d’origine neurologique
complexe nécessitant l’intervention d’un psychologue agréé pour
ce qui est de l’évaluation, du diagnostic et de l’intervention.
Comme pour toute activité complexe pouvant comporter des ris-
ques pour le patient, il est utile d’avoir des lignes directrices pour
informer les praticiens des meilleures procédures qui soient
lorsqu’ils offrent des services aux individus souffrant de ce trou-
ble. Des lignes directrices de pratique sont en effet une ressource
utile pour les professionnels de la santé mentale mais seulement
dans la mesure où elles sont élaborées dans les normes les plus
strictes. L’Ordre des psychologues du Québec a publié les Lignes
directrices pour l’évaluation de la dyslexie chez les enfants (Ordre
des psychologues du Québec, 2014) afin de fournir des directives
Aux psychologues dans l’évaluation de la dyslexie. L’objectif de
cet article était d’évaluer la qualité de ces recommandations a
`
l’aide du Appraisal of Guidelines for Research and Evaluation II
guideline evaluation instrument [Grille d’évaluation de la qualité
des recommandations pour la pratique clinique II] (AGREE,
2009). Les résultats montrent que cette directive comprend de
nombreuses failles méthodologiques. Quelques améliorations pro-
posées a
`l’élaboration de la méthodologie a
`l’aide de l’approche
Appraisal of Guidelines for Research and Evaluation II guideline
evaluation instrument y sont discutées.
Mots-clés : lignes directrices, élaboration de recommandations,
dyslexie.
References
AGREE Collaboration. (2003). Development and validation of an interna-
tional appraisal instrument for assessing the quality of clinical practice
guidelines: The AGREE project. Quality & Safety in Health Care, 12,
18 –23. http://dx.doi.org/10.1136/qhc.12.1.18
AGREE II. (2009). The AGREE Next Steps Consortium. Appraisal of
Guidelines for Research Evaluation II [Online]. Retrieved from http://
www.agreetrust.org/wp-content/uploads/2013/10/AGREE-II-Users-
Manual-and-23-item-Instrument_2009_UPDATE_2013.pdf
Alonso-Coello, P., Irfan, A., Sola
`, I., Gich, I., Delgado-Noguera, M.,
Rigau, D.,...Schunemann, H. (2010). The quality of clinical practice
guidelines over the last two decades: A systematic review of guideline
appraisal studies. Quality & Safety in Health Care, 19, e58.
American Psychiatric Association. (2013). Diagnostic and statistical man-
ual of mental disorders: DSM-5. Washington, DC: Author.
Ansari, S., & Rashidian, A. (2012). Guidelines for guidelines: Are they up
to the task? A comparative assessment of clinical practice guideline
development handbooks. PLoS ONE, 7, e49864. http://dx.doi.org/10
.1371/journal.pone.0049864
Barriocanal, A. M., López, A., Monreal, M., & Montané, E. (2016).
Quality assessment of peripheral artery disease clinical guidelines. Jour-
215
EVALUATION OF GUIDELINE FOR DYSLEXIA
nal of Vascular Surgery, 63, 1091–1098. http://dx.doi.org/10.1016/j.jvs
.2015.12.040
Beauchamp, S., Drapeau, M., & Dionne, C. (2015). The development of
practice guidelines in the social and human sciences. Canadian Psychol-
ogy, 56, 357–367. http://dx.doi.org/10.1037/a0038811
Bergman, D. A. (1999). Evidence-based guidelines and critical pathways
for quality improvement. Pediatrics, 103(Suppl. E1), 225–232.
Berrigan, L., Marshall, S., McCullagh, S., Velikonja, D., & Bayley, M.
(2011). Quality of clinical practice guidelines for persons who have
sustained mild traumatic brain injury. Brain Injury, 25(7– 8), 742–751.
http://dx.doi.org/10.3109/02699052.2011.580317
Blozik, E., Nothacker, M., Bunk, T., Szecsenyi, J., Ollenschläger, G., &
Scherer, M. (2012). Simultaneous development of guidelines and quality
indicators— how do guideline groups act? A worldwide survey. Inter-
national Journal of Health Care Quality Assurance, 25, 712–729. http://
dx.doi.org/10.1108/09526861211270659
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F.,
Feder, G.,...theAGREE Next Steps Consortium. (2010). AGREE II
: Advancing guideline development, reporting and evaluation in health
care. Canadian Medical Association Journal, 182, E839 –E842. http://
dx.doi.org/10.1503/cmaj.090449
Burgers, J. S., Cluzeau, F. A., Hanna, S. E., Hunt, C., & Grol, R. (2003).
Characteristics of high-quality guidelines: Evaluation of 86 clinical
guidelines developed in ten European countries and Canada. Interna-
tional Journal of Technology Assessment in Health Care, 19, 148 –157.
http://dx.doi.org/10.1017/S026646230300014X
Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H.,
Abboud, P. A., & Rubin, H. R. (1999). Why don’t physicians follow
clinical practice guidelines? A framework for improvement. Journal of
the American Medical Association, 282, 1458 –1465. http://dx.doi.org/
10.1001/jama.282.15.1458
Cahill, N. E., & Heyland, D. K. (2010). Bridging the guideline-practice gap
in critical care nutrition: A review of guideline implementation studies.
Journal of Parenteral and Enteral Nutrition, 34, 653– 659. http://dx.doi
.org/10.1177/0148607110361907
Chang, S. G., Kim, D. I., Shin, E. S., Jang, J. E., Yeon, J. Y., & Lee, Y. S.
(2016). Methodological quality appraisal of 27 Korean guidelines using
a scoring based on the AGREE II instrument and a web-based evalua-
tion. Journal of Korean Medical Science, 31, 682– 687. http://dx.doi.org/
10.3346/jkms.2016.31.5.682
Daniel, S. S., Walsh, A. K., Goldston, D. B., Arnold, E. M., Reboussin,
B. A., & Wood, F. B. (2006). Suicidality, school dropout, and reading
problems among adolescents. Journal of Learning Disabilities, 39, 507–
514. http://dx.doi.org/10.1177/00222194060390060301
Davis, D. A., & Taylor-Vaisey, A. (1997). Translating guidelines into
practice: A systematic review of theoretic concepts, practical experience
and research evidence in the adoption of clinical practice guidelines.
Canadian Medical Association Journal, 157, 408 – 416.
Del Campo, P. D., Gracia, J., Blasco, J. A., & Andradas, E. (2011). A
strategy for patient involvement in clinical practice guidelines: Method-
ological approaches. BMJ Quality & Safety, bmjqs-2010.
Drapeau, M., & Hunsley, J. (2014). Where’s the science? Introduction to a
special issue of Canadian psychology on science in psychology. Cana-
dian Psychology, 55, 145–152. http://dx.doi.org/10.1037/a0037321
Gordon, J., & Cooper, B. (2010). Talking knowledge—practising knowl-
edge: A critical best practice approach to how social workers understand
and use knowledge in Practice. Practice, 22, 245–257. http://dx.doi.org/
10.1080/09503153.2010.495768
Graham, I. D., Beardall, S., Carter, A. O., Tetroe, J., & Davies, B. (2003).
The state of the science and art of practice guidelines development,
dissemination and evaluation in Canada. Journal of Evaluation in Clin-
ical Practice, 9, 195–202. http://dx.doi.org/10.1046/j.1365-2753.2003
.00385.x
Grol, R. (2001). Successes and failures in the implementation of evidence-
based guidelines for clinical practice. Medical Care, 39(8, Suppl. 2),
II46 –II54.
Leach, J. M., Scarborough, H. S., & Rescorla, L. (2003). Late-emerging
reading disabilities. Journal of Educational Psychology, 95, 211.
Lerner, J. W. (1989). Educational interventions in learning disabilities.
Journal of the American Academy of Child & Adolescent Psychiatry, 28,
326 –331.
Lyon, G. R., Fletcher, J. M., Shaywitz, S. E., Shaywitz, B. A., Torgesen,
J. K., Wood, F. B.,...Olson, R. (2001). Rethinking learning disabilities.
Rethinking Special Education for a New Century, 259 –287.
Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2003). A definition of
dyslexia. Annals of Dyslexia, 53, 1–14.
MacQueen, G., Santaguida, P., Keshavarz, H., Jaworska, N., Levine, M.,
Beyene, J., & Raina, P. (2017). Systematic review of clinical practice
guidelines for failed antidepressant treatment response in major depres-
sive disorder, dysthymia, and subthreshold depression in adults. Cana-
dian Journal of Psychiatry, 62, 11–23.
Montori, V. M., Brito, J. P., & Murad, M. H. (2013). The optimal practice
of evidence-based medicine: Incorporating patient preferences in prac-
tice guidelines. Journal of the American Medical Association, 310,
2503–2504. http://dx.doi.org/10.1001/jama.2013.281422
Murdoch, D. D., Gregory, A., & Eggleton, J. M. (2015). Why psychology? An
investigation of the training in psychological literacy in nursing, medicine,
social work, counselling psychology, and clinical psychology. Canadian
Psychology, 56, 136 –146. http://dx.doi.org/10.1037/a0038191
Norris, S. L., Holmer, H. K., Ogden, L. A., & Burda, B. U. (2011). Conflict
of interest in clinical practice guideline development: A systematic
review. PLoS ONE, 6, e25153. http://dx.doi.org/10.1371/journal.pone
.0025153
Ordre des psychologues du Quebec (OPQ). (2014). Lignes directrices pour
l’évaluation de la dyslexie chez les enfants. Retrieved from www
.ordrepsy.qc.ca/documents/26707/0/Lignes⫹directrices⫹pour⫹l%27
%C3%A9valuation⫹de⫹la⫹dyslexie⫹chez⫹les⫹enfants/139edf81-
9163-4060-9c66-06f02eee0ed8
Rashid, A., Thomas, V., Shaw, T., & Leng, G. (2016). Patient and public
involvement in the development of healthcare guidance: An overview of
current methods and future challenges. Patient-Patient-Centered Out-
comes Research, 1– 6.
Rosenfeld, R. M., Shiffman, R. N., & Robertson, P. (2013). Clinical
practice guideline development manual (3rd.): A quality-driven ap-
proach for translating evidence into action. Otolaryngology—Head and
Neck Surgery, 148(Suppl. 1), S1–S55. http://dx.doi.org/10.1177/
0194599812467004
Ruszczy´
nski, M., Horvath, A., Dziechciarz, P., & Szajewska, H. (2016).
Cow’s milk allergy guidelines: A quality appraisal with the AGREE II
instrument. Clinical and Experimental Allergy, 46, 1236 –1241. http://
dx.doi.org/10.1111/cea.12784
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., &
Richardson, W. S. (1996). Evidence based medicine: What it is and what
it isn’t. British Medical Journal, 312, 71–72. http://dx.doi.org/10.1136/
bmj.312.7023.71
Shaneyfelt, T. M., Mayo-Smith, M. F., & Rothwangl, J. (1999). Are
guidelines following guidelines? The methodological quality of clinical
practice guidelines in the peer-reviewed medical literature. Journal of
the American Medical Association, 281, 1900 –1905. http://dx.doi.org/
10.1001/jama.281.20.1900
Spilka, M. J., & Dobson, K. S. (2015). Promoting the internationaliza-
tion of evidence-based practice: Benchmarking as a strategy to eval-
uate culturally transported psychological treatments. Clinical Psy-
chology: Science and Practice, 22, 58 –75. http://dx.doi.org/10.1111/
cpsp.12092
Stamoulos, R., Reyes, A., Trepanier, L., & Drapeau, M. (2014). The
methodological quality of clinical practice guidelines in psychology: A
216 TREPANIER, STAMOULOS, AND REYES
pilot study of the guidelines developed by the Ordre des Psychologues
duq Uébec. Canadian Psychology, 55, 177–186. http://dx.doi.org/10
.1037/a0037287
Steinert, T., Richter, D., & Bergk, J. (2010). Sources of bias in guideline
development. Psychiatric Services, 61, 946 –947. http://dx.doi.org/10
.1176/ps.2010.61.9.946
Ward, J. E., & Grieco, V. (1996). Why we need guidelines for guidelines:
A study of the quality of clinical practice guidelines in Australia.
Medical Journal of Australia, 165, 574 –576.
Ye, Z.-K., Liu, Y., Cui, X.-L., & Liu, L.-H. (2016). Critical appraisal of the
quality of clinical practice guidelines for stress ulcer prophylaxis. PLoS
ONE, 11, e0155020. http://dx.doi.org/10.1371/journal.pone.0155020
Received March 21, 2017
Revision received May 25, 2017
Accepted May 26, 2017 䡲
217
EVALUATION OF GUIDELINE FOR DYSLEXIA
A preview of this full-text is provided by American Psychological Association.
Content available from Canadian Psychology/Psychologie canadienne
This content is subject to copyright. Terms and conditions apply.
