Content uploaded by Lisa Didion
Author content
All content in this area was uploaded by Lisa Didion on Jan 13, 2020
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=uree20
Journal of Research on Educational Effectiveness
ISSN: 1934-5747 (Print) 1934-5739 (Online) Journal homepage: https://www.tandfonline.com/loi/uree20
Teacher Professional Development and Student
Reading Achievement: A Meta-Analytic Review of
the Effects
Lisa Didion, Jessica R. Toste & Marissa J. Filderman
To cite this article: Lisa Didion, Jessica R. Toste & Marissa J. Filderman (2019): Teacher
Professional Development and Student Reading Achievement: A Meta-Analytic Review of the
Effects, Journal of Research on Educational Effectiveness
To link to this article: https://doi.org/10.1080/19345747.2019.1670884
Published online: 19 Dec 2019.
Submit your article to this journal
View related articles
View Crossmark data
THEORY, CONTEXT, AND MECHANISMS
Teacher Professional Development and Student Reading
Achievement: A Meta-Analytic Review of the Effects
Lisa Didion
a
, Jessica R. Toste
a
and Marissa J. Filderman
a
ABSTRACT
Research-based instruction is necessary to support students’reading
development, and professional development (PD) is a critical avenue
to ensuring high-quality instructional practices. To date, there has
been no comprehensive review focused on the effects of teacher PD
on student reading achievement. As such, the purpose of the pre-
sent meta-analysis was to examine the impact of teacher PD on
reading outcomes for students in kindergarten through 8th grade. A
secondary aim was to determine whether study, program, or partici-
pant characteristics were potential moderators of these effects. A
comprehensive search of published and unpublished research
between 1975 and 2017 resulted in 28 studies that met prescribed
criteria. Results indicate that teacher PD has a moderate and signifi-
cant, positive average effect on reading achievement. However,
moderator analyses did not explain the variance in the effects of PD
on student outcomes. Primarily, studies represented typically-devel-
oping students in the elementary grades, with only three studies
including middle school students and four studies including reading
outcomes of students with or at-risk for reading disabilities.
Recommendations for improved study designs that allow for more
in-depth investigation of the characteristics of effective PD and
mechanisms of change in student outcomes are discussed.
KEYWORDS
professional development
reading
teacher education
In order to improve students’academic performance, teachers need to be able to effect-
ively deliver instruction. Teachers require adequate training to implement evidence-
based practices so that students can receive high-quality instruction across content areas.
Teacher professional development (PD) is the means by which we train in-service teach-
ers on current research-based methods, yet there is a lack of consensus in the literature
on what teacher PD components are necessary to contribute positively to student out-
comes (i.e., Blank & Alas, 2009; Kennedy, 1998; Yoon, Duncan, Lee, Scarloss, &
Shapley, 2007). In the current study, we expand on findings from previous reviews of
teacher PD and student-level academic outcomes (i.e., Blank & Alas, 2009; Yoon et al.,
2007) to investigate the effects of PD on the reading achievement of students in
CONTACT Lisa Didion lisa-didionjohnston@uiowa.edu Department of Teaching and Learning, The University of
Iowa, 246N Lindquist Center, Iowa City, IA, 52242, USA.
a
Department of Special Education and the Meadows Center for Preventing Educational Risk, The University of Texas,
Austin, Texas, USA
Lisa Didion is now affiliated with the Department of Teaching and Learning, The University of Iowa, Iowa, USA.
ß2019 Taylor & Francis Group, LLC
JOURNAL OF RESEARCH ON EDUCATIONAL EFFECTIVENESS
https://doi.org/10.1080/19345747.2019.1670884
kindergarten through 8th grade. Our goal was not only to determine the effect of PD
on elementary and middle school-age students’reading achievement, but also to gain
knowledge about components that may maximize effectiveness or require further explor-
ation. Thus, we examined moderators of methodological interest, such as study design,
and moderators of theoretical interest, such as reading outcomes being measured, as
well as, characteristics of PD and participants. In the following sections, we describe the
background and rationale for examining the effects of teacher PD on student outcomes
specific to reading, as well as the moderators under investigation in the current review.
Negative long-term consequences are associated with lack of reading proficiency
including dropout, un/underemployment, and involvement in the criminal justice sys-
tem (Cortiella & Horowitz, 2014; Reynolds & Ou, 2004;Wagner,Newman,Cameto,
Garza, & Levine, 2005). Although efforts to improve student outcomes in reading
across grade levels have been made, the number of students who have reading diffi-
culties is remaining stable. Spanning the past two decades, national data indicate
that the average reading score for fourth and eighth- grade students fall below profi-
ciency cutoffs (U.S. Department of Education, National Center for Education
Statistics, 2017). Specifically, since 1992, only about one-third of fourth and eighth
-grade students are performing at or above proficiency levels (U.S. Department of
Education, National Center for Education Statistics, 2017). Further concerning is that
students with disabilities demonstrate an average score below basic levels in reading
performance with a similar, flat trend over the past 15 years (U.S. Department of
Education, National Center for Education Statistics, 2017). Students who struggle to
read proficiently will continue to fall behind if teachers are not versed in high-qual-
ity, evidence-based instruction.
High-Quality Professional Development
High-quality teachers, both regular and special education, play a larger role in contribu-
ting to student achievement than any other school factor (Hattie, 2009,2012;
Leithwood, Harris, & Hopkins, 2008; Sanders & Rivers, 1996; Shen & Cooley, 2008).
Research is evolving on how best to teach students with reading difficulties; accordingly,
teachers that were sufficiently trained previously may not have the knowledge to imple-
ment current, effective, research-based methods to reduce achievement gaps (Connor,
Alberto, Compton, & O’Connor, 2014). Therefore, teacher learning is paramount to
building capacity to provide instruction that improves reading achievement and support
overall school effectiveness (Hochberg & Desimone, 2010).
Desimone (2009) proposes a widely accepted conceptual framework for studying the
effect of PD on teacher and student outcomes. The framework suggests that when high-
quality PD is provided, teachers’knowledge and skills improve and a change is noted in
their attitudes and beliefs. This, in turn, influences and changes their instruction, which
has the potential to positively impact student outcomes. The conceptual framework is
grounded in theories of teacher change (i.e., PD influences teacher knowledge and prac-
tice) and theories of instruction (i.e., improved teachers’practices improve student out-
comes; Supovitz, 2001; Wayne, Yoon, Zhu, Cronen, & Garet, 2008). Student
achievement is influenced by the quality of the training that teachers receive, the extent
2 L. DIDION ET AL.
with which teachers were influenced, and the degree to which teachers’reformed
instruction (Desimone, 2009).
Core Features of High-Quality Professional Development
In Desimone’s conceptual framework, there are four core features posited as central
to what she identifies as “high-quality”PD: intensity, relevance, active learning, and
collective participation (i.e., collaborative learning processes). This aligns with
research findings and policy initiatives that agree teacher PD should combine inten-
sity and relevance [to teachers needs] through collaborative learning processes (Bos,
Mather, Narr, & Babur, 1999; Garet, Porter, Desimone, Birman, & Yoon, 2001; U.S.
Department of Education, National Center for Education Statistics, 2001). However,
little consensus has been reached on what makes these core features effective for
teacher learning.
Intensity
Intensity is most often referred to as the number of hours spent in PD (Kennedy, 2016).
The number of contact hours needed for optimal intensity to elicit teacher change
remains inconclusive (Blank & Alas, 2009) as published reviews have suggested different
levels of intensity as being most effective. For example, Yoon et al. (2007) reported that
PD programs with >14 contact hours had positive, significant effects on student
achievement, while studies with 5–14 h of PD showed no statistically significant effects
on student learning outcomes. Other research suggests that the most effective PD is
spread over time, but findings range from 20 contact hours (Desimone, 2009), over 30 h
(Guskey & Yoon, 2009), to 49 h (Darling-Hammond, Wei, Andree, Richardson, &
Orphanos, 2009). These findings vary and, therefore, duration and dosage variables need
to be further reported and analyzed in order to build PD that is intense enough to elicit
change at both the teacher- and student-level.
Relevance
Another suggestion for PD is that it be relevant, or coherent. That is to say, PD should
operate in conjunction with student and teacher characteristics and preferences
(Hochberg & Desimone, 2010). Previous research has suggested that teachers’beliefs
about teaching are associated with their satisfaction of PD, which in turn influences
their practices. Specifically, when new strategies taught as PD are farther from their own
instructional practices, the less likely teachers are to adopt the new techniques into their
instructional routines (Dingle, Brownell, Leko, Boardman, & Haager, 2011; Snyder,
Denny, Pasia, Rakap, & Crowe, 2011). Also, the impact of teacher PD varies depending
on other factors within the school environment (i.e., other teachers’beliefs, grade level
cohort relationships, school climate) that influence individual teachers’interpretation of
reformed practices (Opfer & Pedder, 2011). The many factors that influence teachers
can create “noise,”such that messages may conflict with one another and result in
EQUITY AND EXCELLENCE IN EDUCATION 3
teachers being unable to determine which practices are most important
(Kennedy, 2016).
Participation
One way to reduce distracting messages is by incorporating collaborative learning proc-
esses within teacher PD. Collaborative learning processes are those that include active or
collective participation (Darling-Hammond, Hyler, & Garder, 2017). Active participation
includes PD activities that allow opportunities for teachers to be involved in the learning
process and participate in reflective dialog, such as opportunities to review data for
instructional decision making, plan instruction in the classroom, or receive instructional
feedback (Garet et al., 2001). There is evidence that when PD allows for active participa-
tion opportunities, teachers self-report higher levels of support for newly learned strat-
egies (Banilower & Shimkus, 2004). Another way for teachers to learn collaboratively is
through collective participation. Collective participation is the amount of time teachers
spend together within the same school working toward a common goal (Brownell,
Adams, Sindelar, Waldron, & van Hover, 2006; Hochberg & Desimone, 2010).
Collaboration facilitates learning through the exchange of ideas and reflection of practi-
ces elicited through conversation between teachers (Leko et al., 2015) and has been
found to be effective in impacting school-wide change (Buczynski & Hansen, 2010).
Activities during collaborative learning processes should consider both relevance and
intensity to be most effective (Bos et al., 1999; Darling-Hammond et al., 2017;
Kennedy, 2016).
Formats of Professional Development
Before policy and research began to focus on principles of high-quality teaching, PD
was generally provided in one-session workshops, where teachers would receive instruc-
tion in a whole group format (Little, 1993). This type of “one-off”PD has generally
been poorly reviewed by teachers and does not provide opportunity to develop their
understanding of a new approach (Hawley & Valli, 1999). PD has since evolved, and
there are various formats to train in-service teachers, which may include: (a) follow-up
workshops, defined as any trainings provided after an initial PD; (b) professional learn-
ing communities (PLCs), defined as small group PD with opportunities to participate in
reflective dialog; (c) coaching, defined as “master”teachers who offer ongoing, in-person
support either one-on-one or in small groups; or (d) online programs, defined as pro-
viding any training, follow-up videos, or coaching correspondence via technology.
Program designs can provide teacher PD using a singular component (e.g., coaching
support) or in conjunction with other formats (e.g., PLCs with coaching support).
Past research has yet to determine the extent to which PD formats affect student out-
comes. For instance, in a longitudinal analysis, Biancarosa, Bryk, and Dexter (2010)
examined a coaching program’s effects on the reading performance of approximately
8,000 students and 250 teachers. Authors used hierarchical cross-level value-added
effects modeling to analyze outcomes of students across teachers within schools over
time by comparing gains during coaching years to gains during the baseline year.
4 L. DIDION ET AL.
During the first implementation year, students exhibited a 16% increase in learning
gains as compared to the growth rate during baseline. During the second implementa-
tion year, a 27% increase in learning gains was observed compared to baseline growth
rate. These results indicate coaching may have significant effects on students’reading
outcomes. In contrast, Garet et al. (2008) found no significant difference between teach-
ers that received PD focused to improve their content knowledge in reading and spell-
ing, and the same PD with added coaching support. Both formats had significant effects
on teachers’content knowledge and their use of explicit instruction as compared to
teachers that received district PD (business-as-usual; BAU), but none had significant
effects on second-grade students’reading outcomes. Research that uses experimental
designs to study the effects of PD formats such as coaching, PLCs, or online learning
remain scarce and offer differing results. Therefore, more consensus is needed on the
effectiveness of various PD formats.
Further, it has been largely posited that teacher PD can increase the intensity by com-
bining multiple PD formats (Connor et al., 2014). Carlisle, Cortina, and Katz (2011)
compared different combinations of PD to determine whether combining formats con-
tributed to improved teacher practices and student outcomes in reading. They compared
workshops focused on: (a) teacher knowledge and practices, (b) added content on using
student data to improve practices, and (c) workshops with student data plus coaching
and PLC opportunities. Results indicated that the last condition, combining all three
components, resulted in the highest student outcomes. Increased review and implemen-
tation of rigorous research designs may provide guidance to design the highest quality
teacher PD.
Impact of Professional Development on Student Achievement
Kennedy (1998) completed one of the first reviews examining the research on the asso-
ciation between teacher PD and student achievement. The goal of the review was to
examine the content of PD and subsequent effects on student outcomes in mathematics
and/or science. Results from 12 studies indicated that teacher PD that was focused on
teacher behavior had smaller effects than programs focused on teachers’knowledge of
the subject, curriculum, or student learning. Kennedy noted that PD differed in intensity
and how contact hours were distributed over time; therefore, a clear relationship
between intensity and student outcomes could not be determined through this review.
Specifically, some studies that had fewer contact hours with teachers in PD showed
greater impact on student academic performance than more intensive PD and vice
versa. Moderators of time distribution, coaching, and delivery format (whole group vs.
individual support) did not provide further explanation of the impact of total number
of hours of PD provided. Kennedy argued that the dimensions of PD advocated for in
the literature (intensity, coaching, whole group vs. individual support) are not relative
to the effects of the program and future investigations should focus on variables that are
more predictive of student performance, such as teaching content knowledge. Further,
Kennedy’s findings are limited to research completed by university academics and do
not represent a sample of studies that included research sponsored by federal, state or
local organizations.
EQUITY AND EXCELLENCE IN EDUCATION 5
Blank and Alas (2009) also sought to compare teacher PD and student outcomes in
mathematics or science. Results from their meta-analysis show that mathematics-focused
PD had significant positive effects on student achievement and that these relations are
moderated by format characteristics of the PD provided (e.g., summer workshops).
Sixteen studies met inclusion criteria: experimental or quasi-experimental studies that
discuss effects of teacher PD on learning outcomes for students in grades K-12 in math
or science. Student outcomes in all included studies reported direct student achievement
and excluded distal outcomes (e.g., impressions, perceptions, opinions). The teacher
learning goals described by the studies were focused on improving teacher knowledge in
order to improve student outcomes. Using Cohen’sdstatistics, results indicated that
student performance on pre-post mathematics measures had a mean effect size of 0.21.
The number of effects sizes reported for students’science outcomes was small and not
significantly different from zero. Moderating analyses indicated that studies that used
randomized controlled trials had significantly larger effect sizes than studies that used
quasi-experimental designs. They further found larger effect sizes for studies that took
place in elementary grades as compared to secondary grades. However, all of their
included studies were completed by researchers in institutions of higher education and
thus, did not reflect research completed from outside sources (e.g., state and federal ini-
tiatives). The mean contact hours for PD within studies was 91 h, but some programs
provided <10 h. The results varied across studies so no consensus on optimal intensity
for maximum effect could be concluded, such that studies with low and high intensity
had medium to large positive effects (e.g., 12 h in Walsh-Cavazos, 1994 and 41 h in
Saxe, Gearhart, & Nasir, 2001); but also other studies with low and high intensity had
null or negative effects (e.g., 14 h in Snippe, 1992 and 540 h in Meyers & Sutton, 2006).
Due to the variability in teacher PD and all of the variables that may impact teacher or
student learning, the authors suggest that randomized controlled trials are essential to
investigating the efficacy of teacher PD on student level academic outcomes.
Yoon et al. (2007) also sought to examine how teacher PD affects student achieve-
ment outcomes. They reviewed over 1,300 studies with What Works Clearinghouse
(WWC) evidence standards. The WWC standards identify educational research aimed
at improving educational outcomes for teachers and students while assessing the overall
quality of the research. Yoon and colleagues identified only nine studies that described
PD with the required rigor to meet WWC standards. Moderate standardized mean dif-
ference effects (M¼0.54) were observed across these nine studies in regards to student
achievement outcomes. These effect sizes were consistent across various content areas.
All nine studies investigated elementary aged students; four focused on reading out-
comes, two on mathematics, one on science, two with combined math and reading, and
one examined all three content areas. Teacher PD was delivered through workshops or
summer institutes, and all but one study provided follow-up with teachers. Studies were
homogeneous in format and heterogeneous in time/duration. PD providing >14 h indi-
cated a positive, significant effect on student outcomes. Three studies provided <14 h of
PD and none showed statistically significant effects on student achievement. Yoon and
colleagues reported difficulty concluding the effects of intensity variables due to consis-
tencies in form (workshops) and variability in intensity, such that no pattern of effects
was detected at differing levels of intensity or for specific PD topics. Further, results
6 L. DIDION ET AL.
should be interpreted with caution as no weighting procedures based on study sample
size were used when calculating effect size. The findings are limited to an elementary
school population only, as authors did not locate any studies investigating PD on sec-
ondary students’reading outcomes.
Professional Development and Reading Outcomes
In their study, Yoon et al. (2007) identified only six rigorous studies that investigated out-
comes in reading. However, evidence was inconsistent across these studies; three of the
studies found statistically significant results on some or all reading outcomes (Cole, 1992;
McCutchen et al., 2002; McGill-Franzen, Allington, Yokoi, & Brooks, 1999) while the,
other three had no impact on student reading performance (Duffy et al., 1986;Sloan,
1993; Tienken, 2003). While duration and intensity may play a role, again, too few identi-
fied studies met required rigor, and effects were inconclusive. The lack of available
research on the relationship between teacher PD and student outcomes in reading leaves
little evidence to determine how teachers develop content knowledge. The National
Reading Panel’s report (National Reading Panel [NRP], 2000) emphasizes the importance
of developing teachers’content knowledge in reading across five essential reading compo-
nents (i.e., phonological awareness, phonics, fluency, vocabulary, comprehension). The
NRP suggests teachers understand the individual development of each of their students
on the essential components so that instruction is appropriate. Consensus has yet to be
reached on how teachers best develop this knowledge.
To develop as proficient readers, students require mastery within two domains: code-
focused skills (i.e., phonological awareness, phonics, fluency), or the ability to read
accurately with fluency, and meaning-focused skills (i.e., vocabulary, comprehension), or
the ability to understand and gain knowledge from text. Instructional techniques teach-
ers learn in PD may look quite different for both of these skill-sets, and therefore may
affect outcomes in these domains differently. In the first years in elementary school, stu-
dents are expected to learn the basic skills required for word reading proficiency, or
“learn to read”while in the later, elementary years, they are expected to gain meaning
from text, or “read to learn”(Chall, 1983). In the later grades, as reading demands con-
tinue to increase (e.g., more complex words to decode, increased speed) comprehension
and vocabulary are likely to be impacted if code-focused skills have not been adequately
mastered (Leach, Scarborough, & Rescorla, 2003). One might consider that meaning-
focused gains are harder to achieve—that is, word reading/decoding skills are more
discrete skills than meaning-focused skills which require efficient decoding, reading flu-
ency, prior knowledge, strategy use, and monitoring (Snow, 2002). Therefore, questions
remain regarding the impact of PD on teachers’application of instructional strategies
within these two domains.
Possible Moderators of Professional Development Effectiveness
Previous reviews and meta-analyses lack investigation of participant-level characteristics,
both teacher and student, as potential moderators of the effects of PD to improve student-
level outcomes. First, there may be reason to believe that PD differentially influences teachers
EQUITY AND EXCELLENCE IN EDUCATION 7
based on their years of experience or whether they have earned graduate degrees. For
instance, novice teachers lack experience but have recently learned evidence-based practices
from their training institutions and therefore may learn PD content differently than teachers
who have several years of experience. Also, experienced teachers may be less willing to
change their standard practices. New strategies presented in PD that teachers have not per-
sonally vetted overtime may be accepted less often (Opfer & Pedder, 2011). For these rea-
sons, teacher experience may play a moderating role in the effectiveness of PD on student
outcomes in reading. Additionally, teachers that earned advanced degrees may differ from
those that earned undergraduate degrees. It may be that teachers with advanced degrees
value continued learning, as suggested by their graduate studies, and are more responsive to
PD instruction. Further, Wayne and Youngs (2003) found that advanced degrees in math
and science are positively related to student outcomes, but the same effects were not
observed for reading. Questions remain regarding how different characteristics of teachers
(e.g., years of experience, advanced degrees) influence the adoption of new practices and
how student learning is impacted.
Second, student characteristics related to disability status may also explain some of
the variability in PD effectiveness. PD offered to general and special educators may have
differential impacts for students with disabilities and typically developing readers.
Students with disabilities have unique needs related to their development as readers and
require individualized intervention (Fuchs & Fuchs, 2015). With 61.2% of students with
disabilities educated in general education classroom settings (U.S. Department of
Education, National Center for Education Statistics, 2017), it is necessary to examine
differences in the PD required to improve the reading outcomes of typically developing
students and students with disabilities. Instructional practices learned through PD may
not be related to strategies that improve reading outcomes for students with disabilities.
These variables may provide answers to important questions related to PD design.
Purpose of the Current Study
National reading achievement data has remained largely unchanged over the past
20 years (U.S. Department of Education, National Center for Education Statistics, 2017),
with the majority of students performing below a proficient level. Teachers across a stu-
dent’s school career need to be able to effectively support the unique needs of students
with evidence-based practices. PD is a common way in which all teachers continually
learn practices that are most effective for their diverse set of learners. Previous research
has noted that teacher PD designed with intensity, relevance, and collaborative learning
processes shows effects on increased student academic outcomes (Bos et al., 1999;
Kennedy, 2016). Since the publication of previous reviews of the PD literature (Blank &
Alas, 2009; Yoon et al., 2007), the volume of literature examining the impact of PD on
student outcomes has grown. Thus, there is a need to reevaluate the evidence and exam-
ine potential moderators related to high-quality PD (Desimone, 2009). Also, the major-
ity of synthesized research on teacher PD and student outcomes is focused on
mathematics and science (e.g., Blank & Alas, 2009). To date, no systematic review exists
that exclusively examines teacher PD and student outcomes in reading. Additionally,
there is little synthesized about how PD differentially affects students and teachers of
8 L. DIDION ET AL.
varying abilities. It may be that PD for teachers of students with disabilities may be
impacted differently than teachers of typically-developing students—that is, special edu-
cation teachers may need PD to learn instructional techniques that are unique to
students with disabilities (i.e., intensive intervention, data literacy, direct instruction).
No previously published systematic reviews have investigated the characteristics of the
teacher or student participants. This, too, may be an important factor in the impact a
particular PD design may have on student outcomes. Therefore, it is the purpose of the
present meta-analysis to examine the effects of PD on student outcomes in reading and
further to examine what characteristics of PD and participants moderate the effects. We
pose four research questions:
1. What are the effects of PD on reading achievement for students in K-8?
2. What characteristics of study design are potential moderators of effects (i.e.,
experimental design, measured outcomes, replicable quality)?
3. What characteristics of professional development (i.e., intensity, collaborative
learning processes, content focus, format) are potential moderators of effects?
4. What characteristics of participants, both teacher (i.e., years of experience, certifi-
cation) and student (i.e., disability status), are potential moderators of effects?
Method
Operational Definitions
Teacher professional development (PD) is defined as any training provided to in-service
teachers. To be considered, the PD could include: (a) any session lasting three hours or
longer; (b) delivery during summer or school months; (c) delivery in the context of cur-
rent practice; (d) whole group meetings, summer coursework, professional learning
communities, on-going coaching, online training, and follow-up sessions; and (e) deliv-
ery by district, school, or university personnel and can include persons hired by entities
to deliver PD. PD does not include pre-service teacher education, staff meetings to
review current school protocol, or graduate-level coursework.
Search Procedures and Study Identification
Quantitative research studies for this meta-analysis were identified through multiple
sources and four search methods. First, a search was completed through four electronic
databases: PsycINFO, ERIC, Academic Search Complete, and Education Source. The
search was completed within abstract, subject terms, and titles of studies and reports
published in peer-reviewed journals between 1975 and May 31, 2017. The search terms
included professional education or professional development or career development or
teacher development or trainor workshop or coachor learning communitor PLC or
teacher education and reading or literacy or comprehenor phonicor phonemic aware-
ness or fluency or vocaband teacheror reading specialistor educatorand academic
achievement or academic performance or academic success or student performance or stu-
dent success or student achievement or reading achievement or reading performance or
EQUITY AND EXCELLENCE IN EDUCATION 9
reading outcomes or academic outcomes or academic improvement. The initial search
yielded 4,308 articles and 3,058 articles after duplicates were removed.
Authors met to discuss screening and coding procedures prior to screening processes
in order to establish inclusion criteria for the present meta-analysis. Studies were
screened in a two-step process by the first author with oversight by the second. First, all
abstracts were reviewed to identify research that appeared to meet inclusion criteria
(described below), with a conservative erring toward the possibility that criteria were
met, and 141 studies qualified. Studies included after this initial step had evidence that
the focus of the research was on understanding the impact of PD on student outcomes
in reading. Next, these articles were screened through a full-text review to establish a
final selection of included studies. The first and second authors met to discuss all poten-
tially included studies, questionable studies, and excluded studies in order to examine
biases in the included list. After full-text review, 24 studies met all inclusion criteria as
determined by the authors (see Figure 1). The majority of removed studies were due to
study design (k¼31). These studies used a single group’s pre- and post-test data to
examine effects and did not provide a control group. Other studies were excluded for
lack of reporting of student-level reading outcomes or lack of sufficient information to
calculate effect size estimates.
Our second search used the same screening procedures, databases and search terms with
anot peer-reviewed limit to identify unpublished dissertations, federally funded projects, and
conference presentations. The second search yielded 3,625 more studies. After reviewing
abstracts and full text, four additional studies were identified for inclusion. Third, a hand
search was completed through the table of contents of Review of Educational Research,
Journal of Teacher Education, and Teaching and Teacher Education from January 1975 to
May 2017; no additional studies were identified for inclusion. Finally, a hand search was
completed through websites of foundations and research organizations to identify potential
projects investigating the effects of professional development on student outcomes. Key cen-
ters included the National Center of Improving Literacy, National Institute for Literacy,
Florida Center for Reading Research, RAND, Research for Better Schools, Center for Social
Organization of Schools, the Consortium of Policy Research in Education, and Campbell
Collaboration. Eight studies were reviewed in full text; however, none met inclusion criteria,
primarily due to student outcomes not being reported.
In total, 28 studies met the following inclusion criteria for this meta-analysis:
1. Participants were practicing teachers in K-8 education (studies of pre-service
teachers or master’s education training programs were excluded).
2. Student participants included elementary and middle school (K-8) students and/
or students with disabilities (i.e., LD, EBD, struggling readers, at risk).
3. The primary independent variable was teacher PD: (a) whole group, defined as
large group PD delivered in some form of lecture that may include active and/or
collective participation; (b) summer coursework, defined as any workshop or
institute delivered during summer months; (c) professional learning communities,
defined as small group PD incorporating reflective dialog with opportunities to
participate, action research projects, or teacher work groups; (d) coaching,
defined as continual support given by PD staff in small group or 1:1 meetings;
10 L. DIDION ET AL.
and (e) online training, defined as any training, follow-up videos, or coaching
correspondence via computer.
4. The dependent variable included at least one measure of students’reading
achievement. Studies had to examine one or more of the following skills: phono-
logical awareness, defined as the ability to focus and manipulate phonemes in
spoken words; decoding, defined as blending and segmenting written words;
word identification, defined as untimed word reading; fluency, defined as timed
Figure 1. Flow of documents.
EQUITY AND EXCELLENCE IN EDUCATION 11
word reading or measures of connected text; vocabulary, defined as identifying
meanings of words; comprehension defined as identifying the meaning from text;
or general reading skills which measure more than one of the previous skills in a
comprehensive assessment (e.g., state end-of-year assessment). Reading skills
were classified as code-focused skills (i.e., phonological awareness, decoding,
word identification, fluency), meaning-focused skills (i.e., vocabulary, comprehen-
sion) or general reading.
5. The study utilized an experimental or quasi-experimental design. Case studies (sam-
ple size of one), qualitative, single-group, and single-case designs were excluded from
the current meta-analysis. Quasi-experimental designs had to specifically describe
their efforts to equate groups either by matching or pretest covariates.
6. The study included information needed to calculate a standardized mean differ-
ence (Hedges’g) between treatment and comparison conditions (i.e., means,
standard deviations and participant numbers for post-test measures,
test statistics).
7. Published in English, in a peer-reviewed journal between 1975 and 2017.
Coding Procedures
An extensive code sheet developed for use with systemic reviews (Vaughn, Elbaum,
Wanzek, Scammacca, & Walker, 2014) was adapted for use in organizing information
on the following study features: (a) research design, (b) teacher participant characteris-
tics, (c) student participant characteristics, (d) PD components, (e) dependent measures,
and (f) effect sizes. The coding sheet used a combination of forced-choice items (e.g.,
research design, assignment method, fidelity of implementation), open-ended items (e.g.,
characteristics used to equate groups, age or grades of participants as described in the
text), and written descriptions of the PD condition.
If insufficient information was provided on key variables (e.g., student outcomes, PD
components) authors were contacted to retrieve the missing data (k¼1). To ensure that
the agreed upon definitions and descriptions were followed consistently throughout the
coding process, each article was independently coded and then double-coded by a
second rater to check for accuracy. If disagreements occurred, meetings were held to
review article information and reach consensus. Across all cells in the coding workbook,
the overall mean interrater agreement was 99.54%. The interrater reliability was 100%
for research design, 99.56% for teacher participant characteristics, 99.44% for student
participant characteristics, 99.02% for PD components, 99.84% for teacher outcome
measures, 99.45% for student outcome measures, and 99.08% for effect size calculations.
Calculating Effect Sizes
Means, standard deviation, and group numbers were used to calculate standardized
mean difference effect sizes (specific analytic strategies are discussed in the next section).
Effect sizes were estimated to compare PD groups to comparison groups—either a no
PD condition or a business as usual (BAU). If a study compared various PD programs
and comparison groups, effect sizes were only calculated between a PD condition and
12 L. DIDION ET AL.
comparison group. In this case, different program designs of PD were not compared
(i.e., Fuchs, Fuchs, Hamlett, & Ferguson, 1992). If a study described two different inten-
sities of PD with no control group (e.g., PD with coaching and PD without coaching),
then the more intense PD was considered the treatment condition (PD with coaching)
and the less intense PD condition (PD without coaching) was treated as the comparison
(e.g., Sailors & Price, 2010). We ran a sensitivity analysis which excluded these studies,
and results were similar to the overall analysis, g¼0.18, p<.001, and a 95% CI of
[0.09, 0.28]. If student outcome data, such as means and standard deviations were
reported disaggregated by participant characteristic (i.e., gender, grade), groups were not
compared if they were both control, cross-grade level, or members of different groups
(e.g., 2nd grade control was not compared to 4th grade control; 1st grade treatment was
not compared to 2nd grade treatment; females were not compared to males; e.g.,
O’Connor, 1999). For studies that did not provide the necessary information to calculate
effect sizes (i.e., post-test means and standard deviations), an effect size calculator
retrieved from the Campbell Collaboration (Wilson, 2017) was used to calculate standar-
dized mean difference (k¼9). In this case, standardized mean differences could be cal-
culated via a conversion process from the information provided; other test statistics (i.e.,
F-tests, chi-square, mean gain scores, t-tests) were translated into standardized mean dif-
ferences with the effect size calculator. And finally, two experimental studies did not
report their student participant demographics, including sample size, separately for
treatment and control groups (i.e., Alexander et al., 1987; Matsumura, Garnier, &
Spybrook, 2013); therefore, total student population was distributed evenly across teach-
ers. For example, Matsumura et al. (2013) had a total sample of 2,983 students and 167
teachers, with 101 teachers receiving PD and 66 teachers serving as control. In this case,
2,983 was divided by 167 (total number of teachers) resulting in 18 students per class-
room. This number was multiplied by the number of teachers in each group resulting
in 1,806 treated students and 1,177 control students. This assumes that the student par-
ticipants are distributed in the same ratio as the teacher participants.
Analytic Strategies
The effect size index used for all student outcome measures was Hedges’g(Hedges,
1981) and corrected for sample size bias. Hedges’gwas chosen as it provides a more
precise estimate of effect for small sample sizes (Fritz, Morris, & Richler, 2012). When
post-test means and standard deviations were provided, the following formulae were
used:
d¼13
4niT þniC
ðÞ
2
YiT YiC
sp
,
where niT and niC are the sample sizes for the treatment and control groups, YiT and
YiC are mean outcome scores for the treatment and control groups, and spis the pooled
standard deviation. spwas computed as
sp¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðniT 1Þs2
iT þðniC 1Þs2
iC
niT þniC
ðÞ
2
s,
EQUITY AND EXCELLENCE IN EDUCATION 13
where s2
iT and s2
iC are the variance for scores in the treatment and control groups. The
estimate of within-study variance for individual studies, vi, was calculated with
vi¼niT þniC
niT niC
þd2
2ðniT þniCÞ
Effects were defined at the level of the student rather than at the classroom- or
teacher-level. In cases where there was assignment to condition at the teacher level, we
corrected the effect size and standard errors for clustering as recommended by Hedges
(2007) to account for multilevel designs, to control for inflated Type I error, and overes-
timated statistical significance (What Works Clearinghouse, 2017).
All eligible, independent effect sizes were included from each study, resulting in some
studies contributing multiple effect sizes when several reading outcomes were reported.
Therefore, to account for the statistical dependencies of correlated effects, random
effects robust standard estimation was used (Hedges, Tipton, & Johnson, 2010) which
corrects standard errors of individual studies to account for correlations between effect
sizes estimated from the same sample (Gleser & Olkin, 2009). A mean correlation (q)
between all pairs of effect sizes within a given study was estimated for the between-
study sampling variance estimates, s2:s2was estimated with q¼.80, and sensitivity
analyses showed findings were robust across different estimates of s2:ROBUMETA in
Stata (Hedberg, 2014) was used to run the analyses, which considered the dependency
of effect sizes within studies. When using robust variance estimation when the number
of studies included in meta-analyses is <40, such as the current study, test statistics and
confidence intervals run the risk of having inflated Type 1 error (Tanner-Smith &
Tipton, 2014). However, the Stata ROBUMETA macro uses small sample size adjust-
ments for hypothesis tests to minimize this risk (Tanner-Smith, Tipton, &
Polanin, 2016).
We hypothesized that characteristics of the study design, PD characteristics, and par-
ticipant characteristics had potential influence on the effects of PD on student outcomes
in reading. Moderator variables were coded on either a continuous scale (i.e., intensity,
years of experience, percentage advanced degrees) or dichotomous (i.e., study design,
participation, format, content focus, disability status). Only variables that could not be
measured on a continuous scale were dichotomized. Moderators were included in a
meta-regression model.
Publication Bias
Potential publication bias was evaluated by testing the asymmetry in the effect sizes as a
function of the standard errors (Egger, Smith, Scheider, & Minder, 1997). Asymmetry of
effect sizes may indicate publication bias (Sterne et al., 2011). Results indicated that the
standard errors of effect sizes did not significantly predict effect sizes among studies
with ROBUMETA in Stata, p¼.11. Therefore, due to the lack of asymmetry, publica-
tion bias was not likely to have influenced the results in the included dataset.
14 L. DIDION ET AL.
Results
First, we report the main effect. Then, results from a subset analysis of interest (study
design and reading outcomes). These analyses provide information about contributions
to the overall average effect from various subgroups. Next, we provide descriptive details
about the 28 included studies. Finally, we report results from the meta-regression model.
Table 1 reports the results, power, and confidence intervals of the meta-regression.
Table 2 summarizes types of publication, conditions analyzed, number of participants,
student outcome measures, and effect sizes for each included study. Table 3 summarizes
formats used in PD, identified learning goals, content focus, mode of delivery and inten-
sity for each included study. Table 4 aggregates study characteristics (i.e., grade, format,
design type, delivery, reading outcomes, teacher characteristics, student characteristics)
across all studies.
Main Effect
To address our primary research question, we examined the effects of PD on the read-
ing achievement of students in K-8 by analyzing the unconditional model—one without
moderator variables. One hundred and ten effect sizes were obtained from 79 independ-
ent samples within 28 studies. Overall, analysis indicates that PD may have impacted
student outcomes in reading, compared to students taught by control teachers, with a
moderate, average significant effect size (see Table 1), Hedges’g¼0.18, p<.001, and a
95% CI of [0.09, 0.27]. However, this is the average effect and there is notable variation
in effect sizes from different studies. The prediction interval is between 0.23 and 0.57,
and included effect sizes ranged from 0.15 to 0.94. Therefore, meta-regression meth-
ods are necessary to study the variation in effects.
Table 1. Main effect and moderation analysis of professional development on student reading
achievement.
ES SE 95% CI pvalue
Main effect 0.18 0.04 [0.09, 0.27] 0.001
Variables Beta SE 95% CI pvalue
Study design (RCT ¼1, QED ¼0) 0.14 0.12 [0.42, 0.14] 0.28
Whole group (yes ¼1, no ¼0) 0.17 0.10 [0.40, 0.05] 0.11
Summer workshop (yes ¼1, no ¼0) 0.03 0.10 [0.23, 0.30] 0.76
Professional learning community (yes ¼1, no ¼0) 0.05 0.22 [0.45, 0.55] 0.83
Coaching (yes ¼1, no ¼0) 0.01 0.10 [0.23, 0.26] 0.91
Construct (code-focused ¼1, meaning-focused ¼0) 0.0001 0.0002 [0.0005, 0.0003] 0.54
Dosage 0 0.0002 [0.0005, 0.0005] 0.92
Content focus (yes ¼1, no ¼0) 0.008 0.12 [0.27, 0.29] 0.95
Active participation/collaborative learning (yes ¼1, no ¼0) 0.06 0.16 [0.31, 0.43] 0.71
Special education included (yes ¼1, no ¼0) 0.05 0.15 [0.29, 0.41] 0.73
Mean % of teachers with advanced degrees 0 0.0001 [0.0002, 0.0002] 0.68
Mean years of teachers’experience 0 0.0001 [0.0003, 0.0002] 0.85
Replicable (yes ¼1, no ¼0) 0.07 0.16 [0.33, 0.46] 0.68
Note. Main effect was generated from an unconditional model; all coefficients from the moderator analysis were entered
in one model. Several models were run for thorough subgroup comparisons among moderators with more than two
categories; Variables are continuous unless otherwise noted.
ES: effect size; RCT: randomized control trials; QED: quasi-experimental design.
p<.01; p<.001.
EQUITY AND EXCELLENCE IN EDUCATION 15
Table 2. Characteristics of study design and effect sizes.
Teachers Students
Study ID Publication type School level Conditions PD NCtrl NPD NCtrl NStudent Outcome Measure(s) # of ES ES range
Abe et al. (2012) RCT; peer reviewed Elementary
(5th)
PD & BAU 95 101 1566 1486 Stanford 10 Achievement 1 0.04
Alexander
et al. (1987)
RCT; peer reviewed Elementary
(4th)
PD, Manual, &
BAU
88
9
210 210
257
Woodcock Reading Mastery Test 2 0.02, 0.10
Angrist and
Lavy (1998)
QED; peer reviewed Elementary
(6th)
PD & BAU NR NR 395 357 JSA testing program 1 0.04
Bos et al. (1999) QED; peer reviewed Elementary (K
& 2nd)
PD & BAU 11 17 K
19
2nd
30
K
46
2nd
54
Informal letter-sound
assessment, WJ-III: Reading
Fluency Test
2 0.35, 0.60
Duffy et al. (1986) QED; peer reviewed Elementary
(5th)
PD & BAU 11 11 130 129 Gates-MacGinitie Reading Test-
2nd edition
1 0.07
Fisher, Frey, and
Lapp (2011)
RCT; peer reviewed Middle
(grade
not
specified)
PD & BAU 8 8 446 482 Gates-MacGinitie 1 0.14
F€
orster and
Souvignier
(2015)
RCT; peer reviewed Elementary
(3rd)
PD þprogress
monitoring,
Progress
monitoring
& BAU
13 15
15
289 340
289
Salzburger Lese-Screening
Ein Leseverstandnistest fur Erst
bis Sechsteklassler
40.01 to 0.03
M¼0.01
Fuchs et al. (1992) RCT; peer reviewed Elementary
and
Middle
(grade
not
specified)
PD1
PD2 & BAU
11
11
11 21
20
22 Comprehensive Reading
Assessment Battery: Words
Correct and Correct
Questions
MAZE: number correct
8 0.02 to 0.59
M¼0.32
Garet et al. (2008) RCT; peer
reviewed report
Elementary
(2nd)
PD1
PD2 & BAU
87
84
87 1789
1606
1661 Standardized district assessment 2 0.03, 0.07
Gersten, Dimino,
Jayanthi, Kim,
and Santoro
(2010)
RCT; peer reviewed Elementary
(1st)
PD & BAU 39 42 217 251 Woodcock Diagnostic Reading
Battery; Reading vocabulary,
Oral vocabulary, passage
comprehension, word attack,
6 0.53 to 0.94
M¼0.77
16 L. DIDION ET AL.
Oral Reading Fluency, Letter
Word ID
Jayanthi
et al. (2018)
RCT; peer
reviewed report
Elementary
(1st)
PD & BAU 94 88 863 817 WJ-III: Oral Vocab, Reading
Vocabulary, GRADE
30
M¼0
Knezek and
Christensen
(2007)
RCT; peer reviewed Elementary
(1st
& 2nd)
PD & BAU NR NR 1st
288
2nd
264
1st
119
2nd
119
Texas Primary Reading Inventory 8 0.06 to 0.65
M¼0.30
Magidin de Kramer,
Masters,
O’Dwyer, Dash,
and Russell,
(2012)
RCT; peer reviewed Middle (7th) PD & BAU 35 45 831 1225 Informal vocabulary and reading
comprehension assessment
2 0.02, 0.03
Matsumura
et al. (2010)
RCT; peer reviewed Elementary
(4th & 5th)
PD & BAU 42 31 736 533 Texas Assessment of Knowledge
and Skills, Degrees of
Reading Power Assessment
2 0.49, 0.87
Matsumura
et al. (2013)
RCT; peer reviewed Elementary
(4th & 5th)
PD & BAU 101 66 1806 1177 State standardized
accountability tests
1 0.03
McGill-Franzen
et al. (1999)
QED; peer reviewed Elementary
(K)
PD
Books & BAU
NR NR 164 139
153
Peabody Picture; Vocab test
Concepts About Print; letter ID,
writing vocab, Ohio Word
Test, Hearing Sounds Test
10 0.24 to 0.87
M¼0.57
McIntyre, Kyle,
Chen, Mu~
noz,
and Beldon
(2010)
QED; peer reviewed Elementary (K
- 5th)
PD & matched
control
23 NR 50 50 Predictive Assessment Scales 1 0.14
McMaster, Han,
Coolong-Chaffin,
and Fuchs (2013)
RCT; peer reviewed Elementary
(K)
PD & BAU 8 8 93 96 Segmentation, Blending, Rapid
Letter naming, Word
identification, Word attack,
Oral Reading
80.03 to 0.09
M¼0.02
Miller and
Ellsworth (1985)
QED; peer-reviewed Elementary
(2nd
–5th)
PD & BAU 47 96 137 135 California Achievement Test 2 0.18, 0.39
Moskowitz, Malvin,
Schaeffer,
Schaps, and
Condon (1981)
QED; state report Elementary
(5th)
PD & BAU 8 11 220 221 Stanford Achievement Test
Intermediate; Levels I and II
20.05, 0.39
Parkinson, Meakin,
and Salinger
(2015)
RCT; peer
reviewed report
Elementary (K
& 1st)
PD & BAU K
218
1st
228
K
218
1st
228
K
2309
1st
2112
K
2012
1st
1825
PAR: Letter Word Reading
Fluency, Vocabulary,
GRADE: Phonemic Awareness,
Word Reading and Meaning,
Comprehension
6 0.03 to 0.08
M¼0.04
(continued)
EQUITY AND EXCELLENCE IN EDUCATION 17
Table 2. Continued.
Teachers Students
Study ID Publication type School level Conditions PD NCtrl NPD NCtrl NStudent Outcome Measure(s) # of ES ES range
Phillips,
McNaughton,
and
MacDonald
(2004)
QED; peer reviewed Elementary
(K)
PD & matched
control &control
73 NR 108 135
100
Letter Identification test,
Concepts about Print, Word
Recognition test, Word
Reading test, Hearing and
Recording Sounds in Words,
Text Reading Levels
12 0.35 to 0.69
M¼0.48
Sailors and
Price (2010)
RCT; peer reviewed Elementary &
Middle
(2nd - 8th)
PD1 & PD2 27 17 200 327 GRADE 1 0.19
Scanlon, Gelzheiser,
Vellutino,
Schatschneider,
and
Sweeney (2008)
RCT; peer reviewed Elementary
(K)
PD
PD plus
intervention &
Intervention only
10
9
9 164
145
124 Phonological Awareness and
Literacy Screening Battery
PALS-K
2 0.12, 0.28
Van Keer and
Verhaeghe
(2005)
RCT; peer reviewed Elementary
(2nd
& 5th)
Intensive PD &
Restricted PD
2nd
7
5th
7
2nd
7
5th
9
2nd
139
5th
146
2nd
133
5th
196
Toetsen Begrijpend Lezen
Een minuut test
3 0.03 to 0.12
M¼0.09
van Kuijk, Deunk,
Bosker, and
Ritzema (2016)
QED; peer reviewed Elementary
(2nd
& 3rd)
PD & BAU 33 35 2nd
231
3rd
189
2nd
210
3rd
189
Cito standard reading
comprehension assessment
2 0.05, 0.07
Vernon-Feagans
et al. (2012)
RCT; federal report Elementary (K
& 1st)
PD & BAU 43 32 K-RD
91
K-typ
95
1st-RD
103
1st typ
111
K-RD
60
K-typ
60
1st -RD
56
1st typ 72
Peabody Picture Vocabulary
Test, WJ-III: Word Attack,
Letter Word ID, Passage
Comprehension
16 0.14 to 0.56
M¼0.25
Witmer, Duke,
Billman, and
Betts (2014)
RCT; peer reviewed Elementary
(1st
& 2nd)
PD & BAU 6 5 73 60 Concepts of Comprehension
Assessment
1 0.18
Note. RCT: randomized control trials; QED: quasi experimental design; DIBELS: Dynamic Indicators of Basic Early Literacy Skills; WJ-III: Woodcock Johnson- 3rd editions; GRADE: Group
Reading Assessment and Diagnostic Evaluation; TOWRE: Test of Word Reading Efficiency; PALS: Peer-Assisted Learning; PAR: Predictive Assessment of Reading.
18 L. DIDION ET AL.
Table 3. Professional development format and learning goals.
Study ID PD format
Teacher learning goal
(text described) Primary content focus Delivered by
Contact hours and duration
(text described)
Abe et al. (2012) Whole group, summer
coursework, PLC,
follow-up, active
participation
To improve instruction for reading
comprehension and
classroom pedagogy
Reading comprehension Local staff 295 contact hours per teacher, 42
days of PD over 2 years
Alexander
et al. (1987)
PD: whole group, active
participation
To be trained in the explicit theory
of analogical reasoning and
procedures for delivering
instruction to students
Analogical instruction Researchers Two 8 h sessions
Ctrl: manual Received instructional materials Analogical instruction Manual None
Ctrl: BAU
Angrist and
Lavy (1998)
Coaching, follow-up To improve instruction for reading,
the pass rate from grade to
grade, increase scores on
achievement tests, and support
overall school climate
Individualized instruction Researcher 11 hours per week for year one
and 16 hours per week for
year two
Bos et al. (1999) Summer
coursework, coaching
To integrate knowledge and skills
while promoting a positive
attitude toward using explicit
instruction for teaching reading
and spelling
Supporting students with
reading difficulties
Researchers, staff
member (coach)
3.5 hour sessions for 2.5 weeks
during the summer and one
hour sessions once per month
over the course of one year
Duffy et al. (1986) Whole group, follow up To learn how to instruct students
to use reading skills including
strategies to explicitly state the
reading skill being taught, when
it would be used and how to
apply it.
Emphasizing mental
processes to students
Researcher Initial training and 10 hours of
follow-up over the course of
the year
Fisher et al. (2011) Whole group, coaching To learn how to model thinking
while reading aloud
literacy strategies School faculty, peer
coach and researcher
NR
F€
orster and
Souvignier
(2015)
Whole group To use data for
instructional decisions
Data literacy Authors Three, two hour group
training sessions
Fuchs et al. (1992) PD1: whole group,
coaching, online
To support teachers CBM use and
consult on instructional
adjustments.
Data literacy Computer program,
doctoral students
Two, 2 hour after school
workshops, 30 min sessions, 4-
8 times per month over
17 weeks.
PD2: whole group, online To support teachers CBM use. Data literacy Computer program Two, 2 hour after school workshops
Ctrl: BAU No CBM
(continued)
EQUITY AND EXCELLENCE IN EDUCATION 19
Table 3. Continued.
Study ID PD format
Teacher learning goal
(text described) Primary content focus Delivered by
Contact hours and duration
(text described)
Garet et al. (2008) Whole group, follow-up,
active participation
PD A: To improve the knowledge
and practice of teachers to
support reading achievement in
high poverty schools
Instruction Language Essentials for
Teachers of Reading
and Spelling facilitators
(researchers)
48 total hours included five
institute days and three seminar
follow up days
Whole group, coaching,
follow-up, active
participation
PD B: To improve the knowledge
and practice of teachers to
support reading achievement in
high poverty schools
Instruction Language Essentials for
Teachers of Reading
and Spelling facilitators
(researchers)
48 total hours, five institute days
and three seminar follow up
days plus an average of
60 hours total of
coaching support
Gersten
et al. (2010)
PLC, collective
participation, active
participation
To improve vocabulary and
comprehension instruction
and knowledge.
Instructional practices Researcher 16 sessions; twice a month from
October to mid June at 75 min
per session
Jayanthi
et al. (2018)
PLC, active participation To improve teaching practice and
knowledge to increase student
vocabulary outcomes
Instructional practices Literacy personnel from
each school
Ten 1.25 h sessions, twice a month
from October to March
Knezek and
Christensen
(2007)
Summer coursework,
follow-up, active
participation,
To integrate technology
in classrooms
Technology Project KIDS staff and
online modules from
the Early
Reading Center
Five full day summer workshops
Magidin de Kramer
et al. (2012)
Online To learn best practices for vocab,
reading comprehension, and
writing instruction
Instruction e Learning for Education
Initiative staff
Three sessions with each session
lasting 7 weeks for
approximately 4-6 hours
each week
Matsumura
et al. (2010)
PLC, coaching, active
participation
To improve the quality of
classroom talk.
reading comprehension coaches NR
Matsumura
et al. (2013)
PLC, coaching, active
participation
To increase pedagogical knowledge
and abilities to plan, teach and
reflect on their lessons with a
specific emphasis on the
questioning the author approach
instruction coaches NR
McGill-Frazen,
Allington, Yokoi,
and
Brooks (1999)
Summer workshop,
follow-up
To put books in children’s hands
with focus on physical design of
classrooms, effective book
displays, read a-louds, interactive
techniques, environmental print,
author genre, content themes,
small group lessons, and literacy
activities during play
Enhancing access
to books
Researchers Three, eight hour sessions during
summer plus seven, two hour
follow-up training sessions
20 L. DIDION ET AL.
McIntyre
et al. (2010)
Whole group, follow-up,
active participation,
collective participation
To use grade level academic
instructional practices that are
more accessible to language
minority students
Instruction for ELLs Researcher Three, eight hour Saturday
sessions, followed by eight,
three hour after school sessions
for a total of 50 hours across
18 months
McMaster
et al. (2013)
Whole group, coaching,
follow up
To help create the school-level
conditions that support
effective teaching
Instruction Researcher, Project
coordinator, &
experienced teacher
Two sessions 30–45 min long, half-
day workshop, one
hour refresher
Miller and
Ellsworth (1985)
Coaching To build teachers’background
knowledge and change their
attitudes toward reading
instruction and subsequently
changing teaching behaviors.
General reading
instruction
University staff Three hour meetings once per
week over two years with less
structure/meeting time as
program continued
Moskowitz
et al. (1981)
Whole group, coaching To make classroom environments
responsive to students’
emotional and cognitive needs
by fostering positive attitudes,
behaviors, and norms regarding
self, peers, and school
Classroom management Researcher Minimum 8 hours total
Two hour sessions once
per week
Parkinson
et al. (2015)
Whole group, summer
coursework, coaching
To improve classroom
environments and literacy
instruction
Instruction Researcher 117.21 hours
Phillips et al. (2004) Whole group, active
participation
To learn how to deliver highly
focused instruction that enables
rapid development of children’s
understanding and participating
through joint activity
Instructional settings Researcher Ten half day sessions
Sailors and
Price (2010)
PD1: summer
coursework, coaching
To improve instructional reading
practices by teaching cognitive
reading strategies specially
related to intentional
comprehension instruction
Instruction University reading
faculty, Coaches
Two, eight hour summer
workshops plus averaged
329 minutes of coaching over
the course of the year
PD2: summer coursework To improve instructional reading
practices by teaching cognitive
reading strategies specially
related to intentional
comprehension instruction
Instruction University reading faculty Two, eight hour
summer workshops
Scanlon
et al. (2008)
PD only: whole group,
summer
coursework, coaching,
To develop teachers knowledge in
order to enable them to fully
understand students’
learning needs
Supporting students with
reading difficulties
Research staff Three day summer workshop, at
least five coaching sessions, and
one hour meetings once
per month
(continued)
EQUITY AND EXCELLENCE IN EDUCATION 21
Table 3. Continued.
Study ID PD format
Teacher learning goal
(text described) Primary content focus Delivered by
Contact hours and duration
(text described)
Supplemental Small
Group Intervention
Researchers delivered Interactive
Strategies Approach intervention
Supporting students with
reading difficulties
NA
Research staff directly
to students
None
Combined (PD and
intervention)
To develop teachers knowledge in
order to enable them to fully
understand students’
learning needs
Supporting students with
reading difficulties
Research staff Three day summer workshop, at
least five coaching sessions, and
one hour meetings once
per month
Van Keer and
Verhaeghe
(2005)
PD1: whole group,
coaching, active
participation
To update teachers’knowledge on
recent research developments in
the field of reading
comprehension and to translate
this knowledge into classroom
practice with the intention of
making reading comprehension
instruction in elementary schools
more effective
Reading practices Researcher Three hour initial session and an
average of 35 hours of coaching
over one year
PD2: Whole group, active
participation
Same as above Reading Practices Researcher Three, three hour meetings
van Kuijk
et al. (2016)
Whole group To improve the quality of reading
comprehension instruction and
to improve the degree to which
teachers attend to reading
strategies, and to learn how to
set student performance goals
and make instructional decisions
based on data
Data literacy/ instruction Researcher 40 hours that included nine after
school meetings 1.5 to 2.5 hours
and completing homework
assignments
Vernon-Feagans
et al. (2012)
Online To acquire key reading diagnostic
strategies relevant to K-1
struggling readers
Data literacy University staff NR
Witmer et al. (2014) Summer workshop, PLC,
follow-up, active
participation
To learn how to administer
Concepts of Comprehension
Assessment and interpret data to
facilitate student development of
fundamental knowledge and
skills for comprehending
informational texts
Test administration and
data literacy
Researcher Two 3.5 h sessions occurred before
school year, three, two hour
follow-up PLC sessions fall,
winter, and spring
Note. BAU: business-as-usual; Ctrl: Control group; PD: professional development group.
22 L. DIDION ET AL.
Subset Analyses
Subset analyses were run to examine differences in the average effect for study design
(randomized controlled trials versus quasi-experimental design) and reading outcome
measured (code-focused versus meaning-focused). First, we examined differences
between study design types in order to determine the impact of including quasi-experi-
mental designs. A total of 21 studies used a randomized controlled trial with 88 effect
sizes among them. Analysis indicates that in randomized controlled trials, PD signifi-
cantly improved student outcomes in reading, compared to control, with a moderate
average effect size, Hedges’g¼0.18, p<.05, and a 95% CI of [0.07, 0.29]. Experimental
effect sizes ranged from 0.15 to 0.94. A total of 7 studies used quasi-experimental
design with 22 effect sizes. Analysis of quasi-experimental design studies indicates that
PD significantly improved student outcomes in reading, compared to control, with an
Table 4. Aggregated study characteristics.
Characteristic k#ES
Grade level
Elementary 25 99
Middle 1 2
Combined (Elementary þMiddle) 2 9
Training formats/activities
Whole group 16 56
Summer coursework 8 31
PLC 6 10
Coaching 13 41
Online 2 26
Content focused 11 26
Active participation and/or Collective participation 12 38
Design type
RCT 21 88
QED 7 22
Delivered by
Researcher 17 73
District staff 4 7
Online 1 8
Both research and district staff 4 12
Other 3 11
Reading outcomes
Phonological awareness 7 24
Word ID 3 14
Decoding 1 2
Fluency 6 13
Vocabulary 7 17
Comprehension 15 24
General reading 7 16
Teacher characteristics
Mean experience (M¼12.34) 10 49
PD % advanced degree (M¼39.42) 13 59
Ctrl % advanced degree (M¼44.72) 13 59
Included special education teachers exclusively 1 8
Included general education teachers exclusively 26 99
Included both general and special education teacher 2 4
Student characteristics
Included special education students exclusively 1 8
Included students with reading difficulties exclusively 3 4
Included general education students exclusively 19 80
Included both general and special education students 5 18
Note. RCT: randomized control trial; QED: quasi-experimental design; PD: professional development; Ctrl: control.
EQUITY AND EXCELLENCE IN EDUCATION 23
average effect size, Hedges’g¼0.19, p<.05, and a 95% CI of [0.003, 0.39]. Quasi-
experimental effect sizes ranged from 0.05 to 0.69. We concluded that the effects of
study design type were similar to one another, as well as to the overall average effect
(g¼.18). Based on these results, there is rationale to include quasi-experimental designs
in the overall analysis of effects; however, it is included as a moderator of effects in the
meta-regression model.
Next, we compared the average effect for studies measuring code-focused outcomes
to studies measuring meaning-focused student outcomes to examine any differences
between the reading constructs. In total, 64 effect sizes measured code-focused
outcomes. The analysis indicates that PD significantly improved code-focused student
outcomes, compared to control, with a moderate average effect size, Hedges’g¼0.22,
p<.01, and a 95% CI of [0.09, 0.35]. Effect sizes ranged from 0.05 to 0.94. In total,
62 effect sizes measured meaning-focused outcomes. The analysis indicates that PD
significantly improved meaning-focused student outcomes, compared to control, with
an average effect size, Hedges’g¼0.17, p<.001, and a 95% CI of [0.07, 0.26]. Effect
sizes ranged from 0.15 to 0.92.
Description of Studies
Participant Characteristics
The 28 studies included in this meta-analysis represent a total of 1,348 teachers receiv-
ing PD (M¼49.93, SD ¼59.09) with 16,677 students (M¼521.16, SD ¼687.77). Across
the control conditions, studies included another 1,264 teachers (M¼50.56,
SD ¼6260.67) with 14,296 students (M¼446.75, SD ¼576.46). Further, the majority of
studies included student populations that were exclusively students in general education
(k¼19; ES n¼80). Five studies included both general education students and students
in special education (ES n¼18), three studies included populations that were exclusively
students with reading difficulties (ES n¼4), and one study included a population that
was exclusively students in special education (ES n¼8). Ten of the studies reported the
mean experience of teacher participants (ES n¼49). Average teacher experience was
12.34 years (SD ¼3.93). Thirteen studies reported the percent of teachers with advanced
degrees included in the participants (ES n¼59). The average percent of teachers with
advanced degrees was 39.42 (SD ¼22.96) in treatment conditions and 44.72
(SD ¼24.75) in control conditions.
Study Design
We were interested in examining what reading constructs were measured in studies, as well
as the quality of study descriptions. Reading constructs were coded into seven different cate-
gories: phonological awareness, word identification, decoding, fluency, vocabulary, compre-
hension, and general reading. More than half of the 28 studies measured comprehension
(k¼15), followed by phonological awareness (k¼7), vocabulary (k¼7), general reading
(k¼7), fluency (k¼6), word identification (k¼3), and decoding (k¼1). Overall study
reporting quality was satisfactory for purposes of this meta-analysis. PD in all studies were
led by staff directly trained by researchers. Students’reading outcomes were measured by
24 L. DIDION ET AL.
standardized assessments. Studies notably lacked on their reporting of teacher experience
(k¼10) and certification levels (k¼13). A quantitative quality code measured study replic-
ability. Results indicate that 64% of all studies could be replicated as described. Further,
results indicate that a higher percentage of quasi-experimental designs were described with
replicable detail (69%) as compared to the randomized controlled trials (60%). Finally, very
few studies (k ¼5) reported the procedural fidelity score of the training and the average
score reported was 90.18%. In terms of the quality of the research design, included studies
provided the pertinent information needed to calculate effect sizes and code important fea-
tures of moderating variables (e.g., intensity, PD focus, format); inferences were not neces-
sary to define studies’variables for the present meta-analysis. Further, 11 studies reported
that there was no differential attrition between treatment and comparisons groups, while the
remaining studies did not report information related to attrition. For randomized controlled
trials, 12 studies clearly described appropriate randomization procedures while the remain-
ing 9 experimental studies did not report information related to the procedures used for ran-
domizing participants into treatment and control conditions. Randomization was more
likely to be described in studies that occurred after 2007.
Components of PD
Intensity was calculated by adding the total hours of the PD in each study. The average
hours of PD was 51.89 (SD ¼65.70) with a range from 4 to 295 h across all 28 studies.
Twelve studies with 38 effect sizes included collaborative learning processes. In sum, 11
studies with 26 effect sizes were content focused.
Delivery Formats
Finally, studies could include one or more of five PD formats (a) whole group, (b) sum-
mer coursework, (c) PLC, (d) coaching, and (e) online learning. While 9 studies used a
single format, studies could receive more than one format code creating multicompo-
nent PD (k¼19). Of the 28, most studies included a whole group format (k¼16), fol-
lowed by coaching (k¼13), summer coursework (k¼8), PLC (k¼6), and online
learning (k¼2). Most effect sizes contributing to the overall pooled effect of PD on stu-
dent outcomes came from studies that used whole group (ES n¼56), followed by
coaching (ES n¼41), summer coursework (ES n¼31), online learning (ES n¼26), and
PLC (ES n¼10).
Moderator Analyses
Our three remaining research questions focus on potential moderators of the effects
of PD on reading outcomes. We consider the following categories in our moderator
analyses: study design, PD characteristics, and participant characteristics. To control
for the confounding effects among moderators, we entered all moderators in one
meta-regression model. As Table 1 shows, study design, intensity (dosage), collabora-
tive learning processes and active participation, content focus, format, reading con-
struct, years of teaching experience, level of certification, disability status, and
EQUITY AND EXCELLENCE IN EDUCATION 25
replicable quality did not significantly moderate the effect between PD and read-
ing outcomes.
Discussion
Teachers need high-quality PD to learn how to implement evidence-based practices that
improve student outcomes in reading. However, many questions remain about what fea-
tures make PD “high-quality.”The purpose of the present meta-analysis was to explore
components of teacher PD and their effects on students’reading achievement.
Specifically, we examined whether participation in teacher PD had an overall effect on
students’reading outcomes. We concluded that students of teachers who had received
PD performed significantly better than students of control teachers on reading measures.
In addition, we tested similar moderators as previous meta-analyses on teacher PD
(Blank & Alas, 2009; Yoon et al., 2007) to determine whether study design or PD char-
acteristics, and we extended the research to determine whether participant characteris-
tics had moderating effects on the impact of teacher PD. There were no significant
moderating effects.
Effect of Professional Development on Students’Reading Outcomes
To answer our primary research question investigating the effects of PD on student out-
comes in reading, effect sizes were calculated across 28 empirical studies, which resulted
in a sample that largely consisted of PD for teachers of typically-developing students in
elementary school. Results indicate that PD had a significant positive, average effect on
student outcomes in reading. Further, there was variation in the effect sizes calculated.
The prediction interval ranged between 0.23 and 0.57. While the confidence interval
provides information related to the true effect in 95% of random samples, the prediction
interval is an estimate that contains 95% of the values of the effect in a population
(Borenstein, Higgins, Hedges, & Rothstein, 2017). It allows us to make inferences about
the distribution of effect sizes (Borenstein et al., 2017). What is particularly important
to note in this meta-analysis is that the prediction interval crosses 0. This indicates that
control students outperformed students of teachers that received PD on reading meas-
ures and therefore, findings were contradictory across studies. The variation of effects
across studies included in this meta-analysis confirms the need to examine moderat-
ing variables.
Moderator Analyses
To answer our remaining questions examining moderate effects of PD on students’reading
outcomes, moderator analyses were explored for study design, PD characteristics, teacher
characteristics, and student characteristics. Overall, these moderators did not significantly
influence the PD programs’overall effectiveness on student reading performance. A discus-
sion of variables related to study design, characteristics of PD and participants follows.
26 L. DIDION ET AL.
Characteristics of Study Design
We first ran a subset analysis to observe differences between included randomized con-
trolled trials and quasi-experimental designs. Both types of study designs had similar
significant effect size estimates. Therefore, to gain a more inclusive and broader answer
to our primary research question, we included the quasi-experimental designs. This
decision was based on a similar effect the PD had on student reading outcomes in these
studies as compared to experimental trials.
Both types of study designs were also compared to examine differences in replicabil-
ity. A subjective measure was used to assess reporting quality of the descriptions of the
professional development to determine whether or not they could be replicated as
described. Results indicated that 64% of all studies could be replicated as described, with
quasi-experimental designs being more replicable (69%) than experimental controlled
trials (60%). It is concerning that studies provided limited information for future
researchers to replicate the methods used. Replication research is important in building
the foundation for evidence-based practices (Travers, Cook, Therrien, & Coyne, 2016).
Further, replication is central to empirical research to confirm and validate effects
(Cook et al., 2015). If studies do not describe their methods of intervention or reform,
as in this case with professional development, replication efforts are hindered. Future
research of professional development should include comprehensive details about the
methods used.
Also, a subset analysis was run to compare the average effect for studies measuring
code-focused outcomes to studies measuring meaning-focused student outcomes to
examine any differences between the reading constructs. Results indicate that code-
focused outcomes had a larger effect than meaning-focused outcomes. We suggest this
may be due to the professional developments provided. Just two studies (Abe, Thomas,
Sinicrope, & Gee, 2012; Matsumura, Garnier, Correnti, Junker, & DiPrima Bickel, 2010)
focused on improving reading comprehension, and none focused on vocabulary instruc-
tion. Yet, more than half of the studies measured reading comprehension (k¼16), and
seven measured vocabulary outcomes. It is expected that code-focused instruction would
promote comprehension and vocabulary to an extent, but teachers still need more
instruction on providing explicit meaning-focused instruction to improve student out-
comes. Without direct focus on improving the instructional strategies teachers use to
improve reading comprehension (and vocabulary), it is likely that the teachers returned
to their traditional practices after PD was complete (Snow, 2002). Further, comprehen-
sion assessments are not grounded in the developmental process of reading comprehen-
sion or as a product of instruction but rather, they are related to proficiency in other
inter-connected reading skills (Snow, 2002). Often, assessments measure comprehension
skills differently. For instance, Keenan and Meenan (2014), examined the results of 1000
students’performance on four standardized reading comprehension assessments which
resulted in a median correlation of .54. This is concerning since these assessments claim
to measure the same skill. This variability in comprehension may require teachers to
instruct comprehension using different strategies to assess individual skills (Catts &
Kamhi, 2017). There is a need to investigate PD to improve teachers’practices in
promoting meaning-focused growth.
EQUITY AND EXCELLENCE IN EDUCATION 27
Characteristics of Professional Development
Research and policy agree that intensity and collaborative learning opportunities are founda-
tional to high-quality PD, yet no moderating effects were found in the current meta-analysis.
Heterogeneity of PD programs may play a role in the non-significant effects on student out-
comes, such that PDs used differing intensities with various combinations of formats. This is
similar to previous meta-analyses looking at teacher PD and student outcomes (Blank &
Alas, 2009; Yoon et al., 2007); differences in learning goals and PD designs create problems
when arriving to a consensus. In this meta-analysis, the intensity variable measured by the
number of contact hours ranged from 4 to 295h across all studies. It has been suggested that
greater than 14 h (Yoon et al., 2007), 20 h (Desimone, 2009), 30 h (Guskey & Yoon, 2009),
and 49 h (Darling-Hammond et al., 2009) all produce best results. Research has yet to agree
on the level of intensity needed to change teacher knowledge and instructional practices to
produce improved student outcomes. This variation suggests that there are other compo-
nents of PD affecting student outcomes regardless of intensity.
Overall, formats observed in this meta-analysis were also relatively heterogeneous
spanning from reliance on a single type of format (e.g., King, Deno, Mirkin, & Wesson,
1983) to multi-component programs (e.g., Abe et al., 2012). Moderator analysis further
examined whether including specific formats (i.e., whole group, coaching, summer
coursework, PLC, coaching) influenced the impact of teacher PD, and it was not
observed that they did. Often studies used one or more of these components together,
so even if effects were found it would be hard to say why. The influence that these for-
mats may have on one another is as important as how each format works alone. Past
meta-analyses suggest that incorporating learning opportunities over the summer
increases intensity (Blank & Alas, 2009), which may impact student learning. We did
not find that summer coursework positively influenced student outcomes in reading.
Further, it was surprising that neither coaching nor PLCs provided a positive significant
impact on student reading outcomes. These formats incorporate increased collaborative
learning opportunities (Connor, 2017; Darling-Hammond et al., 2017) and are increas-
ingly used in policy initiatives (Darling-Hammond et al., 2009). It is possible that such
methods did not support teachers as intended (e.g., procedural fidelity; discussed below)
and more investigation is needed.
Further, our analyses did not reveal moderating effects when active or collective par-
ticipation was embedded even though research and policy support this as best practice
(Hochberg & Desimone, 2010). Collaborative learning engages teachers in the learning
process versus experiencing PD through traditional lecture (Darling-Hammond et al.,
2017) where teachers are expected to sit, listen, and attend. Previous meta-analyses
investigating effects of teacher PD on student outcomes in mathematics and science
provide evidence that active methods that engage participants improve teacher learning
(Blank & Alas, 2009). Studies included in this meta-analysis did not provide ample
information about the quality of collaborative learning processes, so it is difficult to
make judgments for why effects were not found. Investigations of PD should consider
assessing the condition of teachers’learning experiences rather than focusing on time
spent learning. For both intensity and collaborative learning processes, the quality of the
learning experiences may be more important.
28 L. DIDION ET AL.
It is difficult to make judgments about quality of format and learning experiences
because very few studies (k¼5) reported the procedural fidelity score of the training.
When investigating teacher PD as the primary independent variable, it is highly advis-
able to develop measures of fidelity of implementation to ensure that goals of PD are
clearly outlined and followed through as intended. Programs without this information
may discover that their PD was not implemented as designed and therefore effects may
be weakened or strengthened by excluded or evolved agendas. Some studies admitted
that structure and continuity lessened over time (Miller & Ellsworth, 1985). Measures of
fidelity would explain more about the impact of this program’s implementation levels.
Characteristics of Participants
Research supports that students with reading difficulties need effective intervention to
prevent persistent problems resulting in long-term personal and societal level benefits
(Connor et al., 2014). Yet, few studies included in this meta-analysis examined how to
strengthen teachers’abilities to support these learners. Only four studies looked exclu-
sively at reading outcomes for students with special needs (k¼1) or reading difficulties
(k¼3), and five studies looked at a mixed population of general and special education
students. The majority of the effect sizes in the pooled estimate were contributed from
populations of general education students. There may be a reason to believe that train-
ing for teachers of students with and at-risk for reading disabilities requires a unique
experience as compared to teachers of typically-developing students. Teachers rarely
receive PD that is focused on addressing the needs of students with disabilities
(Darling-Hammond et al., 2009). Prevention efforts may be enhanced if teachers are
provided with quality PD on how to best support learners who struggle with reading.
Also, teachers in elementary school are the first step in preventing students from hav-
ing persistent difficulties throughout their schooling. The majority of studies included in
this meta-analysis were from elementary school. Four studies looked at the effects of
student outcomes for middle school students, exclusively (k¼1) and with elementary
students (k¼2). Research has indicated that letter-word identification and comprehen-
sion growth slows over time for students with disabilities (Wei, Darling-Hammond,
Andree, Richardson, & Orphanos, 2009). Perhaps these outcomes can be attributed to
the lack of research on PD and secondary reading teachers’students. Teacher training at
the secondary level may be important to improve post-school outcomes for students
with reading difficulties. Considering that teachers of secondary students often share the
same caseloads, PD that includes collaborative learning opportunities is a worthwhile
investment (Bryant, Linan-Thompson, Ugel, Hamff, & Hougen, 2001; George, 2001).
Relevance may also play an important role in PD effects. Without teacher buy-in,
reformed practices go by the wayside. Unfortunately, nearly half of teachers in the U.S. are
dissatisfied with their learning experiences through PD (Darling-Hammond et al., 2009).
Negative effects observed on student outcomes may be a result of teachers’negative emo-
tional responses, like resistance to adopt new strategies (Kennedy, 2016). Factors like teach-
ers’willingness to participate influence the likelihood they engage with and benefit from
training (Brady et al., 2009). PD is only as effective as teachers’desires to incorporate new
practices into their instruction (Buczynski & Hansen, 2010). More research is needed to
EQUITY AND EXCELLENCE IN EDUCATION 29
understand what motivates teachers to engage with PD in a meaningful way. Understanding
teachers’motivation to learn is crucial. Acquired through informed experiences, teachers’
disposition to learn influences how teachers learn (Opfer & Pedder, 2011). Important ques-
tions related to what professional learning experiences matter to teachers go unanswered.
Limitations and Implications for Practice
Although our results suggest an overall positive average effect of teacher PD on the
reading achievement of students in grades K-8, our analyses are limited by lack of infor-
mation on the effects of teacher knowledge and study reporting quality. There are sev-
eral limitations to the present study, which we describe below. We also propose avenues
for future research that will assist in addressing the gaps in our knowledge, such as con-
sidering other factors that mediate the relationship between PD’s effects on student out-
comes and examining populations of teachers of students with disabilities and
secondary teachers as well.
Similar to past meta-analytic research (Blank & Alas, 2009; Yoon et al., 2007), the pre-
sent study examined the effect of teacher PD on students’achievement; therefore, we
were unable to examine variables that are assumed to underlie these relations. For
example, there are questions that remain about the impact of PD on teacher-level out-
comes, such as increased content knowledge or improved instructional practices. One
would assume that these outcomes would mediate the relations between PD and student
outcomes. Most measures used to evaluate teacher-level variables are not standardized or
are self-report (Kang, Cha, & Ha, 2013). Teacher perception data has bias, and developing
standardized measures should be considered. Similar to studying outcomes for students in
kindergarten through 12th grade, more research is needed to develop measures that can
be comparable across populations of teachers. With a meta-analysis that looks closer at
teacher outcomes, like reading knowledge, instructional practices and relevance, we can
align teacher PD to meet policy goals. More research is needed that investigates the rela-
tionship of PD and teachers’knowledge, attitudes and beliefs (Kang et al., 2013).
Also, the quality of these studies was not evaluated using procedural standards, such
as those provided by WWC, which are designed to identify the most valid and scientif-
ically defensible evidence available. It is the researchers’responsibility to take care in
designing rigorous research to develop the field of professional learning (Guskey &
Yoon, 2009). Valid conclusions about PD design components that are effective may be
better answered if studies included had the highest quality.
As previously noted, only one study examined reading outcomes for students with
disabilities. As students with disabilities are at the most substantial risk of reading fail-
ure, there is an urgent need for high-quality experimental studies that allow for the
investigation of teacher PD in this area and potential causal relations. Students with
reading difficulties require expertise to individualize instruction (Lemons, Al Otaiba,
Conway, & Mellado De La Cruz, 2016), and it is unknown whether PD reviewed here
directly addressed this need. Further, with few studies at the secondary level (k¼4),
there is a significant need to examine effective PD for these teachers. Processes, such as
collaborative learning, may have a stronger effect for this population. Our overall ana-
lysis is more of a reflection of the effects of PD on elementary student outcomes.
30 L. DIDION ET AL.
Overall, results should be interpreted with caution; we are not suggesting that compo-
nents related to PD do not have influence on results, but the current body of evidence
is inconclusive and raises important questions about the study of mechanisms of change
in PD. The results of this meta-analysis may suggest that while single studies have found
connections between moderating variables (e.g., study design, intensity, collaborative
learning processes, content focus, format), aggregating studies within a meta-analysis
diminishes moderating variables’overall average effect. It is unclear what happens
between when teachers receive instruction through PD and the distal outcome of stu-
dent reading performance. Is there a lag between when teachers learn strategies to when
they incorporate them into instruction? How long do teachers continue to use new tech-
niques after PD has ended? Are teachers implementing new techniques correctly?
Future research is needed to understand what mediates the relationship between effect-
ive PD and improved student reading performance.
Although moderating variables did not offer explanation for the variation across stud-
ies, it is important to report these null effects to avoid reporting bias (Kirkham et al.,
2010). Including the present meta-analysis, there is now evidence across disciplines
(Blank & Alas, 2009; Yoon et al., 2007) that heterogeneity in PD programs results in
inconclusive and inconsistent moderating effects on student outcomes for variables such
as intensity, learning experiences, and program format. It has been argued that research
on teacher training move beyond investigating components that make PD effective and
instead explore how it is that teachers learn (Webster-Wright, 2009). As participants in
PD, teachers have a dual perspective as both teacher and learner (Bates, Swennen, &
Jones, 2011; Darling-Hammond & McLaughlin, 2011). Research is flush with how stu-
dents learn, but little is known about teacher learners. Teachers as learners need guid-
ance to implement new methods of instruction into current practices (Kennedy, 2016;
Loughran, 2014). Program design should take both roles into account with an emphasis
placed on what teachers need to know, what they are able to do as a result of training
(Richmond, Bartell, & Dunn, 2016) and methods to measure their knowledge and per-
formance. PD study design should consider that there may be a difference between pro-
fessional development and professional learning (Groundwater-Smith & Mockler, 2009;
Hardy, 2010). The challenge of PD is three-fold: to provide teachers with increased lev-
els of pedagogical knowledge, to create teacher buy-in to a new set of practices, and to
provide adequate training for teachers to effectively deliver the new practices (Hochberg
& Desimone, 2010). These challenges are met with understanding teachers’learning
how to learn and using their newly acquired knowledge to change their instructional
practices for improved student outcomes (Avalos, 2011). PD should focus on working
with teachers rather than presenting new methods. Teachers learn by doing and reflect-
ing on their experiences similar to students (Darling-Hammond & McLaughlin, 2011).
The experience of teacher learning is not fully understood (Webster-Wright, 2009).
Conclusion
This meta-analytic review documents the effectiveness of teacher PD in improving reading
outcomes for students in grades K through 8. Questions remain, such as the quality of
included studies, the effects of PD on teacher outcomes, and defining/coding other core
EQUITY AND EXCELLENCE IN EDUCATION 31
components (e.g., quality of learning experiences). It should be emphasized that findings
from this meta-analysis are not claiming that components such as collaborative learning or
formats such as coaching and PLCs are ineffective. More so, it is important to take away
from results that study design should be carefully considered to more accurately answer
research questions about these topics. Future research designs should be set up to provide
BAU conditions that include more than the status quo. For example, to better answer ques-
tions about active participation, PD with similar content and learning goals could be used,
however, the treatment group could receive opportunities to actively participate in their
learning. Further, components that make up teacher PD may not be the best predictors of
the effectiveness of programing (Kennedy, 2016). Effective PD may have more to do with
quality of experiences rather than intensity. PD should focus on quality experiences since
conclusions about intensity or activities cannot be drawn across academic disciplines (Blank
&Alas,2009; Yoon et al., 2007). Teachers need time in order to deepen their understanding
of reformed practices, but if that time is not used wisely, the benefit will be minimal (Guskey
&Yoon,2009). Effective PD is important and we should design high-quality studies focused
more on coherent programing and goals that target teacher learning.
Funding
This work was supported in part by grant H325H140001 from the Office of Special Education
Programs, U.S. Department of Education. Nothing in the article necessarily reflects the positions
or policies of the federal government, and no endorsement by it should be inferred.
ORCID
Lisa Didion http://orcid.org/0000-0001-5218-2449
Jessica R. Toste http://orcid.org/0000-0002-6327-0054
Marissa J. Filderman http://orcid.org/0000-0002-8498-7345
ARTICLE HISTORY
Received 3 July 2018
Revised 12 July 2019
Accepted 27 August 2019
References
Abe, Y., Thomas, V., Sinicrope, C., & Gee, K. A. (2012). Effects of the pacific CHILD professional
development program. Final report. National Center for Education Evaluation and Regional
Assistance (NCEE 2013-4002). Washington, DC: U.S. Department of Education.
Alexander, P. A., Wilson, A. F., White, C. S., Willson, V. L., Tallent, M. K., & Shutes, R. E.
(1987). Effects of teacher training on children’s analogical reasoning performance. Teaching
and Teacher Education,3(4), 275–285. doi:10.1016/0742-051X(87)90020-5
Angrist, J. D., & Lavy, V. (1998). Does teacher training affect pupil learning? Evidence from
matched comparisons in Jerusalem public schools (NBER-WP-6781). Cambridge, MA: National
Bureau of Economic Research.
Avalos, B. (2011). Teacher professional development in teaching and teacher education over ten
years. Teaching and Teacher Education,27(1), 10–20. doi:10.1016/j.tate.2010.08.007
32 L. DIDION ET AL.
Banilower, E., & Shimkus, E. (2004). Professional development observation study. Chapel Hill, NC:
Horizon Research.
Bates, T., Swennen, A., & Jones, K. (2011). Teacher educators –A professional development per-
spective. Professional Development in Education,36(1–2), 719. doi:10.1080/19415250903457653
Biancarosa, G., Bryk, A. S., & Dexter, E. R. (2010). Assessing the value-added effects of literacy
collaborative professional development on student learning. The Elementary School Journal,
111(1), 7–34.
Blank, R., & Alas, N. (2009). Effects of teacher professional development on gains in student
achievement. Report prepared for the Council of Chief State School Officers.
Borenstein, M., Higgins, J. P., Hedges, L. V., & Rothstein, H. R. (2017). Basics of meta-analysis:
I2 is not an absolute measure of heterogeneity. Research Synthesis Methods,8(1), 5–18. doi:10.
1002/jrsm.1230
Bos, C. S., Mather, N., Narr, R. F., & Babur, N. (1999). Interactive, collaborative professional
development in early literacy instruction: Supporting the balancing act. Learning Disabilities
Research and Practice,14(4), 227–238. doi:10.1207/sldrp1404_4
Brady, S., Gillis, M., Smith, T., Lavalette, M., Liss-Bronstein, L., Lowe, E., …Wilder, T. D.
(2009). First grade teachers’knowledge of phonological awareness and code concepts:
Examining gains from an intensive form of professional development and corresponding
teacher attitudes. Reading and Writing,22(4), 425–455. doi:10.1007/s11145-009-9166-x
Brownell, M. T., Adams, A., Sindelar, P., Waldron, N., & van Hover, S. (2006). Learning from
collaboration: The role of teacher qualities. Exceptional Children,72(2), 169–187. doi:10.1177/
001440290607200203
Bryant, D. P., Linan-Thompson, S., Ugel, N., Hamff, A., & Hougen, M. (2001). The effects of pro-
fessional development for middle school general and special education teachers on implemen-
tation of reading strategies in inclusive content area classes. Learning Disability Quarterly,
24(4), 251–264. doi:10.2307/1511114
Buczynski, S., & Hansen, C. B. (2010). Impact of professional development on teacher practice:
Uncovering connections. Teaching and Teacher Education,26(3), 599–607. doi:10.1016/j.tate.
2009.09.006
Carlisle, J. F., Cortina, K. S., & Katz, L. A. (2011). First-grade teachers’response to three models
of professional development in reading. Reading and Writing Quarterly,27(3), 212–238. doi:10.
1080/10573569.2011.560482
Catts, H. W., & Kamhi, A. G. (2017). Prologue: Reading comprehension is not a single ability.
Language, Speech, and Hearing Services in Schools,48(2), 73–76. doi:10.1044/2017_LSHSS-16-
0033
Chall, J. S. (1983). Learning to read: The great debate. New York, NY:McGraw-Hill.
Cole, D. C. (1992). The effects of a one-year staff development program on the achievement test
scores of fourth grade students (Unpublished doctoral dissertation). University of Mississippi,
Oxford, MS.
Connor, C. M. (2017). Commentary on the special issue on instructional coaching models:
Common elements of effective coaching models. Theory into Practice,56(1), 78–83. doi:10.
1080/00405841.2016.1274575
Connor, C. M., Alberto, P. A., Compton, D. L., & O’Connor, R. E. (2014). Improving reading out-
comes for students with or at risk for reading disabilities: A synthesis of the contributions from
the institute of education sciences research centers. National Center for Special Education
Research (NCSER 2014-3000). Washington, DC: U.S. Department of Education.
Cook, B. G., Buysse, V., Klingner, J., Landrum, T. J., McWilliam, R. A., Tankersley, M., & Test,
D. W. (2015). CEC’s standards for classifying the evidence base of practices in special educa-
tion. Remedial and Special Education,36(4), 220–234. doi:10.1177/0741932514557271
Cortiella, C., & Horowitz, S. H. (2014). The state of learning disabilities: Facts, trends and emerg-
ing issues. New York, NY:National Center for Learning Disabilities.
Darling-Hammond, L., Hyler, M. E., & Garder, M. (2017). Effective teacher professional develop-
ment. Palo Alto, CA:Learning Policy Institute.
EQUITY AND EXCELLENCE IN EDUCATION 33
Darling-Hammond, L., & McLaughlin, M. W. (2011). Policies that support professional develop-
ment in an era of reform. Phi Delta Kappan,92(6), 81–92. doi:10.1177/003172171109200622
Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009).
Professional learning in the learning profession. Washington, DC:National Staff Development
Council.
Desimone, L. M. (2009). Improving impact studies of teachers’professional development: Toward
better conceptualizations and measures. Educational Researcher,38(3), 181–199. doi:10.3102/
0013189X08331140
Dingle, M. P., Brownell, M. T., Leko, M. M., Boardman, A. G., & Haager, D. (2011). Developing
effective special education reading teachers: The influence of professional development, context,
and individual qualities. Learning Disability Quarterly,34(1), 87–103. doi:10.1177/
073194871103400106
Duffy, G. G., Roehler, L. R., Meloth, M. S., Vavrus, L. G., Book, C., Putnam, J., & Wesselman, R.
(1986). The relationship between explicit verbal explanations during reading skill instruction
and student awareness and achievement: A study of reading teacher effects. Reading Research
Quarterly,21(3), 237–252. doi:10.2307/747707
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a
simple, graphical test. BMJ (Clinical Research ed.),315(7109), 629–634. doi:10.1136/bmj.315.
7109.629
Fisher, D., Frey, N., & Lapp, D. (2011). Coaching middle-level teachers to think aloud improves
comprehension instruction and student reading achievement. The Teacher Educator,46(3),
231–243. doi:10.1080/08878730.2011.580043
F€
orster, N., & Souvignier, E. (2015). Effects of providing teachers with information about their
students’reading progress. School Psychology Review,44(1), 60–75. doi:10.17105/SPR44-1.60-75
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations,
and interpretation. Journal of Experimental Psychology: General,141(1), 2–18. doi:10.1037/
a0024338
Fuchs, D., & Fuchs, L. S. (2015). Rethinking service delivery for students with significant learning
problems: Developing and implementing intensive instruction. Remedial and Special Education,
36(2), 105–111. doi:10.1177/0741932514558337
Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Ferguson, C. (1992). Effects of expert system consult-
ation within curriculum-based measurement, using a reading maze task. Exceptional Children,
58(5), 436–450. doi:10.1177/001440299205800507
Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., …Zhu, P. (2008). The
impact of two professional development interventions on early reading instruction and achieve-
ment. National Center for Education Evaluation and Regional Assistance (NCEE 2008-4030).
Washington, DC: U.S. Department of Education.
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes pro-
fessional development effective? Results from a national sample of teachers. American
Educational Research Journal,38(4), 915–945. doi:10.3102/00028312038004915
George, P. S. (2001). The evolution of middle schools. Educational Leadership,58(4), 40–44.
Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. E. (2010). Teacher study group:
Impact of the professional development model on reading instruction and student outcomes in
first grade classrooms. American Educational Research Journal,47(3), 694–739. doi:10.3102/
0002831209361208
Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes: Random-effects model. In
H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and
meta-analysis (pp. 357–376). New York, NY: Russell Sage Foundation.
Groundwater-Smith, S., & Mockler, N. (2009). Teacher professional learning in an age of compli-
ance: Mind the gap. Dordrecht, The Netherlands: Springer Science & Business Media.
Guskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta
Kappan,90(7), 495–500. doi:10.1177/003172170909000709
Hardy, I. (2010). Critiquing teacher professional development: Teacher learning within the field
of teachers’work. Critical Studies in Education,51(1), 71–84. doi:10.1080/17508480903450232
34 L. DIDION ET AL.
Hattie, J. A. (2009). Visible learning: A synthesis of 800þmeta-analyses on achievement.
Abingdon, UK: Routledge.
Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. New York, NY:
Routledge.
Hawley, W. D., & Valli, L. (1999). The essentials of effective professional development: A new
consensus. In Teaching as the learning profession: Handbook of policy and practice
(pp. 127–150). New York, NY: Teachers College Press.
Hedberg, E. C. (2014). ROBUMETA: Stata module to perform robust variance estimation in meta-
regression with dependent effect size estimates. Chestnut Hill, MA: Statistical Software
Components.
Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estima-
tors. Journal of Educational Statistics,6(2), 107–128. doi:10.3102/10769986006002107
Hedges, L. V. (2007). Effect size estimates in cluster-randomized designs. Journal of Educational
and Behavioral Statistics,32(4), 341–370. doi:10.3102/1076998606298043
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regres-
sion with dependent effect size estimates. Research Synthesis Methods,1(1), 39–65. doi:10.1002/
jrsm.5
Hochberg, E. D., & Desimone, L. M. (2010). Professional development in the accountability con-
text: Building capacity to achieve standards. Educational Psychologist,45(2), 89–106. doi:10.
1080/00461521003703052
Jayanthi, M., Dimino, J., Gersten, R., Taylor, M. J., Haymond, K., Smolkowski, K., & Newman-
Gonchar, R. (2018). The impact of teacher study groups in vocabulary on teaching practice,
teacher knowledge, and student vocabulary knowledge: A large-scale replication study. Journal
of Research on Educational Effectiveness,11(1), 83–108. doi:10.1080/19345747.2017.1327625
Kang, H. S., Cha, J., & Ha, B. W. (2013). What should we consider in teachers’professional
development impact studies? Based on the conceptual framework of Desimone. Creative
Education,04(04), 11–18. doi:10.4236/ce.2013.44A003
Keenan, J. M., & Meenan, C. E. (2014). Test differences in diagnosing reading comprehension
deficits. Journal of Learning Disabilities,47(2), 125–135. doi:10.1177/0022219412439326
Kennedy, M. (1998). Form and substance in inservice teacher education. Research Monograph
(No–13). Madison, WI: National Institute for Science Education.
Kennedy, M. M. (2016). How does professional development improve teaching? Review of
Educational Research,86(4), 945–980. doi:10.3102/0034654315626800
King, R. P., Deno, S., Mirkin, P., & Wesson, C. (1983). The effects of training teachers in the use
of formative evaluation in reading: An experimental-control comparison (Research Report No.
111). Retrieved from https://files.eric.ed.gov/fulltext/ED236189.pdf
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson,
P. R. (2010). The impact of outcome reporting bias in randomised controlled trials on a cohort
of systematic reviews. BMJ,340, c365. doi:10.1136/bmj.c365
Knezek, G., & Christensen, R. (2007). Effect of technology-based programs on first-and second-
grade reading achievement. Computers in the Schools,24(3–4), 23–41. doi:10.1300/J025v24n03_
03
Leach, J. M., Scarborough, H. S., & Rescorla, L. (2003). Late-emerging reading disabilities. Journal
of Educational Psychology,95(2), 211–224. doi:10.1037/0022-0663.95.2.211
Leithwood, K., Harris, A., & Hopkins, D. (2008). Seven strong claims about successful school
leadership. School Leadership and Management,28(1), 27–42. doi:10.1080/13632434.2019.
1596077
Leko, M. M., Kiely, M. T., Brownell, M. T., Osipova, A., Dingle, M. P., & Mundy, C. A. (2015).
Understanding special educators’learning opportunities in collaborative groups: The role of
discourse. Teacher Education and Special Education: The Journal of the Teacher Education
Division of the Council for Exceptional Children,38(2), 138–157. doi:10.1177/0888406414557283
Lemons, C. J., Al Otaiba, S., Conway, S. J., & Mellado De La Cruz, V. (2016). Improving profes-
sional development to enhance reading outcomes for students in special education. New
Directions for Child and Adolescent Development,154,87–104. doi:10.1002/cad.20177
EQUITY AND EXCELLENCE IN EDUCATION 35
Little, J. W. (1993). Teachers’professional development in a climate of educational reform.
Educational Evaluation and Policy Analysis,15(2), 129–151. doi:10.2307/1164418
Loughran, J. (2014). Professionally developing as a teacher educator. Journal of Teacher
Education,65(4), 271–283. doi:10.1177/0022487114533386
Magidin de Kramer, R., Masters, J., O’Dwyer, L. M., Dash, S., & Russell, M. (2012). Relationship
of online teacher professional development to seventh-grade teachers’and students’knowledge
and practices in English language arts. The Teacher Educator,47(3), 236–259. doi:10.1080/
08878730.2012.685795
Matsumura, L. C., Garnier, H. E., Correnti, R., Junker, B., & DiPrima Bickel, D. (2010).
Investigating the effectiveness of a comprehensive literacy coaching program in schools with
high teacher mobility. The Elementary School Journal,111(1), 35–62. doi:10.1086/653469
Matsumura, L. C., Garnier, H. E., & Spybrook, J. (2013). Literacy coaching to improve student
reading achievement: A multi-level mediation model. Learning and Instruction,25,35–48. doi:
10.1016/j.learninstruc.2012.11.001
McCutchen, D., Harry, D. R., Cox, S., Sidman, S., Covill, A. E., & Cunningham, A. E. (2002).
Reading teachers’knowledge of children’s literature and English phonology. Annals of Dyslexia,
52(1), 205–228. doi:10.1007/s11881-002-0013-x
McGill-Franzen, A., Allington, R. L., Yokoi, L., & Brooks, G. (1999). Putting books in the class-
room seems necessary but not sufficient. The Journal of Educational Research,93(2), 67–74.
doi:10.1080/00220679909597631
McIntyre, E., Kyle, D., Chen, C. T., Mu~
noz, M., & Beldon, S. (2010). Teacher learning and ELL
reading achievement in sheltered instruction classrooms: Linking professional development to
student development. Literacy Research and Instruction,49(4), 334–351. doi:10.1080/
19388070903229412
McMaster, K. L., Han, I., Coolong-Chaffin, M., & Fuchs, D. (2013). Promoting teachers’use of
scientifically based instruction: A comparison of university versus district support. The
Elementary School Journal,113(3), 303–330.
Meyers, S. J., & Sutton, J. T. (2006). Linking teacher characteristics to student mathematics out-
comes: Preliminary evidence of impact on teachers and students after participation in the first
year of the Math in Middle Institute Partnership. Paper presented at the MSP Evaluation
Summit II. Minneapolis, MN, Unpublished manuscript.
Miller, J. W., & Ellsworth, R. (1985). The evaluation of a two-year program to improve teacher
effectiveness in reading instruction. The Elementary School Journal,85(4), 485–495. doi:10.
1086/461416
Moskowitz, J. M., Malvin, J. H., Schaeffer, G. A., Schaps, E., & Condon, J. W. (1981). The effects
of a classroom management teacher training primary prevention program on fifth-grade students.
Napa, CA: Pacific Institute for Research and Evaluation.
National Reading Panel (NRP). (2000). Teaching children to read: An evidence-based assessment of
the scientific research literature on reading and its implications for reading instruction (National
Institute of Health Pub. No. 00-4769). Washington, DC: National Institute of Child Health and
Human Development.
O’Connor, R. E. (1999). Teachers learning ladders to literacy. Learning Disabilities Research &
Practice,14(4), 203–214.
Opfer, V. D., & Pedder, D. (2011). Conceptualizing teacher professional learning. Review of
Educational Research,81(3), 376–407. doi:10.3102/0034654311413609
Parkinson, J., Meakin, J., & Salinger, T. (2015). Longitudinal impacts of the children’s literacy ini-
tiative professional development, coaching, and model classroom intervention on early literacy
achievement. Evanston, IL: Society for Research on Educational Effectiveness. Retrieved from
http://eric.ed.gov/?id=ED56235.7
Phillips, G., McNaughton, S., & MacDonald, S. (2004). Managing the mismatch: Enhancing early
literacy progress for children with diverse language and cultural identities in mainstream urban
schools in New Zealand. Journal of Educational Psychology,96(2), 309–323. doi:10.1037/0022-
0663.96.2.309
36 L. DIDION ET AL.
Reynolds, A. J., & Ou, S. R. (2004). Alterable predictors of child well-being in the Chicago longi-
tudinal study. Children and Youth Services Review,26(1), 1–14. doi:10.1016/j.childyouth.2003.
11.005
Richmond, G., Bartell, T., & Dunn, A. H. (2016). Beyond “tinkering”: Enacting the imperative for
change in teacher education in a climate of standards and accountability. Journal of Teacher
Education,67(2), 102–104. doi:10.1177/0022487116628837
Sailors, M., & Price, L. R. (2010). Professional development that supports the teaching of
cognitive reading strategy instruction. The Elementary School Journal,110(3), 301–322. doi:10.
1086/648980
Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student
academic achievement. Knoxville, TN: The University of Tennessee Value-Added Research
Center.
Saxe, G. B., Gearhart, M., & Nasir, N. I S. (2001). Enhancing students’understanding of mathem-
atics: A study of three contrasting approaches to professional support. Journal of Mathematics
Teacher Education,4(1), 55–79. doi:10.1023/A:1009935100676
Scanlon, D. M., Gelzheiser, L. M., Vellutino, F. R., Schatschneider, C., & Sweeney, J. M. (2008).
Reducing the incidence of early reading difficulties: Professional development for classroom
teachers versus direct interventions for children. Learning and Individual Differences,18(3),
346–359. doi:10.1016/j.lindif.2008.05.002
Shen, J., & Cooley, V. E. (2008). Critical issues in using data for decision-making. International
Journal of Leadership in Education,11(3), 319–329. doi:10.1080/13603120701721839
Sloan, H. A. (1993). Direct instruction in fourth and fifth-grade classrooms (Doctoral dissertation).
Retrieved from Purdue University database (UMI No. 9334424).
Snippe, J. (1992). Effects of instructional supervision on pupils’achievement. Paper presented at the
annual meeting of the American Educational Research Association, San Francisco, CA,
Unpublished manuscript.
Snow, C. (2002). Reading for understanding: Toward an R&D program in reading comprehension.
Santa Monica, CA: Rand Corporation.
Snyder, P. A., Denny, M. K., Pasia, C., Rakap, S., & Crowe, C. (2011). Professional development
in early childhood intervention. In C. Groark & L. Kaczmarek (Eds.), Early childhood interven-
tion program policies for special needs children (Vol. 3, Emerging Issues, pp. 169–204) Santa
Barbara, CA: Praeger.
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., …Higgins,
J. P. T. (2011). Recommendations for examining and interpreting funnel plot asymmetry in
meta-analyses of randomised controlled trials. BMJ,343, d4002. doi:10.1136/bmj.d4002
Supovitz, J. A. (2001). Translating teaching practice into improved student achievement.
Yearbook-National Society for the Study of Education,2,81–98.
Tanner-Smith, E. E., & Tipton, E. (2014). Robust variance estimation with dependent effect sizes:
Practical considerations including a software tutorial in Stata and SPSS. Research Synthesis
Methods,5(1), 13–30. doi:10.1002/jrsm.1091
Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling complex meta-analytic data
structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-
Course Criminology,2(1), 85–112. doi:10.1007/s40865-016-0026-5
Tienken, C. H. (2003). The effect of staff development in the use of scoring rubrics and reflective
questioning strategies on fourth-grade students’narrative writing performance (Doctoral disserta-
tion). Retrieved from Dissertation Abstracts International (UMI No. 9334424).
Travers, J. C., Cook, B. G., Therrien, W. J., & Coyne, M. D. (2016). Replication research and spe-
cial education. Remedial and Special Education,37(4), 195–204. doi:10.1177/0741932516648462
U.S. Department of Education, National Center for Education Statistics. (2001). Teacher quality.
A report on the preparation and qualifications of public school teachers (NCES 1999-080).
Washington, DC: Author.
U.S. Department of Education, National Center for Education Statistics. (2017). The Digest of
Education Statistics, 2016 (NCES 2017-094), Table 204.60. Washington, DC: Author.
EQUITY AND EXCELLENCE IN EDUCATION 37
Van Keer, H., & Verhaeghe, J. P. (2005). Comparing two teacher development programs for
innovating reading comprehension instruction with regard to teachers’experiences and student
outcomes. Teaching and Teacher Education,21(5), 543–562. doi:10.1016/j.tate.2005.03.002
van Kuijk, M. F., Deunk, M. I., Bosker, R. J., & Ritzema, E. S. (2016). Goals, data use, and
instruction: The effect of a teacher professional development program on reading achievement.
School Effectiveness and School Improvement,27(2), 135–156. doi:10.1080/09243453.2015.
1026268
Vaughn, S., Elbaum, B. E., Wanzek, J., Scammacca, N., & Walker, M. A. (2014). Code sheet and
guide for education-related intervention study syntheses. Austin, TX:The Meadows Center for
Preventing Educational Risk.
Vernon-Feagans, L., Kainz, K., Amendum, S., Ginsberg, M., Wood, T., & Bock, A. (2012). The
targeted reading intervention: A classroom teacher professional development program to pro-
mote effective teaching for struggling readers in kindergarten and first grade. Learning
Disability Quarterly,35(2), 102–114.
Wagner, M., Newman, L., Cameto, R., Garza, N., & Levine, P. (2005). After high school: A first
look at the post-school experiences of youth with disabilities. A report from the National
Longitudinal Transition Study-2 (NLTS2) (ERIC Document Reproduction Service No.
ED494935). Menlo Park, CA: SRI International.
Walsh-Cavazos, S. (1994). A study of the effects of a mathematics staff development module on
teachers’and students’achievement (Doctoral dissertation). Texas Tech University.
Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., & Garet, M. S. (2008). Experimenting with teacher
professional development: Motives and methods. Educational Researcher,37(8), 469–479.
doi:10.3102/0013189X08327154
Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A
review. Review of Educational Research,73(1), 89–122.
Webster-Wright, A. (2009). Reframing professional development through understanding authentic
professional learning. Review of Educational Research,79(2), 702–739. doi:10.3102/
0034654308330970
Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., & Orphanos, S. (2009).
Professional learning in the learning profession: A status report on teacher development in the
US and abroad. Technical Report. National Staff Development Council.
What Works Clearinghouse. (2017). Procedures handbook (version 4.0). Washington, DC:
Institute of Educational Sciences, US Department of Education. Retrieved from https://ies.ed.
gov/ncee/wwc/docs/referenceresources/wwc_procedures_handbook_v4.pdf
Wilson, D. B. (2017). Practical Meta-Analysis Effect Size Calculator [Online calculator]. Retrieved
from https://www.campbellcollaboration.org/this-is-a-web-based-effect-size-calculator/explore/
this-is-a-web-based-effect-size-calculator
Witmer, S. E., Duke, N. K., Billman, A. K., & Betts, J. (2014). Using assessment to improve early
elementary students’knowledge and skills for comprehending informational text. Journal of
Applied School Psychology,30(3), 223–253. doi:10.1080/15377903.2014.924454
Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. L. (2007). Reviewing the evi-
dence on how teacher professional development affects student achievement. Issues & answers.
Regional Educational Laboratory Southwest (REL 2007-No. 033). Washington, DC: U.S.
Department of Special Education.
38 L. DIDION ET AL.
