Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences

Abstract

Prior to the COVID-19 pandemic, online credit recovery (OCR) was the most popular use of distance learning in high schools in the United States. With high course failure rates during the height of the COVID-19 pandemic, high schools have turned to OCR to help students recover lost credit. This study examined the potential consequences of increasing OCR enrollment at the school level using administrative data from North Carolina and found that increasing OCR enrollment is associated with higher rates of passing previously failed courses but with diminishing returns after about three-quarters of students who failed courses enrolling in OCR. Consistent OCR enrollment increases over four years is associated with higher graduation rates. Contrary to prior research, this study finds no evidence that school-level OCR enrollment increases are associated with lower test score proficiency rates. Using pre-pandemic data to help inform post-pandemic decision making, the results suggest that increasing OCR enrollment might address increased pandemic-induced course failure rates by expanding opportunities to re-earn course credit, but this would not necessarily translate to higher graduation rates.

Full text

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
324
Online Credit Recovery School-Level Enrollment:
Intended and Unintended Consequences
Samantha Viano
George Mason University, USA
Abstract
Prior to the COVID-19 pandemic, online credit recovery (OCR) was the most popular use of
distance learning in high schools in the United States. With high course failure rates during the
height of the COVID-19 pandemic, high schools have turned to OCR to help students recover lost
credit. This study examined the potential consequences of increasing OCR enrollment at the school
level using administrative data from North Carolina and found that increasing OCR enrollment is
associated with higher rates of passing previously failed courses but with diminishing returns after
about three-quarters of students who failed courses enrolling in OCR. Consistent OCR enrollment
increases over four years is associated with higher graduation rates. Contrary to prior research, this
study finds no evidence that school-level OCR enrollment increases are associated with lower test
score proficiency rates. Using pre-pandemic data to help inform post-pandemic decision making,
the results suggest that increasing OCR enrollment might address increased pandemic-induced
course failure rates by expanding opportunities to re-earn course credit, but this would not
necessarily translate to higher graduation rates.
Keywords: Online credit recovery, high schools, test scores, course failure, administrative data,
fixed effects
Viano, S. (2023). Online credit recovery school-level enrollment: Intended and unintended
consequences. Online Learning, 27(2), 324-342.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
325
While the rapid shift to online learning during the COVID-19 pandemic was extremely
disruptive, it was not the first time most high schools in the United States used distance learning.
Over the last 20 years, high schools have increasingly turned to online credit recovery (OCR) for
students who fail traditional courses and need credit remediation (Watson & Gemin, 2008). OCR
potentially provides cost savings and efficiency by allowing students to rely on software instead
of a traditional instructor to remediate credit (Heinrich & Darling-Aduana, 2021). As course
failure rates have increased in the wake of pandemic schooling (Borter & O’Brien, 2021; St.
George, 2020; Thompson, 2020), educational leaders are likely considering whether to scale
OCR to meet growing demand for course remediation. This study provides evidence on the
effects of increasing OCR enrollment at the school level.
Decisions about appropriate OCR enrollment levels will be especially complex post-
pandemic because many students failed courses in which they had no access to the traditional
face-to-face format (National Trends in School Openings Since January 2021). However, many
OCR tradeoffs are well known to school leaders. Mainly, pervasive criticism from the media,
academics, and the NCAA has focused on whether OCR helps students to graduate high school
without acquiring the appropriate academic knowledge (Kohli, 2017; Loewenberg, 2020;
Sproull, 2018). Using pre-pandemic data to help inform post-pandemic decision making, I
investigate both potential benefits and negative side effects of increasing OCR enrollment to
assess whether it could narrow opportunity gaps by providing access to credit recovery or
whether it exacerbates opportunity gaps already widened by the pandemic if the online courses
are of low quality (Goldberg, 2021).
Literature Review
Why Schools Use Online Credit Recovery
Beginning in 2010-11, the U.S. Department of Education (ED) required high schools to
report their graduation rates and required states to hold high schools accountable for their
graduation rates (No Child Left Behind High School Graduation Rate Non-Regulatory Guidance,
2008). At the same time, states competed for significant federal funds through Race to the Top,
which offered large financial incentives to states that developed accountability systems that
heavily sanctioned low-performing schools as determined by test scores and high school
graduation rates (Overview Information; Race to the Top Fund; Notice Inviting Applications for
New Awards for Fiscal Year 2010; Notice, 2010). As federal-level school accountability
mandates placed greater emphasis on graduation rates, school districts responded by offering
credit recovery options along with other reforms, including expanded access to OCR.
Due to evidence indicating that failure to accumulate credits in a timely manner is a
major barrier to high school graduation (see Allensworth & Easton, 2005; Bowers, 2010; Mac
Iver & Messel, 2013), educational policymakers may view OCR as a means to remove this
barrier in response to accountability pressure to increase graduation rates. The hypothesis is that
students are less likely to drop out of high school (and, therefore, more likely to graduate) if they
can more easily obtain credits for required courses they failed, i.e., without the constraints of the
typical face-to-face (F2F) course (Murin et al., 2015; Watson et al., 2008). While F2F was the
traditional method for credit recovery, OCR represented a shift to distance education away from
the traditional F2F strategy of repeating the failed course after school, during the summer, or
during the following school year. In 2013, Connecticut became the first state to mandate that
high schools offer OCR to all students who fail a course if the school has a dropout rate of eight

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
326
percent or higher, representing a shift in the locus of decision-making about credit recovery from
local districts to the state (Murin et al., 2015).
OCR is a very popular tool for school districts. In the 2009-10 school year, before the ED
required that high schools be held accountable for their graduation rates, nationwide enrollment
in OCR was estimated at over 1.1 million (Queen & Lewis, 2011). In surveys from Iowa,
Wisconsin, and New York, high school leaders reported that their most common use of distance
learning was for OCR (Clements, Stafford, et al., 2015; Clements, Zweig, et al., 2015). During
the 2015-16 school year, 72 percent of high schools reported offering credit recovery, and one in
10 high schools enrolled 20 percent or more of the student body in credit recovery (Tyner &
Munyan-Penney, 2018). Also in the 2015-16 school year, schools in North Carolina were as
likely to enroll students in OCR as F2F credit recovery for the first time (Viano, 2021). The most
recent OCR enrollment estimates come from the federal Civil Rights Data Collection from 2017-
18. Based on author calculations, about 60% of high schools at that time offered credit recovery
to an average of 78 students per school. While press outlets have reported that districts greatly
expanded OCR to respond to high course failure rates during the first two years of the COVID-
19 pandemic (Gross, 2021; Belsha, 2022), no estimates of actual recent enrollments are available
at this time.
The Effects of Online Credit Recovery
Despite large enrollments nationally, a policy mandate in Connecticut, and evidence of
high utilization in at least one state, only recently has evidence begun to emerge on the effects of
OCR on students (Carr, 2014; Heppen et al., 2016; Stallings et al., 2016). Recently, several
studies have investigated, with mixed results, how students individually fare when enrolled in
OCR compared to F2F credit recovery courses. This includes two experiments in Chicago and
Los Angeles where students who failed Algebra I (both cities) or English 9 (Los Angeles only)
were randomly assigned to take OCR or F2F credit recovery over the summer. Immediately after
taking the OCR/F2F course, students randomly assigned to OCR sometimes had a lower
likelihood of passing the credit recovery course (Chicago experiment and English 9 in the Los
Angeles experiment) with lower exam scores in the Chicago experiment than those randomly
assigned to F2F (Heppen et al., 2016; Rickles et al., 2023). The Chicago experiment took place
in 2011 and 2012, allowing for analysis of longitudinal outcomes but, in this experiment, OCR
did not lead to differential outcomes later in high school like high school graduation and grade
point average (Rickles et al., 2018).
Quasi-experimental research has found that OCR students were more likely to pass the
course, had lower test scores, and were more likely to graduate from high school than F2F credit
recovery students (Hart et al., 2019; Heinrich et al., 2019; Stallings et al., 2016; Viano & Henry,
2023). At the same time, these students were less likely to enroll in a four-year university and
had lower earnings as young adults (Heinrich & Cheng, 2022; Heinrich & Darling-Aduana,
2021). These studies use a variety of adjustments to regression models to try to account for
complex selection bias in student selection into OCR. The main challenge when establishing
whether OCR causes differential student outcomes is the likelihood of strategic OCR enrollment
approaches by school administrators (Viano, 2021). While it appears that OCR students are more
likely to earn course credit but have lower test scores than similar F2F credit recovery students,
this might reflect the fact that administrators assign highly motivated but lower achieving
students to OCR courses who would have had the same test scores and graduation rates if they
had been assigned to a F2F course.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
327
Compared to the quasi-experimental studies, experiments on OCR have not as
definitively found OCR to effectively increase high school graduation probability while
decreasing test scores, so it remains unclear what results schools are getting from using online
learning for credit recovery. Since OCR remains popular (Tyner & Munyan-Penney, 2018), it is
possible that school leaders continue to use OCR because they observe positive trends school-
wide that they attribute to OCR. However, prior research has not examined OCR enrollment
effects from a school-level perspective.
Conceptual Framework
At its most basic level, OCR is a popular use of distance learning in high schools to allow
students the option of retaking failed courses online instead of F2F. As described above, schools
are incentivized to use OCR to increase credit accumulation to raise graduation rates. Within the
context of this study’s framework, two outcomes are the intended consequences of successful
OCR enrollment: students will pass more classes that they previously failed, and they will be
more likely to graduate high school as OCR enrollment rates increase. However, critics are
skeptical that these positive gains come for free (see Carr, 2014). Specifically, several studies
indicate that the population of students who fail courses in high school would be particularly ill-
suited to succeed in an online learning environment (Viano, 2018a). Students who fail courses in
high school, often labeled as at-risk students, are more likely to have lower technological and
online skills than students who do not fail courses in high school (Judge, 2005; Kuhlemeier &
Hemker, 2007; Oliver et al., 2009; Valadez & Duran, 2007). Also, students who fail one class are
more likely to have failed other courses as well, perhaps indicating multiple skill deficits that
could make it challenging to succeed on a complicated online platform (Bowers & Sprott, 2012;
Judge, 2005; Roderick, 1994). As summed up by Huett and colleagues in a review of knowledge
about K-12 online learning, “We fear that distance education may become little more than a
'dumping ground' for credit recovery...the exact opposite population the research says tends to
thrive in the distance environment” (Huett et al., 2008, p. 64).
Prior research has confirmed some of these concerns about OCR student experiences.
Studies of OCR students enrolled in the North Carolina Virtual Public Schools (NCVPS) report
challenges with reading comprehension, navigating the online platform, and motivation (Lewis
et al., 2014, 2015). Students simultaneously appreciated that courses were self-paced while
disliking the challenges related to time management (Lewis et al., 2014). Another study of high
school students in an online course offered by a university found that OCR students were more
likely to seek help with their course from a parent while non-OCR students in the online course
were more likely to go to teachers or peers (Oviatt et al., 2018). This strategy could undermine
student performance if OCR students seek assistance with content the parent is not familiar with
(Oviatt et al., 2018). While schools across the country turn to OCR as a way for students to earn
course credits, there are significant reasons to doubt that an online learning approach would be
successful with the population of students who fail courses in high school.
Further, the negative effects of interventions designed to quickly meet accountability
targets are well documented (see Balfanz et al., 2007; Dee et al., 2013; Jennings & Bearak,
2014). As an intervention implemented to respond to federal accountability pressure to increase
high school graduation rates, OCR has the potential to introduce unintended negative side-effects
like lower test scores. OCR could lead to lower end-of-course exam scores if OCR courses are
low-quality and/or students learn less in OCR courses than if they had repeated the F2F (Fong et

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
328
al., 2014; Heinrich et al., 2019; Heppen et al., 2016). While research has examined these
connections at the student level, scaling to the school level would help to understand broader
consequences of schools’ decisions about OCR enrollment levels.
Purpose and Research Questions
This study represents the first known attempt to quantify the impact of offering OCR at
the school level on outcomes like graduation rates and exam proficiency rates. Prior OCR
research has not investigated whether student-level associations scale. Educational leaders are
likely making broader decisions about how many OCR licenses to purchase, computer labs to
devote to OCR, and staff to supervise OCR. In other words, educational leaders also benefit from
information on the appropriate level of OCR enrollment. While it might be tempting to assume
that prior evidence scales to the school level, this would not necessarily be expected based on
prior evidence on treatment effect heterogeneity (Olsen, 2017). Specifically, OCR might be
associated with positive outcomes, on average, with diminishing returns when scaled to the point
where schools are unable to provide the same level of support (e.g., teacher assistance, high-
quality software), or, conversely, scaled too small for supports to be provided. Correspondingly,
I address the following research question using administrative data from the state of North
Carolina: to what extent is increasing OCR enrollment at the school level associated with the
intended consequences of increased passing rates of previously failed courses and high school
graduation rates and the unintended consequence of lowered proficiency rates on end of course
exams?
Methods
Data and Sample
The data for this project come from an administrative database maintained by the
Education Policy Initiative at the University of North Carolina at Chapel Hill (EPIC) including
all public schools in North Carolina (NC). I include datasets on course rosters/grades and school
demographics/performance for the 2012-13 through 2016-17 school years. I only include OCR
enrollment in core courses required to graduate from high school (i.e., English, mathematics,
science, and social studies; see High School Graduation Requirements, n.d.). Schools are the unit
of analysis.
There are about 600 schools with high school-level grades in NC. Of these, about 400 are
traditional high schools with grades 9-12 while the other 200 schools contain other grades in
addition to 9-12 (Facts and Figures 2015-16, 2016). The schools’ racial makeup ranges from
100 percent white to 99 percent non-white, with the median school having 54 percent white
students, 28 percent Black students, and 12 percent Hispanic students. The percentage of
economically disadvantaged students ranges from zero to 100 percent, with the median school
having half of the student body classified as economically disadvantaged. Overall, NC contains
many high schools with a diverse array of racial and socioeconomic demographics.
Schools in NC have two options for OCR: publicly run NCVPS or privately run online
course providers. This is a very common configuration of OCR options nationally; at least 40
states have a state-run virtual school and privately provided courses are ubiquitous (Watson et
al., 2008). NCVPS courses are available across the state, and schools pay per course enrollment,
between $310 and $510 (North Carolina Virtual Public School, n.d.). Private providers have
contracts with schools/districts to provide OCR, usually charging per course or based on the
number of students logged in at one time. Schools in NC during 2012-13 through 2016-17 school

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
329
years tended to offer OCR during the school day as part of the students’ schedule with an in-
class monitor, although this was not a state requirement (Viano, 2018b). While it would be useful
to include the OCR provider type (i.e., NCVPS or private) or the actual provider (e.g., an
indicator for the company), this information is not available in secondary data, and private
providers have been unwilling to share information on their clients (i.e., school districts) with
researchers (Stallings et al., 2016).
Measures
I include mean values on the independent variables, outcomes, and covariates in Table 1.
These values help to communicate the typical demographics of high schools in NC during this
time. In addition, the mean values are meant to aid in interpretation of the findings on the
average OCR enrollment and the outcomes. In other words, the models predict the change in
outcomes in response to changes in the independent variable which can be interpreted in
reference to the mean values in Table 1.
Table 1
Descriptive Statistics
Mean
Key Independent Variable
Percent of Students Who Failed Courses Enrolled in OCR
18.434
(0.310)
Outcomes
High School Graduation Rate
85.917
(0.277)
EOC Proficiency Rate
48.778
(0.430)
Passing Rate of Failed Courses
12.737
(# Passed Courses that Were Previously Failed/# Failed Courses*100)
(0.684)
Covariates
Enrollment (in 100s)
8.334
(0.121)
Percent of Black Students
27.658
(0.451)
Percent of Hispanic Students
12.777
(0.187)
Percent of LEP Students
2.900
(0.064)
Percent of SPED Students
12.963
(0.180)
Percent of Gifted Students
16.260
(0.216)
Percent Economically Disadvantaged
49.985
(0.407)
Course Failure Rate
0.074
(# of Failed Courses / # of Initial Course Enrollments)
(0.001)
Observations
2561
Note. Standard errors in parentheses. OCR—online credit recovery, EOC—end of course exam, LEP—limited
English proficient, SPED—receives special education services.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
330
Online Credit Recovery Enrollment
I define OCR enrollment as the percent of students who failed a core, required F2F
course and subsequently enrolled in OCR. This is a measure of how often schools assign OCR to
the target population, i.e., students who lost course credit. On average, schools assign 18.4% of
students who previously failed courses to OCR (see Table 1). This percentage increased
throughout this time with the median school assigning 9.9% of students who failed courses to
OCR in 2012-13 to 18.9% in 2016-17.
Dependent Variables
The study includes three dependent variables. (1) The passing rate of previously failed
courses represents the number of courses students passed in OCR or F2F each school year
divided by the number of courses students failed in the current or previous school year. While I
term this the passing rate, this more closely resembles a ratio that can be above 1 if schools
assign OCR/F2F for courses failed more than a year prior (data limitations only allow the
calculation of the number of failures with a one-year lag). (2) Graduation rates are the state’s
official four-year cohort graduation rate, indicating the percent of first-time ninth graders who
graduate within four years. (3) The end-of-course exam (EOC) proficiency rate is the percent of
students in the school who scored proficient on the Math I, Biology, and English II EOCs (the
only subjects high school students are tested on in NC).
Empirical Framework
I assess the effect of changes in OCR enrollment and the outcomes using the following
model: 󰇛󰇜        
where  represents one of the three outcomes, standardized by year, for each school s in year t.
The variable  represents the student-level OCR enrollment measure (divided by 10 to
ease interpretation). I also fit models with lagged versions of  to examine whether the
associations between the OCR enrollment and the outcomes are cumulative over time. Based on
an F test, I fit the passing rate outcome with quadratic and trinomial terms of  , but I
determined that linear models better fit (and did not meaningfully change the findings of) the
graduation and EOC proficiency rate outcomes. The model includes a vector of time-varying
covariates, , including school enrollment, demographics (percent Black, Hispanic), percent of
limited English proficient students, percent of students with disabilities, percent gifted, and
percent economically disadvantaged. I include the course failure rate for initial course
enrollments as a covariate to account for the association between the preponderance of course
failures and the outcomes, independent of OCR enrollment. See Table 1 for a full list of
covariates included in this vector.
This model exploits changes over time within schools in their levels of OCR enrollment
due to the school fixed effects, , which subtract the group mean of each variable in the model.
The school fixed effect restricts the comparisons to within-school, such that each coefficient
represents the effect of a one unit increase within school on the outcome. This eliminates
between school variation that might affect the outcome like neighborhood crime levels or
availability of afterschool activities. To distinguish these changes from annual trends, like
changes in graduation rates over time, I include a year fixed effect, . Thereby,  represents the

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
331
associated difference in the outcome for each 10-percentage point increase, within school, in
OCR enrollment.
Limitations
The ideal method for assessing the impact of school-level OCR enrollment would be to
randomly assign different OCR enrollment levels to schools. Given that this is impossible to do
using secondary data and the likely reticence of school leaders to forego autonomy on course
assignment across the whole school, I attempt to isolate the effect of changes in OCR enrollment
as much as possible. This modeling strategy has inherent limitations and does not truly
approximate the causal interpretation afforded by random assignment in several ways. First, the
school fixed effect is helpful in that it removes between-school variation, but it also means that I
can only assess changes in outcomes associated with changes in OCR enrollment over time. If a
school was already using OCR strategically to raise graduation rates but did not change OCR
enrollment levels during this time, then this strategic use would be undetected in this model.
Second, it is possible the estimates include omitted variable bias if schools tend to implement
other interventions alongside changes in OCR enrollment that affect the outcomes. In this case, it
would appear that OCR enrollment changes are affecting the outcomes while actually, this other,
unobserved intervention is causing the outcomes. I minimize this risk through the covariates and
the year fixed effect, but it remains present in any quasi-experimental design.
Results
Passing Rate of Previously Failed Courses
The results from the school and year fixed effects model with the outcome of the passing
rate of previously failed courses (standardized) are in Table 2. This model, with standardized
passing rate as the outcome, includes the quadratic and trinomial terms for OCR enrollment since
an F test determined the linear, quadratic, and trinomial terms are jointly significant. These
results show that for each 10-percentage point increase in OCR enrollment, the passing rate of
failed courses is predicted to increase with a 0.128 coefficient on the linear term, although the
negative coefficient (-0.009) on the quadratic percent OCR variable indicates that there are
diminishing returns to this positive effect.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
332
Table 2
School and Year Fixed Effects Models with Full Covariates Estimating the Association Between
Within-school Changes in the Percent of Students who Failed Courses Enrolled in OCR and the
Standardized Passing Rate
(1)
Passing
Rate
PercOCRst
0.128***
(0.036)
PercOCR2st
-0.009
(0.007)
PercOCR3st
0.0003
(0.0001)
School Year 2013-14
-0.062
(0.045)
School Year 2014-15
-0.077
(0.050)
School Year 2015-16
-0.090
(0.066)
School Year 2016-17
-0.093
(0.075)
Enrollment (in 100s)
0.002
(0.010)
Percent of Black Students
0.009
(0.006)
Percent of Hispanic Students
-0.003
(0.006)
Percent of LEP Students
0.014
(0.012)
Percent of SPED Students
0.012
(0.007)
Percent of Gifted Students
0.022
(0.022)
Percent Economically Disadvantaged
-0.0002
(0.001)
Course Failure Rate
0.761
(0.408)
Constant
-0.982
(0.674)
Observations
2561
R2
0.09
Note. Standard errors in parentheses, are clustered by school. All outcomes are standardized by year. PercOCRst is
divided by 10 such that each unit increase corresponds to a 10-percentage point increase in the percent of students
who failed courses enrolled in OCR.
* p < 0.05, ** p < 0.01, *** p < 0.001
To ease interpretation given the higher order variables, I predict the passing rate of
previously failed courses across the typical range of OCR enrollment in Figure 1 with the
unstandardized version of the outcome. The black line shows the predicted value surrounded in
grey by the 95% confidence interval. The horizontal line represents the average passing rate of

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
333
previously failed courses across schools in the sample. Figure 1 shows the positive association
between OCR enrollment and the passing rate with higher OCR enrollments associated with
higher passing rates of previously failed courses. The slope decreases across values of OCR
enrollment due to the negative quadratic term. According to the first derivative of this model, the
first turning point in this trinomial is around 72.5%. This turning point is shown visually on
Figure 1 where the curve starts to trend downwards (the other turning point would occur above
100%). This indicates that the passing rate is predicted to increase with higher OCR enrollment
until about three-quarters of students who fail courses are assigned to OCR, with diminishing
returns thereafter.
Figure 1
Predicted Values for the Passing Rate Outcome from Table 2 with the 95% Confidence Interval
Test Score Proficiency
To examine potentially unintended consequences on EOC proficiency rates, the results
with this outcome (standardized) are in Table 3. These results are mostly null such that I fail to
find a significant relationship between OCR enrollment and EOC proficiency rates. OCR
enrollment could have a delayed effect on EOC proficiency rates. Students might be taking non-
EOC OCR courses, so the negative effects on EOC proficiency rates would occur downstream,
as students enter EOC courses less prepared if they had learned little in OCR. To examine this
potential lagged effect of OCR enrollment, I estimate models in Table 3, columns 2-4, assessing
whether increases in OCR enrollment rates over one (column 2), two (column 3), or three
(column 4) years are associated with changes in EOC proficiency. The row labeled Combined

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
334
PercOCR Coefficients represents the cumulative effect of increasing OCR enrollment 10-
percentage points (i.e., one unit change on the coefficient) every year for one, two, or three years,
respectively. This combined coefficient in column 2 is -0.0003, indicating that a 10-percentage
point increase in OCR enrollment in the current year and the previous year is associated with a
very small, not statistically significant decrease of -0.0003 standardized units on EOC
proficiency rates. I find no evidence in this table that OCR enrollment changes are associated
with decreased EOC proficiency rates.
Table 3
School and Year Fixed Effects Models with Full Covariates Estimating the Association Between
Within-school Changes in the Percent of Students who Failed Courses Enrolled in OCR and the
Standardized EOC Proficiency Rate
(1)
(2)
(3)
(4)
EOC
Proficiency
Rate
EOC
Proficiency
Rate
EOC
Proficiency
Rate
EOC
Proficiency
Rate
PercOCRst
-0.0002
0.002
0.013
0.023
(0.006)
(0.006)
(0.008)
(0.013)
PercOCRs(t-1)
-0.002
-0.008
0.002
(0.006)
(0.008)
(0.012)
PercOCR s(t-2)
0.004
0.004
(0.007)
(0.015)
PercOCR s(t-3)
0.002
(0.010)
Combined PercOCR Coefficients
-0.0002
-0.0003
0.009
0.031
(0.006)
(0.009)
(0.017)
(0.032)
School Year 2013-14
-0.13***
(0.015)
School Year 2014-15
-0.09***
0.034*
(0.020)
(0.015)
School Year 2015-16
-0.15***
-0.027
-0.08***
(0.023)
(0.019)
(0.015)
School Year 2016-17
-0.16***
-0.035
-0.09***
0.001
(0.025)
(0.022)
(0.022)
(0.017)
Enrollment (in 100s)
-0.014
-0.011
-0.011
-0.015
(0.010)
(0.011)
(0.014)
(0.020)
Percent of Black Students
-0.02***
-0.02***
-0.015**
-0.014*
(0.004)
(0.005)
(0.005)
(0.006)
Percent of Hispanic Students
0.006
-0.0001
0.006
0.009
(0.004)
(0.004)
(0.008)
(0.008)
Percent of LEP Students
-0.023*
-0.019*
-0.014
-0.018
(0.009)
(0.009)
(0.009)
(0.010)
Percent of SPED Students
-0.005
-0.005
-0.002
0.001
(0.004)
(0.004)
(0.005)
(0.005)
Percent of Gifted Students
0.010**
0.011***
0.018*
0.017*
(0.003)
(0.003)
(0.008)
(0.007)
Percent Economically
-0.001
-0.002*
-0.002
0.001
Disadvantaged
(0.001)
(0.001)
(0.001)
(0.002)

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
335
Course Failure Rate
-0.70***
-1.50***
-1.54***
-1.74***
(0.165)
(0.322)
(0.373)
(0.414)
Constant
0.625**
0.711***
0.367
0.073
(0.216)
(0.208)
(0.323)
(0.361)
Observations
2561
1968
1440
930
R2
0.10
0.12
0.12
0.11
Note. Standard errors in parentheses, are clustered by school. All outcomes are standardized by year.
PercOCRst is divided by 10 such that each unit increase corresponds to a 10 percentage point increase in
the percent of students who failed courses enrolled in OCR.
* p < 0.05, ** p < 0.01, *** p < 0.001
High School Graduation Rates
To test whether schools strategically increase their OCR enrollment rates to increase their
graduation rates, I present the results from the school and year fixed effect models in Table 4. As
shown in column 1, I do not find evidence that increases in OCR enrollment are associated with
significant increases in high school graduation in the year the increase occurred. However, I do
find evidence that consistent increases over time in OCR enrollment lead to increases
downstream in high school graduation. As shown in column 4 of Table 4, a 10-percentage point
increase in OCR enrollment over four years is associated with a 0.187 standard deviation
increase in the graduation rate. In other words, if schools continually assign more students who
failed courses to OCR every year, then after four years they would be predicted to have higher
graduation rates.
Table 4
School and Year Fixed Effects Models with Full Covariates Estimating the Association Between
Within-school Changes in the Percent of Students Who Failed Courses Enrolled in OCR and the
Standardized Cohort Graduation Rates
(1)
(2)
(3)
(4)
Graduation
Rate
Graduation
Rate
Graduation
Rate
Graduation
Rate
PercOCRst
0.009
0.018
0.009
0.056
(0.009)
(0.012)
(0.017)
(0.032)
PercOCRs(t-1)
0.005
0.018
0.057
(0.013)
(0.019)
(0.031)
PercOCR s(t-2)
0.017
0.055
(0.015)
(0.038)
PercOCR s(t-3)
0.019
(0.020)
Combined PercOCR Coefficients
0.009
0.023
0.044
0.187*
(0.009)
(0.018)
(0.031)
(0.086)
School Year 2013-14
0.010
(0.025)
School Year 2014-15
-0.0002
-0.005
(0.031)
(0.028)
School Year 2015-16
-0.056
-0.083*
-0.082*
(0.037)
(0.036)
(0.033)
School Year 2016-17
0.201***
0.153***
0.185***
0.267***
(0.044)
(0.042)
(0.044)
(0.040)

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
336
Enrollment (in 100s)
-0.023
-0.027
-0.029
-0.037
(0.017)
(0.015)
(0.021)
(0.050)
Percent of Black Students
-0.001
0.003
-0.009
-0.042**
(0.009)
(0.011)
(0.016)
(0.016)
Percent of Hispanic Students
0.008
0.009
-0.010
-0.006
(0.010)
(0.010)
(0.015)
(0.026)
Percent of LEP Students
-0.021
-0.024
-0.027
0.051
(0.023)
(0.029)
(0.042)
(0.028)
Percent of SPED Students
-0.002
-0.011
-0.021
-0.034
(0.011)
(0.015)
(0.021)
(0.029)
Percent of Gifted Students
-0.004
0.002
0.003
-0.014
(0.004)
(0.004)
(0.007)
(0.015)
Percent Economically
-0.001
-0.002
-0.003
0.009**
Disadvantaged
(0.001)
(0.002)
(0.003)
(0.003)
Course Failure Rate
-0.769
-0.481
-0.147
-1.466
(0.473)
(1.119)
(1.654)
(1.739)
Constant
0.373
0.357
1.055*
1.322
(0.397)
(0.444)
(0.514)
(0.829)
Observations
2561
1968
1440
930
R2
0.07
0.08
0.13
0.26
Note. Standard errors in parentheses, are clustered by school. All outcomes are standardized by year. PercOCRst is
divided by 10 such that each unit increase corresponds to a 10-percentage point increase in the percent of students
who failed courses enrolled in OCR.
* p < 0.05, ** p < 0.01, *** p < 0.001
Discussion and Conclusion
As educational leaders consider how to structure credit recovery post-pandemic, the
findings support the potential of increasing OCR enrollment to address large increases in course
failure rates during the pandemic with the caveat that the last year of data was the 2016-17
school year, three years before the beginning of the pandemic. The results mirror prior studies
that found, at the student-level, OCR enrollment was associated with increased likelihood of
recovering course credits compared to F2F, such that the individual-level effects appear to scale
to the school level (Hart et al., 2019; Heinrich et al., 2019; Viano, 2021). I did not find evidence
to support OCR being the only way students should recover course credits, with diminishing
returns to increased OCR enrollments after about three-quarters of students who failed courses
enrolled in OCR.
I found little evidence that the increased credit accumulation from higher OCR
enrollments translated into higher graduation rates, at least in the short term. This result more
closely matches findings from experimental studies which found that OCR did not lead to higher
rates of high school graduation, as opposed to prior quasi-experimental work which found
significant relationships (Hart et al., 2019; Heinrich & Darling-Aduana, 2021; Rickles et al.,
2018; Stallings et al., 2016). This could also reflect that the prior associations between high
school graduation probability and OCR were not high enough to lead to significant increases in
school-level graduation rates. Similarly, negative evidence on lower test scores of OCR courses
compared to F2F might not have impacted proficiency rates if the higher test scores of F2F
students were still below proficiency. In other words, if F2F credit recovery is associated with
higher test scores than OCR but with averages still below test score proficiency cutoffs, then
these higher scores would not translate into changes in proficiency rates. This hypothesis is

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
337
supported by findings reported elsewhere that OCR and F2F credit recovery students have
average standardized assessments scores well below the mean (Viano & Henry, 2023). The null
findings on the relationship between OCR and test score proficiency could be reassuring for
educational leaders concerned that increasing their use of distance learning through OCR
enrollment would harm school-level performance as would have been indicated by prior research
on OCR assignment at the student-level (Heinrich et al., 2019; Heppen et al., 2016; Viano &
Henry, 2023).
Overall, these results encourage educational leaders to carefully consider resources
appropriated to using OCR to recover credits students lost during the pandemic. While OCR
might be effective at initially solving an obvious problem caused by the COVID-19 pandemic,
higher course failure rates, I did not find evidence that OCR will provide a comprehensive
strategy to lessen the impact of high course failure rates on graduation rates.
Declarations
The author declares that there are no potential conflicts of interest with respect to the research, authorship,
and/or publication of this article.
Ethical approval for this study was granted by the Office of Human Research Ethics Internal Review
Board at the University of North Carolina at Chapel Hill.
This project was supported by the National Academy of Education and the National Academy of
Education/Spencer Dissertation Fellowship Program.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
338
References
Allensworth, E. M., & Easton, J. Q. (2005). The on-track indicator as a predictor of high school
graduation. Consortium on Chicago School Research, University of Chicago.
Balfanz, R., Legters, N., West, T. C., & Weber, L. M. (2007). Are NCLB’s measures, incentives,
and improvement strategies the right ones for the nation’s low-performing high schools?
American Educational Research Journal, 44(3), 559–593.
https://doi.org/10.3102/0002831207306768
Belsha, K. (2022, May 12). “We need to double down”: After a surge in ninth graders held back,
schools step up support. Chalkbeat.
https://www.chalkbeat.org/2022/5/12/23068627/ninth-grade-retention-credit-recovery-
pandemic
Borter, G., & O’Brien, B. (2021, March 29). Another danger for kids in the age of COVID:
Failing grades. Reuters. https://www.reuters.com/article/us-health-coronavirus-usa-
students-insig-idUSKBN2BL1BF
Bowers, A. J. (2010). Grades and graduation: A longitudinal risk perspective to identify student
dropouts. The Journal of Educational Research, 103(3), 191–207.
https://doi.org/10.1080/00220670903382970
Bowers, A. J., & Sprott, R. (2012). Why tenth graders fail to finish high school: A dropout
typology latent class analysis. Journal of Education for Students Placed at Risk
(JESPAR), 17(3), 129–148. https://doi.org/10.1080/10824669.2012.692071
Carr, S. (2014). Credit recover hits the mainstream. Education Next, 14(3), 30–37.
Clements, M., Stafford, E., Pazzaglia, A. M., & Jacobs, P. (2015). Online course use in Iowa and
Wisconsin public high schools: The results of two statewide surveys (REL 2015–065).
U.S. Department of Education, Institute of Education Sciences, National Center for
Education Evaluation and Regional Assistance, Regional Educational Laboratory
Midwest. https://www.ies.ed.gov/ncee/edlabs/regions/midwest/pdf/REL_2015065.pdf
Clements, M., Zweig, J., & Pazzaglia, A. M. (2015). Online course use in New York high
schools: Results from a survey in the Greater Capital Region (REL 2015–075). U.S.
Department of Education, Institute of Education Sciences, National Center for Education
Evaluation and Regional Assistance, Regional Educational Laboratory Northeast &
Islands. http://files.eric.ed.gov/fulltext/ED555633.pdf
Dee, T. S., Jacob, B., & Schwartz, N. L. (2013). The effects of NCLB on school resources and
practices. Educational Evaluation and Policy Analysis, 35(2), 252–279.
https://doi.org/10.3102/0162373712467080

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
339
Facts and Figures 2015-16. (2016, January).
http://www.dpi.state.nc.us/docs/fbs/resources/data/factsfigures/2015-16figures.pdf
Fong, A. B., Jaquet, K., & Finkelstein, N. (2014). Who repeats algebra 1, and how does initial
performance relate to improvement when the course is repeated? (REL 2015-059).
Regional Educational Laboratory West. http://eric.ed.gov/?id=ED548534
Goldberg, S. B. (2021). Education in a pandemic: The disparate impacts of COVID-19 on
America’s students. U.S. Department of Education Office for Civil Rights.
Gross, B. (2021). Credit recovery isn’t enough: How to manage a surge of failing course grades
(The Evidence Project). Center for Reinventing Public Education.
Hart, C. M. D., Berger, D., Jacob, B., Loeb, S., & Hill, M. (2019). Online learning, offline
outcomes: Online course taking and high school student performance. AERA Open, 5(1),
1–17. https://doi.org/10.1177/2332858419832852
Heinrich, C. J., & Cheng, H. (2022). Does online credit recovery in high school support or
stymie later labor market success? Journal of Policy Analysis and Management,
pam.22376. https://doi.org/10.1002/pam.22376
Heinrich, C. J., & Darling-Aduana, J. (2021). Does online course-taking increase high school
completion and open pathways to postsecondary education opportunities? Educational
Evaluation and Policy Analysis, 43(3), 367–390.
https://doi.org/10.3102/0162373721993485
Heinrich, C. J., Darling-Aduana, J., Good, A., & Cheng, H. (Emily). (2019). A look inside online
educational settings in high school: P and pitfalls for improving educational opportunities
and outcomes. American Educational Research Journal, 56(6), 2147–2188.
https://doi.org/10.3102/0002831219838776
Heppen, J. B., Sorensen, N., Allensworth, E., Walters, K., Rickles, J., Taylor, S. S., &
Michelman, V. (2016). The struggle to pass algebra: Online vs. face-to-face credit
recovery for at-risk urban students. Journal of Research on Educational Effectiveness,
00–00. https://doi.org/10.1080/19345747.2016.1168500
High School Graduation Requirements. (n.d.).
http://www.ncpublicschools.org/docs/curriculum/home/graduationrequirements.pdf
Huett, J., Moller, L., Foshay, W. R., & Coleman, C. (2008). Implications for instructional design
on the potential of the web. TechTrends, 52(5), 63.
Jennings, J. L., & Bearak, J. M. (2014). “Teaching to the test” in the NCLB era: How test
predictability affects our understanding of student performance. Educational Researcher,
43(8), 381–389. https://doi.org/10.3102/0013189X14554449

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
340
Judge, S. (2005). The impact of computer technology on academic achievement of young
African American children. Journal of Research in Childhood Education, 20(2), 91–101.
https://doi.org/10.1080/02568540509594554
Kohli, S. (2017, July 2). Schools are boosting graduation rates by offering “credit recovery.” But
what are students learning? Los Angeles Times.
http://www.latimes.com/local/education/la-me-edu-pass-credit-recovery-20170701-
htmlstory.html
Kuhlemeier, H., & Hemker, B. (2007). The impact of computer use at home on students’ Internet
skills. Computers & Education, 49(2), 460–480.
https://doi.org/10.1016/j.compedu.2005.10.004
Lewis, S., Whiteside, A. L., & Dikkers, A. G. (2014). Autonomy and responsibility: Online
learning as a solution for at-risk high school students. International Journal of E-
Learning & Distance Education/Revue Internationale Du e-Learning et La Formation à
Distance, 29(2).
Lewis, S., Whiteside, A. L., & Dikkers, A. G. (2015). Providing chances for students to recover
credit: Is online learning a solution? In M. Rice (Ed.)Exploring pedagogies for diverse
learners online (Vol. 25, pp. 143–157). Emerald Group Publishing Limited.
https://doi.org/10.1108/S1479-368720150000027007
Loewenberg, D. (2020, Winter). A digital path to a diploma. Education Next, 20(1).
https://www.educationnext.org/digital-path-to-diploma-online-credit-recovery-classes/
Mac Iver, M. A., & Messel, M. (2013). The ABCs of keeping on track to graduation: Research
findings from Baltimore. Journal of Education for Students Placed at Risk (JESPAR),
18(1), 50–67. https://doi.org/10.1080/10824669.2013.745207
Murin, A., Powell, A., Roberts, V., & Patrick, S. (2015). Using Online Learning for Credit
Recovery: Getting Back on Track to Graduation. Evergreen Education Group.
National Trends in School Openings Since January. (2021, May 13). FutureEd.
https://www.future-ed.org/national-trends-in-school-openings-since-november-2020/
North Carolina Virtual Public School. (n.d.). https://ncvps.org/
Oliver, K., Osborne, J., Patel, R., & Kleiman, G. (2009). Issues surrounding the deployment of a
new statewide virtual public school. Quarterly Review of Distance Education, 10(1), 37–
49.
Olsen, R. B. (2017). Evaluating educational interventions when impacts may vary across sites.
Journal of Research on Educational Effectiveness, 10(4), 907–911.
https://doi.org/10.1080/19345747.2017.1364554

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
341
Oviatt, D. R., Graham, C. R., Borup, J., & Davies, R. S. (2018). Online student use of a
proximate community of engagement at an independent study program. Online Learning,
22(1). https://doi.org/10.24059/olj.v22i1.1153
Queen, B., & Lewis, L. (2011). Distance education courses for public elementary and secondary
school students: 2009-10 (NCES 2012-008). U.S. Department of Education, National
Center for Education Statistics.
Rickles, J., Clements, M., Brodziak de los Reyes, I., Lachowicz, M., Lin, S., & Heppen, J.
(2023). A multisite randomized study of an online learning approach to high school credit
recovery: Effects on student experiences and proximal outcomes. Journal of Research on
Educational Effectiveness. https://doi.org/10.1080/19345747.2023.2198524
Rickles, J., Heppen, J. B., Allensworth, E., Sorensen, N., & Walters, K. (2018). Online credit
recovery and the path to on-time high school graduation. Educational Researcher, 47(8),
481–491. https://doi.org/10.3102/0013189X18788054
Roderick, M. (1994). Grade retention and school dropout: Investigating the association.
American Educational Research Journal, 31(4), 729–759.
Sproull, N. (2018, December 1). NCAA and credit recovery. The Thomas B. Fordham Institute’s
Flypaper Blog. https://fordhaminstitute.org/national/commentary/ncaa-and-credit-
recovery
St. George, D. (2020, December 3). Failing grades double and triple—Some rising sixfold—
Amid pandemic learning. Washington Post.
https://www.washingtonpost.com/local/education/montgomery-county-failing-
grades/2020/12/03/913affd0-34fb-11eb-8d38-6aea1adb3839_story.html
Stallings, D. T., Weiss, S. P., Maser, R. H., Stanhope, D., Starcke, M., & Li, D. (2016).
Academic outcomes for North Carolina Virtual Public School credit recovery students
(REL 2017-177). U.S. Department of Education, Institute of Education Sciences,
National Center for Education Evaluation and Regional Assistance, Regional Educational
Laboratory Southeast. http://ies.ed.gov/ncee/edlabs
Thompson, C. (2020, December 6). Schools confront “off the rails” numbers of failing grades.
AP NEWS. https://apnews.com/article/distance-learning-coronavirus-pandemic-oregon-
7fde612c3dbfd2e21fab9673ca49ad89
Tyner, A., & Munyan-Penney, N. (2018). Gotta give ‘em credit: State and district variation in
credit recovery participation rates. Thomas B. Fordham Institute.
https://edexcellence.net/publications/gotta-give-em-credit
U.S. Department of Education. (2008). No Child Left Behind high school graduation rate: Non-
regulatory guidance.

Online Credit Recovery School-Level Enrollment: Intended and Unintended Consequences
Online Learning Journal – Volume 27 Issue 2 – June 2023
342
U.S. Department of Education. (2010). Overview information; Race to the Top fund; notice
inviting applications for new awards for fiscal year 2010; notice.
Valadez, J. R., & Duran, R. (2007). Redefining the digital divide: Beyond access to computers
and the Internet. The High School Journal, 90(3), 31–44.
Viano, S. (2018a). At-risk high school students recovering course credits online: What we know
and need to know. American Journal of Distance Education, 32(1), 16–26.
https://doi.org/10.1080/08923647.2018.1412554
Viano, S. (2018b). Online learning as a remedy for course failure: An assessment of credit r as
an intervention to earn credits and graduate from high school [Vanderbilt University].
https://etd.library.vanderbilt.edu//available/etd-05312018-143058/
Viano, S. (2021). A choice between second chances: An analysis of how students address course
failure. American Journal of Education, 128(1), 29–58. https://doi.org/10.1086/716549
Viano, S., & Henry, G. T. (2023). Online credit recovery as an intervention for high school
students who fail courses. Educational Policy.
https://doi.org/10.1177/08959048231153597
Watson, J., & Gemin, B. (2008). Using online learning for at-risk students and credit recovery.
North American Council for Online Learning.
Watson, J., Gemin, B., & Ryan, J. (2008). Keeping pace with K-12 online learning: A review of
state-level policy and practice. Evergreen Consulting Associates.