A new study shows that the comprehensive and controversial education reform strategies carried out in Denver Public Schools over a decade dramatically improved student learning in math and English. It calls the reforms “among the most effective in U.S. history.”
“It definitely shows that improvement is possible and it is possible in large school systems with high levels of student need,” said CU Denver’s Center for Education Policy Analysis’ director Parker Baxter, who hopes the center’s study can provide evidence to the ongoing debate of whether the reforms helped or harmed students.
“These effects are among the largest ever observed in educational research using state of the art extremely rigorous methodology that observes individual student achievement over time.”
In 2007, DPS was one of the lowest performing districts in the state. Its graduation rate was 39 percent. From 2008 to 2019, the district, beginning under the leadership of now U.S. Sen. Michael Bennet, embarked on a radical restructuring.
The new strategy relied on school choice and competition, closing low-performing schools, creating new schools, empowering educators, and holding everyone accountable for test results. It also created a student-based funding model that allocates dollars based on need.
The study finds that compared to similar peers in surrounding school districts, DPS students who experienced two years of reform received the equivalent of at least six months and as much as two years of additional schooling. Everyone improved, not just students who experienced a specific reform strategy like leaving a closed school.
“This compounds over time,” he said. “The longer a student was enrolled in DPS during the reforms, the more they benefited from them.”
Students who were in DPS for five years of reforms received the equivalent of between two and three years of additional schooling.
Baxter noted that existing achievement gaps between racial and ethnic groups stayed the same, but everyone benefited, not just wealthier students and not all to the same degree.
“We need to be able to hold two things in our heads at the same time…the idea that the racial and economic achievement gap is a moral crime. And yet we need to celebrate dramatic improvement, including for students who have historically been left behind.”
International research shows even modest gains in standard deviations of learning—if sustained over time—may have sizable impacts on earnings and poverty reduction, health and well-being.
New study looked at data from 20,000 DPS students
A previous study by Baxter two years ago showed that Denver Public Schools went from one of the worst in Colorado to one that outperformed most of the state’s districts. It found significant gains in test scores and graduation rates went from 39 percent in 2007 to 71 percent in 2019. That study looked at district and school level data.
But researchers acknowledged, and critics noted, it was possible the improvements in performance were due to changes in the student population rather than reforms themselves. During that time Denver’s student population exploded from 70,000 to 90,000, with many new students enrolling who were affluent and white.
So, the next step for Baxter was to complete a study tracking individual student performance during the reform era. After a battle, he received permission from the state Board of Education to obtain access to anonymized data from students who attended schools that were closed for poor performance, schools that were newly created and district-led turnaround schools.
“Our study finds that these improvements are independent of any improvement due to the arrival of thousands of new students.”
The new study examined data from more than 20,000 students enrolled in DPS and another 20,000 enrolled in 11 comparison districts in the metro area. Students had to be enrolled in DPS for at least two years prior to the reforms and for at least two years during them.
Researchers controlled for multiple variables like demographics, prior achievement and the possibility that student performance is influenced by the district a student lives in.
“The whole point of this process is to ensure that we really are comparing ‘like students’ to ‘like students’ so that we can observe the performance of these students before the reforms began and then after they were implemented.”
What about gains for students directly affected by different reform strategies?
The gains found in the first part of the study were for the average DPS student. A second analysis evaluated the impact of three specific reform strategies – school closures, new schools, or district-led turnaround.
“The positive effects are much smaller at the level of (schools that had) intervention,” he said.
During the study period, the district opened 65 new schools, and closed, replaced, and restarted 35 others. It found that DPS students who left a closed school and then attended a new school opened during DPS reform saw the largest positive effects in math overall, and performed the same in literacy.
Students who stayed in low-performing schools that were ordered to “turnaround” – often teachers lost their jobs while new teachers embarked upon district-led interventions using federal dollars – those students ended up performing worse.
“Approximately $700,000 per school, it did not in fact result in… benefits.”
English language learners districtwide didn't see significant improvement in literacy for much of the reform period, but the study showed positive results in math and literacy for those who enrolled in new schools and left closed schools.
Lessons from the study
Many observers would describe the reform years as turbulent, marked by turmoil and turnover at the school level, disruptions and community pushback. The temporary 2006 closure of Manual High School disenfranchised and displaced Black and Latino students. The district was divided and teachers complained the intense focus on test scores and accountability stripped classrooms of freedom and creativity. It still has many critics.
“Dramatic changes are going to be controversial even if they create improvement, because those changes often require disrupting existing power structures, existing networks of privilege,” said Baxter.
He said one of the great lessons of dramatic reforms is the importance of leadership in helping the public recognize the tradeoffs when confronting difficult challenges. The two superintendents who led the change were now-U.S. Senator Michael Bennet and Tom Boasberg, who resigned after 10 years as superintendent in 2018.
DPS has rolled back some of the Boasberg-era reforms and is now grappling with the fallout of rapidly opening so many schools just prior to major demographic decline in school-aged children.
Baxter said many of the challenges that caused DPS to launch its strategy – chronic underperformance, declining enrollment, and financial instability – are many of the same challenges school districts across the country face now.
He said the study doesn’t provide a roadmap of specific strategies but said the evidence is important for “anyone seeking hope in the face of really, really big obstacles.”
“Part of the lesson here is that facing these challenges in a proactive, intentional and strategic way can actually result in dramatic improvement for students. These challenges do not have to result in harm, they can become opportunities for transformation and positive change.”
