As schools struggled to overcome the chaos and academic damage wrought by COVID, many turned to tutoring as a simple, if sometimes costly, solution. By the end of 2023, the vast majority of states were funding tutoring programs, and by one estimate at least $7.5 billion in federal relief funds were directed to new offerings.
The flow of resources was supported by an extensive body of evidence. Dozens of studies conducted before the pandemic showed that the positive effects of tutoring were among the largest ever seen in education policy. To help a generation of young students return to their pre-COVID-19 trajectory, advocates argued, there appeared to be no more effective strategy than recruiting thousands of teachers to provide regular supplemental instruction.
With an investment of $8.5 million, New Mexico is once again trying to get the right guidance
But a report shared exclusively with The 74 casts doubt on whether the remarkable learning gains measured in previous studies can actually be produced by the kind of large-scale initiatives launched since 2020. The comprehensive overview of more than 250 high-quality research shows that as tutoring programs grow, their impact steadily decreases.
The findings, which come mainly from pre-COVID documents, echo disappointing results from some local efforts undertaken in the wake of the pandemic. They also reflect the widely recognized reality – observed in education research and the social sciences generally – that the enormous benefits sometimes observed in highly controlled environments are rarely, if ever, transferred to larger populations.
Study author Matthew Kraft, an economist at Brown University who has enthusiastically supported the spread of tutoring, said the promise of the approach should not be overshadowed by the “high, and sometimes excessive, expectations” associated with it.
“We need to be realistic about how difficult it is to do anything good in education, let alone make fundamental changes to the core structures of teaching and learning,” he said.
Previous estimates of the boost from “high-impact” tutoring, which emphasizes individual or small-group instruction, were significant — about as much as a full year of reading growth for elementary school students, and twice as much as seen by high school freshmen, as quantified by standardized test scores. By comparison, the benefits students received from larger interventions ranged from one-third to one-half that magnitude.
Professor Beth Schueler of the University of Virginia, co-authored by Kraft, argued that these results remained “quite impressive,” if not on par with what had been previously measured.
We need to be realistic about how difficult it is to do good in education out of the gate.
Matthew Kraft, Brown University
“Even though the large-scale programs did not replicate the massive impacts found in small-scale studies, the magnitude of impact we find for these more policy-relevant studies is still quite meaningful.”
Strikingly, the 265 studies included in Schueler and Kraft’s analysis were all built around randomized control trials, which are considered the empirical gold standard in quantitative research. All were conducted in the countries that make up the Organization for Economic Co-operation and Development, a group of wealthy, industrialized countries whose education systems are often compared.
Across the entire sample of studies, the average effects of tutoring were approximately equal to those found in previous research reviews. But the improvements in test scores shrank significantly when the authors looked only at programs with between 400 and 999 students; they became even smaller when they were limited to those who enrolled more than 1,000.
For the sake of America’s schoolchildren, Congress must keep AmeriCorps going
Robert Balfanz, a professor at the Johns Hopkins University School of Education, noted that the early hype that promoted tutoring as a “silver bullet” for COVID-related learning loss was destined to fade as school districts began using it to help thousands of struggling students. reaches. Still, he added, even high enrollment efforts resulted in important growth for children.
“This study just shows the reality [tutoring] is a very effective intervention, but it will take a lot of time and patience and a lot of learning to make it work at scale,” said Balfanz, who has contributed to the U.S. Department of Education’s effort to recruit 250,000 teachers and mentors to to work in. schools. “Even then, scale will always detract from what you can do for a smaller group.”
A matter of scale
Emerging research on COVID-era tutoring initiatives has highlighted the complexities facing state and district leadership.
Kraft itself published a study last month on Nashville’s program, which was created in 2021 and has grown to serve about 10 percent of the district’s total students. During its first two years of operation, students’ reading performance improved only modestly, with no corresponding gains in math. Another low-touch experiment, aimed at high school students in suburban Chicago, found only a slight increase in standardized test scores from a handful of tutoring sessions offered via Zoom.
Stuck funding for high-dose tutoring is wreaking havoc on New Jersey students
But some warn that it may be premature to measure the impact of tutoring systems that only started during a public health emergency. Hampered by school closures and an uncertain budget picture, the initial transition to tutoring was difficult in many areas. Districts found it challenging to coordinate with families who had withdrawn from school, and an extremely hot job market made recruiting tutors particularly difficult.
Ashley Bencan is the Chief Operating Officer of the New Jersey Tutoring Corps, which launched as a pilot in the summer of 2021. Since then, the organization has grown to include 10 district and charter school partners in more than 30 locations. But even buoyed by federal and state funding, Bencan said, local schools have struggled to build tutoring systems on top of their typical organizational demands.
This study shows the reality that (tutoring) is a very effective intervention, but it will take a lot of time, patience and learning to make it work on a large scale.
Robert Balfanz, Johns Hopkins University
Even collecting data on which students are participating in the tutoring — a crucial step in determining whether the efforts are actually working, Bencan said — could test the capacity of both school districts and state educational agencies.
“When you’re juggling all the different things you have to work with to start the school year – looking at data, grouping kids, filling vacancies – they have to meet those basic needs first, and then figure out what else they can do. she said. “Tutoring is not designed to meet these basic needs, and we need to think about how we can make it part of the school model.”
Studies: Pandemic relief boosted scores, but not enough to make up for lost learning outcomes
The logistical challenges of fitting tutoring into already full school schedules, finding locations where sessions can take place, and connecting families with teachers can be significant. Although Kraft and Schueler write that the design of successful tutoring programs can be effectively duplicated on a larger scale, they also find that the quality of implementation sometimes deteriorates over the course of expansion. Polls of district leaders have found that larger schools consistently saw lower student participation rates, and only about a sixth of principals in one survey reported that they had encountered no barriers to offering tutoring.
Encouragingly, Kraft and Schueler’s analysis shows that some program structures can withstand the pressures of scale. When programs provided in-person tutoring during school hours, with a student-teacher ratio of no more than 3:1, and met at least three times per week (along with other conditions), effects were more robust across larger numbers of students. students. Although the average impact for a program serving 100 to 399 students was 42 percent smaller than for a program serving fewer than 100 students, those who implemented the above high-quality practices saw their impacts decrease by only 18 percent.
We find suggestive evidence that these implementation challenges are real, and policymakers should consider how to address these issues well.
Beth Schueler, University of Virginia
Schueler said the reduced, though still significant, effects of scaled-up tutoring may simply indicate that policymakers have underestimated both the magnitude of learning loss and the barriers to producing new learning resources from scratch.
“We find suggestive evidence that these implementation challenges are real, and policymakers need to think about how to get that right.”
