Does Program Quality Matter for Head Start?

The Head Start Impact Study demonstrated that the nation’s largest federally funded preschool program could improve children’s learning and development. But critics often note that the positive effects seen at the end of the Head Start experience disappear by the end of third grade. To counter that criticism, program advocates point out that the quality of Head Start programs varies significantly between centers and communities.

But how much does program quality really impact children’s learning and development in Head Start classrooms?

Apparently not all that much, according to a recent study from the Office of Planning, Research, and Evaluation at the U.S. Department of Health and Human Services.

Using data from the Head Start Impact Study (HSIS) and follow-up reports, researchers analyzed how differences in program quality influence children’s cognitive and social-emotional development. They found “little evidence that quality matters to impacts of Head Start,” according to the report.

“I was disappointed,” admits co-researcher Stephen Bell. “We’re not really very far ahead in making Head Start better or understanding which variants of Head Start are worth emphasizing now.”

Bell and co-researcher Laura Peck, research scientists at Abt Associates, consulted with early childhood education experts to identify the three dimensions of quality they would examine: program resources; teacher-child interactions; and children’s exposure to academic activities (practicing letter sounds, playing math games, or writing their names). They evaluated each of these areas using data measured by the Early Childhood Environment Rating Scale (ECERS), the Arnett Caregiver Interaction Scale, and contained in teacher reports. One thing to note about the academic exposure element: some experts question whether increased exposure to academic activities benefits preschool children or is even age appropriate. In this study, a higher score (and high quality rating) indicates that the program exposed children to academic content more frequently.

The researchers found some positive connections for 3-year-olds between high-quality teacher-child interactions and certain cognitive outcomes and between high-quality program resources and children’s behavior. But the report notes that those connections may not reflect true differences between the impacts of high and low quality programs, but simply result from chance. There is one exception: The researchers found that 3-year-olds who received less exposure to academic activities (considered a low quality characteristic in this study) demonstrated better behavior outcomes during the Head Start years and into kindergarten. The researchers did not find any positive connections between program quality and the learning and development of 4-year-olds. They also did not find evidence that program quality influences whether Head Start’s initial impacts last through third grade.

Additionally, Bell and Peck’s work indicates that Head Start programs might not vary as much as originally thought. Bell and Peck determined that the majority of Head Start children in the impact study attended high quality programs: approximately three-quarters of participants attended classrooms with high quality resources and teacher-child interactions.

“What we see, thankfully, is there aren’t a lot of programs that are abysmal,” says Peck. “We’re looking at small variations in these programs and most are clustered at the better end of the scale.”

The real challenge then, is not distinguishing between “high quality” and “low quality” programs, but rather uncovering the subtle differences between a “high quality” program and an exemplary one.

The real challenge then, is not distinguishing between “high quality” and “low quality” programs, but rather uncovering the subtle differences between a “high quality” program and an exemplary one. Most of the measurement tools traditionally used to evaluate program quality, though, simply don’t capture those nuances.

As I noted in an earlier post, many of the measurement tools typically used to evaluate program quality do not even connect to child outcomes. Recent research finds little evidence that ECERS scores, for instance, correlate with children’s academic, language, and social-emotional development. Meanwhile, a study published last year by a team from the University of Illinois at Chicago likewise found little association between scores on the Arnett Caregiver Interaction Scale and children’s development. This is partly because scale scores generally cluster toward the high end, obscuring meaningful distinctions between the skill levels of early childhood teachers.

Peck acknowledges that the analytical approach she and Bell used in the Head Start study had limitations that potentially could have obscured some impacts of program quality. Their research relied on the quality measures included in the HSIS data set and, as the report acknowledges, “improvements in measuring quality have developed in the intervening decade, justifying alternative measurements of ‘quality’.”

“The quality gradient we were able to examine only gives us what’s different from one Head Start center to another,” explains Bell. “Maybe there isn’t real variability in how much of an effect Head Start is making from one part of the country or one community to another. If there is a big difference, we can’t trace it to these quality measures.”

Considering that the data and quality measures used in the HSIS are more than a decade old, “it might be time to refresh the sample with a set of children entering Head Start or other child care today because it is a different configuration,” says Peck. The current version of Head Start operates in a new context, so it deserves a fresh look to determine whether the quality of its programs measures up to modern standards.