Minnesota’s QRIS is Effectively Measuring Pre-K Quality- Sometimes

Over the last few years, almost all states have been developing or expanding their Quality Rating and Improvement Systems (QRIS), monitoring tools that evaluate the quality of child care and early education programs. These tools have multiple purposes. Primarily they are used to assess the quality of programs and to help those programs improve their services. But even though QRISs have been expanding quickly, there is little research around how effectively they evaluate programs and improve quality. Questions about whether the ratings, or tiers, accurately reflect program quality or if higher rated programs are actually associated with better outcomes for children have been difficult to answer.

Minnesota now has some answers: while the tiered ratings of state’s QRIS appear to differentiate quality of center-based programs, they do not differentiate quality of family (home-based) pre-K programs.

QRISs can help parents and policymakers make informed decisions about early education. If ratings accurately reflect quality, they can empower parents when choosing where their children attend pre-K. Policymakers can also use QRIS ratings to inform reimbursement rates of federal child care subsidies or to determine whether a program should be eligible for state funding. QRISs can help inform decisions about where to invest resources by identifying common areas across programs that need improvement.

The Obama Administration’s Race to the Top - Early Learning Challenge (RTT-ELC) grant program placed a large focus on creating tiered QRISs as a method to strengthen state early learning systems. RTT-ELC required that each state conduct a validation study to help determine whether the tiers accurately represent meaningful differences in program quality and whether different levels of quality lead to different child outcomes. Minnesota, a RTT-ELC grantee, released findings from its initial validation study in February. The study on the state’s QRIS called Parent Aware, conducted by Child Trends, shows that the state is headed in the right direction but has room for improvement.

Child Trends deemed that the indicators used to rate programs in Parent Aware are based in research around what constitutes quality early education. For instance, Parent Aware evaluates programs on the quality of teacher-child interactions and curriculum and assessment practices. The validation study looks at a sample of three- and four-year-olds in Parent Aware-rated programs across program types and income levels. It finds that children in Parent Aware-rated programs made gains during their pre-K year in numerous essential areas: math, language and literacy, social competence, persistence, and executive function. Some of these gains were more pronounced for children from lower-income families. But despite more significant growth, these children did not surpass their peers from higher-income families. These children were less likely to master basic concepts and were more likely to be over- or under-weight.

As mentioned, the researchers concluded that Parent Aware is successfully differentiating quality in center-based programs: higher Parent Aware ratings (ratings are between one and four stars) are associated with higher quality programs. Child Trends used ECERS-R to measure quality and ECERS-E to measure practices related to children’s math, literacy, and individualized learning. Programs that scored higher on these measures had higher star ratings. Still there were no observed differences between CLASS scores measuring the quality of teacher-child interactions in center-based programs.

The study finds no difference in quality by star ratings for home-based pre-K programs. This could be because of the low sample size of family programs or because family programs are essentially choosing to earn lower ratings. The report explains that “Programs may set a lower goal rating than they could otherwise achieve because they want to work through each level of Parent Aware incrementally, either to provide feasible, attainable goals for their program or to access the maximum amount of quality improvement grants. The implication of this finding is that the lower rating levels of Parent Aware are likely to have greater variation in quality than the higher levels.” Researchers conclude that more research is needed to improve the rating process for family pre-K programs and that the incentives may need to be changed so that programs are encouraged to work toward the highest possible rating.

While all school-based pre-K programs and all Head Start programs in Minnesota now have a star rating, most center-based and home-based pre-K programs have yet to join. There are two ways that programs in Minnesota can earn a Parent Aware rating-- through the full rating pathway or through the Accelerated Pathway to Rating (APR) process. Accredited programs, Head Start programs, and school-based programs can all use the APR process, which only takes about six to eight weeks. These programs must show that their curricula and assessment tools align with Minnesota standards and provide evidence that staff are prepared to implement curricula and assessments appropriately. All other programs, including the overwhelming majority of family pre-K programs, must go through the full rating pathway. This can be time intensive since they may have much to do to show they meet Parent Aware's standards.

The  chart below shows that programs eligible for the APR process have been much more likely to earn a Parent Aware rating. The state has been working diligently to encourage non-accredited programs to join Parent Aware and assisting them through the process.

Programs rated through the APR process and programs that earn three or four-star ratings through the full-rating pathway both “engage in quality practices” that support children’s school readiness, according to the report.

Perhaps most importantly, the study finds that higher ratings are indeed linked to greater gains for children in certain (although limited) areas. Higher ratings didn’t predict better outcomes across the board, but they were associated with greater gains in select measures, including language development, social competence, and attention/persistence. Additionally, low-income children in highly rated programs saw particularly large gains on print knowledge (a measure of literacy) and social competence. Children in school-based programs and Head Start programs had significantly higher scores than those in other programs in literacy and math.

These findings are especially important because Parent Aware plays an essential role in Minnesota’s growing Early Learning Scholarship program. (Check out New America’s recent paper that looked at early learning in Minnesota). Early Learning Scholarships are essentially vouchers that help lower-income families send their children to school-based, center-based, or home-based pre-K. The scholarships can go directly to parents or directly to select programs. Parent choice is a fundamental aspect of the program.

But parents can’t just send their children anywhere. To regulate quality, the scholarships can only be used at Parent Aware-rated locations. All rated programs are eligible to receive scholarships, but only four-star programs can receive the full amount of $7,500.

The effectiveness of the scholarship program largely depends on the validity of Parent Aware, since the ratings determine where children can attend pre-K. The findings suggest that children from low-income families are experiencing significant gains in Parent Aware programs, but they are still entering kindergarten behind their peers. It’s also evident that quality varies among rated programs. Scholarship recipients need to have access to three and four star programs if they are to have a real chance of catching up with their peers. Programs serving children from low-income families might need to provide them with more supports.

Child Trends suggests that even the highest rated programs can benefit from quality improvement efforts. Specifically, many highly rated programs  scored low on the CLASS Instructional Support measure, specific math and literacy practices, and planning for individualized needs. Coaching might help teachers improve in these areas. The study found that teachers who received coaching in CLASS showed improved Instructional Support scores. Currently, CLASS coaching in Minnesota is limited to center-based programs working towards a three- or four-star rating in the full-rating pathway. Expanding coaching to other programs or using it for quality improvement purposes could support child development across programs.

Minnesota began rolling out Parent Aware in 2012 in select counties, and the tool was only made available statewide a little over a year ago. It’s still new in many parts of the state and it’s natural for there to be growing pains. The report suggests that the patterns might change as the tool becomes more cemented into the state’s early learning system. The state should utilize these results to reform Parent Aware so that it evaluates all types of programs effectively and supports child development and school readiness to the greatest extent possible. It’s unclear what steps policymakers will take to improve Parent Aware and whether the state will choose to devote more resources to the tool, especially as RTT-ELC funds come to an end. "