Checking the Temperature: Measuring Organizational Climate in Early Education

“The ECE [Early Childhood Education] field has the potential to broaden what it considers as key levers for positive change,” posits the Ounce of Prevention Fund (The Ounce) and the University of Chicago Consortium on School Research (UChicago Consortium). The two groups are referring to the organizational climate and conditions of early education sites. A number of states have already included K-12 school climate initiatives in their ESSA plans as more policymakers are recognizing that a positive school climate is an indicator of quality. Research has found that students in schools with strong school climates, or work environments, were 10 times more likely to make substantial academic gains than their peers in schools with weak climates. Similarly, early education programs with positive climates exhibit strong classroom quality, including high-quality teacher-child interactions. However, there are few tools to measure organizational climate in early education programs and pinpoint areas for improvement.

To fill this void, the Ounce and the UChicago Consortium created the Early Education Essential Organizational Supports measurement system (Early Ed Essentials). The Early Ed Essentials are a set of teacher and parent surveys that measure the organizational climate of publicly funded programs serving three-and four-year-olds, with the goal of providing these programs with actionable information for improving organizational conditions. The tool measures:

• Effective Instructional Leaders
  
• Collaborative Teachers
  
• Supportive Environment
  
• Ambitious Instruction
  
• Involved Families
  
• Parent Voice 

The Early Ed Essentials are adapted from the existing K-12 Five Essentials survey, helping to align the framework for organizational excellence across the PreK-12 education continuum.

Last month, the Ounce and the UChicago Consortium released Organizing Early Education for Improvement: Testing a New Survey Tool, a report on the validation study for the Early Ed Essentials that asked three questions:

1. Are the Early Ed Essentials surveys psychometrically sound (do they accurately gauge viewpoints of respondents)?  
   
2. Are the survey responses related to outcomes in early education programs?
   
3. And do early education programs with high early ed essential scores differ qualitatively from programs with low scores?

(A shorter snapshot of the study and its results can be found here.)This post will focus on the latter two questions.

The study’s research team looked at two site-level outcomes and their relation to a site’s Early Ed Essentials score. One of these outcomes was student attendance. A 2013 UChicago Consortium study on pre-K attendance in Chicago Public Schools found that almost half of three-year-olds and one-third of four-year-olds in their sample were chronically absent (missed 10 percent or more of school). These students ended the year with fewer skills (such as math, literacy, letter recognition, and social-emotional skills), were more likely to be chronically absent in second grade, and had lower reading scores at the end of second grade. The Early Ed Essentials validation study found that sites that scored higher on the following essentials also had higher rates of student attendance: Effective Instructional Leaders, Collaborative Teachers, Involved Families, and Supportive Environment.

The research team also looked at sites’ scores from the Classroom Assessment Scoring System (CLASS), an observational tool designed to measure the quality of teacher-student interactions. They found that two essentials were significantly related to CLASS scores: Effective Instructional Leaders and Collaborative Teachers.

The research team wanted to establish whether there were qualitative differences between sites with the highest Early Ed Essential scores and sites with the lowest scores. Not surprisingly, sites with the highest scores were more supportive of teachers, children, and families than sites with the lowest scores. For example, when looking at the “Supportive Environment” essential, which gauges whether leaders and teachers create an emotionally supportive environment for children and families, a parent at a high scoring site said this about her experience, “Supportive environment? Definitely. It’s not only from the teachers and the staff to the kids, or from the director down. It’s not just for the kids. I think it funnels all the way up from the children to the parents… They want parents to be able to facilitate things on their own. They started a teacher assistant program, and there are a couple parents who are participating in that…” While a teacher at a lower-scoring site said this when sharing her feelings on supporting diverse learners, “[It’s difficult] when you don’t have a child that wants to participate: [a child] that’s just stubborn and doesn’t want to write… and doesn’t want to cooperate. Again, it comes with a language barrier, and you have [children] that just sit there and look at you like, ‘what are you saying to me?’ So it becomes difficult.” The qualitative evaluation shows that site climates affect the attitudes of every stakeholder and those attitudes can manifest in classroom practices and student outcomes.

It is commonly recognized that an organization's climate is the product of the systems and practices put in place by that organization’s leader. Debra Pacchiano, vice president of translational research at the Ounce and a principal investigator on the recent Early Ed Essentials study, told me that an important differentiating factor between the highest scoring sites and the lowest scoring sites, that came to light in the qualitative study, was the mindset of school and site leaders. Pacchiano said that leaders of the highest scoring sites were “deeply rooted in the purpose and promise of ECE.” These leaders then created systems around teaching practices and family engagement that reflected that mindset. On the other hand, leaders of the lowest scoring sites, according to Pacchiano, were “completely driven by the program standards and consumed by compliance and implementation.” The qualitative study of the Early Ed Essentials demonstrates the need for strong instructional leaders in early education.

The Ounce and the UChicago Consortium plan on making the Early Ed Essentials a holistic tool for organizational improvement. Pacchiano and Stacy Ehrlich, a senior research scientist at NORC at the University of Chicago and the other principal investigator on the recent study, told me that several protocols are being created to guide sites on how to use the survey data they receive.

Currently, policies intended to improve the quality of early education still focus mostly on structural features (e.g., lowering student/teacher ratios) and teacher quality (e.g., providing professional development for early educators and increasing early educator qualifications). But if early education sites do not have strong organizational climates, then these policies will likely be poorly implemented if implemented at all, and their intention of improving child outcomes will not be fully realized. The Early Ed Essentials tool aims to provide program leaders with standards for a strong organizational climate and show how their sites measure up to these standards and how they can improve. It is important to note, though, that while this study found a relationship between organizational climate and site-level outcomes, it is still not known whether a strong organizational climate is the catalyst for more engaged families and high-quality teacher-child interactions.