Teacher quality scores change depending on students, school

Dara Shifrer, the study's author and assistant professor of sociology at PSU, said these findings are especially salient now as the public education system is dismantled by a pandemic, fundamentally altering the lives of children and their parents.

The study looked at the value-added scores of 4,500 teachers in a large urban district as they moved between schools between 2007 and 2013. The idea of value-added models is to measure the impact a teacher has on student learning by comparing new test scores to previous ones, and comparing test score gains across teachers. Supporters of the approach say looking at how much a student has progressed in a year, regardless of where they started from, captures the effectiveness of a teacher.

Shifrer said that if that's true, then a teacher's score should remain constant regardless of the school they work in or the type of students they teach.

Instead, when she tracked individual teachers, she found that their scores increased when they switched into high-performing (often low-poverty) schools and decreased when they switched into low-performing (often high-poverty) schools.

"If the teacher moved to a school where there were more poor students, more English language learners, or generally more challenging teaching environments, then their scores dropped," Shifrer said. "The changes in individuals' teachers' scores over time were consistent with shifts in the economic status and race of students in their classrooms and schools."

The study focused on scores from the Education Value-Added Assessment System (EVAAS), which is used in a dozen states, including Arkansas, California, Delaware, Georgia, Indiana, North Carolina, Ohio, Pennsylvania, South Carolina, Tennessee, Texas and Virginia.

Shifrer said the study is more evidence that factors beyond a teacher's control -- socioeconomic background, the impact of social inequality, racial segregation -- affect students' learning and whether we view teachers and schools as 'effective.'

For example, the COVID-19 pandemic forced schools nationwide to pivot to online learning, which left behind many students who didn't have computers or home internet access or whose families had to deal with health issues, layoffs and other concerns.

"These value-added scores haven't solved the problem of how to evaluate teacher quality and they limit our ability to understand how students learn and what affects it," she said. "Policymakers need to recognize how powerfully social inequality outside of schools affects what's happening inside of schools."

Shifrer said that educational disparities will be most effectively addressed by targeting inequality in society. The study recommends that teachers who work in high-poverty schools be paid a higher salary and schools should be sufficiently funded to provide the social services that are necessary to support their students. She said the pandemic has only made it more clear how society relies on schools not just for educating children but also for facilitating parents' ability to work, and by providing children with food and other social supports.