Study on the assessment of students: Overcoming bias in decision making

Numerous academic studies suggest that people make "differential" judgments and decisions under the influence of expectations. Work from the University of Luxembourg (in The effect of students' ethnicity on teachers' judgments and recognition memory) demonstrates that differential expectations among school teachers can lead to errors in the way they make decisions. The research team led by Prof. Sabine Krolak-Schwerdt probed this further (in Accuracy of teachers' tracking decisions: short- and long-term effects of accountability),

International studies (For example, see the Programme for International Student Assessment, OECD) show consistently that minority students perform below average. Teachers experience this reality daily, hence group expectations may develop in their minds. These expectations can influence key decisions about each student. For example, Luxembourg has a selective secondary school system, with students assigned to different secondary schools with more or less academically challenging curriculums. The largest ethnic minority is Portuguese, a group which has tended to have lower than average levels of academic achievement.

In the study, experienced teachers were given fictitious student reports and profiles, and were asked to take a mock decision about assigning each to an appropriate secondary school. Luxembourgish students were correctly assigned in 90% of cases, but for Portuguese students only 67% of decisions were correct, with both under- and overestimation of academic ability. Teachers were then asked to rate how accountable they felt regarding these decisions and were then asked to assess more pupil reports. In this instance their decisions were accurate regardless of ethnicity.

"Asking teachers to assess their responsibility for their performance appeared to trigger a more reflective approach to their decision making," noted Dr Ineke Pit-ten Cate, a member of the research team. "Thus the teachers were shown to be good decision makers, but errors may occur under conditions of low accountability due to biased expectations," she said. In normal circumstances teachers are likely to feel highly accountable for decisions about their pupils, so the risk of bias is relatively low. "This study suggests that when we make important choices we should strive to invest mental effort to reduce errors related to involuntary bias," she added. The study was funded by grant C10/LM/784116 from the Fonds National de la Recherche (FNR).