Lacking other meaningful data, university faculty devise their own evaluation systems

But filling in bubbles on institutionalized end-of-term teacher evaluation questionnaires, a standard practice at many large research universities, often fails to produce timely and meaningful data for improving instruction, according to a new study from Oregon State University that explores how faculty are evaluating their teaching practice.

The researchers found that faculty teaching in the STEM disciplines at large research universities are devising their own systems to collect instructional data from their classrooms and using that data to inform their teaching.

Instructors are using quantitative data about their students' classroom performance as well as qualitative data such as student feedback from mid-term surveys or informal conversations, said Jana Bouwma-Gearhart, an associate professor of science and mathematics education at OSU and lead author of the study.

"Some faculty have really created these elaborate data and analysis systems, even though they are not yet required to," said Bouwma-Gearhart, who also serves as an associate dean in OSU's College of Education. "They are using these systems to talk about their instruction and to help inform decisions about programs and curriculum."

The findings were published recently in the Journal of Higher Education Management. The research was supported by a grant from the National Science Foundation.

As universities continue to look at ways to improve student success, there is a growing emphasis in higher education for teaching faculty to be good educators who spend time developing and improving their skills in response to valid and reliable data. This may be particularly critical to retaining students in the STEM fields because students who struggle often opt to leave these disciplines altogether, Bouwma-Gearhart said.

In large enrollment universities -those with 15,000 or more students enrolled -- many undergraduate courses in the STEM disciplines are taught by full- time educators who would likely be considered experts in that discipline.

However, Bouwma-Gearhart and colleagues found that instructors who teach these courses often lack access to structured or formal opportunities to reflect on meaningful data about their teaching beyond the typical end-of-term evaluations -- the results of which often arrive too late; are too vague; and have too little student participation to be of much use.

Bouwma-Gearhart and co-author Matthew T. Hora, of the University of Wisconsin-Madison, interviewed 59 STEM faculty and 20 administrators at large research universities. They wanted to know whether teaching faculty were using data to inform their teaching, and, if so, what kinds of data. They also wanted to better understand how educators are using data while in their classrooms.

"We found that faculty are gathering and responding to data, often using their own systems, and they are sometimes using it to advance their teaching in ways that go beyond what is required of them," Bouwma-Gearhart said

Given the push from policymakers and education leaders toward more data-driven decision-making, documenting and understanding these real-world practices could ultimately lead to the design of more systematic data collection that is useful and helpful for faculty as well as administrators, she said.

"People for the most part buy into the idea that data is good and that decision-making based on data is good," Bouwma-Gearhart said. "But we also see that faculty need flexibility with respect to what kind of data is useful and how the data is used. One-size-fits-all doesn't work when it comes to data to inform teaching practices."

Bouwma-Gearhart said additional research is needed to better understand what motivates some faculty, more than others, to use data to inform their teaching practice. She is currently studying what factors may induce organizational changes related to instructional data use and teaching practice within departments, programs or institutions.

"We found that there is no lack of STEM faculty caring about their teaching or saying that data is ultimately relevant to evaluate and inform their teaching," Bouwma-Gearhart said. "There is a rich conversation and effort underway. The question is how do we engage more faculty in meaningful conversations?"

Find the journal and report online at: http://aaua.org/journals/pdfs/JHEM%2031-1%202016.pdf