Study focuses on use of instructional videos to aid problem solving

The research features a new experimental format that uses data from a system that tracks students' "eye gaze" with cameras, also documenting individual problem-solving methods with "smartpens" that record handwriting and audio. Unlike previous research, the new approach defines the order in which a student performs specific actions, which could be crucial to the success of problem-solving methods, said Edward Berger, a Purdue University associate professor of engineering education and mechanical engineering.

Twenty-four students participated in the study.

"We give them a problem to solve, and we record their actions as they engage with the video," he said. "Everybody in the world makes instructional videos now. But what we don't currently understand is exactly how students use videos in support of their problem solving. So we watch them use a video while they solve a problem, which has not been done before in any significant depth."

The results suggest that students' problem-solving methods can be broadly described by several "archetypes."

"If we can identify which archetypes are more successful, then we can apply that knowledge to better coach students," Berger said. "We can say, here is the pattern of usage we see in students who are really good at solving this type of problem."

The research is detailed in a paper being presented during the American Society for Engineering Education's Annual Conference & Exposition, June 26-29 in New Orleans. The paper was authored by Berger and doctoral student Michael Wilson. A video about the research is available at https://youtu.be/y8jbsf2Abio.

The students were graded on the problem, and their grades also were recorded during a sophomore-level mechanical engineering course that allows students to access hundreds of instructional videos and animations.

The experiment takes about 30 minutes to complete.

"We have a code structure to characterize various actions and quantify exactly what the student is doing," Berger said. "These actions could be something they write, something they do with the video and even something they say because we instruct the students to speak out loud during the exercise. We are identifying and trying to understand these archetypes."

Among the 24 students, the researchers have identified about four such archetypes thus far.

"Say they took 25 minutes to do this problem, they spent eight minutes looking at the video, and here is exactly what they were looking at in the video while they solved the problem. Did they watch the same part of the video more than once? Did they focus on a diagram or the equation, or something else? Then we look at all of the data and say, 'what are the key things they did when they were solving this problem and how did they engage with the video? ' "

The actions are then charted using a system of "code stripes" to visualize the specific approaches and tools students used to solve the problem, and in what order.

The experiments are related to an area of research called cognitive load theory, which attempts to break down how hard the brain is working at a particular task.

"The challenge is that it's really hard to accurately measure cognitive load, and we are using one of several available approaches," Berger said. "We are trying to understand how access to a video helps a student solve problems. Does it help them manage their cognitive load while they are doing the problem?"

Future work will aim to conclusively define the various archetypes.

Purdue researchers also are conducting a new project funded by the National Science Foundation to understand how students use the videos in the Purdue Mechanics Freeform Classroom, pioneered by Charles Krousgrill, a professor of mechanical engineering. The Freeform Classroom approach has been used since 2011 in two mechanical engineering core courses with hundreds of students enrolled annually. The new study is led by Jennifer DeBoer, an assistant professor of engineering education.