Improve learning by taming instructional complexity

Carnegie Mellon University and Temple University researchers scoured the educational research landscape and found that because improved learning depends on many different factors, there are actually more than 205 trillion instructional options available.

In the Nov. 22 issue of Science, the researchers break down exactly how complicated improving education really is when considering the combination of different dimensions -- spacing of practice, studying examples or practicing procedures, to name a few -- with variations in ideal dosage and in student needs as they learn. The researchers offer a fresh perspective on educational research by focusing on conclusive approaches that truly impact classroom learning.

The findings were published only a week after CMU launched the Simon Initiative to accelerate the use of learning science and technology to improve student learning. Named to honor the work of the late Nobel Laureate and CMU Professor Herbert Simon, the initiative will harness CMU's decades of learning data and research to improve educational outcomes for students everywhere.

"There are not just two ways to teach, as our education debates often seem to indicate," said lead author Ken Koedinger, professor of human-computer interaction at Carnegie Mellon, director of the Pittsburgh Science of Learning Center (PSLC) and co-coordinator of the Simon Initiative. "There are trillions of possible ways to teach. Part of the instructional complexity challenge is that education is not 'one size fits all,' and optimal forms of instruction depend on details, such as how much a learner already knows and whether a fact, concept, or thinking skill is being targeted."

For the paper, Koedinger, Temple's Julie Booth and CMU's David Klahr investigated existing education research to show that the space is too vast, with too many possibilities for simple studies to determine what techniques will work for which students at different learning points.

"As learning researchers, we get frustrated when our work doesn't seem to make an impact on the education system," said Booth, assistant professor of educational psychology at Temple who received her Ph.D. in psychology from Carnegie Mellon. "But much of the work on these learning principles has been conducted in laboratory settings. We need to shift our focus to determine when and for whom these techniques work in real-world classrooms."

To tame instructional complexity and maximize the potential of improving research behind educational practice and student learning, the researchers offer five recommendations:

1. Because trying all educational options -- more than 205 trillion -- to find out what works best is impossible, research should focus on how different forms of instruction meet different functional needs, such as which methods are best for learning to remember facts, which are best for learning to induce general skills, and which are best for learning to make sense of concepts and principles.

2. More experiments are needed to determine how different instructional techniques enhance different learning functions. For example, the optimal way to memorize facts may be a poor way to learn to induce general skills.

3. Take advantage of educational technology to further understand how people learn and which instructional dimensions can or cannot be treated independently by conducting massive online studies, which use thousands of students and test hundreds of variations of instruction at the same time.

4. To understand impact, build a national data infrastructure in which data collected at a moment-by-moment basis (i.e., cognitive tutors tracking daily how a student learns algebra over a school year) can be linked with longer-term results, such as state exams and performances in a next class.

5. Create more permanent school and research partnerships to facilitate interaction between education, administration and researchers. For example, the PSLC, funded by the National Science Foundation (NSF), gives teachers immediate feedback and allows researchers to explore only relevant theories.

"These recommendations are just one of the many steps needed to nail down what's necessary to really improve education and to expand our knowledge of how students learn and how to best teach them," said Klahr, the Walter Van Dyke Bingham Professor of Psychology at CMU who directs PIER, the university-wide graduate training program in education research. "They're also in line with how Carnegie Mellon -- an educational research powerhouse -- approaches education by studying the intersection of instruction, cognitive psychology, computer science, statistics, philosophy and policy."