Brief interventions help online learners persist with coursework

A study, set to be published in the Jan. 20 issue of Science, found that people in less-developed countries are completing MOOCs at a lower rate than those in the more developed parts of the world.

But, the researchers found, brief psychological interventions that affirm class takers' sense that they belong can help close the global achievement gap.

"MOOCs have expanded access to education but this doesn't guarantee equal opportunities for people around the world," said René Kizilcec, the lead author of the study and a doctoral candidate in the Department of Communication. "Providing access to the Internet and courseware is not enough. People need to feel welcome in online-learning environments to reach their potential."

Discovering the global gap

Online education gained momentum in 2011, when institutions and entrepreneurs began developing different MOOC platforms such as Coursera and edX. The higher-education landscape shifted and online course delivery became a viable option to extend learning opportunities to a global audience. But learning data made available by the courses themselves revealed that while many people enrolled in free, online courses, far fewer completed them.

Kizilcec began researching MOOCs at the onset. Together with an interdisciplinary group of graduate students, he co-founded Stanford's Lytics Lab in the fall of 2012. Over the past five years, with support from the Office of the Vice Provost for Teaching and Learning and the Graduate School of Education, the lab has focused on studying the experiences of people who take MOOCs to advance the science of learning and instruction.

Kizilcec said his previous research showed that achievement rates in online courses varied based on gender and education level of the learners. But the biggest gap was geographical.

The geographic gap was quantified and visualized in the new study with data on completion rates of about 1.8 million people who have enrolled in Stanford's MOOCs between 2012 and 2015 and with the United Nations' Human Development Index, which measures the countries' level of human development based on factors such as life expectancy, education and standard of living. The lower the development index was for the country associated with a learner's IP address, the less likely the learner was to complete the course.

"Though many had inklings that the gap was there, being able to identify it consistently across so many courses and learners was profound and provided us the foundation to dig deeper and explore interventions that could address this gap at such a scale," said Andrew Saltarelli, a co-author of the research and a senior director of teaching design and practice in the Office of the Vice Provost for Teaching and Learning.

Psychological interventions

While limited Internet access and low English proficiency may contribute to the global achievement gap, there may be more to it, according to researchers.

"It's not the average person in less-developed countries who is taking online courses," Kizilcec said. "These learners tend to be well educated and highly motivated. So, instead of focusing on structural barriers that are complex and expensive to overcome, we tested a psychological explanation."

Kizilcec and others theorize that a psychological barrier contributes to the global gap in MOOCs, namely social identity threat, which is a fear of being seen as less competent because of a social identity. Research has demonstrated that social identity threat can impair a person's working memory and academic performance.

But Graduate School of Education and psychology Professor Geoffrey Cohen, who is a co-author on the study, showed in his previous work that simple interventions that encourage people to feel like they belong in the group helped reduce the effects of the social identity threat.

"Psychology matters even in a diverse worldwide population over the Internet, and a little gesture of inclusion can have a big effect on learning outcomes," Cohen said.

Successful interventions Cohen used in face-to-face experiments were adapted for the online environment and implemented into the design of two MOOCs researched as part of the new study.

In both classes, learners were asked to complete an online activity before starting the MOOC. Some learners were randomly assigned a social belonging activity, which had them read and summarize testimonials from previous students about how they worried about belonging in the course at first but felt more comfortable with time. Others were assigned to an affirmation activity, which had them write about how taking the course reflects and serves their most important values.

The first experiment, which was conducted in a computer science MOOC offered by Stanford, collected data on 2,286 learners, of whom 16 percent were in less developed countries. The experiment was replicated in a public policy MOOC offered by Harvard with 1,165 learners.

The results showed that both types of interventions had a large effect on the performance of learners in less-developed countries, doubling their persistence and essentially eliminating the global achievement gap. The affirmation intervention even reversed the gap by raising completion rates for learners in less-developed countries from 17 percent to 41 percent.

"It is an impressive result which suggests that social identity threat can be a barrier to performance in international learning contexts, even in online environments with little social interaction," Kizilcec said. "It goes to show that a small change to the online experience can have profound and lasting effects if it influences people's perceptions of an environment."

In one of the classes, the affirmation intervention, which helped those in less-developed countries the most, slightly reduced the completion rates of those in developed parts of the world. More research needs to be done to understand why, but some studies have shown that affirmation can cause people who aren't under a psychological threat to disengage.

Improving the online environment

Using this new information about interventions, the next step for educators is to design exercises that target the right groups of people.

"Our hope is that through encouraging learners to bring a healthy frame of mind to our courses, we can increase persistence and success and reduce gaps in opportunities between students from different backgrounds," said Justin Reich, a co-author on the study and a research scientist at the Massachusetts Institute of Technology.

Over time, researchers believe that online courses at Stanford and other institutions will evolve as more studies from the Lytics Lab shine light on how best to approach teaching and learning in virtual environments.

"Many of us at Stanford believe that education is a public good," Saltarelli said. "If there are groups of people who can't be successful in learning experiences because of certain barriers, we have a responsibility to identify and remove the barriers."