Effect of facial expression on emotional state not replicated in multilab study

The RRR project, proposed by University of Amsterdam psychology researchers Eric-Jan Wagenmakers, Titia Beek, Laura Dijkhoff, and Quentin Gronau, aimed to replicate a 1988 study conducted by psychological scientists Fritz Strack, Leonard Martin, and Sabine Stepper.

In the 1998 paper, Strack, Martin, and Stepper reported two studies in which they surreptitiously changed participants' facial expressions. Their goal was to test the idea that our facial expressions can trigger emotional reactions -- the so-called "facial feedback hypothesis" -- even when people are unaware that they are making that expression. Participants who held a pen between their teeth, inducing a smile, rated cartoons as funnier than did those who held a pen between their lips, inducing a frown.

The study is cited frequently in the scientific literature and in introductory psychology courses and textbooks. Although other studies have tested the facial feedback hypothesis using different methods, this influential study had not been directly replicated with the same design and outcome measure. This RRR paper describes a rigorous, multilab replication of that study, with each lab following a vetted protocol that was registered online prior to data collection.

The aim was to replicate the original study as closely as possible, but the RRR differed in several ways from the original. Fritz Strack provided the materials from the original study, including the original Far Side cartoons. The RRR study also used a set of Far Side cartoons after first conducting a study to ensure that they were moderately funny by today's standards. The RRR protocol also standardized the instructions to participants and stipulated that they be delivered via computer in order to minimize interactions with the experimenter. Based on guidance from an expert reviewer during the protocol vetting process, participants were recorded on video during the experiment to ensure that they were holding the pen correctly on each trial.

All of the materials, the protocol, the data, and the analysis scripts are publicly available on the Open Science Framework.

As in the original study, participants were told they would be completing different tasks with parts of the body not normally used for those tasks. Per the instructions provided, they held the pen in their mouth (between their teeth or between their lips) and completed the tasks presented in a booklet, which included drawing lines between various numbers, underlining vowels, and indicating how amused they were by cartoons.

The combined results from 1,894 participants were inconsistent with the findings reported in the original study. The data provided no evidence that inducing participants to have particular facial expressions led them to rate the cartoons differently.

"This RRR did not replicate the [Strack, Martin, Stepper] results and failed to do so in a statistically compelling fashion," the contributing researchers write in their report.

"Nevertheless, it should be stressed that the RRR results do not invalidate the more general facial feedback hypothesis," they conclude.

In a commentary accompanying the RRR report, Strack commends the efforts of those involved in the RRR. He notes his surprise that the original finding was not replicated, especially given that his and colleagues' labs have confirmed the results in "numerous operational and conceptual replications." Strack speculates about some possible reasons for the different outcomes, including that the presence of a camera in the RRR experiments might have affected how participants reacted to the cartoons.

Daniel Simons, the acting editor for this RRR project, commended the care taken by the proposing authors: "This team's exceptional rigor and care in developing the study protocol, teaching other researchers how to follow it, and fully documenting every step of the process set a standard that I hope future large-scale studies like this one will emulate."