Multilab replication project examines cooperation under time pressure

In the original study, participants who did not follow the instructions to respond quickly (or slowly) were excluded from the data analyses. In the replication project, about 65% of participants failed to adhere to the time pressure constraints, and the results depend on whether or not such noncompliant participants are included in the analysis. When noncompliant participants were excluded, as in the original study, the pattern replicated the original results: People responding under time pressure cooperated at a higher rate than those who had to delay their responses. However, when those non-compliant participants were included in the analysis, time pressure had little or no effect on cooperation. The findings raise questions about the best approach to measuring the relationship between time pressure and cooperation.

The results are published as part of a Registered Replication Report (RRR) in Perspectives on Psychological Science, a journal of the Association for Psychological Science.

The RRR, proposed by psychological scientists Samantha Bouwmeester and Peter Verkoeijen of Erasmus University Rotterdam in the Netherlands, aimed to replicate a study conducted in 2012 by David G. Rand (Yale University), Joshua D. Greene (Harvard University), and Martin A. Nowak (Harvard University).

The original study tested one prediction of the social heuristic hypothesis, which holds that when people make cooperative decisions intuitively, they default to the behavior that is typically optimal in their daily lives. Because cooperation usually pays off in the long run for most study participants, this means that their automatic responses should favor prosocial behavior. Given time to make a more deliberative decision, though, decision makers tend to make more selfish responses.

In the original study, Rand and colleagues manipulated the amount of time participants had to decide how much money they would contribute to a communal pot. Participants randomly assigned to the "intuition" group were told to make their decision about how much to allocate in less than 10 seconds, whereas participants in the "deliberation" group were told to wait 10 seconds before deciding.

Many of the participants in the intuition group in the original study did not respond quickly enough, and the researchers chose to exclude data from noncompliant participants in their analyses. Among compliant participants, those who were told to decide quickly contributed more money to the communal pot than did those who were told to wait.

Some follow-up studies replicated the original study's result, while others did not. With this in mind, Bouwmeester and Verkoeijen proposed a preregistered replication of the original 2012 study. The replication's primary analysis examined data from all participants, compliant and noncompliant, because of concerns that excluding data from noncompliant participants undermines random assignment. More participants tend to be excluded from the time pressure condition, and those excluded participants may differ in systematic ways from the ones who adhered to the time constraints. For example, it's possible that fast participants are also more likely to cooperate -- if this is true, excluding participants who respond too slowly would lead to a time pressure group with a disproportionate number of the more cooperative, "fast" participants.

The researchers developed the RRR protocol, with input and guidance from Rand, first author on the original study.

A total of 21 labs from 12 countries participated in the RRR, testing a total sample of 3,596 participants. All labs used the same experimental materials and scripts (translated when necessary) and preregistered their implementation of the protocol.

A meta-analysis combining data from all participating labs showed no effect of time pressure on cooperation when all participants were included in the analyses.

When only compliant participants were included, however, the results indicated that participants under time pressure cooperated more than those who delayed their response, in line with the original study.

"The most straightforward interpretation of the pattern of results is that the difference in the compliant-only analysis resulted from selection biases and that the RRR does not provide evidence for an effect of speeded versus delayed responses on cooperation," the RRR authors conclude.

Ultimately, the RRR authors conclude that "future studies of the effect of intuition on cooperation should strive to avoid high rates of non-compliance."

In a commentary accompanying the RRR report, Rand thanks the replicators for their time and effort, noting that he was glad to see that the RRR results with compliant participants echoed those reported in the original 2012 study.

He also acknowledges that "the lack of effect when including noncompliant participants in the RRR does raise important questions about the existence of a causal effect of time pressure on cooperation," because the RRR results are ambiguous: they are consistent with both a true causal effect and with selection bias. Rand then provides additional analyses to support the argument that the positive effect when excluding noncompliant participants reflects a real effect of time pressure and not just selection bias.

"I hope the results of this RRR, and the questions that they raise, will inspire future work to more fully illuminate the impact of time pressure -- and intuition more broadly -- on human cooperative behavior," Rand concludes.