Paying online community members to write product reviews backfires badly

The study, "Motivating User Generated Content: Social Connectedness Moderates the Effects of Monetary Rewards," is authored by Yacheng Sun of University of Colorado and Tsinghua University, and Xiaojing Dong and Shelby McIntyre of Santa Clara University. The authors examine user response from a sample of customers following the introduction of a monetary payment program for user reviews by a social shopping platform in China. The payment was roughly the equivalent of 25 cents per review in credit for purchases from sellers affiliated with the platform. To the company's surprise, the number of user reviews declined by over 30 percent in the month after the payment program was introduced, relative to the month before. "The familiar "Law of Supply," that implies supply increases in response to higher prices, does not seem to hold true when it comes to paying for reviews on a social platform," said Sun.

The paper explores why reviews drop in response to the monetary payments. The authors conjecture that the drop in reviews could be the result of community members' concerns that their honest reviews -- motivated by an intrinsic motive to either help others with relevant information or to present themselves as knowledgeable about the product or service -- may now be interpreted by the community as simply driven by the less honorable extrinsic motivation of making money. If this were true, the drop in user reviews would be greater among users who had more friends on the social network, who could potentially misinterpret the user's motivation for writing reviews.

The authors empirically test their conjecture by comparing the change in reviewing behavior among "socialites" (more than five friends on network) against "loners" (no friends on network) after the introduction of the payment scheme. Indeed, the reviews from socialites drop 85 percent, from just over 0.4 reviews a month to just under 0.06 reviews a month. In contrast, the loners who had little to lose in terms of social capital increase their reviews from close to zero to about 0.03 per month. The increase in reviews from the loners however does little to offset the massive drop among the socialites, who are the heavy contributors overall. Hence the aggregate drop of 30 percent.

"Nobody wants to be seen as a paid shill for brands, so the users with more friends and followers, who were likely more influential and wrote more originally, are the ones who stop writing. A real double-whammy," said Dong.

"Our results support the approach of industry leaders like Yelp or Amazon, who do not compensate for reviews. In fact, they tap into the intrinsic motive for social recognition through status badges for frequent contributors," said McIntyre. "There may be still 'under the radar' ways to pay only the less socially active users for their reviews, but such targeting can be risky as the heavy reviewers may perceive it to be unfair and therefore stop writing reviews, if and when they learn about it."