Study finds top reviews, not average ratings, sway consumer decision-making

But new research shows that the half-star chasm may not be all that important.

It turns out top reviews carry more sway in a customer's final buying decisions when they are comparing products. The research debunks a widely held notion that serious online consumers buy products with a higher rating.

"It's surprising because as a researcher, a business, or a consumer, we typically believe that when we go to Amazon, the most important piece of information is the average rating," said researcher Dezhi (Denny) Yin, who co-authored the study.

Yin said since the starred average rating is an aggregate of hundreds, sometimes thousands, of online reviews, it is the most comprehensive window into a product's quality.

"And what we found was that when they read some reviews, just a few reviews can overturn the impact of average ratings," said Yin, an associate professor in the School of Information Systems and Management in the Muma College of Business at the University of South Florida.

Using a clever "trade-off" design, the research team conducted three studies to disentangle the effects of online ratings and top reviews on consumer decision-making.

The first study was based on real-world daily data of 538 apps ranked in the top 100 from Apple's App Store for two months. The apps covered 21 categories, such as games, business, finance, and news.

In the two other experiments, undergraduate students were asked to make a purchase decision between two digital camera options whose "stars" from average ratings and top reviews did not align with each other. The studies not only provided converging evidence for the swaying effect of top reviews but also pointed to a possible source of this effect.

Review details matter

"It's the text of the top reviews that made a difference," Yin said. "This swaying effect only happened for the text reviews. Without text, people are not swayed. It's the concrete details that are driving this impact."

Yin explained that the research is not saying that average ratings don't matter. If a product has a low average rating, consumers will not consider the product, much less read the product reviews.

But in the cases where buyers are comparing different products and reading their reviews, a few top reviews can easily sway their purchase decisions, he said, adding that the study findings are not limited to app or product reviews.

The ratings game

What are the takeaways for online retailers?

Yin recommends retailers spend less effort on writing or soliciting fake reviews to try to bump up their average star rating.

"Businesses should not spend a lot of time gaming the rating system. That effort is actually not very meaningful or effective, based on our findings," Yin said. "Our findings suggest that as long as your average ratings were fine, what matters is the top reviews."

In addition, retailers would be smart to respond to any negative top reviews, because those are the reviews that most consumers are likely to read. Retailers might counter the swaying effects of such reviews by, for example, explaining that the criticism was an isolated case or that the concern has been resolved, he said.

Also, researchers also recommend online review platforms, such as Yelp and Amazon, could benefit consumers by designing a layout that spotlights individual reviews with less emphasis on average ratings.

Aside from Yin, the article's co-authors include Zhanfei Lei, University of Massachusetts Amherst; Saby Mitra, University of Florida; and Han Zhang, Georgia Institute of Technology.