'Emotional' reviews predict business success, new study shows

That's because most products online are now rated positively, making it harder than ever to truly discern whether they will succeed in the marketplace.

A new study from Northwestern University Kellogg School of Management and the University of Massachusetts Boston was able to predict the success of movies, commercials, books and restaurants by relying on the "emotionality" of reviews instead of the star rating.

The researchers explored box office revenue of 2,400 movies, sales of 1.6 million books and real-world reservations at more than 1,000 restaurants. In their analysis, emotion reigned supreme; in fact, higher star ratings were either a weaker signal of success or negatively related to success.

"One attractive aspect of online purchasing is that consumers have reviews at their fingertips," said Derek D. Rucker, the Sandy & Morton Goldman Professor of Entrepreneurial Studies in Marketing at Kellogg and co-author of the study.

"However, this research demonstrates a 'positivity problem': Most reviews are positive. In this 'sea of positivity,' how do people distinguish products?" Rucker said.

On Amazon.com, for example, the average star rating is approximately 4.2 out of 5 with well over half of the reviews being five-star ratings. Nearly half of all Yelp reviews are five-star ratings, and recent research indicates that nearly 90% of Uber ratings may be five stars.

Indeed, the study found that review ratings were overwhelmingly four and five stars, a few one-star ratings, and few ratings in between.

Clues to success, it turns out, may rest in the very language consumers use.

Across four large-scale studies, the researchers used computational linguistics to assess the emotionality of reviews -- that is whether the reviews suggested the person writing it had an emotional reaction to the product.

In one study, they looked at the movie industry and examined 13 years of Metacritic reviews from 2005 to 2018. Star ratings, they found, were a significant negative predictor of box office success. Emotionality, however, was a significant positive predictor.

The researchers also were able to predict the success of books on Amazon using the same model based on 20 years of data from 1995 to 2015. They found that 91% of books received a positive rating (four or five stars). The average star rating was an unreliable and sometimes even negative predictor of purchases; however, greater emotionality was predictive of more sales in 93% of genres.

They also examined television commercials. For both the 2016 and 2017 Super Bowls, they obtained real-time tweets the day those ads aired and they measured the subsequent gain in Facebook followers. Then, they compared their own model, which incorporates text emotionality, to the USA Today Ad Meter survey, a popular Super Bowl rating system. The USA Today rating was not predictive of the number of new followers. But the greater the emotionality of tweets, the more Facebook followers a company gained over the following two weeks.

Another domain they turned their attention to was restaurants. Relying on Yelp reviews, the researchers examined whether star ratings or emotionality was a better predictor of table reservations.

Although the average star rating did predict more table reservations, the emotional analysis of restaurant reviews eclipsed the predictive ability of star ratings.

A positive and highly emotional review might read: "I love this restaurant, it is absolutely wonderful." In contrast, a position, but less emotional review might read: "This restaurant is excellent, dinner was flawless."

"This research shows just how pervasive the positivity problem is in online reviews. This work also advances our understanding of emotionality -- a construct considered of significant importance across the social sciences -- by revealing it can predict behavior and marketplace success," explained lead author Matt Rocklage, assistant professor of marketing at University of Massachusetts Boston.

"This work could also lay the foundation for alternatives to star ratings. Firms might ask for a future 'emotional star rating' to understand the attention to the emotionality of individuals' attitudes," said Loran Nordgren, professor of management and organizations at Kellogg and co-author of the paper.