Consumers' trust in online user ratings misplaced, says study

Yet almost all retailers provide user ratings on their websites and many consumers rely on the information when making purchase decisions, according to the paper, published this month in the Journal of Consumer Research.

For the study, researchers examined user ratings for 1,272 products across 120 product categories, such as car seats, bike helmets, sunblock, air filters, smoke alarms and blood pressure monitors. Their analyses show a very low correspondence between average user ratings of products on Amazon.com and product ratings, based on objective tests, found in consumer reports.

"The likelihood that an item with a higher user rating performs objectively better than an item with a lower user rating is only 57 percent," said Bart de Langhe, author of the study and professor of marketing at CU-Boulder's Leeds School of Business. "A correspondence of 50 percent would be random, so user ratings provide very little insight about objective product performance."

In addition, user ratings do not predict the resale value of used products, found the study.

"Products with better reliability and performance retain more of their value over time," said de Langhe. "If average user ratings reflect objective quality, they should correlate positively with resale values. The fact that they don't casts more doubt on the validity of user ratings."

Philip Fernbach and Donald Lichtestein, professors of marketing at the Leeds School, co-authored the study, which also examined what information consumers rely on when judging the quality of products on Amazon.com. Consumers rely very heavily on the average user rating, which is presented front and center, according to the paper. They do this regardless of whether the average rating is based on a small or a large sample of consumers.

"This is a mistake," said de Langhe. "Oftentimes, there are just not enough ratings for a product or there is too much disagreement among reviewers. In this case, consumers should not trust the average very much, but they do nonetheless."

In light of their findings, the authors recommend consumers be more cautious when they make inferences about product quality based on user ratings.

"Accurately evaluating product performance is not an easy task," said de Langhe. "Different alternatives need to be evaluated side by side under the same conditions using objective measurement instruments. You can't assume that people follow such a scientific approach before they rate products online."

What are the implications for marketing and branding in the age of user-generated content? Because consumers comparing two products with the same objective qualities tend to give higher ratings to the one that is more expensive or from a brand with a premium reputation, marketing still counts.

"Brand managers should not fear for their jobs just yet," said de Langhe. "User ratings do not shield consumers from the influence of good old-fashioned advertising and price-signaling."