Ranked lists skew decision-makers' choices toward top option

"We find that ranking helps only those in the top spot, and the rest suffer from it," said study co-author Jinseok Chun, PhD, of Sungkyunkwan University in the Republic of Korea. "The implications of this can be found in almost every corner of society, because ranking is practically everywhere -- movies, novels, cars, vacuum cleaners, even academic institutions such as universities are typically ranked."

The research was published in the Journal of Personality and Social Psychology.

Chun and co-author Richard P. Larrick, PhD, of Duke University, conducted seven experiments involving historical sports data and more than 3,600 total online participants.

In the first experiment, the researchers examined 70 years of data on which National Basketball Association players were elected to the all-NBA team by a vote of sports journalists each year. The researchers looked at players' stats, including whether a player ranked first in one of three categories (points, assists and rebounds per game), as well as the players' "win shares," a more global reflection of overall performance. They found that players who ranked first in points, assists or rebounds were more likely to be elected to the all-NBA team compared with players with a similar overall performance record who did not rank first in any of the categories.

In the next six experiment, the researchers examined how ranking affected online participants' choices in a variety of scenarios, including making hiring decisions and choosing restaurants, gyms and plumbing products.

For example, in one experiment, participants were shown a list of six restaurants from the website OpenTable. One group of participants saw the restaurants' ratings (on a five-point scale) in three categories, "food," "service" and "overall." Another group of participants saw only a ranked list of the restaurants (first through sixth) that was based on the restaurants' ratings in the "overall" category. The final group saw both the ratings and the ranking. All of the participants knew that the ranked list was based only on the restaurants' rating in the "overall" category.

Among the six restaurants, restaurant 5 had the highest "overall" rating (and the second highest "food" rating), while restaurant 4 had the highest "food" rating (and second highest "overall" rating). The researchers found that participants were significantly more likely to choose restaurant 5 in the ratings-and-ranking condition compared with the ratings-only condition (42% vs. 30%) -- even though the ranking did not provide any actual additional information that was not available in the ratings-only condition.

In other experiments in the study, the researchers found that the ranking effect held true in decisions involving choosing a gym, choosing a drain unclogging product and making a hiring decision. It also held true even when the options were labeled "#3" through "#8" -- a top-ranked option did not have to be labeled "#1" to gain the benefits of the top ranking.

The researchers also found one possible explanation for the ranking effect -- participants spent more time reading about and paying attention to the top-ranked option on a list.

Because they invest a disproportionate amount of attention to the top-ranked option when given rank information, decision-makers tend to learn the strength of the top-ranked option, but ignore the strengths of the other options, according to the researchers. "Ranking makes us focus only on those at the top of the hierarchy and disregard the rest," Chun said.

Ranking has benefits and drawbacks, according to the researchers. Decision-makers face an expanding sea of information and ranking can help people make decisions more quickly when faced with an overwhelming amount of data, but it can also lead them to ignore useful information and make less-than-optimal decisions. Further research can help clarify when ranking helps and when it hurts in decision-making, according to the researchers.