When “5 stars” does no one any service

HACK

When “5 stars” does no one any service

From Amazon to Airbnb, the starrating system has become a standard metric, tapping the wisdom of the crowd to give online service providers instant feedback while consumers get to see who the top-rated providers are. But stars can be misleading, depending on the nature of the rating system and the dimensions being assessed.

In seven experiments, participants from Australia, Canada, England, Liechtenstein, New Zealand and the United States were asked to evaluate restaurants, movies and universities. Some ratings were single-dimensional, meaning customers were just asked to rate their experience overall; others were multidimensional, meaning each individual element was rated separately, so food, service, atmosphere, etc.

Where individual dimensions were rated highly, the overall rating tended to be higher than on a platform that only asked for a single, all-encompassing rating. The opposite was also true: Lower scores for individual dimensions pulled down the overall score, even if decor, for example, was less important to a customer primarily interested in the food.

This bias is relevant for platform providers. Although multidimensional ratings provide a fuller picture, they may unduly push ratings in one direction or the other. So, instead of asking for dimensional and overall ratings, platform providers might be better off creating their own aggregates, for less biased results. Also, as in the previous example, providers should consider their “choice architecture” carefully, knowing which dimensions are better to ask (or not ask) their customers to rate. Consumers, too, will benefit from being able to evaluate the dimensions most helpful to them.

“When the stars shine too bright: the influence of multidimensional ratings on online consumer ratings” by IESE’s Christoph Schneider et al. is published in Management Science (2020).