You’re on a road trip and getting hungry, so you decide to check restaurant reviews for the next town you’ll drive through. Slim pickings: there are only two spots that sound the least bit appetizing. One has two stars with 500 reviews; the other has two stars with only 25 reviews. Which is more likely to be a good experience? We’ll tell you right now: you probably chose wrong. According to research, most people do.
To examine how people use online reviews, researchers from Indiana University Bloomington and University of California, Los Angeles first had to prove what the best way to use them really was. They used a statistical model to find patterns in millions of reviews for products on Amazon. They found that there was no relationship between the number of reviews and the average rating — that is, a product with 5,000 reviews was just as likely to have two stars as to have five stars, and a product with five stars was just as likely to have 5,000 reviews as to have five.
Next, they recruited 132 people to look at pairs of phone cases, presented the way they would on an actual online store. After checking out the average user rating and the total number of reviews, they were asked to select which of each pair they would buy. Across the board, the participants chose the product with more reviews.
What’s wrong with that? The number of reviews isn’t a sign of how good a product is; it’s just an indicator of how accurate the average rating is. Statistically, a two-star product with 5,000 reviews is more likely to be of two-star quality than a two-star product with five reviews. That means that when faced with two low-rated products, you should choose the one with fewer reviews because that rating is more likely to be wrong. It’s no guarantee, but the odds are in its favor.
The problem people get into is judging a product — or restaurant, or hotel — by its popularity, not its rating. But plenty of things that are popular aren’t necessarily good (Smoking! Sleep deprivation! Smooth jazz!). As the motherly adage goes, if all your friends jumped off a bridge, would you? We tend to go along with the pack because of a concept called social proof. If we see all our friends driving Ford Pintos, we feel like we’d be missing out if we didn’t buy a Ford Pinto. Herd instinct anyone?
Check out my related post: Can you get rid of the bias in your head?