Do you trust 5 star reviews?

[LSLGuy]

Here’s a recent thread on a closely related topic: http://boards.straightdope.com/sdmb/showthread.php?t=810492

[Enkel]

I recently bought a LED Lantern w/USB port to charge your phone from. I wrote one of those soppy reviews for it because it is amazing. When I opened it up, the design is beautiful… but I love touching the material that it’s made from. It’s a slightly brushed surface that feels both sturdy and soft and the grippy area has a rubber-ish feel to it. Even while I was writing the review, I knew people would blow it off, but I just love this lantern and almost want to cuddle it because it feels so nice and is so nice looking. Something about it just strikes all the right chords in me.

So, then of course I told everyone about it and the they all got the lantern that I’d paid $16 for for only $10 because of black friday sales.

[Broomstick]

Yeah, I did a glowing review for the shoes I bought for work - but they really were amazing. Not only did they have my actual size (8.5 EE) but I can stand/walk on concrete floors for 8 hours, even while carrying boxes and stuff of some weight all day, and go home and my feet don’t hurt. Which, really, is fantastic and much more rare than you might think. I like to think that because I was clear on why I thought they were so fantastic - available in an odd size and their performance - people would be inclined to think there was some substance there. For me they really were worth 5 stars.

[Jackmannii]

There is nothing intrinsically more reliable about 1-star reviews than 5-star reviews.

If someone can coherently describe why they love or hate something then it’s relevant.

Too often, the reviewer adores/hates the company/book author for various reasons unconnected with the quality of the product and is there to shill/pan.

With electronics, I’ve learned to pretty much discount a low percentage of 1-star reviews, no matter how alarmist. There’s always going to be someone who got a defective item and can’t hate it enough. A recent favorite: I was hinting to someone about a weather station as a potential gift, but was concerned to see a couple of recent reviews talking about how it wasn’t very good quality and touting a top-of-the line model for lots more $$. Then I went to the reviews of the top-of-the line station and there were people bitching about the same issues.

[Jophiel]

@RickJay

Eventually all ratings systems become a Ten Point Must system. Logically, if you think about it, the standard rating for a product should be the midpoint. If I buy a coffee maker and it works, I should give it three out of five stars. Four out of five stars should, logically, be a coffeemaker that is substantially superior to other coffee makers of comparable price.

The way I see it, an item mainly just needs to hit its marks. Does it have functions comparable to its price? Do the functions work correctly? Does it do its job? It’s like scoring a middle school paper about polar bears: Did they cite their stuff, did they use proper grammar, did they have an opening, body and proper conclusion? I don’t need it to be the Alpha & Omega of devices to get four or five stars, it just needs to perform competently in all the fields. If I feel it’s truly exceptional, I’ll say as much in the review title and body.

[Thudlow_Boink]

@RickJay

Logically, if you think about it, the standard rating for a product should be the midpoint. If I buy a coffee maker and it works, I should give it three out of five stars.

Or you would just not bother to rate it at all.

If ratings are voluntary and self-selected, they’re going to be skewed toward the extremes because only people who feel strongly about something (positively or negatively) will bother to register a rating/review.

[gnoitall]

@Backwater_Under_Duck

I needed an fan to replace one that had died, and I researched idiot reviews for the one I was looking at. Lady says (paraphrasing), this fan is terrible. It doesn’t cool the room at all. You have to be right in front of it for it to work… You know, like a fan.

Those are the people for whom you wish the Koreans were right.

My strategy for reading reviews is a lot like the other wise and good-looking posters here. Read the five-stars, mostly looking for signs of shillery. Look at the one-stars, but discount obvious ignorants who complain for ignorant and irrelevant reasons. (Some “protest” 1-star reviews are amusing, but that’s the most I can hope for.)

The longer mid-score reviews are sometimes the most enlightening, especially when the writeup is balanced, including pros and cons.

It’s a lot of work.

[kopek]

Like some others I start at the 1 star and work my way up to maybe 3; I get/have gotten the best results that way.

[Omar_Little]

I usually look for items (Amazon) or businesses (Yelp) that have the most reviews rather than the highest avg. review. Products or businesses with the most reviews tend to be a better indicator than the avg. review of a well liked product or business. Yes, I use common sense and would ignore a product or businesses that had the most reviews and the avg. was one star. But those are normally remote.

[DCnDC]

I trust detailed, comprehensive reviews, regardless of the star rating.

Generally, brief 5-star reviews are worthless and I find most 1-star reviews to not be worth much more. There’s always shills and assholes who aren’t satisfied with anything, are willing to lambaste items for truly petty issues, or clearly didn’t understand what they were buying. 1-star reviews can be entertaining, though.

[Driver8]

@Fretful_Porpentine

I don’t trust ANY individual review, although I give more weight to ones that are detailed, supported by evidence, and don’t seem to be written by complete nut jobs. I do trust patterns, and once a few reviews of an item have been posted it’s not usually hard to see what patterns are emerging.

This is effectively my strategy. I don’t trust any specific review, no matter the star rating, although sane and detailed do get a little more weight. Patterns of fact are important. Recently I decided against a key-less entry (using Bluetooth and proximity to your phone) because an overwhelming number of reviews pointed out that recognition was flaky.

The other thing that I do take into account is if the manufacturer is replying. In particular, if they respond to negative reviews in a friendly and helpful manner I consider that a major plus. It suggests that (a) they’re organized and (b) they have the right attitude, which in my opinion greatly improves the odds of being the sort of organization that cares about putting out a good product.

[Eonwe]

@RickJay

The problem with anything like this - product reviews, customer satisfaction surveys - is always the same; the scores drift to perfect.

I call this the Boxing Problem. In boxing, the traitional professional scoring system is called “Ten Point Must.” The idea of Ten Point Must is the boxer who wins the round gets ten points no matter what. The loser is awarded nine points, unless he was knocked down once, and then he gets eight, or knocked down twice, in which case he gets seven, and so on. Theoretically a judge could decide no one won the round and they both get ten.

Eventually all ratings systems become a Ten Point Must system. Logically, if you think about it, the standard rating for a product should be the midpoint. If I buy a coffee maker and it works, I should give it three out of five stars. Four out of five stars should, logically, be a coffeemaker that is substantially superior to other coffee makers of comparable price. Five stars should be a coffee maker that is clearly and remarkably better, by a wide margin, than a good coffee maker. I really like my coffee maker; it is easily used, has some settings for small pots and dark brews, and it’s worked fine for over a year. But that is what I expected. It’s a 3/5. It makes nice coffee and has a timer, but that’s why I paid money for it. It isn’t better than I expected or paid for. Yet 90% of the reviews for it on Amazon are 4/5 or 5/5, which is ridiculous, it’s a budget level coffee maker. Unless you were previously making coffee with dirt and an old bedpan, this is not a perfect coffee maker.

But it never, ever works that way. All ratings systems will drift to perfect or near-perfect scores unless something is wrong, and then the ratings start taking a point off. I’ve read innumerable reviews scoring a product or service 4/5 or 9/10 when the product/service was, judging by the comments, clearly subpar.

You can expand this to almost any ratings system… hell, performance reviews. In my company, “Meets Expectations” is considered a profound disappointment. You expect “Exceeds expectations.” and yet, logically, that cannot actually be true; the vast majority of our employees would logically be rated as Meets Expectations. Very few would be Exceeds or we’d be rolling in money.

If a ratings system is not deliberately, periodically calibrated, this ALWAYS happens.

I don’t know if I’d call this a “problem” though. If you understand how to read the system (that the best rating is equivalent with “completely meets expectations”), then I don’t think there’s an issue.

And frankly, for many product reviews, a maximum score for “meets expectations” makes sense to me, because unless I am constantly buying various models/makes/versions of the product, I don’t have enough knowledge (and neither do most customers) to make meaningful comparisons between similar items.

Bottom line, if I buy a widget, and I am totally happy with the way it widgets, well, why shouldn’t that be a 5/5? My level of customer satisfaction is complete.

This is not a case of inflated ‘everyone’s a winner’ reviews, but a misunderstanding of the kind of information that aggregated user reviews can reasonably collect.

[Staggerlee]

@jtur88

Sometimes I read hotel reviews just for amusement. Somebody downgrades a hotel because there was a thumb print on a mirror. They check into an airport hotel and complain about the loud planes flying over. There was no little plastic stirrer with the packet of Nescafe.

Last summer I spent a month in Asia, using Bangkok as a base, and checked in and out of the same airport hotel four times. I loved the place, they treated me like family. Users panned it, said it was in a poor neighborhood (in Bangkok!!).

When I read hotel reviews, I say to myself, I sure hope I never meet any of those assholes when I travel there.

I love reading 1 star reviews on Tripadvisor.co.uk - there was a chap so disappointed with his trip to the Clifton Suspension Bridge in Bristol, he wished for its collapse (which would be several hundred feet onto a busy road). And my favourite responses to 1 star reviews on there are from the Balmer Lawn Hotel in Brockenhurst, which rather than apologising, accuse the reviewers of dirty tricks and lying. Marvellous.

[Chimera]

Depends. If a place has 1-3 absolutely glowing (boilerplate) reviews and most of the rest are lower, I’ll actually count them against them since it is fairly obvious that they wrote them themselves.

But as others, I’ll usually read through the low reviews for potential issues, especially with machines. For example, I own a Hoover Elite Rewind and most of the low reviews focused on the belt breaking. Well, after about two years of use, I thought my belt had broken, so I ordered 4 more, super cheap (@ $5 if I recall correctly)) and then followed a Youtube video to replace. Was easy and took about 5 minutes. Only when I opened mine, I found that the belt wasn’t broken, it had just come off. Popped it back on and have my four spares. Even then, if I’d have bothered checking the belt first, it wouldn’t have cost me anything but 5 minutes to put it back on.

But again, if you’re reading a restaurant review and there are two glowing boilerplate reviews and a bunch of 2-3 star reviews complaining about the service, you can pretty much expect the service to suck balls.

[VinylTurnip]

---

This thread is great and very useful!! I recommend to all who want good advice about reviews. The discuss of stars and ratings of product gives much help!! Best thread EVER!!!

[ticker]

The number of stars matter less to me than the text. If the reviewer makes substantive points about issues that are relevant to me I take them on board, otherwise I ignore them.

[Jackmannii]

@Staggerlee

And my favourite responses to 1 star reviews on there are from the Balmer Lawn Hotel in Brockenhurst, which rather than apologising, accuse the reviewers of dirty tricks and lying. Marvellous.

I’ve always wondered why some business owners think it’s a sound strategy to viciously attack negative reviewers, or better yet, threaten to sue them.

Makes me eager to purchase their products and services. :dubious:

[bump]

@Fretful_Porpentine

I don’t trust ANY individual review, although I give more weight to ones that are detailed, supported by evidence, and don’t seem to be written by complete nut jobs. I do trust patterns, and once a few reviews of an item have been posted it’s not usually hard to see what patterns are emerging.

Exactly. I tend to look at the relative ratio of the various reviews, and the total number, and then if that’s favorable, I usually read the 1-2 star reviews, and a handful of the 4-5 star reviews to see what the complaints and compliments are.

That way, if something has a lot of 1 and 5 star reviews, you can either deduce that it sucks and has shills recommending it, or that it’s a great product with serious QC issues. (have seen both). If it’s heavily weighted 3,4,5 star reviews, with a large number of reviews, you can usually rely on that product to be solid, if not very good.

And so on…

Just because an individual review may be a 1 or 5 star review doesn’t make it immediately suspect; I’ve reviewed things as 5 star because they really are excellent, and things as 1 star because they truly are terrible.

[LSLGuy]

@Jackmannii

I’ve always wondered why some business owners think it’s a sound strategy to viciously attack negative reviewers, or better yet, threaten to sue them.

Makes me eager to purchase their products and services. :dubious:

A lot of small business owners are small business owners because their personality won’t let them work for anyone else. IOW, they’re “I’m always right” jerks.

I recall an Olde Tyme real hardware store where the owner was quite the one-way jerk. But he had one of everything or could honestly explain why you didn’t need what you thought you needed but needed *this * thingy instead. Just don’t start to dispute with him; that never ended well.

ISTM the post WWII era was huge for guys coming out of the service with an anti-authority chip on their shoulder and going into small retail. Which businesses stayed small since they couldn’t manage people. At all.

[minor7flat5]

My biggest gripe against the star rating system is that all products start life with a very high or very low star rating, and there is no way to filter out that noise when searching for a good widget.

For example, if I am searching for a guitar tuner on Amazon, I will sort by star rating high-to-low, but that just means that there are 500 weird quirky one-off gadgets with three reviews cluttering the good end of the spectrum. I have to click through a dozen pages of flaky obscure sellers with questionable merchandise before I find the item I really want.
I don’t want to see the gizmo with 3 five-star ratings…I want to see the solid performer with 453 ratings that averages 4.8 stars.

But I suspect that a weighting system for sorting reviews is not in Amazon’s best interests, otherwise they would have created such a system long ago.

ETA: LSLGuy, I agree…so many B&M owners are tedious jerks of the “it’s my way or the highway” variety. I like supporting local businesses when I can, but the arrogance of some small business owners sure makes it a challenge.