Reply
 
Thread Tools Display Modes
  #1  
Old 02-16-2017, 07:44 PM
Shagnasty Shagnasty is offline
Charter Member
 
Join Date: May 2000
Posts: 26,771
What is the deal with score inflation in web culture?

I have noticed that current web culture has a strong aversion to bell curves. Anything but a great score is bad whether it is on a five point scale, ten point or anything else. It reminds me of the fictional Lake Wobegon - "Well, that's the news from Lake Wobegon, where all the women are strong, all the men are good looking, and all the children are above average."

3 points (or stars) just means average on a 5 point scale yet many people consider that an insult. It isn't. It just means perfectly fine in the same way that 5 points should mean the same on a 10 point scale. That gives reviewers the most freedom indicate relative ranking in either direction.

Uber is the most notorious offender in this regard. Anything less than a 4.6 on a 5 point scale can be cause for driver suspension. That is crazy. I am rating a given driver against other Uber drivers and not the great limo that I once took or the falling apart car that picked me up in Costa Rica when I needed help. There shouldn't be many 5's in a fair rating system unless they think it is just a checklist of basic competencies.

It isn't just Uber though. The whole idea has spread so that many companies expect people to defy basic statistics and consumer psychology. I consider it abusive, unrealistic and and mathematically unsound. An average score (meaning one in the true middle) should be fine for most purposes by definition.

What do you think?

Last edited by Shagnasty; 02-16-2017 at 07:46 PM.
Advertisements  
  #2  
Old 02-16-2017, 07:51 PM
Jophiel Jophiel is offline
Guest
 
Join Date: Mar 1999
Location: Chicago suburbia
Posts: 16,631
Quote:
Originally Posted by Shagnasty View Post
3 points (or stars) just means average on a 5 point scale yet many people consider that an insult. It isn't. It just means perfectly fine in the same way that 5 points should mean the same on a 10 point scale. That gives reviewers the most freedom indicate relative ranking in either direction.
Of course, if you think about it like an academic scale, 50% is failing and you need a 7/10 to be average at best. Granted, in your Uber example, a 4.6 would still be a 92%.

I think movie, music, video game, etc reviews tend more towards the A-F scale though so average movies still pull 7 stars (or a 7.2 score, etc).
  #3  
Old 02-16-2017, 08:05 PM
Dr. Strangelove Dr. Strangelove is offline
Guest
 
Join Date: Dec 2010
Posts: 5,925
It's not limited to "web culture", whatever that is. Car salesmen also say that "anything less than an A is an F" for their satisfaction surveys.

I think it's pretty obvious why inflation happens; whoever successfully games the system first ruins it for the rest by compressing the scale. I think that's part of why many rating services are just "like" and "dislike".

Ideally, these kinds of surveys wouldn't use star ratings at all, but have internal metrics for failure on objective grounds. If Uber users report "the driver was X minutes late", then they can feed that into a system for evaluating tardiness. Etc.
  #4  
Old 02-17-2017, 03:34 AM
polar bear polar bear is offline
Guest
 
Join Date: Jul 2011
Posts: 989
Here in the netherlands we use a 10 point scale in school and 6/10 is a passing grade, which in practice equals to a 5.5 since exam grades don't get rounded to a whole number.

I have noticed this "either 1 star or 5" all over the place. Last time I went to pick up a car from the dealerschip they had me fill in this list of questions about the service while receiving the papers/explanation of the car/etc. It was perfectly fine so I was going to give 4 out of 5 on most of them... the guy there however asked me if I was satisfied, and after my "sure" he went on to explain the formula that the head office uses to determine if the dealership did its job right. Turns out that 4 out of 5 on every item does not meet that standard (saw the norms on the form and all) and can lead to a reprimand. Ridiculous, so I changed a couple to 5/5.
  #5  
Old 02-17-2017, 05:24 AM
Teuton Teuton is offline
Guest
 
Join Date: Jan 2012
Location: Plymouth, UK
Posts: 1,177
At my job, customers can rate our visits on a 1-5 scale, and anything less than a 5 raises eyebrows.

However, our yearly appraisals also use a 5 point scale, but the appraisers don't give out 1s or 5s at all - they say if you deserve a 1 you'd have at least had written warnings, and 5 means perfection, and no one is perfect. So it's essentially a 3 point scale. Of course, anything less than 4 is bad.
  #6  
Old 02-17-2017, 05:29 AM
hogarth hogarth is offline
Guest
 
Join Date: Apr 2009
Location: Toronto
Posts: 6,568
Quote:
Originally Posted by polar bear View Post
I have noticed this "either 1 star or 5" all over the place. Last time I went to pick up a car from the dealerschip they had me fill in this list of questions about the service while receiving the papers/explanation of the car/etc. It was perfectly fine so I was going to give 4 out of 5 on most of them... the guy there however asked me if I was satisfied, and after my "sure" he went on to explain the formula that the head office uses to determine if the dealership did its job right. Turns out that 4 out of 5 on every item does not meet that standard (saw the norms on the form and all) and can lead to a reprimand. Ridiculous, so I changed a couple to 5/5.
I bought a new car recently and they gave me the same story. I found that annoying enough that I lowered one of the ratings from 10/10 to 8/10 ("salesman annoyed me by begging for ratings").
  #7  
Old 02-17-2017, 07:27 AM
jtur88 jtur88 is online now
Guest
 
Join Date: Aug 2011
Location: Cebu, Philippines
Posts: 12,836
This might have started with the popularity of the Olympics. Skaters and gymnasts would get at least a 7.5 just for finishing their routine. When Nadia Comeneci got a ten, the sun stood still, and from then on, it was assumed that somebody in every event deserves a ten.
  #8  
Old 02-17-2017, 08:12 AM
Mangetout Mangetout is offline
Charter Member
 
Join Date: May 2001
Location: England
Posts: 56,560
Quote:
Originally Posted by Shagnasty View Post
Uber is the most notorious offender in this regard. Anything less than a 4.6 on a 5 point scale can be cause for driver suspension. That is crazy. I am rating a given driver against other Uber drivers and not the great limo that I once took or the falling apart car that picked me up in Costa Rica when I needed help. There shouldn't be many 5's in a fair rating system unless they think it is just a checklist of basic competencies.

It isn't just Uber though. The whole idea has spread so that many companies expect people to defy basic statistics and consumer psychology. I consider it abusive, unrealistic and and mathematically unsound. An average score (meaning one in the true middle) should be fine for most purposes by definition.
eBay is another one - buyers are asked to rate their experience on a 1 to 5 scale, and it's sort of assumed that 3 is 'OK', but in reality, once a seller's rating drops below 5, all sorts of quite harsh things happen (and happen in a way that makes them very hard to recover from - can't get your average back up if you can't sell stuff any more).

I think it comes from process-driven ideas of 'continuous improvement', where, instead of actually measuring improvement from a previous baseline, you measure shortfall from perfection (and in real-world cases that I have to deal with, something can be measurably better than last week, but if it's still measurably short of perfection, that's how much *worse* it is argued to be, vs last week)
It's that, and shitty communication about what the ratings in a system actually mean.
Quote:
Originally Posted by Dr. Strangelove View Post
It's not limited to "web culture", whatever that is. Car salesmen also say that "anything less than an A is an F" for their satisfaction surveys.
In the employee performance review process in one of my previous jobs, someone in upper management actually said "We expect most of our people to exceed expectations". Yeah, because it's not like the word 'expect' really has a meaning.

Last edited by Mangetout; 02-17-2017 at 08:12 AM.
  #9  
Old 02-17-2017, 09:31 AM
Rysto Rysto is offline
Guest
 
Join Date: Jun 2002
Posts: 6,905
For online reviews, I suspect that the cause is sampling bias: people who really like or really dislike a product are far more motivated to rate it.
  #10  
Old 02-17-2017, 09:53 AM
Velocity Velocity is offline
Guest
 
Join Date: Jun 2014
Posts: 10,046
Same thing happened with Xanga, back when it was still the thing, a decade ago. For every comment you made about someone's blog post, you could leave either:

0 "eprops"
1 "eprops" - average rating
2 "eprops" - exceptionally good approval


But overwhelmingly, 2 eprops was the most common rating given, to the point that if you gave 1 eprop - denoting average approval of someone's post - people would wonder what they did wrong and why you were "disapproving" of them.
  #11  
Old 02-17-2017, 10:01 AM
Thudlow Boink Thudlow Boink is offline
Charter Member
 
Join Date: May 2000
Location: Lincoln, IL
Posts: 23,862
Quote:
Originally Posted by Rysto View Post
For online reviews, I suspect that the cause is sampling bias: people who really like or really dislike a product are far more motivated to rate it.
I think this is one big part of it.

Another factor is that, in many cases, people are asked to rate things, but there's no stated guide to what the different levels mean. So, to one person, five stars may mean "truly exceptional; one of the best ever," while to another person it means "there wasn't anything wrong with it."
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 04:23 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2017, vBulletin Solutions, Inc.

Send questions for Cecil Adams to: cecil@chicagoreader.com

Send comments about this website to: webmaster@straightdope.com

Terms of Use / Privacy Policy

Advertise on the Straight Dope!
(Your direct line to thousands of the smartest, hippest people on the planet, plus a few total dipsticks.)

Publishers - interested in subscribing to the Straight Dope?
Write to: sdsubscriptions@chicagoreader.com.

Copyright 2017 Sun-Times Media, LLC.

 
Copyright © 2017