They are business admins. Their thoughts and concepts have little to do with reality.
The system doesn’t use 3 ratings. It uses 5. Your reason for excluding 1 and 5 doesn’t make sense and your “system” doesn’t give any useful information. If you are rating 5 restaurants on whether or not they have good food, your system requires you to give one a 1. Even if they all are excellent and you’d love to eat there again. Therefore your ratings are not of any use to anyone. That makes no sense.
Yes, but some systems are more arbitrary than other.
But “this item is complete crap/perfect” is important information, and if your scale doesn’t include those endpoints, it’s inherently incomplete. If you insist on using 1s and 5s equally to 2s through 4s, you obscure the endpoints. “Is this really perfect/crap, or just relatively perfect/crap?”
It depends what I’m rating. If it’s possible for something to be perfect, then that gets a 5. For instance, an ebay vendor is usually perfect, and back when I used ebay, I mostly rated them “5”. (The thing was what they said it was, and they delivered it promptly.) If it’s something more subjective, like a book or a restaurant, then a 5 means “really excellent”, and I don’t give out many of them.
ebay vendor:
5 – perfect transaction
4 – minor problem that was corrected
3 – problems that were ultimately dealt with
2 – serious problems
1 – vendor lies or cheats or steals
I don’t think I ever gave any rating other than 4 or 5, and almost all ratings were 5.
Movie:
5 – excellent movie
4 – very good movie
3 – this movie has virtues, and some people will enjoy it despite its flaws
2 – this is a bad movie, but there might be reasons to watch it.
1 – I turned it off, and don’t expect anyone to want to watch it.
Most movies I see get 4s, with a scattering of 5s and 3s. I don’t see all that many movies, and usually only go to ones that are pretty good.
Restaurants:
This is trickier, because fancy restaurants, when they are good, are a lot better than the local shawarma place. But the local shawarma place can be a really satisfying meal. I rarely rate restaurants, and if I did, I would want to clarify how I was dealing with that.
Personally, I feel that 5 point Likert scales only rate about 3 or 4 on a 7 point Likert scale.
You’re not following what I’m saying.
If I’m rating five restaurants and I feel all five of them belong in the top twenty percent of restaurants, then I would give them all a five. And if I felt they all belong in the bottom twenty percent of restaurants, I would give them all a one.
Well, at that point we’re just quibbling about what 1s and 5s mean. We seem to be in agreement that, in any real-world collection of items, we could have virtually any distribution.
It doesn’t have to be “perfect” or “I’d rather eat glass” but giving 20% of the items a 1 , 20% a 2 and so on is sort of a bastardized mix of ranking and rating. Let’s take restaurants. I’m going to rate the last 20 restaurants I’ve been to. Using your equal distribution, that should result in four in each category. So the four restaurants in the lowest category consist of 2 in which I really don’t like the selections on the menu and although I did find something to eat and it was edible, it wasn’t particularly good and I would never go to those restaurants again unless I had no options. The other two however, I didn’t even get as far as ordering and eating. I left when I saw a mouse run across the floor or when I saw an employee scooping coleslaw into a bowl from a big bucket with his bare hands. A rating system that puts those four restaurants in the same category isn’t very useful. Two of them should be lumped in with the places where I like the menu selections better and the food was edible but not particularly good and I might go to that restaurant again if my dining companions liked it.
And the same goes for the highest rating - maybe there are one or two restaurants where the menu, the service, the food preparation are far above all the others. I don’t think a system where the “excellent” is lumped in with the merely “very good” is all that useful.
Edited:
How could you possibly know whether a restaurant is in the top (or bottom) twenty percent of restaurants? Or do you means the top twenty percent of restaurants that you’ve been to? - which is not going to be useful to anyone else.
I disagree. I feel my system conveys more information.
Let’s say you and I are rating a hundred restaurants and we’re in perfect agreement about their quality. But we use two different systems for assigning ratings.
My list might look like this:
1 star - twenty restaurants
2 stars - twenty restaurants
3 stars - twenty restaurants
4 stars- twenty restaurants
5 stars - twenty restaurants
Your list might look like this
1 star - two restaurants
2 stars - thirty-two restaurants
3 stars - thirty-two restaurants
4 stars - thirty-two restaurants
5 stars - two restaurants
Now I think we’d both agree that there’s no reason to eat at any of the worst restaurants. But I give a list of twenty restaurants that are the worst while your list only identifies two as the worst. Eighteen of the restaurants I rated as among the worst group are given two star ratings in your system.
The same is true at the other end. Your system gives the same rating to the third best restaurant in town as it does to the thirtieth best restaurant. But I feel there’s going to be a noticeable difference between the third best and the thirtieth best restaurants.
Granted, your ratings system is better at picking out the extremes. Yours is better at picking out the two best restaurants and the two worst restaurants in town. But I feel my ratings system conveys better information about the other ninety-six restaurants we reviewed.
I disagree. I feel that there’s a difference between saying “this movie approaches perfection” and “this movie is in the top fifth of all movies”. I can think of a lot of movies in the latter group that aren’t in the former.
All ratings systems are arbitrary.
If I think Casablanca is the best movie ever made, it doesn’t mean Casablanca is the best movie ever made. It just means I think it is.
If you think Star Wars is the best movie ever made, it doesn’t mean my rating is wrong and your rating is right.
I understand that - what I meant is that you have no basis to think a restaurant is in the top 20% of even a decent sized town because you surely haven’t eaten in all of them , and if you mean that a restaurant is in the top 20% of restaurants you’ve been to, that doesn’t mean very much unless I know most of the restaurants you’ve been to.
And actually, I would never think that any movie was the best movie ever made - that’s kind of what I’m trying to get at . At most, it would be the best movie I’ve ever seen , which is very different.
This is true. If you’re rating things you have to have some kind of foundation of experience in the things you’re rating. Otherwise you might think your experiencing greatness when all you’re really experiencing is a first time encounter.
But I feel once you’ve built up a foundation of experience, you can judge new items based upon past encounters. You may not have eaten at all the restaurants but you’ve eaten at enough restaurants to have a general idea of the spectrum of restaurants.
Something can catch you by surprise though. Maybe, for example, you’ve never eaten Thai cuisine before. You go to a Thai restaurant and you love the food. You then rave to people about how great the restaurant is. But later, you’ve eaten at several Thai restaurants and you realize the first one you went to was actually a fairly average one.
How would it do a better job of communicating if I lied in order to produce such a result?
Put it this way, sure, for an item that I usually find some sort of level of satisfactory, I’m rarely going to give a lowest-possible rating. So if you look at a pattern of my answers and see that most things were rated 2 to 4 and only a rare one is rated 1 or 5 – you’ve got more information about those few things that I rated 1 or 5. If instead in an attempt to use all the numbers I’d taken some things I’d usually rate 3 or 4 and rated them 5 instead – you’ve got less information, not more; because you can’t tell the real 5’s from the faked ones, which means that you have no idea whether any of my claimed 5’s are actually 5’s – or for that matter whether any of my claimed numbers are accurate. You really don’t have any information at all, if I’m using that technique.
Then I’m really not following what you’re saying. Because you also said:
If that doesn’t mean that if you were rating five restaurants on a five-number scale you’d give each of them a different number, then what does it mean?
In addition:
I might have seen, say, twenty movies which I thought were really really good – but they were really really good in different ways, and may have all had aspects that were pretty bad. Suppose Movie A had fantastic acting except for the second protagonist, fascinating theme, really well written script, so-so scenery, and really bad historical accuracy. Movie B has fantastic acting by the primary characters, interesting theme, really great scenery except that it doesn’t look like where the movie’s supposedly set, wonderful music, innovative and effective use of camera techniques, and the secondary characters are mostly wooden caricatures. Et cetera. A number rating isn’t going to tell you which is a “better” movie. It isn’t even going to with any sort of accuracy going to tell you which I think is a “better” movie.
Because they aren’t the worst. You are just arbitrarily assigning 20 to each level under some bizarre idea that there are an equal amount of good and bad things.
oh boy, do I disagree. Your rating is conflating the 2 restaurants where the food is actually spoiled with 16 restaurants that serve boring sandwiches with too much mayonnaise on stale bread, and two that are actually fine, if unexciting. If I’m stuck picking where to eat out of limited options, I REALLY want to know whether that place is just “I won’t like the food” or “spoiled food”. And if I’m making a special trip for my birthday, I want to know whether it’s a great place or just a very good place.
Most real-life distributions are kinda a bell curve. I want to know whether a place is in the tails or in the center. Heck, some real-life distributions, like “does that e-bay vendor deliver as advertised” are highly skewed. I don’t want you to carefully take the ones that wrote answers to my questions in elegant language and set them above the others that were perfectly fine. I want to know whether the vendor delivers on time as advertised. And if 80% of them do, then 80% of them should get the top rating.
Do you feel that people who use a different system that yours are lying?
And I have less information about the things that aren’t rated 1 or 5. And as you point out, most of the things aren’t rated 1 or 5.
So overall, less information.
You don’t understand. Just because I am placing an approximately equal amount of items in every rating doesn’t mean any random collection of five items will each have a different number.
Let’s say I’ve seen a thousand movies in my lifetime. That means around there are around two hundred movies I would rate a five.
But that doesn’t mean the next five movies I see will be rated one, two, three, four, and five. If they’re all great movie, they would all be added to the top five-star group. And it they were all really bad, they might all be placed in the bottom one-star group. But in the long run, this sizes of the groups will remain approximately equal.
It’s like flipping coins. If you flip a hundred coins, about fifty of them will be heads and about fifty of them will be tails. But that doesn’t mean if you flip two coins, one has to be heads and one has to be tails.
No, you’re wrong.
I’m not placing some arbitrary group of items in the lowest level. I am placing the fifth of the items that I feel are the lowest in quality in the group which I then label as being the lowest.