It doesn’t have to be an objective standard. It has to be a transparent standard.
That is, it doesn’t matter how you assign the rating, so long as we know what the criteria you used are, because that lets us translate your ratings into our ratings. Using the example from earlier of the person who doesn’t like pasta, this means we know their ratings of Italian Restaurants will tend to always be lower than other restaurants, even if all else is equal. So 3-star Italian Restaurant on that scale might be 4 or 5 on another scale.
But your system doesn’t have any transparency. You’ve ranked them all 1 to 100, and then imposed a 5-star system on top of it based on nothing more than your desire to have even-sized groups. Your 5-star system adds no new information, and may actually obscure some information.
I feel there is a need for some level of comparison. Otherwise how do you know if the thing you are rating is a good or bad example of what it is?
I mentioned Thai cuisine in an earlier post. If you go to a restaurant and eat Thai food for the first time in your life, how can you judge whether this particular restaurant is serving good or bad Thai food? The meal was, by definition, both the best Thai food and the worst Thai food you ever ate.
Any system in which you give items different numbers is going to involve an element of ranking. The items I rate as a three are ranked lower than the items I rate as a four and so on. The only way to avoid ranking while rating is to give everything the same rating, in which case I feel the ratings approach meaninglessness.
The value a person assigns to a rating can vary. One person might assign a rating of one to any restaurant that they feel you shouldn’t eat at. Another person might assign a rating of one only to restaurants that will harm you via food poisoning. These are both legitimate assignments. I just feel one is more useful than the other.
Yes, I was going to say this. As I’ve noted several times, there are no objective numbers that belong to items being rated. The numbers we assign to them are based on the values we choose. So the values we choose to assign can result in a bell curve, an equal distribution, or every items having the same number. These are all legitimate choices even if, as I have said, I feel some choices are preferable to others.
I agree with what you’re saying here. I’m just drawing a different conclusion from it.
A one-to-five rating has limits. The topic of the thread is how do you work within those limits to convey the most information. In other words, what principles do you use when ranking items?
Comparing the 20-20-20-20-20 and 2-7-48-37-6 distributions shows why I feel the equal distribution principle conveys more information. With the equal distribution system, a person knows that a rating of three means the restaurant is in the middle twenty percent of restaurants. In the other system, they know that a rating of three means the restaurant is in the middle forty-eight percent of restaurants. The information from the first system is more than twice as precise as the information from the second system.
But in most situations where you care, the values aren’t evenly distributed, and tend to cluster. More than half of my ebay transactions were perfect. Only a very few restaurants will make you sick, but that’s hugely valuable information to have.
I want the ratings to carry the most useful information, which doesn’t map with splitting the entities into even groups.
In that example, you have placed 11 items which should have been rated a 3, into the group rated 1. Just because you have arbitrarily decided each group needs an equal amount. That is not useful information. That is actually misinformation. Instead of being fine, you now say these 11 are terrible.
Yes, if we allowed for more possible ratings, there would be the possibility of assigning one number to “restaurants you shouldn’t eat at because they will give you food poisoning” and a different number to “restaurants you shouldn’t eat at because the food tastes really bad”. But that’s like saying Charades would be an easier game if people were allowed to talk while giving clues.
The point is how do you convey the most information within the limits that have been set. And I feel that using one number for “restaurants you shouldn’t eat at because they’re in the bottom fifth” is a better use of a limited set of five numbers than using two of those numbers to cover two different categories of restaurants you shouldn’t eat at.
I understand other people have different systems. Somebody might, for example, assign the following meanings to numbers.
1: “Don’t eat here. The food here will actually harm you.”
2: “Don’t eat here. The food won’t harm you but it tastes really bad.”
3: “Meh.”
4: “This is a good restaurant. I recommend going here if you’re looking for a meal.”
5: “This place is amazing. You should go eat here even if you weren’t planning to go to a restaurant.”
That’s a valid system. And I understand that rating restaurants by a system like this would not produce an equal distribution of numbers because the restaurants that fit those assignments do not exist in equal numbers. I do get what other people are saying.
You, on the other hand, appear to be making no effort to understand anyone else’s point of view. You just keep repeating variations of “There is only one way it’s supposed to be done. All other ways are wrong.”
I think that’s kind of what people are saying - not that your system is wrong, but that it doesn’t give them the information they want and therefore they don’t find it useful. I know that I, for one , couldn’t care less if the restaurant is in the top 20% or the bottom 20%. But I’m that way in a lot of things - I remember a childhood friend of mine. Her mother gave her a lot of crap over her grades - but it wasn’t over the somewhat objective actual grades. It was about how she did in comparison to one other person - if Nora got a 95 and Jeanne got a 96, then Nora’s 95 was unacceptable. If Nora got that same 95 and Jeanne got a 94 , then Nora’s 95 was just fine. In a letter grade system (without plus and minus) they both would have had A’s. Didn’t and still doesn’t make any sense to me for two reasons The first is that for most purposes, the absolute grade should matter- if I’m happy with a 95, then I should be happy with the 95 no matter what grades the other kids had. Even if 95 is the lowest grade in the class. The second is that number grades like that sometimes imply a precision that doesn’t exist - a 95 on a math test with objectively correct answers might be precise, but a 95 as a course grade for an English course where the graded assignments are essays and where the course grade take into account attendance and class participation? It’s hard to say a 95 is distinctly different from a 94 in that situation. And I don’t think you can necessarily distinguish between the 15th best restaurant and the 30th - at some point, when you get past health hazard and bad tasting food , most people are going to distinguish between the “meh” or “good but not great” restaurants for idiosyncratic reasons - the Italian restaurant and the Japanese restaurant are similar quality but Italian is my favorite cuisine and Japanese my least favorite so that’s why the Italian is number 15 and the Japanese restaurant is number 30. Or there are six Italian restaurants and I end up distinguishing them based on minor factors - I like the salad dressing at this one the best so it’s number 15 and number 30 is number 30 because they serve Coke and I prefer Pepsi. But that means that number 30 is in the second quintile simply because I prefer Pepsi to Coke - how useful is that to anyone else if it’s not part of a detailed review that makes that explicit?
These are valid points. As I’ve said this is an issue of conveying information using a limited set of tools.
So if you see that somebody only gave a five star rating to two restaurants do you know that those are the two best restaurants in town? Or do you know that the rater really likes redheads and those are the only two restaurants in town that have redheaded waitresses? The answer is you don’t. You have no way of knowing what the criteria was the rater was assigning to numbers. All you can look at is the numbers that were given and try to figure out what they were intended to mean. And then decide if the information you think they are conveying is useful to you in making your own decision.
I figure my equal distribution plan works better than most. It’s relatively simple and direct at conveying the information on where I feel a particular restaurant fits in the overall spectrum of possibilities.
Other systems are generally based on the principle of assigning different criteria to five numbers and then giving restaurants the number that best matches the characteristics of that particular restaurant. But this presumes that the person seeing only the number will be able to decode its meaning and determine the underlying criteria and that the underlying criteria will be related to things that the reader also cares about when selecting a restaurant. And that even if we share the same criteria for selecting a restaurant, our opinions on how well this restaurant met those criteria match.
And there is no way of conveying perfect information. Even if I set aside the one-to-five rating system and give a long written review where I describe the restaurant, its menu, its staff, its presentation, and relate in excruciating detail the taste, smell, appearance, and texture of everything I consumed, I’m still only describing what I experienced on that particular occasion. You may be able to judge how much you would appreciate the experience I had but you won’t know how similar it will be to the experience you will have if you go to that restaurant.
I don’t - which is why I actually pay no attention to ratings. If there’s a detailed review in which I can see that the rater is crazy and gave a hotel a poor rating simply because it didn’t have the premium cable channel the reviewer wanted* , I pay attention to the review and not the rating. If it’s just a rating, I really don’t pay attention at all except maybe to stay away from places with the lowest rating - which might in fact be perfectly fine.
* And that sort of thing does happen - I’ve seen people who downgrade cruise ships because they didn’t show hockey/baseball/football games.
I play board games. So I need to choose which board games I would enjoy playing.
Tom Vassel is a prominent reviewer of board games. But I have mostly stopped watching his reviews because I have found they are of limited use to me. Vassel does describe what the game he is reviewing is like and does state whether or not he likes the game. But my experience has led me to the conclusion that he interjects too much of his own personal preferences into his reviews. And I don’t share his personal preferences. So I might really love a game that he hates and I might really hate a game that he loves. And I can’t tell from his reviews how much of what he’s saying about a game derives from the game itself and how much of it derives from his preferences.
I don’t know what you’re doing. I’m not in your head.
And I don’t think anything is “the one true system”. What I’m discussing is what I think is, and isn’t, a more or less useful system for actual practical situations.
It reads to me as if you’re the one who is saying that there’s “one true system” that conveys the most information for all situations.
This sort of “there must be One True Way” argument is often used for religions, yes. It’s also often used for politics, gender behavior, financial systems, and all sorts of other things – I’ve run into it used for styles of music.
And it does damage, no matter what field it’s used in.
I think I might see what’s going on. You’re talking about a spherical cow.
The rating system that makes the most sense for talking about a spherical restaurant isn’t the one that makes the most sense for deciding whether, and under what circumstances, to eat at an actual one. And I believe a number of people in this thread have presented the arguments.
What if which song you love best depends on what mood you’re in? What if you love Restaurant A’s cacchiatore, Restaurant B’s wine list, and Restaurant C’s service, and all of those things matter to you?
Yup.
And saying that the salmonella restaurant should get the same rating as the stale-bread restaurant just because there aren’t enough salmonella restaurants to fill a category does not appear to me to provide a useful comparison.
You’ve missed the point of my post which included the person who doesn’t like pasta. I didn’t mean that I’d rank all pasta restaurants lower; I meant that there are very often contradictory criteria at work, and because of those contradictions I can’t do that sort of rating. I might say ‘Pasta Restaurant X has a really good reputation, excellent chefs, excellent service and ambiance, very good drinks selection; I’m sure it’s a really great restaurant. I’m not going to go there very often because I don’t like pasta, but if you love pasta and are visiting from out of town maybe we ought to go there tonight.’
It’s only more precise in a purely mathematical sense: spherical cow. The system which lets me distinguish between ‘fantastic, pretty good, average, poor, actively dangerous’ is more precise in a different fashion; and, for me at least, much more useful when deciding whether to eat at a particular place on a particular day. And that system is going to have a whole lot more restaurants in the second and third categories than in the first and fifth, and probably also more than in the fourth.
Seconding this – or possibly repeating it; it may be a better wording of something I was trying to get across earlier.
I wonder if part of this is that you’re assuming that on every given occasion on which you want to eat at a restaurant, you’ve got a choice of hundreds or at least multiple dozens to choose from, so of course you’re never going to choose from anywhere even in the bottom half.
Whereas I’m presuming that at least most of the time it’s a situation like one of these below:
– We want to go out to dinner but we don’t want to drive more than 20 minutes, and oh shit, it’s Monday. There are only two places open that fit that criteria and you’ve put them both in the same 20% but a rating not using equal numbers would have rated one of them average and the other is one of their two Terribles. And because they only put 2 in the Terrible category you know that probably means something.
– Or, I just got off the road at x exit in a strange place and want somewhere to eat that’s right near my exit – and none of these places got good ratings, but which one isn’t actually going to poison me, and might have at least one thing on the menu that’s adequate?
– Or, we need to go out to dinner tonight but we’re low on money and can’t afford any of these places higher in the list – which of the cheap places is only poor and not dangerous?
Seconding this, also.
Or maybe I’m wrong.
Yup. It’s not One True Way. It’s not “This is The Only System and anybody using anything else must be lying!” It’s “this is a more valid system for this particular sort of use.”
It’s not entirely clear to me why you find your system more valid for this particular sort of use; but you do you.
So you agree with what Eyebrows said. You agree that there are items “which should have been rated a 3” and that anyone who gives rates a different number is spreading “misinformation”.
How does that work exactly? How do you know what number an item should be rated? When you look at the item, do you see a glowing number hovering above it? Or do you hear a voice telling you what number the item is supposed to have?
No, you’re the only one bringing spherical cows into it. Which makes me think you don’t understand that reference.
I’m saying that ratings given to specific restaurants by somebody who ate meals at those restaurants can be used by other people who are thinking about eating meals at those restaurants. I don’t know where you think the abstraction is here.
Maybe you could try reading my posts where I explained why I find my system more valid.
It’s essentially a sorting exercise, and the question is what criteria are you using to sort your items.
I’d probably do something similar to what @Sam_Stone describes- give everything as objective of a score as I could, then sort them highest score to lowest. That’s ranking items.
Now if you’re talking about how you score or rate something, that’s a different question. Generally speaking, I tend to have a sort of informal rubric where I judge how something performs its essential function. So ranking things like dishwashing soap might be judged by how much soap it takes to clean dishes, and how well it gets grime off when soaking dishes overnight.
It’s entirely possible that everything could score relatively high or low. For example, I’d rate most dishwashing liquids in the 6-8 range, with only some really awful ones getting below that, and only one or two getting higher than 8.
And vice versa. “Perfection” only means that it doesn’t have any flaws; it doesn’t necessarily mean that it has any positive qualities. There are plenty of imperfect things I prefer over perfect ones.
There are so many problems with this whole explanation that it’s hard to begin.
First, your argument also ignores math on a basic, fundamental level. You’re claiming that yours is superior for 96 restaurants because it conveys more information, yet 52 restaurants are in the same categories in both systems, not just the four. Every single three star restaurant in your grouping is included in the middle group of the “fake” bell curve system. 14 each of your two- and four-star restaurants are rated exactly the same in the other system.
Next, this is a gross mischaracterization of a bell curse. No one in marketing, statistical analysis or any other of a wide variety of people who actually use bell curves would ever divide a group of 100 items into groups of 2, 32, 32, 32, 2.
What you are actually doing is modifying your system of equal numbers of items and simply chopping off the extreme endpoints, then equally expanding the middle three groups. Three of the groups are all equal sized while only two of the groups are miniscule. That’s absurd.
Ratings using bell curves don’t do this, and it’s disingenuous to claim otherwise. Bell curve systems have the greatest number in the middle, then smaller and smaller numbers in the outer groups. A much more realistic grouping would be something like
1 star - ten restaurants
2 stars – twenty restaurants
3 stars - forty restaurants
4 stars - twenty restaurants
5 stars - ten restaurants
Of course you don’t like having 32 restaurants with four stars. No one does and that’s why no one creates said systems. It’s ridiculous.
Note that my system has exactly the same number of two star and four star restaurants as yours, but only half the number of one star and five star restaurants. It also has twice the number of three star restaurants, which is more natural because most things follow some sort of bell curve distribution.
You aren’t making a logical argument against a bell curve argument. You have only created a strawman and argued against that.
I feel there is an element of cognitive dissonance being shown in some posts in this thread.
I openly recognize and admit the arbitrary and artificial basis of my ratings system. But I’m seeing other people who appear to be saying variations of “There’s nothing arbitrary or artificial about my ratings system. I give each item that rating it deserves. And then whatever collective pattern arises from those individual ratings is a purely natural phenomena. I didn’t impose any pattern.”
Those people are wrong. An equal grouping ratings system is artificial. A bell curve system ratings system is artificial. A reverse bell curve ratings system is artificial. A constant slope ratings system is artificial. Because all ratings systems are artificial.
This is because items do not have a numerical rating. We, as human beings, are simply assigning a numerical rating to items based on criteria we invent. There is no natural ratings process.
When you recognize that, you can examine the ratings system you’re using and look for the artificial principles you used to invent that system. But if you don’t recognize that you invented the system you use, those artificial principles will still exist. You just won’t be conscious of their existence.