# What principle do you use when ranking items? **URL:** \ **Category:** In My Humble Opinion\ **Created:** [December 9, 2022, 6:51pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271 "2022-12-09T18:51:05Z")\ **Posts on this page:** 20\ **Page:** 4
**Author:** ![Horatius](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/horatius/32/3572_2.png) [@Horatius](https://boards.straightdope.com/u/Horatius)\ **Post date:** [December 10, 2022, 1:37pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/61 "2022-12-10T13:37:31Z")
> [@Little\_Nemo](#): > > But as I have said, there’s no objective standard. It doesn’t have to be an _objective_ standard. It has to be a _transparent_ standard. That is, it doesn’t matter how you assign the rating, so long as we know what the criteria you used are, because that lets us translate your ratings into our ratings. Using the example from earlier of the person who doesn’t like pasta, this means we know their ratings of Italian Restaurants will tend to always be lower than other restaurants, even if all else is equal. So 3-star Italian Restaurant on that scale might be 4 or 5 on another scale. But your system doesn’t have any transparency. You’ve ranked them all 1 to 100, and then imposed a 5-star system on top of it based on nothing more than your desire to have even-sized groups. Your 5-star system adds no new information, and may actually obscure some information. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 2:58pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/62 "2022-12-10T14:58:02Z")
> [@kenobi\_65](#): > > Again, a **rating** does _not_ require a comparison, except against a (likely arbitrary) scale, and each item being rated should be given its rating based solely on its own merits. I feel there is a need for some level of comparison. Otherwise how do you know if the thing you are rating is a good or bad example of what it is? I mentioned Thai cuisine in an earlier post. If you go to a restaurant and eat Thai food for the first time in your life, how can you judge whether this particular restaurant is serving good or bad Thai food? The meal was, by definition, both the best Thai food and the worst Thai food you ever ate. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 3:07pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/63 "2022-12-10T15:07:27Z")
> [@I\_Love\_Me\_Vol.I](#): > > > [@Little\_Nemo](#): > > > > Now I think we’d both agree that there’s no reason to eat at any of the worst restaurants. But I give a list of twenty restaurants that are the worst while your list only identifies two as the worst. Eighteen of the restaurants I rated as among the worst group are given two star ratings in your system. > > What you’re doing then is _ranking_ the restaurants, not _rating_ them. Any system in which you give items different numbers is going to involve an element of ranking. The items I rate as a three are ranked lower than the items I rate as a four and so on. The only way to avoid ranking while rating is to give everything the same rating, in which case I feel the ratings approach meaninglessness. The value a person assigns to a rating can vary. One person might assign a rating of one to any restaurant that they feel you shouldn’t eat at. Another person might assign a rating of one only to restaurants that will harm you via food poisoning. These are both legitimate assignments. I just feel one is more useful than the other. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 3:12pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/64 "2022-12-10T15:12:04Z")
> [@I\_Love\_Me\_Vol.I](#): > > Also, on re-reading, it occurs to me that opinions or ratings don’t naturally chart as a normal distribution. Yes, I was going to say this. As I’ve noted several times, there are no objective numbers that belong to items being rated. The numbers we assign to them are based on the values we choose. So the values we choose to assign can result in a bell curve, an equal distribution, or every items having the same number. These are all legitimate choices even if, as I have said, I feel some choices are preferable to others. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 3:24pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/65 "2022-12-10T15:24:08Z")
> [@Horatius](#): > > You can’t do that with restaurants or movies, though. There’s no objective underlying score that determines the rank of every movie. It’s purely personal, and relative. So there’s no underlying score that can be evaluated outside the 5-star rating system, and the system is worthless unless we have some idea as to what criteria determine a 1 or a 5, or any other score. > > If you don’t think that 5 levels is sufficiently granular, then you need to use more stars, not impose an arbitrary distribution. Of course, at the extreme of this, we end up back with just a plain ranked choice list, with the top movie out of one hundred movies getting one hundred stars, and so on. I agree with what you’re saying here. I’m just drawing a different conclusion from it. A one-to-five rating has limits. The topic of the thread is how do you work within those limits to convey the most information. In other words, what principles do you use when ranking items? Comparing the 20-20-20-20-20 and 2-7-48-37-6 distributions shows why I feel the equal distribution principle conveys more information. With the equal distribution system, a person knows that a rating of three means the restaurant is in the middle twenty percent of restaurants. In the other system, they know that a rating of three means the restaurant is in the middle forty-eight percent of restaurants. The information from the first system is more than twice as precise as the information from the second system. ---
**Author:** ![puzzlegal](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/puzzlegal/32/3827_2.png) [@puzzlegal](https://boards.straightdope.com/u/puzzlegal)\ **Post date:** [December 10, 2022, 3:36pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/66 "2022-12-10T15:36:26Z")
But in most situations where you care, the values aren’t evenly distributed, and tend to cluster. More than half of my ebay transactions were perfect. Only a very few restaurants will make you sick, but that’s hugely valuable information to have. I want the ratings to carry the most useful information, which doesn’t map with splitting the entities into even groups. ---
**Author:** ![Eyebrows\_0f\_Doom](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/eyebrows_0f_doom/32/3119_2.png) [@Eyebrows\_0f\_Doom](https://boards.straightdope.com/u/Eyebrows_0f_Doom)\ **Post date:** [December 10, 2022, 3:38pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/67 "2022-12-10T15:38:12Z")
> [@Little\_Nemo](#): > > Comparing the 20-20-20-20-20 and 2-7-48-37-6 distributions shows why I feel the equal distribution principle conveys more information. In that example, you have placed 11 items which should have been rated a 3, into the group rated 1. Just because you have arbitrarily decided each group needs an equal amount. That is not useful information. That is actually _misinformation_. Instead of being fine, you now say these 11 are terrible. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 3:57pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/68 "2022-12-10T15:57:07Z")
> [@puzzlegal](#): > > But in most situations where you care, the values aren’t evenly distributed, and tend to cluster. More than half of my ebay transactions were perfect. Only a very few restaurants will make you sick, but that’s hugely valuable information to have. > > I want the ratings to carry the most useful information, which doesn’t map with splitting the entities into even groups. Yes, if we allowed for more possible ratings, there would be the possibility of assigning one number to “restaurants you shouldn’t eat at because they will give you food poisoning” and a different number to “restaurants you shouldn’t eat at because the food tastes really bad”. But that’s like saying Charades would be an easier game if people were allowed to talk while giving clues. The point is how do you convey the most information within the limits that have been set. And I feel that using one number for “restaurants you shouldn’t eat at because they’re in the bottom fifth” is a better use of a limited set of five numbers than using two of those numbers to cover two different categories of restaurants you shouldn’t eat at. I understand other people have different systems. Somebody might, for example, assign the following meanings to numbers. 1: “Don’t eat here. The food here will actually harm you.” 2: “Don’t eat here. The food won’t harm you but it tastes really bad.” 3: “Meh.” 4: “This is a good restaurant. I recommend going here if you’re looking for a meal.” 5: “This place is amazing. You should go eat here even if you weren’t planning to go to a restaurant.” That’s a valid system. And I understand that rating restaurants by a system like this would not produce an equal distribution of numbers because the restaurants that fit those assignments do not exist in equal numbers. I do get what other people are saying. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 4:00pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/69 "2022-12-10T16:00:50Z")
> [@Eyebrows\_0f\_Doom](#): > > In that example, you have placed 11 items which should have been rated a 3, into the group rated 1. Just because you have arbitrarily decided each group needs an equal amount. That is not useful information. That is actually _misinformation_. Instead of being fine, you now say these 11 are terrible. You, on the other hand, appear to be making no effort to understand anyone else’s point of view. You just keep repeating variations of “There is only one way it’s supposed to be done. All other ways are wrong.” ---
**Author:** ![doreen](https://avatars.discourse-cdn.com/v4/letter/d/858c86/32.png) [@doreen](https://boards.straightdope.com/u/doreen)\ **Post date:** [December 10, 2022, 4:26pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/70 "2022-12-10T16:26:34Z")
I think that’s kind of what people are saying - not that your system is wrong, but that it doesn’t give them the information they want and therefore they don’t find it useful. I know that I, for one , couldn’t care less if the restaurant is in the top 20% or the bottom 20%. But I’m that way in a lot of things - I remember a childhood friend of mine. Her mother gave her a lot of crap over her grades - but it wasn’t over the somewhat objective actual grades. It was about how she did in comparison to one other person - if Nora got a 95 and Jeanne got a 96, then Nora’s 95 was unacceptable. If Nora got that same 95 and Jeanne got a 94 , then Nora’s 95 was just fine. In a letter grade system (without plus and minus) they both would have had A’s. Didn’t and still doesn’t make any sense to me for two reasons The first is that for most purposes, the absolute grade should matter- if I’m happy with a 95, then I should be happy with the 95 no matter what grades the other kids had. Even if 95 is the lowest grade in the class. The second is that number grades like that sometimes imply a precision that doesn’t exist - a 95 on a math test with objectively correct answers might be precise, but a 95 as a course grade for an English course where the graded assignments are essays and where the course grade take into account attendance and class participation? It’s hard to say a 95 is distinctly different from a 94 in that situation. And I don’t think you can necessarily distinguish between the 15th best restaurant and the 30th - at some point, when you get past health hazard and bad tasting food , most people are going to distinguish between the “meh” or “good but not great” restaurants for idiosyncratic reasons - the Italian restaurant and the Japanese restaurant are similar quality but Italian is my favorite cuisine and Japanese my least favorite so that’s why the Italian is number 15 and the Japanese restaurant is number 30. Or there are six Italian restaurants and I end up distinguishing them based on minor factors - I like the salad dressing at this one the best so it’s number 15 and number 30 is number 30 because they serve Coke and I prefer Pepsi. But that means that number 30 is in the second quintile simply because I prefer Pepsi to Coke - how useful is that to anyone else if it’s not part of a detailed review that makes that explicit? ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 4:48pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/71 "2022-12-10T16:48:11Z")
> [@doreen](#): > > I think that’s kind of what people are saying - not that your system is wrong, but that it doesn’t give them the information they want and therefore they don’t find it useful. I know that I, for one , couldn’t care less if the restaurant is in the top 20% or the bottom 20%. > > … > > And I don’t think you can necessarily distinguish between the 15th best restaurant and the 30th - at some point, when you get past health hazard and bad tasting food , most people are going to distinguish between the “meh” or “good but not great” restaurants for idiosyncratic reasons - the Italian restaurant and the Japanese restaurant are similar quality but Italian is my favorite cuisine and Japanese my least favorite so that’s why the Italian is number 15 and the Japanese restaurant is number 30. Or there are six Italian restaurants and I end up distinguishing them based on minor factors - I like the salad dressing at this one the best so it’s number 15 and number 30 is number 30 because they serve Coke and I prefer Pepsi. But that means that number 30 is in the second quintile simply because I prefer Pepsi to Coke - how useful is that to anyone else if it’s not part of a detailed review that makes that explicit? These are valid points. As I’ve said this is an issue of conveying information using a limited set of tools. So if you see that somebody only gave a five star rating to two restaurants do you know that those are the two best restaurants in town? Or do you know that the rater really likes redheads and those are the only two restaurants in town that have redheaded waitresses? The answer is you don’t. You have no way of knowing what the criteria was the rater was assigning to numbers. All you can look at is the numbers that were given and try to figure out what they were intended to mean. And then decide if the information you think they are conveying is useful to you in making your own decision. I figure my equal distribution plan works better than most. It’s relatively simple and direct at conveying the information on where I feel a particular restaurant fits in the overall spectrum of possibilities. Other systems are generally based on the principle of assigning different criteria to five numbers and then giving restaurants the number that best matches the characteristics of that particular restaurant. But this presumes that the person seeing only the number will be able to decode its meaning and determine the underlying criteria and that the underlying criteria will be related to things that the reader also cares about when selecting a restaurant. And that even if we share the same criteria for selecting a restaurant, our opinions on how well this restaurant met those criteria match. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 4:55pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/72 "2022-12-10T16:55:56Z")
And there is no way of conveying perfect information. Even if I set aside the one-to-five rating system and give a long written review where I describe the restaurant, its menu, its staff, its presentation, and relate in excruciating detail the taste, smell, appearance, and texture of everything I consumed, I’m still only describing what I experienced on that particular occasion. You may be able to judge how much you would appreciate the experience I had but you won’t know how similar it will be to the experience you will have if you go to that restaurant. ---
**Author:** ![doreen](https://avatars.discourse-cdn.com/v4/letter/d/858c86/32.png) [@doreen](https://boards.straightdope.com/u/doreen)\ **Post date:** [December 10, 2022, 4:56pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/73 "2022-12-10T16:56:50Z")
> [@Little\_Nemo](#): > > So if you see that somebody only gave a five star rating to two restaurants do you know that those are the two best restaurants in town? Or do you know that the rater really likes redheads and those are the only two restaurants in town that have redheaded waitresses? The answer is you don’t. You have no way of knowing what the criteria was the rater was assigning to numbers. I don’t - which is why I actually pay no attention to ratings. If there’s a detailed review in which I can see that the rater is crazy and gave a hotel a poor rating simply because it didn’t have the premium cable channel the reviewer wanted\* , I pay attention to the review and not the rating. If it’s just a rating, I really don’t pay attention at all except maybe to stay away from places with the lowest rating - which might in fact be perfectly fine. \* And that sort of thing does happen - I’ve seen people who downgrade cruise ships because they didn’t show hockey/baseball/football games. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 10, 2022, 5:04pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/74 "2022-12-10T17:04:20Z")
> [@doreen](#): > > I don’t - which is why I actually pay not attention to ratings. If there’s a detailed review in which I can see that the rater is crazy and gave a hotel a poor rating simply because it didn’t have the premium cable channel , I pay attention to the review and not the rating. If it’s just a rating, I really don’t pay attention at all except maybe to stay away from places with the lowest rating - which might in fact be perfectly fine. This is what for me is the Tom Vassel effect. I play board games. So I need to choose which board games I would enjoy playing. Tom Vassel is a prominent reviewer of board games. But I have mostly stopped watching his reviews because I have found they are of limited use to me. Vassel does describe what the game he is reviewing is like and does state whether or not he likes the game. But my experience has led me to the conclusion that he interjects too much of his own personal preferences into his reviews. And I don’t share his personal preferences. So I might really love a game that he hates and I might really hate a game that he loves. And I can’t tell from his reviews how much of what he’s saying about a game derives from the game itself and how much of it derives from his preferences. ---
**Author:** ![thorny\_locust](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/thorny_locust/32/431_2.png) [@thorny\_locust](https://boards.straightdope.com/u/thorny_locust)\ **Post date:** [December 10, 2022, 5:41pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/75 "2022-12-10T17:41:13Z")
> [@Little\_Nemo](#): > > Yes, you think you’re telling the truth when you use your system. And you would be lying if you didn’t use your system. > > But the question is what do you think the rest of us are doing? Some of us aren’t using your system. Do you think we know your system is the one true system and we are choosing to lie by using a different one? > > Or do you think we believe our system is the true one and we are doing what we think is true when we use it? I don’t know what you’re doing. I’m not in your head. And I don’t think anything is “the one true system”. What I’m discussing is what I think is, and isn’t, a more or less useful system for actual practical situations. It reads to me as if you’re the one who is saying that there’s “one true system” that conveys the most information for all situations. > [@Little\_Nemo](#): > > I realize this argument usually isn’t applied to rating systems. It’s an argument usually used for religions. This sort of “there must be One True Way” argument is often used for religions, yes. It’s also often used for politics, gender behavior, financial systems, and all sorts of other things – I’ve run into it used for styles of music. And it does damage, no matter what field it’s used in. > [@Little\_Nemo](#): > > The general rule is the smaller the group is, the more precise the information about the things in that group is. And the way to put the most things in the smallest possible groups, is to make the groups as close to equal in size as possible. I’ve spoken about the arbitrariness of ratings systems but what I’ve said in this paragraph isn’t arbitrary. It’s a mathematical fact. > > That said, maybe there are reasons outside of mathematics why a different ratings system is superior. But people making that claim need to present the argument. I think I might see what’s going on. You’re talking about a spherical cow. The rating system that makes the most sense for talking about a spherical restaurant isn’t the one that makes the most sense for deciding whether, and under what circumstances, to eat at an actual one. And I believe a number of people in this thread have presented the arguments. > [@Little\_Nemo](#): > > Do you love “Yesterday” the same amount you love “I Call Your Name”? What if which song you love best depends on what mood you’re in? What if you love Restaurant A’s cacchiatore, Restaurant B’s wine list, and Restaurant C’s service, and all of those things matter to you? > [@Little\_Nemo](#): > > I feel that rating everything the same isn’t rating. It’s just saying something exists. Rating requires comparison. Yup. And saying that the salmonella restaurant should get the same rating as the stale-bread restaurant just because there aren’t enough salmonella restaurants to fill a category does not appear to me to provide a useful comparison. > [@Horatius](#): > > it doesn’t matter how you assign the rating, so long as we know what the criteria you used are, because that lets us translate your ratings into our ratings. Using the example from earlier of the person who doesn’t like pasta, this means we know their ratings of Italian Restaurants will tend to always be lower than other restaurants, even if all else is equal You’ve missed the point of my post which included the person who doesn’t like pasta. I didn’t mean that I’d rank all pasta restaurants lower; I meant that there are very often contradictory criteria at work, and because of those contradictions I can’t do that sort of rating. I might say ‘Pasta Restaurant X has a really good reputation, excellent chefs, excellent service and ambiance, very good drinks selection; I’m sure it’s a really great restaurant. I’m not going to go there very often because I don’t like pasta, but if you love pasta and are visiting from out of town maybe we ought to go there tonight.’ > [@Little\_Nemo](#): > > Comparing the 20-20-20-20-20 and 2-7-48-37-6 distributions shows why I feel the equal distribution principle conveys more information. With the equal distribution system, a person knows that a rating of three means the restaurant is in the middle twenty percent of restaurants. In the other system, they know that a rating of three means the restaurant is in the middle forty-eight percent of restaurants. The information from the first system is more than twice as precise as the information from the second system. It’s only more precise in a purely mathematical sense: spherical cow. The system which lets me distinguish between ‘fantastic, pretty good, average, poor, actively dangerous’ is more precise in a different fashion; and, for me at least, much more useful when deciding whether to eat at a particular place on a particular day. And that system is going to have a whole lot more restaurants in the second and third categories than in the first and fifth, and probably also more than in the fourth. > [@Eyebrows\_0f\_Doom](#): > > > [@Little\_Nemo](#): > > > > Comparing the 20-20-20-20-20 and 2-7-48-37-6 distributions shows why I feel the equal distribution principle conveys more information. > > In that example, you have placed 11 items which should have been rated a 3, into the group rated 1. Just because you have arbitrarily decided each group needs an equal amount. That is not useful information. That is actually _misinformation_. Instead of being fine, you now say these 11 are terrible. Seconding this – or possibly repeating it; it may be a better wording of something I was trying to get across earlier. > [@Little\_Nemo](#): > > I figure my equal distribution plan works better than most. It’s relatively simple and direct at conveying the information on where I feel a particular restaurant fits in the overall spectrum of possibilities. I wonder if part of this is that you’re assuming that on every given occasion on which you want to eat at a restaurant, you’ve got a choice of hundreds or at least multiple dozens to choose from, so of course you’re never going to choose from anywhere even in the bottom half. Whereas I’m presuming that at least most of the time it’s a situation like one of these below: – We want to go out to dinner but we don’t want to drive more than 20 minutes, and oh shit, it’s Monday. There are only two places open that fit that criteria and you’ve put them both in the same 20% but a rating not using equal numbers would have rated one of them average and the other is one of their two Terribles. And because they only put 2 in the Terrible category you know that probably means something. – Or, I just got off the road at x exit in a strange place and want somewhere to eat that’s right near my exit – and none of these places got good ratings, but which one isn’t actually going to poison me, and might have at least one thing on the menu that’s adequate? – Or, we need to go out to dinner tonight but we’re low on money and can’t afford any of these places higher in the list – which of the cheap places is only poor and not dangerous? > [@kenobi\_65](#): > > At this point, I don’t think either of us are going to convince the other of much of anything. Your preferred approach makes you happy, so enjoy it. Seconding this, also. Or maybe I’m wrong. > [@Little\_Nemo](#): > > I understand other people have different systems. Somebody might, for example, assign the following meanings to numbers. > > 1: “Don’t eat here. The food here will actually harm you.” > 2: “Don’t eat here. The food won’t harm you but it tastes really bad.” > 3: “Meh.” > 4: “This is a good restaurant. I recommend going here if you’re looking for a meal.” > 5: “This place is amazing. You should go eat here even if you weren’t planning to go to a restaurant.” > > That’s a valid system. And I understand that rating restaurants by a system like this would not produce an equal distribution of numbers because the restaurants that fit those assignments do not exist in equal numbers. I do get what other people are saying. Yup. It’s not One True Way. It’s not “This is The Only System and anybody using anything else must be lying!” It’s “this is a more valid system for this particular sort of use.” It’s not entirely clear to me why you find your system more valid for this particular sort of use; but you do you. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 12, 2022, 4:17am UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/76 "2022-12-12T04:17:06Z")
> [@thorny\_locust](#): > > I don’t think anything is “the one true system”. You say that but… > [@thorny\_locust](#): > > > [@Eyebrows\_0f\_Doom](#): > > > > In that example, you have placed 11 items which should have been rated a 3, into the group rated 1. Just because you have arbitrarily decided each group needs an equal amount. That is not useful information. That is actually _misinformation_. Instead of being fine, you now say these 11 are terrible. > > Seconding this – or possibly repeating it; it may be a better wording of something I was trying to get across earlier. So you agree with what Eyebrows said. You agree that there are items “which should have been rated a 3” and that anyone who gives rates a different number is spreading “misinformation”. How does that work exactly? How do you know what number an item should be rated? When you look at the item, do you see a glowing number hovering above it? Or do you hear a voice telling you what number the item is supposed to have? > [@thorny\_locust](#): > > I think I might see what’s going on. You’re talking about a spherical cow. > > The rating system that makes the most sense for talking about a spherical restaurant isn’t the one that makes the most sense for deciding whether, and under what circumstances, to eat at an actual one. And I believe a number of people in this thread have presented the arguments. No, you’re the only one bringing spherical cows into it. Which makes me think you don’t understand that reference. I’m saying that ratings given to specific restaurants by somebody who ate meals at those restaurants can be used by other people who are thinking about eating meals at those restaurants. I don’t know where you think the abstraction is here. > [@thorny\_locust](#): > > It’s not entirely clear to me why you find your system more valid for this particular sort of use Maybe you could try reading my posts where I explained why I find my system more valid. ---
**Author:** ![bump](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@bump](https://boards.straightdope.com/u/bump)\ **Post date:** [December 12, 2022, 4:41am UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/77 "2022-12-12T04:41:50Z")
Yeah that’s what I was thinking too. It’s essentially a sorting exercise, and the question is what criteria are you using to sort your items. I’d probably do something similar to what @Sam_Stone describes- give everything as objective of a score as I could, then sort them highest score to lowest. That’s ranking items. Now if you’re talking about how you _score_ or rate something, that’s a different question. Generally speaking, I tend to have a sort of informal rubric where I judge how something performs its essential function. So ranking things like dishwashing soap might be judged by how much soap it takes to clean dishes, and how well it gets grime off when soaking dishes overnight. It’s entirely possible that everything could score relatively high or low. For example, I’d rate _most_ dishwashing liquids in the 6-8 range, with only some really awful ones getting below that, and only one or two getting higher than 8. ---
**Author:** ![Alessan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/alessan/32/465_2.png) [@Alessan](https://boards.straightdope.com/u/Alessan)\ **Post date:** [December 12, 2022, 10:17am UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/78 "2022-12-12T10:17:50Z")
> [@Little\_Nemo](#): > > I disagree. I feel that there’s a difference between saying “this movie approaches perfection” and “this movie is in the top fifth of all movies”. I can think of a lot of movies in the latter group that aren’t in the former. And vice versa. “Perfection” only means that it doesn’t have any flaws; it doesn’t necessarily mean that it has any positive qualities. There are plenty of imperfect things I prefer over perfect ones. ---
**Author:** ![TokyoBayer](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tokyobayer/32/13989_2.png) [@TokyoBayer](https://boards.straightdope.com/u/TokyoBayer)\ **Post date:** [December 12, 2022, 1:29pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/79 "2022-12-12T13:29:32Z")
> [@Little\_Nemo](#): > > Your list might look like this > > 1 star - two restaurants > > 2 stars - thirty-two restaurants > > 3 stars - thirty-two restaurants > > 4 stars - thirty-two restaurants > > 5 stars - two restaurants > > Now I think we’d both agree that there’s no reason to eat at any of the worst restaurants. But I give a list of twenty restaurants that are the worst while your list only identifies two as the worst. Eighteen of the restaurants I rated as among the worst group are given two star ratings in your system. > > The same is true at the other end. Your system gives the same rating to the third best restaurant in town as it does to the thirtieth best restaurant. But I feel there’s going to be a noticeable difference between the third best and the thirtieth best restaurants. > > Granted, your ratings system is better at picking out the extremes. Yours is better at picking out the two best restaurants and the two worst restaurants in town. **But I feel my ratings system conveys better information about the other ninety-six restaurants we reviewed.** my bolding There are so many problems with this whole explanation that it’s hard to begin. First, your argument also ignores math on a basic, fundamental level. You’re claiming that yours is superior for 96 restaurants because it conveys more information, yet 52 restaurants are in the same categories in both systems, not just the four. Every single three star restaurant in your grouping is included in the middle group of the “fake” bell curve system. 14 each of your two- and four-star restaurants are rated exactly the same in the other system. Next, this is a gross mischaracterization of a bell curse. No one in marketing, statistical analysis or any other of a wide variety of people who actually use bell curves would ever divide a group of 100 items into groups of 2, 32, 32, 32, 2. What you are actually doing is modifying _your_ system of equal numbers of items and simply chopping off the extreme endpoints, then equally expanding the middle three groups. Three of the groups are all equal sized while only two of the groups are miniscule. That’s absurd. Ratings using bell curves don’t do this, and it’s disingenuous to claim otherwise. Bell curve systems have the greatest number in the middle, then smaller and smaller numbers in the outer groups. A much more realistic grouping would be something like 1 star - ten restaurants 2 stars – twenty restaurants 3 stars - forty restaurants 4 stars - twenty restaurants 5 stars - ten restaurants Of course you don’t like having 32 restaurants with four stars. No one does and that’s why no one creates said systems. It’s ridiculous. Note that my system has exactly the same number of two star and four star restaurants as yours, but only half the number of one star and five star restaurants. It also has twice the number of three star restaurants, which is more natural because most things follow some sort of bell curve distribution. You aren’t making a logical argument against a bell curve argument. You have only created a strawman and argued against that. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [December 12, 2022, 2:20pm UTC](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271/80 "2022-12-12T14:20:50Z")
I feel there is an element of cognitive dissonance being shown in some posts in this thread. I openly recognize and admit the arbitrary and artificial basis of my ratings system. But I’m seeing other people who appear to be saying variations of _“There’s nothing arbitrary or artificial about my ratings system. I give each item that rating it deserves. And then whatever collective pattern arises from those individual ratings is a purely natural phenomena. I didn’t impose any pattern.”_ Those people are wrong. An equal grouping ratings system is artificial. A bell curve system ratings system is artificial. A reverse bell curve ratings system is artificial. A constant slope ratings system is artificial. Because all ratings systems are artificial. This is because items do not have a numerical rating. We, as human beings, are simply assigning a numerical rating to items based on criteria we invent. There is no natural ratings process. When you recognize that, you can examine the ratings system you’re using and look for the artificial principles you used to invent that system. But if you don’t recognize that you invented the system you use, those artificial principles will still exist. You just won’t be conscious of their existence. [Previous page](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271.md?page=3) [Next page](https://boards.straightdope.com/t/what-principle-do-you-use-when-ranking-items/976271.md?page=5)