So here at the Dope, we use the “Let it all hang out” post ranking scheme, where civilized geniuses post the Internet’s most worthy comments, so every single post is naturally worth reading.
In the more barbaric corners of the web, however, they have to resort to different comment ranking schemes like:
Facebook "like"s that magically increase the chance, using some unknown algorithm, that more people will see the post
Reddit’s up and downvotes that are then subject to more magical algorithms that try to mix good recent posts with all-time-top posts
Firefox’s “frecency” metric that tries to find relevant autocomplete entries for your history
Slashdot’s moderation system, which allows up and down votes but only to limits of -1/+5
Google PageRank, the inner workings of which constitute a multi-million dollar detective market
Etc.
Have there been any studies, mathematical proofs, user surveys, whatever that compare these systems against one another to see which ones work the best in given scenarios? Given that this is going to be an increasingly problematic area as humans continue to accumulate knowledge, I just thought it’d be interesting to look at how to best sort out all the information. When three billion people are all whispering, how do we choose who to listen to?
This is a subfield of recommender systems, so that general literature has some stuff on selecting posts to show. But it’s a very young field, and I get the impression that it’s still pre-Copernican.
Really, I am not sure there can be any straightforward, easy answers, because everyone has different purposes. Humankind has been working on this problem ever since we invented language, and other ways to communicate ideas. Each solution (of which the internet, and and the systems you mention are amongst the latest) may work, to an extent, for some purposes, but at the same time they always seem to just make the overall problem bigger.
I don’t have an answer, just an observation about Slashdot’s system. Once in a while I have deep knowledge on the specific topic and it’s interesting to watch posts that are simply wrong (but sound intelligent and logical) get voted up.
Sometimes those incorrect posts are countered with someone with accurate information and the counter post is modded up also so you see both at the same time.
Sometimes, when there are much fewer people that have deep knowledge in the area, you see one or two lone posts countering the misinformation and everyone else agreeing with the incorrect post and slapping themselves on the back in support of each other.
But I do think in general, good/accurate posts do rise up in Slashdot’s system.
It depends on what sort of communication you want. If your content consists entirely of one person asking a question and a bunch of people answering it, then it makes sense to have the “best” answer (assuming you can tell which one that is, but we’ll leave that issue aside for the moment) at the top, with all the others below it and possibly even hidden. For something like that, Google’s algorithm, or Firefox’s, works pretty well. On the other hand, if you want to have some sort of conversation between the different responders, like we have here, then you have to leave the posts in their original order, and not hide any but the absolute worst, lest you disrupt the flow of the conversation.
Weather Underground has an interesting compromise for this. Users can upvote/downvote if they choose to.
You have the option of hiding all posts that are below a certain threshold. I don’t think there is any technical impact from a post being upvoted, other than keeping it above these thresholds.
The Dope’s format precludes truly multi-threaded discussions in one thread (which is why we have the concept of hijacking here), but I don’t think that’s an unsolvable problem.
Other sites experiment with systems nonetheless – for example, showing the first ancestor of a highly voted comment, or maybe just its immediate parent, but hiding all the siblings and children. Slashdot “reparents” certain comments proximal to highly-rated ones to retain the context, for example, whereas Reddit (with an addon that many users use) will allow easy expansion of collapsed threads surrounding highly-rated comments.
Of course, that’s assuming that all posts have a clear parentage. But a post can also be “All this talk of X reminds me of Y”, deriving vaguely from any or all of the other posts in the thread, or it can be “I take a middle view between A and B”, where A and B could both legitimately be argued to be parents, and so on. Or a post could have a number of subsections each of which has a single clear parentage, but a different one for each subsection. And even if a post does have a clear logical parent, the software won’t necessarily know that, unless the poster indicates it in some way.
Most modern forums, and even Facebook, let you reply to specific comments. And most people actually use that functionality (unless they’re trying to jump the hierarchy for a recentness bonus), and so tracking lineage isn’t that hard for the software.
I guess what I’m saying is that yeah, lineage is an issue, but I don’t think it’s one of the harder ones. Reddit, Slashdot, the discourse forum that was unsuccessfully pitched here a while back, etc. all deal with this to some level of success. It’s necessary when 300+ responses are the norm, not the exception as it is here.
IOW, the structure of a discussion and the structure of a forum are connected by a two-way street, and I’m sure each shapes the other.
Edit: And I do find the SDMB’s “flat” format easier to read, but it is unusable for discussions more than 3-4 pages long.
The SDMB has an option for threaded mode displays, but because the default view is linear, the vast majority of people don’t post in a way that would make threading work.