Statisticians or electoral process specialists - question on polling

[Arnold Winkelried]

I am planning on opening a poll to vote on the proposed smilies (see thread in ATMB for more details.) The vBulletin polling process allows 10 choices. We have 50 submissions. I plan on having 5 polls with ten smilies each, and the top 10 from those 5 polls will form the final poll.
My options are: pick the top two from each of the five polls, or pick the 10 most-voted-for smilies.
Example:
Poll 1: Smiley 1A gets 40 votes, Smiley 1D gets 30 votes, Smiley 1G gets 20 votes, etc...
Poll 2: Smiley 2J gets 15 votes, Smiley 2H gets 10 votes, Smiley 2E gets 5 votes, etc...
Poll 3: Smiley 3F gets 30 votes, Smiley 3C gets 29 votes, Smiley 3B gets 25 votes, etc...


If I pick the top two from each poll, I would have 1A, 1D, 2J, 2H, 3F, 3C as finalists.
If I pick the top vote-getters (and assuming I limit the number of finalists to 6), the finalists would be 1A, 1D, 3F, 3C, 3B, 1G.
Which is the "correct" method? In poll 2, 2J was the winner, but less people voted in poll 2. If I have 5 polls, people might start voting in polls 1 and 2, but then get bored and skips polls 3, 4, 5 (for example.) Does this mean that the winner of poll 5 should be skipped, and instead I should include the smiley in third or fourth position in poll 1, if that smiley has more votes than the top vote-getter in poll 5?

[erislover]

How many smilies will make it through your process.. just one?

[erislover]

Also, will you allow a vote for "no new smileys whatsoever"?

[hawthorne]

There is no "correct" method. That being said, the situation would seem to suit the Borda count very well (although this would require more than the vB poll feature). See here. If that's too complicated, you could try a "lucky loser" scheme: have a slightly larger ballot (could you go down to 4*12 or stretch to 4*13?), then take the top two from each poll plus the two highest vote-getting non-place getters.

[hawthorne]

Erm, I see that stretching to more than 10 per ballot isn't possible. Oops. But some weight to # of votes could be given by letting the winner in each bracket through and then choosing the next 5 highest vote getters regardless of place through.

[Big Kahuna Burger]

AP Statistics "Scholar" checking in. Finally, a chance to use all the technical jargon I know.

Obviously, due to the constraints of the vB system there are many confounding factors. An option would be to continually eliminate the lowest polling smilies. For example, after the first five rounds you'd only eliminate the ten lowest polling overall or two lowest in each group, then proceed with four groups, then three, etc. However, this would probably take too long and has some of the same problems. One thing you should do is allow people to sign up before you start polling and make them vote in every poll. That would definitely eliminate one of the confounding factors. Also, if you get some major borderline cases feel free to open another poll with 2nd and 3rd place smilies and pick the top 5 to join the 1st place finishers.

-BKB, who will get his AP Stat results July 1

[Arnold Winkelried]

erislover - yes, for the first poll we will add only one smiley, the one that gets the most votes. I also plan on having the possibility to vote for "no new smiley", which raises another question - would that be part of the first round of polling, with the possibility of elimination of that question from the final vote, or would it be in the last round along with the winners?

hawthorne, Big Kahuna Burger - I'll think about your posts and comment later.

[Arnold Winkelried]

hawthorne - I don't think I could implement a Borda count with a vBulletin poll - for one thing doesn't that mean that people would have to assign various "weights" to each vote?

Big Kahuna Burger - I don't think forcing the people to vote in every poll is feasible.

After reading the Scientific American articles at the link provided by hawthorne, I think I'm going to try the method described as "approval voting", hoping I understand it correctly. From hawthorne's link:

Sam Merrill in the department of mathematics and computer science at Wilkes University in Wilkes-Barre, Penn., provides more details:

"The method of voting in multicandidate elections referred to in the question is, I believe, approval voting. Under this procedure, each voter can vote for one, two, three or any number of candidates. In effect, the voter considers each candidate one by one and casts a vote for those whom she (or he) approves but not for those whom she does not approve. The winner is the one approved by the most voters.

"Approval voting is now used by several national and international professional societies in their election of officers. These include the Institute of Electrical and Electronics Engineers, the American Statistical Association, the Mathematical Association of America and the Institute for Management Science.

So here's my plan. You can create a vBulletin poll that allows multiple selections. The number of entries will be (# of smilies submitted) + 1 (for 'no new smilies!' option). I will divide that number more or less equally amongst different polls since there's a limit of 10 choices per poll. E.g. 51 choices (50 smilies + 'no smiley') could be 6 polls: 9, 9, 9, 8, 8, 8. Each poll would be multiple choice, e.g. you could vote for all 9 in poll A, seven in poll B, etc... At the end of the polling period, the option with the most votes would win. If there's a tie then we would have a second vote to break the tie.

How does that sound?

Another question - how long should the poll be open? I was thinking either 7 or 14 days, with a preference for 7.

[White Lightning]

While approval voting sounds good, I actually liked the idea of the two-round poll. Seems like that safeguards against random hiccoughs of polling. The wildcard scheme seemed effective, taking the top picks from each group and some number of top picks from the remaining pool. That could still be implemented under the approval voting scenario.

[knock knock]

The problem with the approval rating poll is that the winner might just be satisfactory to a lot of people, not anyone's real preference. "no new smilie" doesn't work well if there are multiple polls - you at least have to make it clear in each poll that people can choose to vote for none of them and go to the other poll to choose "no new smilie."
The problem with deciding directly from the 6 separate polls is that people might not look at all 6. This might not spread randomly, especially if a poll that gets voted in moves to the top. (Do these polls work like that?)
I recommend the 2-round poll. 5 polls of 10 smilies each, have people either pick 2 or do the approval voting - pick up to 3. Then have 1 final poll with the winner of each of the 5 polls and the next 4 highest vote-getters and "no new smilie".

[hawthorne]

The nice thing about the approval method is that you really aren't running 5 or 6 different polls - you're just running one poll with multiple pages. If this is made clear to people then no second round is required.

Again, the answer to knock knock's concern that the winner "might just be satisfactory to a lot of people" is to point out this to people in the voting thread and suggest that they consider this when deciding how many votes to cast.

[Measure for Measure]

[hijack]Since this is thread on voting procedures, I'm going to present a slight hijack.

The first round should be devoted to emoticon concepts. (eg. Yuk, dubious, whatever). The second round can pick the actual smilies.

I see no reason why additional smilies within that same emoticon concept could not be offered in the 2nd round.

The 2nd round could also contain the choice, "None of the above", to reflect the preferences of the Anti Smilie Coalition. [/hijack]

Dang. I was going to recommend approval voting. Glad to see we're doing this right.

(Oh, and it amuses me to draw this process out as long as possible. Then again, I don't have to work out all the bureaucratic issues. Thanks and kuddos to our Adminstrator.)

[Arnold Winkelried]

After reading the arguments in favour of the two rounds of polling, and considering that people might not vote in all polls in the first round (especially if they get frustrated when the server hits a rough patch and times out), I think I'll do the following:

1. Take # of smilies proposed, divide more or less equally in x groups of y smilies each, where y <= 10 (vBulletin limits you to 10 choices per poll) and x <=5 (limit of 50 proposals)
2. Set up x polls (approval voting method). Polling period will be 7 days.
3. At end of 7 days, create a second-round poll with the top winner in each group (x smilies) plus the other top vote-getters treating all the polls as one (9 - x smilies) and the "no new smilies" option for a 10-choice poll that will also expire in 7 days. This round would be "single-choice" voting (or should it also be approval voting?)
4. If the winner of the second round is a combination of 2 or more smilies (e.g. dubious has two options), one final poll to choose the best smiley for that category, with perhaps a shorter polling period (5 days or so, covering at least several weekdays and one week-end.)
5. If there are ties in round one or round two, we could have some "intermediate rounds", or else, to avoid drawing out the voting process too much, it could be a coin flip. I favour the coin flip method.

flowbark - I think that by combining smilies representing the same concept prior to the voting process, we accomplish your suggestion of voting by concept in the first round.

[Measure for Measure]

Originally posted by Arnold Winkelried [*]At end of 7 days, create a second-round poll with the top winner in each group (x smilies) plus the other top vote-getters treating all the polls as one (9 - x smilies) and the "no new smilies" option for a 10-choice poll that will also expire in 7 days. This round would be "single-choice" voting (or should it also be approval voting?)

Whenever you have greater than 2 choices, you have a "split the ballot" problem. Approval voting is always superior, IMHO. (This conclusion has not been (can not be?) demonstrated rigorously and unequivocably, to my knowledge however).

Furthermore: Hey, we're fighting ignorance. I'd like to see how an approval voting process operates in practice. Note that those who have a strong antipathy for certain designs can "close them out", by voting for other popular designs and "none of the above".
---
Yes, the design of the 1st round is going well: I consider this close enough to "voting by concept". (Actually it's better, since the viewers can see examples of the underlying concepts.)
---
I would permit additional submissions towards the end, if anybody thinks they can improve upon the execution of the underlying concept. (Impossible w.r.t "Smackie", IMHO. Quite possible with "dubious".)

[Fenris]

Rather than have a vB poll which doesn't really accomplish the goal and seems to make thing...um...tedious (7 rounds of voting, minimum?!), might I suggest a plain old "Post only in this thread to vote and Bold your vote?

I believe I'm agreeing with what Flowbark is suggesting.

What I'd like to see is a thread as decribed above. Each person has three votes (one for first choice, one for second and one for third) First choice is worth say...15 points, second 10 points, third 5 points.

The advantage to this is that it satisfy more people than a normal "pick your fav" poll would. In other words, if Smiley #89 recieves 26 1st place votes, Smiley #88 recieves 25 1st place votes and and #87 recieves 24 1st place votes, #89 will win (of course).

Under the method I propose, say #89 recieves 6 1st, 5 2nd and 3 3d place votes, #88 recieves 4 1st, 2 2nd and 3 3d place votes, but #87 recieves 4 1st place, 3 2nd place and 12 3d place votes,
that would give #89 118 points, #88 would get 83 and #87 would get 150: it wasn't everyone's first choice but it was the one that the most people liked to some degree.

I grant you the bookkeeping is a pain with this method, I'd strongly recommend it as a way to satisfy more people.

I'd also suggest if you use this method that the thread be used ONLY for voting: no lobbying etc, and lobbying posts be deleted (it'll make bookkeeping easier.) Also insist that a standard form be used, something easly cut-and-pasted, something like:

Vote:
1st Choice: 89
2nd Choice: 70
3d choice: 55


Then all that has to be done is to input the thread results into an Excel spreadsheet with some formulas and volia! (I'd also say 7 days is more than enough!)

Fenris

[Measure for Measure]

I once read a paper which compared voting systems: it used data from a British Union election in which the voters recorded their preferences in some detail. This allowed the researchers to compare various different voting schemes.

The results: First past the post gave very different results. The remainder gave very similar results. This is encouraging, since it suggests 2 rules:

1) Avoid winner-take-all / first past the post voting.

2) Choosing the simplest of the remaining systems is permissible.

This leads me to support Approval voting. It's not winner-take-all. And it's reasonably simple.

Although admittedly I may be putting too much weight on a single empirical study.

[hawthorne]

I think Fenris is suggesting some form of Borda count, which Arnold has decided is a bit much of a fuss. What is being suggested will be fine I think, because the big problem is weighing up what a fair performance in a good bracket means v a good performance in a fair bracket. Approval voting should sort that out. I'm still of the view that additional rounds are unnecessary and that later rounds will be subject more to rabble-rousing than preference revelation, but that's fine too.

[Arnold Winkelried]

Originally posted by Fenris
Rather than have a vB poll which doesn't really accomplish the goal and seems to make thing...um...tedious (7 rounds of voting, minimum?!), might I suggest a plain old "Post only in this thread to vote and Bold your vote?

...

I grant you the bookkeeping is a pain with this method, I'd strongly recommend it as a way to satisfy more people.

There would actually be three rounds at most; the first round would however be split amongst several threads because of the upper limit of choices in a vBulletin poll.

Asking people to post their vote would be a lot of extra work - people would change their minds and include a second post saying "please please please change my first vote", it's easy to make typos, and you have to read multiple pages in a thread. Halfway through, how would you know who's in the lead?

The main reason I opposite is the fact that, as you say, "the bookkeeping is a pain." Whereas with the vBulletin poll, we have an already automated feature that will do all this manual work for us. Once someone has voted, there is no way to "undo" their vote AFAIK, so if someone e-mails me saying "please please please change my vote" I can answer "sorry, no can do" with a clear conscience.

flowbark - I see no strong reasons to prevent the use of approval voting in the second round.

[OpalCat]

There is the option of using off-site polling. The one that I use often is http://www.freepolls.com/

[Arnold Winkelried]

Thank you for the suggestion OpacCat, but I don't see any convincing reasons to use another site when we can do it using the vBulletin polling feature.

[Arnold Winkelried]

The first round will be split up into 5 vBulletin polls of 10 smilies each. At the submissions page I just split up the table of submissions by columns - first column will be Poll 1, second Poll 2, etc...