I give my audience 10 choices and ask each person to choose
The most important
The 2nd most important
The 3rd most important
How do I test the data to come up with the overall 1st, 2nd and 3rd choice? The trick is that the same choice may be at different rankings for different people. I don’t think a Borda score would work since rankings are not proportional. I read up on Friedman’s Test but I’ve never used it. Is that the right test for this data?
You are wanting to choose a ranked voting system. But Arrow’s Impossibility Theorem says there is no perfect voting system. More specifically, there is no voting system that satisfies the three seemingly simple conditions:
So there is no “right choice” of a voting system. You’ll just have to choose among the various possibilities, which are all sub-optimal in some way.
Do you have a complete ranking of all 10 choices for each member of the audience or just the top 3?
If its the latter then I’m not sure that there is an optimal choice since it really depends a lot on what you ultimate goal is. Does it matter most that a given call is their top choice, or is it an honor just to be nominated and even coming in third for a person indicates a significant preference?
I would probably just assign a point value to each rank, and calculate the average number of points that a given choice receives across all surveys.
If you want to emphasize being picked as number 1, then going with a weight of 3,2,1 for first second and third choice respectively would work. If you want to emphasize participation, then weights of 6, 5, 4 for 1st, 2nd, 3rd would be good, for somewhere in between maybe weights of 4, 3, 2.
A general approach, that does not depend on scoring for each rank: compare each choice head-to-head. Do more people prefer choice A or choice B? A person prefers a choice if they rank it strictly higher than the other choice. (So, if a person does not rank either choice as important, they have no preference.) This gives you a ranking matrix of every choice vs every other choice. It won’t solve the problem for you, but it will give you a clear picture of the audiences preferences without depending on an arbitrary scoring system.
You’ll have to decide how to use the ranking matrix. For example: if there’s a choice that “wins” every head-to-head match up. it’s your clear winner. If not, eliminate the choice which loses the most match ups (or more than one if tied for worst). Then look for a single choice that wins all remaining match ups, ignoring eliminated choices. Repeat until there’s only one.
There will be many edge cases from transitive loops and ties that you’ll have to consider eventually. Exercise left to each reader’s preferences.
As an aside, what @Pleonast describes here is what’s called a “Condorcet” system.
Actually, just his first step (if there’s someone who wins every head-to-head matchup, they win) is the Condorcet condition; any voting system that satisfies that is a Condorcet system. There are many different Condorcet systems, that vary in the details of what you do when there isn’t that clear overall winner right away.
But of course, if it does happen that your vote tallies do lead to a Condorcet winner, then that simplifies things considerably.
I’m not sure what this sentence means or why some form of modified Borda count wouldn’t be good enough. A points for 1st place votes, B points for 2nd place votes, C points for 3rd place votes with some values for A, B, and C that you feel are fair.