Any way to make this poll statistically meaningful?

Factual Questions
1

A dumb question here. I edit a small poetry magazine; for issue #8 I had the idea for a poll: name your three favourite books of poetry published from 1995 to the present. The results are in…plenty of responses, but virtually all the contributors completely ignored the rules. Some named books published before 1995, some named books that don’t even exist, & quite a lot named a lot more than 3 books…plus ancillary honourable mentions, alternatives, &c. Arghh! I’ve tried reining in some of the more dubious entries (e.g. one guy who named two books he admitted he’d not finished reading yet!) but I’m now scratching my head as to how I might try enumerating the results to come up with a top ten list. Any suggestions? My initial thoughts were: (1) award each book fractional points (thus if someone names 3 books, each gets 1/3rd a point; if they name 6, each gets 1/6th). (2) establish a cutoff–say, 5 titles–& simply discard the books named after #5. – Do either of these approaches make sense?

To give you an idea of what I’m contending with here are two of the thornier entries:

However much these are interesting to read, I’m a little bewildered as to what I can do to produce any meaningful statistics from them! --N

2

Not sure how to recover this, but for a true statistical test I think that you would need to completely invalidate any entry that did not fit the form and just run the stats on the remaining entries. Of course, the smaller number of entries would reduce the meaning of the resulting statistics. Kind of a catch-22 eh?

3

Well, look, it’s not like this is rocket science here, is it? I mean, you’re not calculating the trajectory for the space shuttle. Was me, I’d tot up the results for the folks who did follow the rules, and then just make passing mention of “all the probably marvelous poets who regretfully will not be mentioned because the people who submitted their names didn’t follow the rules–better luck next time, folks.”

Why do you feel a need to extract statistics for a poetry journal? :confused:

4

You’re not going to get a “statistically meaningful” poll anyway, since your sample in inherently biased. However, what you need to do is determine a set of rules and then abide by them.

My approach (having done lots of surveys) would be to accept the first three named, period. That was the rule you set up, that’s what I’d stick to. Then I’d assign one vote for each. If a response is ambiguous or unclear (“Any book by Ferdinand Magellan”) or doesn’t fit the rules (“published in 1847”) or doesn’t exist, you discard that entry altogether. If someone named more than three, and you discard one of their top three, take their fourth.

Score one point for each of the three. If someone only names two, they’ve only cast two votes. If someone names seventeen, their first three get their three votes.

That would give me the basis of my top ten.

By the way, I would NOT discard an entry because someone says they haven’t finished reading the book. Could be that the first few poems alone were worthy of making it a favourite book. Don’t penalize someone for giving you too much information. The rules (as you’ve cited them to us) didn’t say that the person had to have read every word, or have memorized three of the poems… just said “name your favourite.”
I’d also compile the side list of additional entries, etc.: the discard pile, so to speak. It wouldn’t affect my top ten list, but there could be stuff of interest. For example, it could be interesting or humourous to have a little paragraph about how Emily Dickinson got more votes than anyone else (or whatever you find), but was disqualified under the rules of the survey.

The golden rule, for me, would be: don’t change the survey rules in midstream. The people who DID follow the rules might have named other works if they thought they would count (even if they counted at less than one vote). If you find that the responses require a rules change (like, if only two people submitted post-1995 entries), I’d try to do it in a way that didn’t penalize the people who did follow the rules. Hope that helps.

5

This is what I’d do:

Pick your favourite poems and publish those as the winners

Then, on the back page of the next issue, explain your “methodology” for determining the result by printing out a HUGE formula (with lots of greek letters, occult symbols and the like).

Make sure it looks “poetic”.

And see if any of your readers realise that you are inventing a new form of “statisical poetry”
K
B = sqrt(1/K Bi2/2) / wK
i=1
B = peak-deviation * sqrt(power-of-modulating-signal)/ highest-modulating-frequency

sin(wct + B1sin(wm1t + B2sin 2m2t))
(* A (oscil carrier (* B (oscil modulator (* C (oscil cascade))))))
6

DuckDuckGoose: Well, I’m sure I’m not the only person to think a top-ten poll might be of interest. The point was not to get a general sample of the public (it’s certainly not going to be “statistically meaningful” in that sense) but to see what the rather smaller community the magazine pertains to (viz. readers of contemporary modernist poetries, especially from the UK & Ireland) might think. – Though I’m indeed tempted to write a testy or flippant note like you describe, the point of the exercise is not to annoy subscribers…

CKDH: Hm, I’ll see how much I can stick to the original rules without disqualifying too much…my estimate is that about 1/4 to 1/3 of the entries are wonky in one way or another. --N

7

Any that don’t meet the original rules are thrown out. It’s that simple. It not a statistical polling , if you try to guess at what somebody might have meant. Flordia is a shamful example of this.

8

Well, depends on what you mean by “wonky”. Let’s take the examples you cited:

  • If they gave you more than three entries, you ain’t tossing them all out, you’re taking the first three listed and you’re just tossing out the excess.

  • If they gave you pre-1995 books, then I think you have to toss them out, otherwise you’re not being fair to the 2/3 to 3/4ths of people who did follow the rules. (“I would have voted for Allen Ginsburg if I knew we could go back pre-1995.”)

  • If they gave you a book that doesn’t exist, don’t you HAVE to throw it out? I mean, how stupid would it look if one of your top ten was a non-existent book?? (Seems to me that would make a cute little comment, though, in the accompanying article.)

Other stuff, I think you have to interpret intent as best you can within the rules framework you set up. You quoted one entry who said “any one of Alice Notley’s recent works, maybe Mysteries of Small Houses…but with a fond backwards glance at The Scarlet Cabinet”. OK, I’d take Mysteries as his vote. I’d put Cabinet in a list on the side, for possible use as a tie-breaker. For instance, if Notley was not going to make the top ten because votes were evenly divided between the two books, you might consider those “side column” votes: the letter quoted above could be taken as a vote for whichever of the Notley books was closest, to see if that additional vote would put her over the top. However, please note, you are doing that at the cost of pushing some other book (author) off the list. There is some subjectivity to that.

If you wanted to make a side list of the entries from people who voted for more than three for the same reason, you could do so, but it goes against my grain.

If you follow that kind of process, I don’t think you’ll feel that you’re unfairly throwing out votes, nor will you throw out that many. You’ll be interpreting the votes within the rules of the survey.

It is true, that approach may annoy some part of the 1/4 to 1/3 of those who didn’t follow the rules (all the votes for Emily Dickinson). Your article can have a sorrowful commentary on how difficult it was to tabulate because…

But any other approach, seems to me, runs the risk of annoying the 2/3 to 3/4 who DID follow the rules. And will make your survey and your results look bad. And your article will have the commentary (in different words, of course), “Oh, sorry, we decided to allow pre-1995 books…”

I’ve done lots of surveys, professionally, and you’re generally stuck with what you asked for and what you get. If you have to throw out 25% of the votes, that’s sad, but I don’t see that you have any choice. We usually expect around 10% of survey responses to be screwed up in some non-repairable way.

[Edited by C K Dexter Haven on 04-17-2001 at 08:01 AM]

9

Or you can just go with what you’ve got and do qualitative rather than quantitative research. Just describing all the different ways people violated the basic rule could make for a humorous article on its own.

10

I think what your contributors are telling you by ignoring the rules is that they don’t really care if your poll produces an answer.

Like the professor played by Robin Williams in “Dead Poets Society”, they don’t think poetry is linear. I believe the premise was that he was supposed to teach that if you lined up all the poets on one axis and the poems on another, that you could draw a line intersecting the best poet with his best peom and get the best poem of all time. He just wasn’t about to teach that.

So I’d say your poll is a success exactly because it produced a lot of rule-breaking responses. And you can repeat it every year and hope for more of the same.

11

I find this thread very intriguing. A couple of simple rules, and the responses from the poets are all over the map. Then you come to GQ for advice, and we literal-mindedly try to force all the responses to conform to the rules. Back in the box, you poets! A nice metaphor for Science Vs. Art.

Your audience is poets, not scientists, and I think you need to take that into account when deciding what to do. As a scientist, I’m not qualified to give advice here, but as a scientist, I can’t help myself. :slight_smile: So ask yourself this: What do you think your readers want more, an accurately ordered list, a sharp cut-off at 1995, and a strict interpretation of the rules, or a list of works of poetry they might enjoy?

I was thinking it’s too bad there isn’t an “artists” forum, where you could get advice more in tune with your readers, but maybe MPSIMS would do. Tell them if they’ve ever posted answers in GQ, or are scientists, mathematicians, or engineers, they’re disqulified from answering. And be sure to change the title: the phrase statistically meaningful may chase away the people you most want to hear from. :wink:

[sup]You might also want to check with the mods; Don’t want you to get dinged for a double post.[/sup]

12

ZenBeam: well, my hope had been to have my cake & eat it too. I should make it clear that it’s not an anonymous poll: the individual entries are to be printed, with the authors’ names attached (part of the interest after all is when a poet writes in with his/her favourite books, so you get to see perhaps a little of what has influenced their thinking & writing lately). But yes, I’m trying to come up with a top-ten list at the end of it, since this seems to me also a useful thing to do. – Anyway, I’ve given it a shot & come up with a top-five list (top-ten proved hard to do, since so few volumes got more than two or three votes so there were too many ties, & given the number of discarded dubious nominations I didn’t think the results at that level were likely to make much sense). My main conclusion from all this is that even in the rather specialized area of the magazine I run (contemporary modernist poetry) there’s a lot of poetry out there & surprisingly little overlap between what people are reading!

(My other conclusion from the poll is that an alarming number of people think that Miles Champion is hot stuff. If you haven’t heard of him don’t worry…) --N

13

I bet no one sent in more than one entry.

14

ndorward, I guess you’ve finished your analysis, but this is probably what I’d do if in your situation:

Assign the first three entries given by respondents as the favored books. Give each of the three entries one point each (a la CKDH). If your rule was to name their 3 favorites, starting with their most favored, weigh the categories as #1 = 3 pts, #2 = 2 pts, #3 = 1 pt.

If the book does not qualify under your rules (earlier than 1995), keep it but assign it to the “Not Applicable” category. Treat this category as a book and see where it comes out in your poll.

If someone gave less than 3 answers, assign the empty slots as “Did Not Answer.” This category may or may not appear in your Top 10 (or 5) list, depending on the frequency of occurance.

Tell your readers how your list was generated so they can form an opinion on the poll and also so they can appreciate your hard work. Repeat the poll year after year and see what happens to the “Not Applicable” category.

What’s the magazine, if I may ask?

15

brachyrhynchos: thanks for the note–yes, I think I’ve managed to come up with something at least reasonably satisfactory for the tabulation, I hope. – The magazine is a little magazine called The Gig (after a Herbie Nichols tune I like, especially in the rendition on a Clusone 3 album); its focus is on contemporary experimental/modernist poetry, with something of a slant towards UK & Irish poetry. Probably few names the general reader would recognize–Tom Raworth & JH Prynne are perhaps the best-known of the UK writers, for instance, & there’s US writers like Clark Coolidge & Rae Armantrout in there too. I’ve been reading this kind of material for years, since coming across the anthology A Various Art, but the magazine’s something of an outgrowth of my work in the past 3 years as a collaborator on Keith Tuma’s Oxford UP Anthology of Twentieth-Century British & Irish Poetry (my role was writing the notes).

More than you needed to know, I’m sure. Those still further curious can find a few promotional pages online at http://www.geocities.com/ndorward/ – there’s a table-of-contents for past issues &c. --N

16

Thanks ndorward for the link and info. Contemporary poetry is one of many areas I have no knowledge about. I’m more of the songbird (avian) type. But you’ve given me something to look into. Thanks!