Is my company being cheated? (Sadistics)

In distributing a product a poll is taken to see if the product was delivered. The population is 550,000, the poll sample was 300. There is no “mean” as this is a one-time poll taken about a one-time distribution - however, our “score” was 94.7% delivered and our bonus depends upon our achieving a 95%.

My problem is that the verification company is saying that their sample size viz their population corresponds to a 95% confidence level and a 2.6% margin of error… and I just don’t see that. But perhaps it’s my source… after all, I’m not a statistician.

http://www.raosoft.com/samplesize.html

This tells me that I need to call 1,400+ people if I want a 95%/2.6% CL/ME, as claimed by the verification company.

Of course, we do our own audits, one far more detailed than the verification company: to call the same job, we contacted 21,000 people, not 300. Of course, our survey is for distribution quality, not just a sample from a population, so it must be far more detailed. Plugging in the numbers in the above calculator shows me that our CL is 99% and our ME is .87%.

Anyway, my question is: Is it possible, on a population of 550,000 and a sample size of 300, assuming a standard response distribution, to achieve a confidence level of 95% with a margin of error of just 2.6%? Is there some higher-order statistics that’s being done here… or are we being cheated?

There is a mean; you just don’t know what it is. The mean is the percentage of people that actually got on-time delivery. You don’t know that value–if you did, you wouldn’t be taking the survey–but you can make a reasonable guess.

Let’s guess that it is in fact 95%. Plugging that number and your other figures into the sample estimator, I get a sample size of 270–pretty much as the survey company says. You left the prior estimate of affirmative responses at 50%, which makes a huge difference.

If you have significant bonuses riding on a .3% shortfall with a 2.8% margin of error you need a more accurate survey. IMHO the range of the margin of error falls well within the shortfall you have earned the bonus unless they want to tighten up that margin with a larger survey.

My next Q is with a breakpoint so close to a bonus line, is there anyone who handles this data with an incentive to see your department/co-workers not recieve their bonus like ops managers who may be charged against their labor dollars for the awarded bonuses, shifting their labor numbers below a breakpoint for their bonuses.

Actually, they’re not calling us on this one… but they have with 1 other (in a 6 year contract totalling ~250 chances at the bonus, meaning we have acheived 99.6% of all possible bonuses). Perhaps “cheating” was a bit strong, but I wasn’t thinking about the monetary aspect as I was the mathematical… were they cheating on their sample size figures to provide the cheapest audit possible? I kept on getting ratios closing on 90% CL and a 10%MoE and figured that I was wrong (most likely), they were doing higher-order statistics than I’m aware of (almost as likely), or that they were fudging the numbers (least likely, but then, we’re not the ones paying for it. )

This is obviously my gap in knowledge… thanks!

(You’ve just reproved Occams razor: The most likely explanation (that I don’t know what I’m talking about) is, in fact, the correct one! )

The “prior estimate” is the part that I didn’t understand… the only thing I noticed was that the further it deviated from 50, the smaller the sample. Since my argument was going to be “your survey company should call more people”, a declining figure just obviously wasn’t going to do!