I want to provide an example of a statistically relevant quantitative milestone where a fictional device performs with a 95% accuracy compared to some gold standard. I want to include a ‘correct’ measure of statistical accuracy…It doesn’t matter what statistic I use in this comparison, as long as whatever I use, mathematicians in the room won’t laugh at me for some non-sequitor.
Example:
Proof of concept will be demonstrated when the device shows 95% accuracy to the gold standard (r=0.90; or whatever…student’s t test; or something.)
I need help with the parenthetical statistical measure.
please feel free to rewrite my statement if that is what it takes.
Well, the first thing is to describe what you mean by “95% accuracy”; do you mean it falls within a certain target range 95% of the time, or it functions with 95% reliability, or what?
For correlation to a linear estimate or approximation, you want the R2 to be high (at least R2>0.5) and the p-value to be low (usually specified to be p<0.10 or p<0.05, depending upon the size and homogeneity of your dataset). If you don’t know what these terms mean and how they are defined, you really shouldn’t be presenting them to an audience of mathematicians (or at least statisticians) because as soon as you throw up a p-value you’re almost certain to have people questioning you about how you arrived at that conclusion and what it means. Don’t start talking about hypothesis testing unless you’ve actually done it because you will get quickly wrapped around the axle about details you are not equipped to answer.
this is purely for entertainment purposes. It doesn’t matter what I put in there or what I really mean by 95% accuracy. I just want to make sure that what I have up on the screen will not be incorrect in any sense.
Most people in the room are not likely to be mathematicians or statisticians. but there could be.
Don’t read to much into it…this is why I have it in ‘humble opinion’ rather then ‘factual questions’
I just don’t want to do something akin to a Chinese character tattoo that I claim says ‘loyalty’ when it actually says ‘soda’
There’s lots of tests, which one you would use would depend on a lot of factors, but a statement using ROC (Receiver Operating Characteristic) analysis could look like:
“The device demonstrated a statistically significant improvement in accuracy compared to the gold standard method, with an AUC of 0.95 (95% CI: 0.92 - 0.98) on the ROC curve, as compared to an AUC of 0.88 (95% CI: 0.84 - 0.91) for the gold standard method. The z-test results indicated a p-value of less than 0.05, indicating that the difference in AUC between the two methods was statistically significant.”