Would you please tell us what the study was trying to show? What was the hypothesis for this experiment?
I thought so at first too. But the p value shown is way too high for that.
If I’ve found the correct paper (and I think I have), the surrounding sentences say
“Mean age was 14.28 ± 1.78 years for the study group.
Among the 87 members of the study group, 64 were girls
(73.56%) and 23 were boys (26.44%). Mean age of girls
was 13.56 ± 0.79, and of the boys was 14.73 ± 1.02
(p = 0.032). There were significantly more migraine sufferers
in the study group than in the contacted group
(p<0.001).”
I think the p value associated with the ages is the p-value for the null hypothesis that the boys and girls were selected from an underlying distribution of headache sufferers with a mean age of 14.28.
How about a link to the paper?
But that’s only true if the age was hypothesized to be a dependent variable. If they just scooped up some middle schoolers and polled them on their ages, then it makes no sense to put a p value on it, because it’s independent.
Secondly, it’s not “these exact numbers”, but rather “these exact numbers or more” I’m sure the chances of getting those exact numbers are infinitesimally small.
That’s simply not true. Any time you have two randomly chosen groups, you can do a test to compare whatever characteristics you’re interested in.
Here is the way I would interpret it.
The age of the Girls had mean 13.56 and standard deviation of 0.79
The age of the Boys had mean 14.73 and standard deviation of 1.02
If there were no real systematic difference between the ages of the Boys and the Girls selected for the study, the probability that we would get such a large difference by chance is 3.2%
The numbers work out right if there were about 13 boys and 13 girls in the study.
As far as how to interpret it, assuming there is no issue of multiple comparisons (see XKCD link above), I like the following cutoffs
p>0.1 Ignore result as probably just random chance
0.01<p<0.1 Result worthy of interest and investigation, but not conclusive.
0.001<p<0.01 Results most likely real (assuming test assumptions hold)
p<0.001 Results definitely real (assuming test assumptions hold)
So in this case, I would say, that there is some indication that there was a selection bias in the study for choosing older boys than girls, but it could just be chance.
You still might want to do a test to make sure that your scooping of middle schoolers was unbiased. This result indicates that there is some evidence that it may not have been.
Sorry. I should have done that - here’s a link http://www.springerlink.com/content/m05442w894377682/
Looking at the paper in context, I’m pretty sure my initial reading of the results was right. They are using a different test than I expected and I made some mistakes in my calculations, so my sample size estimates were off otherwise what I said looks correct.
It is not uncommon for a study to report any anomalies in the sample selection that are found significant p<0.05, even if it doesn’t have a great deal of effect on their conclusions.
[QUOTE=Buck Godot]
It is not uncommon for a study to report any anomalies in the sample selection that are found significant p<0.05, even if it doesn’t have a great deal of effect on their conclusions.
[/QUOTE]
This makes the sentence in the OP make a lot more sense, at least to me.
Any ideas about “A total of 87 subjects completed the study: 64 girls (73.56%) and 23 boys (26.44%) (p = 0.016).” from the abstract in the link? Is that correct for that distribution of boys and girls when randomly selecting from a 50/50 mix? (Or the actual mix of 12 to 17 year-olds?)
I don’t think the subjects were randomly selected. I believe the participants were recruited based on their medical history and enrolled based on their willingness to participate.
mmm
I’m not suggesting they were randomly selected, I’m just wondering if that’s the calculation the author’s made to get p=0.016. I’m also not suggesting that it makes sense to make that calculation, just wondering if they did.
I would also read it like that, but it doesn’t add up. I calculate the probability of getting such a large age difference by chance to be much lower.
This is a bit more unclear. It can’t be assuming a 50/50 mix since that p-value would be much more extreme. I suspect that it might be the p-value for the difference in the percentages of those that enrolled in the study vs those that completed it, but it’s really not clear. They don’t indicate how many boys and girls entered the study so I can’t check.
I agree that a t-test gives far too significant a p-value. But the methods section of the paper says they used a Wilcoxon rank test rather than a t-test to estimate the difference in ages, so we can’t recompute their results with just with the information provided. If there were a lot of ties in the ages (likely if ages were truncated by years) it’s possible that they could get that p-value.