The details you ask about can be extracted from the stored records, but the 10 years of personal observations are just my memories.
I know my observations are just anecdote, and I want it that way as a contrast to the archived records. I’m trying to get at when does something like my archived records become data instead of records of anecdotes?
So far, for the archived records we have as details that the records are of a self selected group of hundreds of thousands of men and women(ratio of men to women undefined) tested with a standardized test over a period of decades using uniform methods of testing.
I’m sure there are some physical tests that women can do better than men. Perhaps agility based things?
Usually for something like that, you’ll have two distributions: one for men, and one for women. The distributions will overlap, with the mean of one being higher than the mean of the other. The standard deviations can also be different. Perhaps women’s scores are all bunched up around the mean, but men’s scores spread out. Then you might have a situation where on average men score higher than women, but the worst scorers are also men.
It is also possible for the one group to be biased, but the other not. Perhaps only the most athletic women join the armed forces, or stay in the armed forces, but men in the armed forces have athleticism similar to the wider population of men.
The overlap means that some men score higher than some women, even if most women score higher than most men (or the opposite). The curves might be mostly identical, except at the extreme high end. So for any two randomly selected people it’s hard to predict if the man or woman is better, but the elite women are much better than the elite men, so the “best in the world” will always be women.
How representational the sample is of the full population is only important if the question you’re asking needs it to be. If you’re asking a question specifically about people in the services, then this seems like the perfect sample. If you want to know about people dissimilar to service members, then it might not be a good example.
The point being, these are all questions that can be asked of the data, and given the right samples can be addressed.
Memories are terrible data. Stored records are generally good data, at least they tend to be sources of data that can be understood, where we can understand the quality of the data and make conclusions that take into account how good the data is.
An aside here, are you telling us that you administered a test for 10 years in the military and found roughly 30 individual women who outperformed some subset of men on the test? I would think, if you administered thousands of these tests, that finding only 30 women capable of beating men would be proof that men are genetically more physically capable than women.
Now, if the statement on prowess is a coelacanth level claim “There are no women in the world who can beat any men at this test”, then your anecdotes are good data to prove this wrong.
Ah, OK, if you have stored records, and those are what you’re basing your conclusions on, then yes, that’s probably good data (especially if, as I suspect, this is something that every servicemember must go through on a set regular basis, rather than some folks just volunteering for testing). Even with good data, though, you need to be a little careful in paying attention to where it’s relevant: You have data on the population of servicemembers, which may or may not be relevant to the population at large. As @echoreply mentions, it’s possible that only the most physically-fit women join the armed forces, but that all men do, or other similar biases.
By the way, my apologies if I sounded condescending when I asked if you had records. Very often, when people say they have lots of experience with something, they don’t have records. (just don’t ask HOW often, because I haven’t kept records on that )
I like thinking of examples, so here’s one where the data is useless to backup the anecdote.
Service members are required to pass a vision test where their corrected or uncorrected vision must exceed a certain level. If they fail, even with new glasses, it’s recorded as a fail. For all others it’s recorded as a pass.
The failures are due to eye disease, or just bad luck of the genetics, and are equal for men and women. So, the pass/fail ratio shows no sex bias when corrected for the proportion of men and women taking the test.
However, as the administer of the test, @DorkVader, might have anecdotal belief that women tend to score better on the eye test than men, but the raw scores were never recorded, just pass/fail.
If women scoring better is an interesting enough hypothesis, then maybe the powers that be will start recording the raw scores, and after a few thousand tests, the question can be answered.
Data, questions, analysis, anecdotes, and such all exist together all mixed up.
If established as accurate (through production of a physical specimen, for example), then it has moved past the phase of anecdote and become evidence. To use the example of the coelacanth, reported sightings (if there were any prior to 1938) would have been anecdotes, but actually catching the fish brought evidence into play (on the flip side, Loch Ness Monster sightings have been and remain unreliable anecdotes, until such time as someone nets an actual Nessie or has it make a guest appearance on TV).
It’s like when “alternative” medicine is shown by good, reproducible science to work (rare as that is), at which point it’s no longer “alternative” but just plain medicine.
No it’s good, I don’t mind. I’m glad you’re participating and I didn’t take it as condescending. I’m trying to clarify a fuzzy notion of the difference between anecdote and data at least a little, to refine my understanding and thinking so I’m not bothered by questions about my questions if they help me.
And the physical prowess of men vs women topic was one that I chose for the example because it seemed easiest to build a believable hypothetical from.
ok, so thinking about this a bit, if I say that all test records as archived contain the final score, plus the raw number of repetitions of the exercise or the time elapsed plus the age and sex of the people taking the test. I can look at that information and say definitively “women perform better at these exercises in these circumstances than men” if that is what is shown. That would be data, but if I then go further and say that this conclusion applies to 18% of women in the population generally, would that make the information anecdotal since we don’t know if the women in the military truly represent a cross section of 18% the general populace? (I’m just making up 18% for the ratio of women in the military, I don’t know if that’s even close to accurate) and to further complicate it, maybe 18% doesn’t represent the ratio of women in the general population as well
The discussion here seems to be just a collection of ideas about the semantics of the word “data”. In its simplest definition, it is nothing other than a collection of recorded observations or facts.
If you have one fact, that’s a datum. To have data, you need at least two facts.
I’m not aware that the concept of “data” contains any implication of being a set of facts sufficient to form any meaningful conclusion. To get that, you need a lot of data – however much it takes. If you only have a few facts, insufficient to form a conclusion, does that mean you have no data at all?
This is like asking, How many grains of sand does it take to make a pile?
I don’t know about anyone else but the definition of the word “anecdote” is a key word for me.
To me that suggests a mere report, naturally second hand. As such, the issue it is reporting on doesn’t rise to the level of a point of data.
e.g. there are anecdotal reports of the Loch Ness monster but all of them taken collectively (no matter how many) have given us no actual data on the Loch Ness monster.
Now we do have data on the number of anecdotes about the monster and you may choose to investigate further on the basis of that but to me that is a different thing entirely.
A general theme that appears to be emerging from the “anecdotes bad, data good” viewpoint is the unreliability of informal testimony about human experience. Urban legends and hand-me-down myths and legends and some person’s misremembered occurrences and all that. And hence why we embrace the formality of scientific studies and a controlled experiment and a structured process of inquiry.
I think most of us who have weighted in on the side of “anecdotal or single-occurrence experience is data” are not attempting to contradict the relevance of the scientific method or the flimsy reliability of hearsay and the mainstream of commonly held beliefs.
I am an actuary, and my job is all about making sense of data. I regularly work with people whose job title is “data scientist”. So I am keenly aware that all data is dirty. People who don’t work with data regularly may not be aware of that, but there are features about the way data gets collected that make it always have issues. That carefully compiled data in scientific studies? It has errors and inconsistencies. It’s just better than most other data sets. And some data is worse than other data. Anecdotal data is especially poor-quality data.
Another often-glossed-over factor is research design, and how failure to think beyond one’s preconceived notions about what is out there to be found can shape the data-gathering process so that relevant data goes missing and biases are confirmed.
The design of questions on surveys can distort the outcome. The most egregious examples are “push polling” (e.g. “On a scale of one to ten, how offended are you about unbathed derelicts lurking outside banks harassing you for money when they should be forced to work for a living like proper people do?”) but those are deliberate, whereas the more insidious and dangerous forms are not intentional, but often just as distorted.
There is no objectivity. Doesn’t mean there’s nothing to be gained by trying to look at matters from every angle and learn more, but we’re always seeing things only from the available viewing angles, and, at that, only from the ones it has occurred to us to attempt a view from.
)