# Statistics Question on Surveys: **URL:** \ **Category:** Factual Questions\ **Created:** [September 10, 2017, 10:22pm UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030 "2017-09-10T22:22:45Z")\ **Posts on this page:** 6\ **Page:** 2
**Author:** ![LSLGuy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lslguy/32/5813_2.png) [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)\ **Post date:** [September 13, 2017, 3:01pm UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/21 "2017-09-13T15:01:50Z")
That’s a more-controlled situation. But it’s not a fully-controlled situation. Did you survey the girls during PE class or during regular class? Was it winter or summer? Rainy today or not? Most of all, what question are you trying to answer? You can only understand the biases and hence the accuracy in detail of your survey methodology versus the specific question you’re trying to answer. You don’t need to know much about the question to identify a crap survey with a crap methodology. But once we get past the crap surveys it gets harder to quantify the true metric of reliability of OK, good, or very good survey designs. ---
**Author:** ![md2000](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@md2000](https://boards.straightdope.com/u/md2000)\ **Post date:** [September 13, 2017, 3:09pm UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/22 "2017-09-13T15:09:34Z")
During the last election, some poll (Los Angeles newspaper?) consistently reported Trump as doing much better with blacks than any other poll, and consequently doing much better overall. Someone dug into this and found - the poll determined what minorities each respondent belonged to; they apparently randomly selected a respondent, young and black, who was extremely favorable to Trump. Since he was one of their only African-American respondents in the small sample in California, he skewed the entire poll to suggest Trump was doing really well there among black people. Then… they used the same people every week for their sample for weekly polls, so this guy consistently skewed the results instead of being a one-time blip. ---
**Author:** ![Thudlow\_Boink](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/thudlow_boink/32/320_2.png) [@Thudlow\_Boink](https://boards.straightdope.com/u/Thudlow_Boink)\ **Post date:** [September 13, 2017, 3:09pm UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/23 "2017-09-13T15:09:54Z")
> [@Mdcastle](#): > > So how does this change the accuracy of things if a person knows the population size and there’s no possible error introduced by the question. > Say there’s 500 girls at high school. A person notes down that 27 out of 50 of them are wearing dresses or skirts as opposed to pants or shorts. You know the population size and there’s no chance of inaccurate answers because they misunderstood the question or lied. Is it possible to quantify how accurate the data is? It’s still important how those 50 were selected. In such a situation it’s at least conceivable you could generate a true random sample, because you could obtain a list of all the girls in the school and randomly select 50 from that list. I believe that in a situation like that, where the sample is more than 5% of the population, the calculations involve a “finite population correction factor”; but the more statistically literate Dopers are probably better able to address that. ---
**Author:** ![Bijou\_Drains](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bijou_drains/32/2814_2.png) [@Bijou\_Drains](https://boards.straightdope.com/u/Bijou_Drains)\ **Post date:** [September 14, 2017, 12:14am UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/24 "2017-09-14T00:14:18Z")
> [@Banksiaman](#): > > A friend whose student job was working on phone surveys said that she often filled her quotas of mothers of young children / students / unemployed within an hour, but would spend the rest of the week trying to populate the waged / full time employed categories because they were never home. And now mobiles have shot that all to hell as well. I assume they were mostly calling people from 9-5 M-F? To get people who are working you need to also call nights/weekends. Most surveys do very little calling 9-5 ---
**Author:** ![Chessic\_Sense](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@Chessic\_Sense](https://boards.straightdope.com/u/Chessic_Sense)\ **Post date:** [September 14, 2017, 12:18am UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/25 "2017-09-14T00:18:15Z")
> [@Chronos](#): > > learn about your neighborhood’s haberdashery habits Pervert. ---
**Author:** ![GMANCANADA](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/gmancanada/32/2837_2.png) [@GMANCANADA](https://boards.straightdope.com/u/GMANCANADA)\ **Post date:** [September 14, 2017, 3:55am UTC](https://boards.straightdope.com/t/statistics-question-on-surveys/796030/26 "2017-09-14T03:55:05Z")
\*\*@mdcastle \*\*- the possible sources of bias I’d talk to the client about in this type of observational study would be: [ul] [li](As noted by Mr. Boink) where / when are the observations take place?[/li][li]locations around the gym class may be distorted since girls leaving gym class late in the day may wear their shorts home[/li]Where are the change rooms relative to the gym class? Do they get changed and walk through the school (where my observers are) to get to the track to excersize? [li]First class in the morning - girls that have gym first class may wear shorts to school to save time changing.[/li][li]Lunch time - girls may wear shorts / jeans if they are leaving the school to go for lunch at the nearby mall. [/li][li]Is there a dance or some other formal activity (awards ceremony?) that may have girls dress up that day?[/li][li]Is there a “casual day” that day? My son normally wears a uniform but they get casual Fridays once a month - so the students wear jeans or shorts that day.[/li][/ul] Also - \*\*@ Banksiaman & Bijou Drain \*\*- we often have that situation, our client will want a valid sample of all provinces in Canada of something like 26 to 28 year old females who completed university and are making +$60,000 / year. We often have the case that we complete places like Ontario and Quebec (lots of people) within a day, but smaller provinces like Newfoundland or PEI can take a couple weeks (simply because there are so few people). We flag that at the start of the project and let the client know we will devote X time and Y dollars and then stop, so the budget doesn’t get blown looking for the last 20 people we’d need to be to be valid. We’d either 1) press on 2) scrap the requirement or 3) relax it (non-uni. grades, less income, wider age group, depending on where we thought we were getting hung up. [Previous page](https://boards.straightdope.com/t/statistics-question-on-surveys/796030.md?page=1)