# What is the term for this kind of statistical mistake? **URL:** \ **Category:** Factual Questions\ **Created:** [January 13, 2013, 9:33pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095 "2013-01-13T21:33:00Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Absolute](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Absolute](https://boards.straightdope.com/u/Absolute)\ **Post date:** [January 13, 2013, 9:33pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/1 "2013-01-13T21:33:00Z")
Let’s say there is an industry of companies that make widgets. There are actually two types of widget, fluttering widgets and oscillating widgets, but the difference in functionality is subtle and not appreciated by most consumers, or anyone else. Most companies make both kind of widget anyway, and they are sold in roughly equal numbers. After receiving scattered reports of deaths due to widget use, the government commissions a study, and said study concludes that the use of widgets leads to a 40% greater chance of a heart attack in both the widget user and anyone nearby. Widgets are quickly banned. However, in reality, the risk of a heart attack is a peculiar consequence of the mode of operation of fluttering widgets only. Fluttering widgets in fact increase your risk of a heart attack by 80%, yet oscillating widgets are perfectly safe. However, because no one appreciated the significance of the two different types of widget, the study did not even gather data on which deaths were associated with which type of widget, and the most accurate statistical correlation was never discovered. Is there a name for this kind of statistical error? “Choosing too broad a sampling variable”, or something? ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [January 13, 2013, 9:41pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/2 "2013-01-13T21:41:56Z")
I would say ill-defined variables. As for the fallacy name, I would say this is a subset of hasty generalization. ---
**Author:** ![Absolute](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Absolute](https://boards.straightdope.com/u/Absolute)\ **Post date:** [January 14, 2013, 1:57am UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/3 "2013-01-14T01:57:12Z")
Is it really as simple as that? It is not simply an ill-defined variable, it is a failure to recognize that a variable exists. ---
**Author:** ![Maserschmidt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/maserschmidt/32/18829_2.png) [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)\ **Post date:** [January 14, 2013, 2:21am UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/4 "2013-01-14T02:21:26Z")
Error in model specification? ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [January 14, 2013, 2:45pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/5 "2013-01-14T14:45:14Z")
> [@Absolute](#): > > Is it really as simple as that? It is not simply an ill-defined variable, it is a failure to recognize that a variable exists. I don’t think that it is lack of recognition of the variable’s existence. For example, crooked tails are common in siamese cats so I do a study of 100 cats (8 of which are siamese) and get 8% of my sample has crooked tails. Are you claiming that I do not recognize that there are siamese and non-siamese cats? Instead I made my variable too general (all cats). ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [January 14, 2013, 4:29pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/6 "2013-01-14T16:29:59Z")
It sounds similar to either the [ecological fallacy](http://en.wikipedia.org/wiki/Ecological_fallacy) or the [fallacy of division](http://en.wikipedia.org/wiki/Fallacy_of_division), but it’s not exactly the same as either. ---
**Author:** ![cerberus](https://avatars.discourse-cdn.com/v4/letter/c/71e660/32.png) [@cerberus](https://boards.straightdope.com/u/cerberus)\ **Post date:** [January 14, 2013, 5:36pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/7 "2013-01-14T17:36:21Z")
It does not have a fancy name. It is a matter of not accounting for reasonably plausible sources of variability, in this case type of widget. In practice one stratifies or adjusts for these sources. ---
**Author:** ![FasterThanMeerkats](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@FasterThanMeerkats](https://boards.straightdope.com/u/FasterThanMeerkats)\ **Post date:** [January 14, 2013, 5:40pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/8 "2013-01-14T17:40:29Z")
[Simpson’s Paradox](http://en.wikipedia.org/wiki/Simpson's_paradox), wikipedia link Average of total vastly different from average of subgroups. ---
**Author:** ![Absolute](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Absolute](https://boards.straightdope.com/u/Absolute)\ **Post date:** [January 14, 2013, 6:45pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/9 "2013-01-14T18:45:25Z")
> [@FasterThanMeerkats](#): > > [Simpson’s Paradox](http://en.wikipedia.org/wiki/Simpson's_paradox), wikipedia link > > Average of total vastly different from average of subgroups. Thank you for that. Following links on that page, I found the following, which I think is closest to the situation I described: > **[Omitted-variable bias](https://en.wikipedia.org/wiki/Omitted-variable_bias)** > > In statistics, omitted-variable bias (OVB) occurs when a statistical model leaves out one or more relevant variables. The bias results in the model attributing the effect of the missing variables to those that were included. > More specifically, OVB is the bias that appears in the estimates of parameters in a regression analysis, when the assumed specification is incorrect in that it omits an independent variable that is a determinant of the dependent variable and correlated with one or more of th... ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [January 14, 2013, 6:56pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/10 "2013-01-14T18:56:11Z")
> [@FasterThanMeerkats](#): > > [Simpson’s Paradox](http://en.wikipedia.org/wiki/Simpson's_paradox), wikipedia link > > Average of total vastly different from average of subgroups. It’s definitely not Simpson’s paradox, which is referring specifically to the situation described in the very first sentence of the article you linked to: > [@](#): > > In probability and statistics, Simpson’s paradox (or the Yule–Simpson effect) is a paradox in which a trend that appears in different groups of data disappears when these groups are combined, and the reverse trend appears for the aggregate data. ---
**Author:** ![cerberus](https://avatars.discourse-cdn.com/v4/letter/c/71e660/32.png) [@cerberus](https://boards.straightdope.com/u/cerberus)\ **Post date:** [January 14, 2013, 7:17pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/11 "2013-01-14T19:17:33Z")
Right, Simpsons requires the aggregate effect to flip relative to the subsets. The error here is not considering the effect of widget type in the analysis. The idea is that your setup is valid if you’ve correctly specified the underlying variables in an appropriate and complete manner, including the right variables and not including the wrong variables. Model miss-specification covers many problems. ---
**Author:** ![Absolute](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Absolute](https://boards.straightdope.com/u/Absolute)\ **Post date:** [January 14, 2013, 8:41pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/12 "2013-01-14T20:41:07Z")
Would you agree that “Omitted-variable bias”, linked above, is an accurate description? ---
**Author:** ![cerberus](https://avatars.discourse-cdn.com/v4/letter/c/71e660/32.png) [@cerberus](https://boards.straightdope.com/u/cerberus)\ **Post date:** [January 14, 2013, 9:01pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/13 "2013-01-14T21:01:42Z")
The problem here is the omission of widget type in the analysis. Variable omission is the specific fault here. I prefer a broader term encompassing incomplete or excessive adjustment to multiple terms describing every particular error, though. Too much jargon. ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [January 14, 2013, 9:21pm UTC](https://boards.straightdope.com/t/what-is-the-term-for-this-kind-of-statistical-mistake/647095/14 "2013-01-14T21:21:05Z")
> [@Absolute](#): > > Would you agree that “Omitted-variable bias”, linked above, is an accurate description? It’s an omitted variable issue for sure, but the word “bias” has a specific technical meaning in statistics that doesn’t fit here.