# Statistical way to properly kickout an outlier **URL:** \ **Category:** Factual Questions\ **Created:** [May 29, 2024, 3:16pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234 "2024-05-29T15:16:58Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 29, 2024, 10:14pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/21 "2024-05-29T22:14:31Z")
And the [geomethic meandian](https://xkcd.com/2435/), or any similar metric, usually ends up looking a lot like the median, as long as the median is one of the measures you throw into the soup. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [May 29, 2024, 11:46pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/22 "2024-05-29T23:46:34Z")
In the past, I’ve use Peirce’s Criterion to remove small numbers of outliers: > **[Peirce's criterion](https://en.wikipedia.org/wiki/Peirce%27s_criterion)** > > In robust statistics, Peirce's criterion is a rule for eliminating outliers from data sets, which was devised by Benjamin Peirce. > In data sets containing real-numbered measurements, the suspected outliers are the measured values that appear to lie outside the cluster of most of the other data values. The outliers would greatly change the estimate of location if the arithmetic average were to be used as a summary statistic of location. The problem is that the arithmetic mean is very sensitive to... I had to implement it from the original paper due to lack of library support at the time. It was remarkably readable, though. Refreshing compared to most modern papers I read. I like it because it doesn’t require any fudge factors. It’s based on this idea: > The principle upon which it is proposed to solve this problem is, that the proposed observations should be rejected when the probability of the system of errors obtained by retaining them is less than that of the system of errors obtained by their rejection multiplied by the probability of making so many, and no more, abnormal observations. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 30, 2024, 12:18am UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/23 "2024-05-30T00:18:43Z")
> [@Pleonast](#): > > post-hoc reasoning Not buying it. Not for the stated goal of throwing out a particular score. > [@Stranger\_On\_A\_Train](#): > > … if you are just sampling data and have a ‘flyer’ that you don’t like, you can’t just throw it out because it is inconvenient. This is the crux of it. No statistician I but lots of the bits described in this thread sound like not only like trying to massage the data but wanting to be sure the massage results in a happy ending. ---
**Author:** ![Pleonast](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pleonast/32/1183_2.png) [@Pleonast](https://boards.straightdope.com/u/Pleonast)\ **Post date:** [May 30, 2024, 2:19am UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/24 "2024-05-30T02:19:02Z")
> [@DSeid](#): > > > [@Pleonast](#): > > > > post-hoc reasoning > > Not buying it. Not for the stated goal of throwing out a particular score. I agree, which is why the first paragraph of my post was > [@Pleonast](#): > > > [@DSeid](#): > > > > There is no acceptable way to look at data and figure out how to throw out a particular score after the fact. > > Definitely true for something like an art contest as in the original example. ---
**Author:** ![Measure\_for\_Measure](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/measure_for_measure/32/557_2.png) [@Measure\_for\_Measure](https://boards.straightdope.com/u/Measure_for_Measure)\ **Post date:** [May 30, 2024, 4:39am UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/25 "2024-05-30T04:39:20Z")
> [@DSeid](#): > > No statistician I but lots of the bits described in this thread sound like not only like trying to massage the data but wanting to be sure the massage results in a happy ending. Your concern can be assuaged if the methodology is put in place before the results are in. Inflation is a pretty good example. During the 1970s Fed officials would homebrew price indices which eliminated outlying sectors as they appeared: the exercises successfully allowed them to fool themselves. But it’s _also_ true that non-representative sectors (like energy) can be poor measures of underlying inflation. Applying robust indices like the trimmed mean or a median index is one approach. Removing food and energy is the most familiar. The first two take care to trim both sides - both the sectors that have high inflation and the sectors that have low or negative inflation. All of them are valid when applied to eras years after they were proposed. Post hoc methods can also be legit. Does anybody think economic measures during COVID are representative of a normally functioning economy? Certainly not: they are representative of an economy in a medical coma. It’s valid to drop such observations from a dateset or a chart, at least if the subject of the inquiry is long run trends. (Cyclic studies are another matter.) Put in another way, the median is a valid measure of central tendency. It is also an extremely trimmed mean, an average of 1 or 2 remaining observations in the dataset after the edges are trimmed all the way to the center. ---
**Author:** ![Sigene](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@Sigene](https://boards.straightdope.com/u/Sigene)\ **Post date:** [May 30, 2024, 11:36am UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/26 "2024-05-30T11:36:15Z")
If it makes any difference. I don’t want to do ‘Post hoc’ analysis. I want to put in place a methodology NOW, so that IF such a scenario occurs in the future, I have a reasonable method established that would allow me to kick out the “5” while keeping the “15.” ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 30, 2024, 11:40am UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/27 "2024-05-30T11:40:29Z")
Thing is it shouldn’t allow you; it does or doesn’t? ---
**Author:** ![Great\_Antibob](https://avatars.discourse-cdn.com/v4/letter/g/e47c2d/32.png) [@Great\_Antibob](https://boards.straightdope.com/u/Great_Antibob)\ **Post date:** [May 30, 2024, 12:23pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/28 "2024-05-30T12:23:59Z")
Sounds like you want to have such a method determined _a priori_ that doesn’t bias the data but is still statistically valid for whatever you are trying to measure without divulging what you are trying to measure. _A priori_ there should not be any such method that fits all your criteria. There just can’t be. We can’t know what is truly an “outlier” without some knowledge of how we expect the measurements to be distributed. The thing is, unless you have additional information - such as some underlying knowledge of what you are measuring or that certain values are likely to be in error - you can’t just look at a random list of samples and automatically _know_ a measurement is truly an outlier rather than a valid but rare (or even not so rare) occurrence. Even the method of tossing out both high and low numbers can be wrong but usually works out ok. But that’s _usually_. There’s no guarantee it doesn’t influence the underlying statistics every time you do it. Especially if you blindly apply it no matter what you are measuring. Basically, without realizing it, you are asking how to fudge the statistics without fudging the statistics. Perhaps with knowledge of what was being measured, some kind of procedure could be determined to validly toss out the occasional extreme value, but there’s no such method that can be determined beforehand with no knowledge of what is being measured and how those measurements are being made. ---
**Author:** ![Napier](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@Napier](https://boards.straightdope.com/u/Napier)\ **Post date:** [May 30, 2024, 1:50pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/29 "2024-05-30T13:50:23Z")
Hope I get this right, from memory and conversations with a statistician friend. There is an iterative univariate analysis method called “Lowess” that considers a distribution, chooses outliers to remove, then reconsiders that distribution to remove more. Also, all these things should be tempered by anything you know or expect about the distribution. Is it Gaussian, or what? For example, it could be a Cauchy distribution. A Cauchy distributed population of values will have a relatively tight cluster of values plus one value relatively far away from the cluster. If you toss that one extreme value, and zoom in on that tight cluster, you will find that it now consists of a relatively tight cluster (much smaller than the first) plus one value relatively far away; it’s still a Cauchy distribution, just a tighter one. An example of a system that generates Cauchy distributions would be a laser pointer on a turntable near an infinitely long straight wall – you spin the pointer and let it come to a stop, then measure the location of its spot on the wall. Interestingly, taking the average of all the values in a Cauchy distribution is a lousy way to estimate its overall value. You’d do better to pick any one value at random. Weird! Though you’d do far better still to evaluate its median. Like I say, I’m saying this from memory and I’m not a statistician. But if it’s interesting, it might be a lead to pursue. Of course anybody who knows these things better than I can come along and straighten me out if they like! ---
**Author:** ![Pleonast](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pleonast/32/1183_2.png) [@Pleonast](https://boards.straightdope.com/u/Pleonast)\ **Post date:** [May 30, 2024, 2:47pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/30 "2024-05-30T14:47:28Z")
> [@Sigene](#): > > If it makes any difference. I don’t want to do ‘Post hoc’ analysis. I want to put in place a methodology NOW, so that IF such a scenario occurs in the future, I have a reasonable method established that would allow me to kick out the “5” while keeping the “15.” Echoing what others are saying, if you want to do this, you need to know the expected form of the distribution. The difficulty is that to reject an extreme outlier (the “5”), but _not_ reject a marginal datum (the “15”), you need to distinguish between the two. And that can only be done if you have some reasonable model of the scores. For example, if you expect a Gaussian distribution, you might decide that anything more than 3 standard deviations from the mean should be rejected. Or maybe it should be 4 or 2.5 standard deviations. All of this requires knowledge and analysis of your scores (and before you have any scores, if you don’t want to be post hoc). Without more specific knowledge of your data, we can’t really give you more specific advice. ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [May 30, 2024, 2:50pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/31 "2024-05-30T14:50:54Z")
> [@Stranger\_On\_A\_Train](#): > > In the case of the o.p.’s example of art reviewers, there is no expectation that their valuations will be normally distributed (random error or variation distributed about a central mean), and in fact you would expect bias both in how each reviewer scales their scores, and in how each reviewer interprets and rates unique works of art. Which is why a non-parametric test may be the best and eliminate the throwing out outliers issues. ---
**Author:** ![Stranger\_On\_A\_Train](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Stranger\_On\_A\_Train](https://boards.straightdope.com/u/Stranger_On_A_Train)\ **Post date:** [May 30, 2024, 3:05pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/32 "2024-05-30T15:05:56Z")
> [@Pleonast](#): > > For example, if you expect a Gaussian distribution, you might decide that anything more than 3 standard deviations from the mean should be rejected. Or maybe it should be 4 or 2.5 standard deviations. All of this requires knowledge and analysis of your scores (and before you have any scores, if you don’t want to be post hoc). Of course, by removing the point from the sample, you will be affecting the parameters of the distribution, including the variance (the standard deviation is the square root of the variance). If the point is truly an outlier it shouldn’t impact the parameters of your estimate significantly, but you would need a criteria for how to evaluate that. If you actually have an expected distribution of comparable size you can back into this by applying the Student’s _t_-test (or the Welch’s _t_-test if your expected distribution is a significantly larger sample) and applying some specified level of confidence. Of course, this assumes that the expected distribution is Gaussian. Most real world samples that are not a result of random variation (or error) about a central mean or measurement are not actually Gaussian, and trying to force a non-Gaussian phenomenon into a Gaussian distribution is a frequent cause of statistical malfeasance and resultant bad predictions. > [@Saint\_Cad](#): > > Which is why a non-parametric test may be the best and eliminate the throwing out outliers issues. Well, except non-parametric distributions require large amounts of data and have a higher degree of subjectivity in terms of what is truly an ‘outlier’ versus a point that is a legitimate measurement that just happens to be at an unexpected extrema of the phenomenon being sampled. The purpose of using a non-parametric approach is to avoid imposing a rigorous presumption of what the distribution will be, so throwing out ‘unexpected’ results without an epistemic justification is even more suspect. This obviously gets into the philosophy of statistical measurement and prediction and is probably beyond the scope of what the o.p. is looking at, but it is important to illustrate that there is no simple or universal method to identifying outliers in a sample. Stranger ---
**Author:** ![Maserschmidt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/maserschmidt/32/18829_2.png) [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)\ **Post date:** [May 30, 2024, 3:13pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/33 "2024-05-30T15:13:54Z")
> [@Napier](#): > > There is an iterative univariate analysis method called “Lowess” that considers a distribution, chooses outliers to remove, then reconsiders that distribution to remove more. LOESS. We talked about it, but never used it… [Did You Know? | National Centers for Environmental Information (NCEI)](https://www.ncei.noaa.gov/access/monitoring/dyk/loess#:~:text=Locally%20estimated%20scatterplot%20smoothing%2C%20or,through%20a%20scatterplot%20of%20data). ---
**Author:** ![Maserschmidt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/maserschmidt/32/18829_2.png) [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)\ **Post date:** [May 30, 2024, 3:15pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/34 "2024-05-30T15:15:57Z")
> [@Stranger\_On\_A\_Train](#): > > Well, except non-parametric distributions require large amounts of data and have a higher degree of subjectivity in terms of what is truly an ‘outlier’ versus a point that is a legitimate measurement that just happens to be at an unexpected extrema of the phenomenon being sampled. The purpose of using a non-parametric approach is to avoid imposing a rigorous presumption of what the distribution will be, so throwing out ‘unexpected’ results without an epistemic justification is even more suspect I agree with this. When I’ve use nonparametric fits, I’ve essentially taken them as given, and assumed things that look like outliers were just extremes still within the distribution. I wouldn’t know how to infer otherwise, which is why it’s not parametric. ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [May 30, 2024, 3:29pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/35 "2024-05-30T15:29:32Z")
> [@Stranger\_On\_A\_Train](#): > > The purpose of using a non-parametric approach is to avoid imposing a rigorous presumption of what the distribution will be, so throwing out ‘unexpected’ results without an epistemic justification is even more suspect. My point exactly. Using a non-parametric test would eliminate “needing” to throw out data. ---
**Author:** ![Omar\_Little](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/omar_little/32/269_2.png) [@Omar\_Little](https://boards.straightdope.com/u/Omar_Little)\ **Post date:** [May 30, 2024, 3:49pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/36 "2024-05-30T15:49:55Z")
> [@Sigene](#): > > I don’t want to do ‘Post hoc’ analysis. I want to put in place a methodology NOW, so that IF such a scenario occurs in the future, I have a reasonable method established that would allow me to kick out the “5” while keeping the “15.” It sounds like you want to manipulate the data IF you have an outlier you don’t like. This is introducing your own bias into the analysis, which invalidates the statistical valid analysis you are attempting to achieve. If you want to put in a methodology that fairly treats all evaluations you have to apply that method, regardless of the evaluations and apply it, not only when there is an outlier that you don’t like. ---
**Author:** ![Sigene](https://avatars.discourse-cdn.com/v4/letter/s/df788c/32.png) [@Sigene](https://boards.straightdope.com/u/Sigene)\ **Post date:** [May 30, 2024, 4:25pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/37 "2024-05-30T16:25:41Z")
That is what I want to do. I don’t want to throw out a number if I don’t like it. I want a means to throw out a number because it is wildly out of step with the others. And I want to be able to use that measure for any number that meets this criteria. I can’t just throw out a number because the reviewer is a jerk and doesn’t like atheists so scores from his bias vs the quality of the work. But I do want a means to throw out a number legitimately without having to say…I don’t like you because you are a jerk. ---
**Author:** ![thelurkinghorror](https://avatars.discourse-cdn.com/v4/letter/t/7c8e57/32.png) [@thelurkinghorror](https://boards.straightdope.com/u/thelurkinghorror)\ **Post date:** [May 30, 2024, 4:28pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/38 "2024-05-30T16:28:18Z")
~~Jackknifing~~ ~~Bootstrapping~~ Handjobbing ---
**Author:** ![Great\_Antibob](https://avatars.discourse-cdn.com/v4/letter/g/e47c2d/32.png) [@Great\_Antibob](https://boards.straightdope.com/u/Great_Antibob)\ **Post date:** [May 30, 2024, 4:35pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/39 "2024-05-30T16:35:05Z")
> [@Sigene](#): > > I want a means to throw out a number because it is wildly out of step with the others. This is the point people have been trying to make. How do you know with certainty it is wildly out of step? This means you already have an idea of what the number “should” be. If you know a particular source of data is not good - you don’t use that source. Right now, you seem to want it both ways, i.e. wanting something you can claim is objective while knowingly including subjective judgments (both yours and the biased reviewer). Something has to give, whether that’s the statistical validity of what you are doing, somebody’s feelings, or something else. Once you decide to include garbage numbers for whatever reason, you’re stuck with them. Better not to have them in the first place. ---
**Author:** ![Omar\_Little](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/omar_little/32/269_2.png) [@Omar\_Little](https://boards.straightdope.com/u/Omar_Little)\ **Post date:** [May 30, 2024, 4:39pm UTC](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234/40 "2024-05-30T16:39:09Z")
> [@Sigene](#): > > That is what I want to do. Then your premise of a statistically valid analysis is incorrect. [Previous page](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234.md?page=1) [Next page](https://boards.straightdope.com/t/statistical-way-to-properly-kickout-an-outlier/1002234.md?page=3)