This is not homework (but thank you for thinking that I’m young enough to be in school), nor is it something for my job. There’s a distribution that I want to test the randomness of. I presume that it’s possible to use something like the chi-square test to tell how likely it is to be random. Can any of you do this quickly? Here’s the distribution: There are 58 objects which can be assigned to any one of 36 categories. I want to know if they have been randomly assigned to those categories or if there is something fishy going on. If I use one of the standard ways of referring to the objects and the categories, there are 58 balls which are thrown at 36 urns.
We have been assuming that each time we throw a ball at the urns, there is an equal chance that the ball will go in each of the 36 urns. We observe after the balls have gone into urns that 9 of the urns have 0 balls in them, 10 of the urns have 1 ball in them, 10 of the urns have 2 balls in them, 4 of the urns has 3 balls in them, 0 of the urns have 4 balls in them, 2 of the urns have 5 balls in them, 1 of the urns has 6 balls in it, and no urn has more than 6 balls in it. Is that likely to be a random distribution or is there probably something fishy going on?
On the other hand, suppose that the distribution is that 14 of the urns have 1 ball in them, 22 of the urns have 2 balls in them, and no urn has more than 2 balls in it… Is that likely to be a random distribution? Is something fishy going on that would indicate that’s it’s probably not random? After I get some answers to these two questions, I will explain how this applies to a well-known question.
Looking at this table of the Chi-Squared distribution[sic *], with 57 degrees of freedom, it appears you needn’t get suspicious until your χ[sup]2[/sup] is in the high 70’s or 80’s.
You’re OK, if my work is correct, because your data have χ[sup]2[/sup] = 48.8
(9* (0 - 58/36)^2 + 10* (1 - 58/36)^2 + 10* (2 - 58/36)^2 + 4* (3 - 58/36)^2 + 0* (4 - 58/36)^2 + 2* (5 - 58/36)^2 + 1* (6 - 58/36)^2) * 36/58 ~= 48.8
I’d be much MUCH more suspicious of the “fishy” even distribution you describe in the final paragraph, but I don’t know a simple way to demonstrate the fishiness with a χ[sup]2[/sup] approach.
(* - I almost corrected the URL’s sqaure to square on proofread. :smack: )
OK, I ran some simulations (1,000,000) to figure out the expected distribution and how to bucket things to run a reasonable Pearson’s chi-squared test.
The expected number of urns that will hold 0, 1, 2, or 3+ balls is as follows.
k Expected
0 7.03
1 11.64
2 9.48
3+ 7.85
These numbers aren’t perfect, but we will see that they are close enough to make a point, and with these numbers in hand we can do a chi-squared test on each of the observed distributions.
O.K., here’s how this applies to a distribution that’s been in the news. Look at the number of black nominees for the Academy Awards in the best lead actor, best lead actress, best supporting actor, and best supporting actress categories. The link below gives all such nominations. Since 1981, there have been 58 such nominees, and it’s been 36 years from 1981 to 1986. I’m going to eliminate all the years before 1981 because there were only 12 nominees in all the years from 1927/1928 to 1980. Note that there are a total of 20 nominees in the acting categories each year.
There were no black nominees in these categories in 2014 and 2015. There were 6 nominees in those categories in 2016. The distribution over the 36 years matches the first one I gave. You might wonder if this is surprising. Shouldn’t the number of nominations be more evenly spread out? Wouldn’t it be more likely that each year there would be more like 1 or 2 nominees each year?
Well, no. In fact, it’s more likely that the number of nominations for black actors and actresses would go up and down a little. As septimus and Lance Turbo have shown, the actual situation (9 of the urns have 0 balls in them, 10 of the urns have 1 ball in them, 10 of the urns have 2 balls in them, 4 of the urns has 3 balls in them, 0 of the urns have 4 balls in them, 2 of the urns have 5 balls in them, 1 of the urns has 6 balls in it, and no urn has more than 6 balls in it) is much more likely than an evenly spread out version (14 of the urns have 1 ball in them, 22 of the urns have 2 balls in them, and no urn has more than 2 balls in it) if it’s a truly random distribution. (Here balls are nominations and urns are years of the Academy Awards.) I understand why people were surprised by the apparently more uneven distribution though.
Suppose you flip a coin several hundred times and write down the string of heads and tails. Then ask someone to write down a random string of heads and tails of the same length, using their own intuition of what random is. If you look at these two strings, you’ll almost certainly see longer substrings of heads or tails in the string created by flipping a coin than you’ll see in the supposedly random string written down by the person based on their intuition. People tend to underestimate the amount of local variation in truly random situations. They tend to even things out too much.
Note that none of this touches on the question of whether 58 nominations over 36 years is a reasonable number. I have no idea of how to calculate an answer to that question. What are the numbers supposed to be based on - the proportion of blacks in the U.S. or in the world (since the nominees come mostly from the U.S. but have a fair number from elsewhere) or the number in movies made in the U.S. or in the world or whatever? The tests mentioned only answer the question of whether the nominations are randomly spread out among the 58 nominations over 36 years.
Note also that part of the problem is that just 20 nominations per year is far too small a sample for most purposes. If you were to claim that you could predict the Presidential election based on 20 randomly chosen voters, you would be rightly laughed at. Year-to-year samples of 20 aren’t large enough. If there were 200 nominees a year you could say something about the number of nominees each year. Even with the situation of 20 nominees that we do have, if there were 15 years in a row with no black nominees, followed by 18 years in a row with 4 or 5 black nominees each year, followed by 11 years in a row with no black nominees, followed by 19 years in a row with 6 or 7 black nominees, followed by 9 years in a row with 1 black nominee, you could say that it didn’t look random. But based on the small sample that we actually have, we don’t have any reason to say that it’s not random.
This is a different problem for various reasons, most simply that you’re now constrained by a maximum urn capacity of 20 balls. One could do a new statistical test on a revised problem, but the full set of required revisions isn’t obvious.
Academy award nominations are not randomly assigned and any resemblance to a random process is coincidental. Who is arguing that nominations for African Americans should be more spread out because that would look more random? That’s a silly argument for a number of reasons but I’m not sure anyone is actually making it.
For two years in a row there were no African American nominees. This got a ton of negative publicity. This was followed by a year with the all time record number of black nominees. One could argue that that is perfectly consistent with a random process but it is also consistent with human voters with acting consciously or subconsciously on bias and then overcorrecting as a response to negative publicity.
Another difference between the Academy Awards problem and the original urn problem is that the number of African-American nominees, or the number who win, aren’t fixed numbers. If the “dice” had rolled differently, we could have had more, or fewer. Probably the best model would be independent draws from a Poisson distribution each year (this would still theoretically have problems with the upper bound, but we’re far enough away from that bound for it to matter very little).
I would think a more reasonable basis would be the proportion of actors that re identified as black. So look into wht percentage of Screen Actors Guild members are black. Or somehow restrict it to ‘major’ actors, since most Academy Award nominees are ‘major’ actors.
