Checking Cooked up Data

[andy_fl]

I heard this sometime back (sorry no cite) on radio that the way data could be checked for “manipulation” is by checking the percentage of zeros (Total number of Zeros occuring anywhere in the data) among all the individual numbers. It claimed that the percentage of zeros are less for cooked up data compared to real data.

The reasoning was that humans when asked to make up figures are less likely to use 0 anywhere in the figure.

Is there any truth to this ? Any cites as to what is the normal ratio of zeros ?

[Squink]

I don’t know much else about it, but yes, there’s some truth to it.

In 1938, Benford found that for many types of data, the digits of the numbers are distributed in a predictable pattern. That is, for certain data sets a specified percentage of the numbers should start with the digit 1, the digit 2 and so on. Accounting data is one of the many types of data which is expected to follow the pattern predicted by Benford (Carslaw 1988; Thomas 1989). It has also been discovered that manufactured, or manipulated, numbers do not have digit patterns which follow a Benford pattern (Hill 1988). Therefore, in theory, the more an observed digit pattern deviates from the expected Benford pattern, the more a data set is suspect.

________Benford’s Law
Also see Google on (“digit distribution” fraud)
If you have access to a library, New Scientist ran an article ondigit distributions and fraud detection a few years ago

[samarm]

Squink, you beat me to it!

I had a whole post perpared but my PC decided to crash just before I submitted.

andy_fl: Try googling “Benford’s George’s random” for some good links.

[andy_fl]

Thanks for the links. Appreciate it.