[QUOTE=jshore]
Look, intention, I agree with you that if you do the same exact mathematical operations on a set of data then, to the limits of machine precision, you will get the same result and this will be the sort of exact mathematical replication that you are speaking of.
However, and this is the important point which is why I am bolding it: In the physical sciences when it is talked about researchers having to give enough detail for others to replicate their results, this is not the standard that is generally demanded. I know of absolutely no journal in the physical sciences that requires authors to give enough details of their work to allow this standard of replication. It may be that you feel that this should be the standard. Then, by all means, it is your perogative to try to fight for a major sea-change in the way that research in the physical sciences is done. However, to demand that Mann et al. live up to a standard that is not in fact the standard that is expected of him by the journals he publishes in or by his peers in the field is simply not realistic.
Look, I gave you an example of Monte Carlo simulation but in fact I could give you an example with no random numbers involved: The code that I use at work has a step where we take experimental data and interpolate it for further use in the modeling. The interpolation routine is complicated…in fact, iterative. To allow people to replicate my calculations to the degree that you are demanding would require me practically publishing the interpolation routine code line-for-line. This is so far from anything that any journal in the field would ever demand from me as to be frankly amusing to contemplate!
Now, in my own work with my own computer codes, I do sometimes like to make sure that a change I make to the code is still resulting in the same results down to this level of accuracy. Other times, I have made changes to the code that I know may actually cause changes in the results that are not physically significant but do not constitute exact replication in this strict mathematical sense. For my own edification, when I am recording modifications to my code, I try to make a special note to myself when I do make a change that I think can make small mathematical changes in the results. However, as I will repeat again for emphasis, when scientists talk about other scientists having to give enough details to allow replication of results, they are not talking about it having to be done to the level of detail necessary for this to be true in a strict mathematical sense. The sense of replication that is meant is the sense of getting the same basic physical result. This is clearly true in experimental work where it would be impossible to have a higher standard. However, even in numerical work such as Mann’s where it is t least in theory possible to demand this higher standard for replication, that is not the standard that is demanded.
[/QUOTE]
jshore, thank you for your thoughtful comments. Mann does not have any iterative routines such as you describe. He doesn’t have any Monte Carlo analysis. He has a mathematical operation which was incorrectly described in the text. This error could not be found without the code, and meant that his work could not be replicated.
Yes, if there is Monte Carlo, if there are iterative routines such as you describe, the final results may differ slightly. But this is a situation where A + B doesn’t equal C.
I truly don’t understand why you would want to defend someone who hid his data and code and misrepresented his procedures and results so badly that he had to issue a corrigendum to his work to (partially) fix his mistakes.
I also don’t understand your defense of the peer-review system. If it worked as well as you seem to think it works … why was it necessary for Mann to issue a corrigendum to Nature to fix his errors? He hadn’t even correctly listed his sites. He extended data back in time to allow it to be used to “prove” his point. This was all revealed in the Corrigendum. Why weren’t these errors caught in the peer-review?
You are defending a man who falsified data. Made it up out of thin air to support his contentions. You are welcome to do so … me, I prefer science.
w.