Yes, it’s a one-in-twenty using the standard criteria, but no it has nothing at all to do with “standard deviation”, although it is statistical mathematics, which sd comes from. In this case it’s the “Q value” or “confidence interval” rather than sd. No, I’m not even going to try to explain what those are. I worked in an industry where those things came up, sometimes, and I continually beat my head against a wall futilely trying to get people with engineering degrees to understand their significance. I ain’t even going to try to explain them in a forum where I can’t count on the reader even knowing basic algebra. Anyone want to know the details, Google is your friend.
And no, thelurkinghorror’s wrong too, just in a different way.
Sorry, should have been more specific. Standard deviation is a descriptive statistic, and as such it doesn’t help you draw testable conclusions. You need to convert it to a standardized score first. I would use hypothesis testing, confidence intervals are great at showing differences graphically, but don’t have quantitative explanatory power.
The advisor didn’t want it published - it would have caused them to retract their previous work. Friend was done with it all by that point and didn’t care anymore.
Too late for editing-
And it should be retracted. They were studying the expression and function of a particular protein that for biochemical reasons was very hard to work with - showed up poorly on gels, no available reagents etc… Previous post-doc generated an antibody to the protein (first person ever to do so) and used it to follow expression profiles, location, induction and other things - got a couple of pretty good publications from it.
Turns out, the antibody didn’t bind to the protein. It bound to E. coli.
Since not many people work on this, it may take a long time before it is corrected. The post-doc is untouchable (like I said, government job in a second-world country) and the advisor will be retired.
It has been estimated that half of all refereed published papers in math contain an error. I don’t know if half my papers do, I hope not, but I have published errors. A good friend, now deceased, was one of the stongest mathematicians I was privileged to know and the joke ran that each of his published papers began with correction of the errors in the previous one. Yet the effect of all this is exceedingly minor. With one exception, described below, either the error is easy to correct and people who use the result quickly discover the error and correct it, or the result is so minor that no one actually notices since no one uses it.
The exception: About 20 years ago, a colleague of mine got a letter from a student in Denmark who discovered that the proof of a theorem published nearly fifty years earlier was wrong and. in fact, the student discovered a counter-example. This theorem had been used in dozens, maybe hundreds, of papers over the years. The student came to McGill and got a PhD with my colleague and is now, I believe, a professor in Copenhagen. But i do not know what happened with all those dozens or hundreds of papers based on this result. Many papers have gaps, usually easily filled, sometimes filled with difficulty (e.g. the Wiles’s original solution to the Fermat conjecture) and sometimes unfillable, but these gaps will be discovered by anyone who reads the paper with sufficient care.
At a guess, there was probably some similar but weaker theorem that could be salvaged from the original flawed one, and many of the other mathematicians who used it probably went back to see if they could use that weaker version instead. Though of course you would know better than I.
I’m pretty happy with the data that get’s published in the chemical literature, although you often see quite speculative mechanisms put forward to explain reactivity etc. This is accepted practice when you’re publishing on new chemistry.
The actual data, though, which in organic chemistry is the characterisation data of the compounds you have made, is pretty good IME. Certainly stuff in the higher profile journals gets picked up and repeated by other research groups -any completely bogus work would get exposed. I guess you have degrees of wrong-ness though - a common one would be publishing a yield of a chemical reaction that’s your best result on your best day - 85% when generally it’s more like 50-60%.
People should also know just how incremental research can be in the more established disciplines - even international leading stuff can be baby steps. It’s pretty rare to see stuff like ‘A new model for thermodynamics’ or ‘A re-assesment of the nature of the Chemical Bond’ being published. These sort of outrageous claims just don’t get made at the top level - it’s more measured and stepwise.