Good God man, don’t do that!
This is the internet. Before you know it, some cowpoke in a legislature somewhere in Texas is going to take your idea above and run with it!
“Hostile scrutiny?”
Part of me wonders what the hell that is, and another part of me already knows that it doesn’t have anything to do with science.
If a black guy can’t withstand a lynch mob does he deserve to live? Science can withstand hostile scientific scrutiny and indeed that leads to good science, but complex science is not very good at withstanding populist attack. Complex truths tend to lose out to simple lies. This doesn’t mean good science isn’t good science.
I disagree: surely this is the acid test? If it withstands such scrutiny, then the case is all the stronger; if it doesn’t withstand such scrutiny, then science still advances.
You’re wrong. Science should face impartial scrutiny. Hostile people make shit up and ignore evidence that doesn’t fit their prejudices.
Most climate change deniers are untrained, ignorant people with an axe to grind.
Impartial scrutiny is best, but that’s not quite what I said.
As opposed to well-meaning people who make shit up and ignore evidence that doesn’t fit their prejudices? Sorry, but I maintain my position: if a piece of science withstands hostile scrutiny, then it is all the stronger.
I’ll grant you the untrained, but I’d like cites for the latter two claims, please.
It depends what you mean by “withstand”. If we are talking about whether it can withstand hostile scrutiny and still be accepted by scientists that’s one thing.
However, because AGW isn’t just of theoretical but also practical importance, there is also the issue of whether it can withstand hostile scrutiny and remain accepted by popular opinion. If you mean “withstand” in that respect, I don’t think you are necessarily correct for reasons already given.
So, you decide that ignoring history is also good?
History shows that it already did.
No people suffered more knowledgeable opposition than the researchers that back in the 40’s an 50’s came with research and papers that pointed out that CO2 and other gases were going to be a problem.
After more than 50 years of further research and evidence that scientific opposition came tumbling down until we reached the agreement levels of today, now most scientists agree that warming is happening, climate is changing and humans are causing the recorded increase.
http://www.aip.org/history/climate/
Good lord, have you paid any attention to the previous discussions showing how deniers are axe murderers of the truth?
http://scienceblogs.com/deltoid/global_warming/mcintyre/
http://scienceblogs.com/deltoid/2009/11/on_those_stolen_cru_emails.php
Of course what remains is an investigation to see if they actually did put action into the words (cherry picked and usually taken out of context by deniers) and check accusations that scientists made underhanded moves to prevent deniers from publishing their stuff, as I have seen, there is a difference between good skeptical research and trash. So far it seems that the scientists were just concentrating in denouncing the trash papers, I would call that “doing their job”.
http://thingsbreak.wordpress.com/2009/12/18/pat-michaels-lying-in-the-wall-street-journal/
I have no dog in the AGW fight, but I do confess to a fundamental misunderstanding of the peer-review process. Maybe I should say an over-all ignorance of the process.
Can you point me to a place to educate myself?
A good video explanation regarding peer review, dealing also with the latest critics of the scientists:
Wikipedia seems to do a good job here, but always check the cites.
Disagreeing with previously published scientific experiments or investigations is a part of the scientific method. It is the reason you decide to to study something. You look at the literature, and say, I don’t think that that is the correct conclusion. I think the method may be flawed, or the sample too small.
What you do then is to do a different experiment, or an different investigation, and get your own data so you can find out. And, if it is a sufficiently different result, two or seven other people will design new experiments, or conduct other investigations, and generate their own data to find out if the difference is one of observational method, or underlying aspects of the phenomenon being studied.
You investigate, and fact check data as part of your own experiment. You don’t waste time fact checking another experiment over which you had no control. If it is sufficiently at odds with your expectation to cause interest, you do another experiment.
Someone else doesn’t have data, they have a report. Publications based on reports is not scientific investigation. A fact is something you observed, not something someone told you.
And when you tell someone, it becomes a report again. It cannot ever be data again.
Now you can’t ever do all the observations or even all the experimental trials, so, you need to maintain the reliability of your collections systems, and your data storage. But that is so you can do your experiments, not to defend your experiments. If your methods are suspect for any reason, your results need to be verified by an entirely different experiment.
When your experiment is an entire world, you can expect high levels of variability on reliability. That means you need to examine many experiments, on many models, and repeat every type of examination many times. In the end, single investigations are going to be of little use, and only the broadest of data sets will be reliable enough to merit confidence.
Peer review is not a trap to catch errors, it is a filter to assure that there is some degree of confidence that the report merits consideration in what should be further investigated by others, with different methodologies. Finding out that a peer reviewed report was not an accurate reflection of the underlying realities isn’t at all rare. It is the point of the process. Finding actual deliberate misrepresentation is a career killer for a scientist. But it doesn’t invalidate the method of peer review, and subsequent experimentation by others, it is just another aberrant data point, and a ruined life.
Science stumbles on. Stumbling is the method.
Tris
Yes, I mean be subjected to hostile scrutiny and still come out on top, science-wise. I’ll grant you that public opinion is a wholly different story.
I know there are some, but you stated ‘most’.
AGW hasn’t been scratched by scientific scrutiny. Political slander is not scientific scrutiny.
Hostile scrutiny by those competent to have an opinion. There is enough dissension in science so that at least one reviewer of any paper is going to be hostile - but fair and honest.
Do you think biology would benefit from “hostile scrutiny” from creationists who will not change their opinion no matter what the data says? What data is going to convince AGW deniers?
Let me pile on here also. I work in Computer Science and Electrical Engineering, in computer design. I’ve got somewhere around 30 papers (I’ve lost track) and have done hundreds of reviews, and more usefully, have read well over a thousand reviews. I’ve program chaired several conferences, been on the program committee of more than I remember, been on the editorial boards of two journals, and have edited somewhere around half a dozen special issues of journals.
Each paper gets sent out to at least three and more often 5 - 8 reviewers. Reviewing is done on a volunteer basis, and the most sought after reviewers are the best known in the field and also the busiest. Even today, some professors reassign papers to grad students. Not all people do reviews in detail - it depends on how well you know the details of a particular area, how much time you have, and how well the paper is written.
Reviews typically consist of check boxes, rating writing style, content, innovativeness, use of references, on some sort of scale. The most important part are the comments. Most review forms have a place for comments to the committee or editor which the author does not see, and also comments to the author. Editors will weigh reviews on the depth of the comments. Someone just saying “it’s great” or “it stinks” is not going to influence the decision as much as someone writing a detailed critique. In our conferences there is also a place for reviewers to give their confidence in the material, because you get stuff in areas you are really on top of, and also get stuff you know but not so well.
In conferences the accept/reject decision is made based on the reviews. In journals it still is, but there is the possibility of the paper being revised and sent for another round.
Now, given how busy reviewers are, asking them or expecting them to look at the raw data is laughable - especially since the data is input to some sort of analysis process. Looking at code is even more absurd. Even given that a researcher would want to release code, it would take weeks or even months to get any degree of confidence in it.
Peer review just forces authors to consider prior work (some of which possibly written by the reviewers) and get their argument in order. Results contradicting other results is going to need a lot more support than results supporting others (but will have a greater chance of being published if it seems correct.) Some academics I know argue for a lower accept rate at our conference, since it seems that their tenure review committees think high reject rates are indicators of higher quality. In industry we don’t get rated on paper writing for the most part, so high reject rates mean fewer submissions, since industry people don’t want to waste their time. It is a dilemma.
Also, most papers will be published and then never be seen again. If a paper is truly significant, which is hard to predict, others will try to reproduce the work. That is the real test! As for not being able to predict what papers are important, I ran a committee selecting the most significant papers from the first 35 years of our conference. Our choices had very low correlation with the best papers selected each year.
Hope this helps. I don’t think you understand a peer review process until you’ve run a few.
What appears is that there is another element into the mix, besides deniers and skeptics: the boneheaded media (And in the end, a media that lies as it does not bother to make any corrections so far).
Imagine that if in a recent genetics scandal the media instead of getting the opinion from skeptical biologists of the research of the guy being discredited, instead got commentary from creationists like Duane Gish!
This is indeed what is happening lately when outfits like The Australian, Daily Mail, the Telegraph and The Wall Street Journal are cited recently. The newsmen are giving a forum to discredited and hatchet men like Mark Morano, Steve McIntyre or Lord (not of the house of lords) Monkton to comment on the “scandals”. You should then be able to see then who’s fault when they are propping up even non scandals like the one mentioned in the OP.
With this, I will agree, though I do object to your citing Steve McIntyre. Unlike the others, he is entirely open about what he does and how he does it.
He admits to being a demagogue and a fraud? He’s the James O’Keefe of climate change.
The accusations of fraud and conspiracy are way over the top, I agree. But on the other hand, someone checking if YOUR methodology applied to the raw data YOU used gives the results YOU published is not “crapping on your research”. The data asked for wasn’t even Jones’: it was from Australian, Russian and Chinese weather stations, and McIntyre wanted to know the specific stations invoved in Jones’ 1990 study and the datasets from them that were used to produce his results. Jones should have had the confidence in his own work to give out the information and let McIntyre do his worst. Incidentally, McIntyre HAD published in the field when he requested the data.
Reportedly Jones has recently stated in an interview with Nature that there WAS a problem with the Chinese station metadata (station locations) but he wasn’t aware of it at the time, and he has said issuing a correction was “worthy of consideration”. Which may or may not affect his original paper’s conclusion (1990 is pretty far back in terms of climate science anyway) but it might have come to light a damned sight sooner if he’d allowed someone to critically review his work.
(The Nature interview is supposedly here: 'Climategate' scientist speaks out : Nature News but it’s a subscription site. I’d be grateful if anyone with access could verify the story and/or add context.)
Did Spencer and Christy conceal which temperature records they had analysed and the way they analysed it? Because according to Jones, that’s standard practice.
Incidentally, AGW skeptic Spencer has recently conducted his own analysis of Northern hemisphere temperature data using methods similar to Jones, come up with similar results, and said so. He has also specifically refuted the claims of AGW skeptics D’Aleo and Watts that the reduction in temperature measurement stations since 1973 has introduced a warming bias into the temperature record by culling of cooler stations. (Cooler stations have indeed been dropped, but the methodology for combining the records prevents this from introducing a warming bias.) His analyses and results are on his own blog: New Work on the Recent Warming of Northern Hemispheric Land Areas - Roy Spencer, PhD.
That’s how things are supposed to be. People should analyse the data, say what they did and how they did it, and publish even if it is in opposition to their ideas. So if say, your tree-ring proxies show declining temperatures in the later 20th century, you PUBLISH IT, say what you think is happening, and justify using those proxies anyway in your paleoclimate analysis. As opposed to chopping off the “decline” part and splicing on instrument data, for example.
Tris, I agree completely, but the truth is, suprisingly little actual experimentation occurs in climate science. Mann didn’t go out and take tree ring cores from old trees all over the world to produce his hockeystick; Jones didn’t go out and establish a whole bunch of weather stations all over the world with GPS position and altitude and urban/rural site data for each one to put together the HADCRUT records. Everyone uses the same data sources, believer and skeptic alike. When these guys publish papers demonstrating some point-or-other, they take a relevant subset of the existing data and mathematically analyse it. Jones took Russian, Chinese and Australian data to demonstrate that the Urban Heat Island effect wasn’t signficant. It’s entirely true that other researchers could have attempted to follow his method with their own subsets of data, possibly in different countries entirely, but why is it so unreasonable to first try and duplicate his results with HIS subset?
The answer to why is a complex set of reasons, scientific, academic, and legal. But, even assuming all those could be overcome, the best you can hope to add to the pile of evidence by re-examining an existing collection of data is that different statistical methodologies produce different conclusions. Now a third party must examine your methodology, and compare it to the original, and offer yet a third set of conclusions, without introducing any new information into the study of the underlying phenomenon.
A far better use of scientific effort, and resources is to gather data from as many sources as you can find independently, perhaps some of the same, perhaps not, and use your own method, then publish your results, and give your conclusions. This way, you have increased the body of knowledge, whether you resolve, or rekindle the debate on the previous research. It might also engender those with differing views to be more receptive to the possibility that combined efforts would benefit both researches. You “looking over my shoulder” and you “examining the same phenomenon through a new perspective” are much different things. Scientists are human.
Tris