You ever hear of a guy named James Randi? He’s not a games journalist – he’s a retired stage magician. But his relevance to this discussion is his post-retirement career as a skeptic with respect to paranormal claims, and his debunking of those who purport to have paranormal powers. He’s most famous for his offer of a million dollars to anyone who can demonstrate those powers.
The central aspect of that million dollar challenge is simple: the claimant must be able to describe, specifically, ahead of time what he can do, and then Randi will propose a set of monitoring or test parameters, to which the claimant agrees in advance are valid. This prevents the claimant from failing when under observation and then saying, “Well, it’s all the negative energy from disbelief that’s foiling my telekinesis,” or “The studio lights create an electric energy field and that dampens my telepathy.”
In other words, Randi explains ahead of time that there will be studio lights, and the claimant then has every opportunity to explain the negative effects of studio lights. If there are, the test will be done in a greenhouse, under natural light. The point is: Randi knows that post-hoc rationalizations are cheap; the only way to show an actual valid prediction is to be able to agree, in advance, on how to measure success.
The same principle holds true here. It’s true that I don’t know what you’ll do. But that’s the point: I don’t. So defining ahead of time what types of outcomes are legitimate indicators of success is precisely the way to rebut any accusations of post-hoc rationalization.
Even worse for your claim that “it will be obvious,” after the strike ends, then. I absolutely believe that they have opinions all over the spectrum, but if that’s the case, how can they lend an authority to this question?
No, because then I’d be guilty of exactly the sin I ascribe to you: waiting until after the events have happened to claim I had some problem with using them as evidence.
I’m saying, NOW, ahead of any actual outcome, what I predict will happen. You’re waiting until after you see what happens to decide which reactions or evidence will be dispositive. If A, B, C, D, E, and F all opine that the strike was a failure, you have all wriggle room to point at G and H’s opinion that it was a noble success. But if A, B, C, G, and H come down on the side of failure, you’re just as free to piously claim that it was always D and E that were the most insightful commentators, and it’s THEIR opinion that’s truly captured the ways this was a win.
That tactic – the unwillingness to commit to a standard ahead of times – is evasive. It was evasive when Sylvia Browne and Uri Geller used it with James Randi, and it’s no less evasive now.
I’m not making any such accusation. I am talking about your argument, here, and how it evades the setting of goals in advance.
You’re reading that as a personal attack, but it’s not. I don’t think you have a plan for deception. I think you’re passionate about the subject and you’re human, and that brings with it the very human thought habits that tend to create the end results I mention . . . not as result of deception or dishonesty at all. Confirmatory bias is the thought pattern, the selection bias, in which decent and honest people have been shown, decently and honestly, to nonetheless seek out and assign more weight to evidence that confirms their hypothesis, and to ignore or underweigh evidence that would weaken or disprove their hypothesis.
It’s not a matter of deception, or something done by people who are trying to be deceptive or dishonest. It’s a bias, an unconscious thought pattern.
And it’s defeated by the agreement, in advance, on what the meaning of given test results will be.