So to be clear, what is being claimed really has no basis. Correct answers are NOT generally achieved by reflecting upon a first intuitive one and changing to the correct one: overwhelmingly the correct answers were obtained by having the answer or approach to the answer in the first place; and many wrong answers were obtained upon reflection.
I am very confident that there are individual variations in tendency to reflect before answering. I am also very sure that those individual variations vary according to task and context and based upon past experiences and knowledge of individuals, both between individuals and for the same individual.
But the CRT, let alone this possibly intentionally garbled misleading and confused “annoying question” does not actually provide evidence about cognitive reflection at all.
So, when posters in this thread wrongly suggested that people usually get the OP video question wrong because they cannot resist their first gut response and fail to reflect adequately, the reason those posters were wrong about people was also probably not because they could not resist their first gut response and failed to reflect adequately.
Correct answers on the CRT usually do not come by suppressing a first incorrect intuitive response. The correct answer/approach usually comes first.
Incorrect answers are often not a first response but an answer come to after reflection.
The CRT does not actually measure how much respondents suppress first incorrect impulsive intuitive responses.
Greater reflection and less use of heuristics may result in fewer failures of reasoning but the CRT does not demonstrate such and the “annoying question” merely demonstrates that is possible to present a problem in a manner garbled and misleading.
Without trying to make this personal, the fact that you think that the people who came up with the answer 8 weren’t “listening carefully” is both insulting and shows an unwillingness to inspect your own assumptions about what clear communication looks like.
I guarantee that only a subset of the people who answered 8 thought that they were answering the question “how many minor accidents would have to happen on road B for the roads to be considered equally dangerous.” And the reason is not because there is some math shortcut that seems intuitive but is wrong, but rather the question was asked poorly. It’s bad communication. Actually, if the question was posed as I just wrote it, you could absolutely point to any “8” answers as examples of cognative reflection. But as is, people are answering any number of questions, doing their best to interpret incomplete and unclear instructions.
There are two other examples in the video:
A bat and a ball cost $1.10 together. The bat costs $1 more than the ball. What does the ball cost?
Lily pads on a pond double every day. They grow to cover the pond in 48 days. How long does it take for the pond to be half covered?
Both of these clearly state a scenario and the associated question. People give “intuitive but wrong” answers because there is easy math that seems kinda right on the surface.
Those questions are not in any way of a kind with the road/accident question.
You misunderstood me, I think. Your prior post was perfectly clear, and very much on point - I was joking that my extremely convoluted sentence was still clearer than the video in the OP.
You are very specifically trying to make it personal and there is absolutely no need to.
All the information was there in the question pointing to some definitions of “equivalent” making far more sense than others. Many people missed that because they did not listen to the video carefully or, in some cases, at all. This is clear from the posts we’ve seen in this thread and the comments under the video itself.
Other people misunderstand what is being asked because of the ambiguity of the language used. I have no problem with admitting that.
It is bad communication, something that we see in real life all the time. A point I (and others) have been making right the way through the thread, multiple times.
There have been back and forth discussions in which I have stated the importance of clarity and the need to check what you are actually being asked and restating info in clear and unambiguous terms. You may have missed those, dare I say (without making it personal) that you haven’t read my posts carefully?
As for “inspecting my own assumptions” as far as I know, only one of us in the thread has contacted the makers of the video to find out whether the ambiguity in the setup is indeed part of the purpose of the question. Also, to know more about how the question was used in a formal study situation by Frederick.
If I find out that it was messed up in the delivery or completely misrepresented in its purpose then I will gladly say so.
I think it is both, there is the ambiguity in the question form and a very strange ratio of major/minor that give people cues and a sporting chance to say, “hang on, this is a weird situation, there isn’t an easy answer”.
But that would only happen if they stop, think deeper and consider what they’ve been told. But then there are the lovely simple numbers which are just begging to have an space filled in. And people think it must need filling in, and people do fill in the space with a specific number when really it needs a big fat question mark.
I think this question shows both that ambiguous wording and a lack of clarity can lead to people answering a question other than that actually being asked, and that temptingly simple calculations can prompt them to do so in predictable ways.
That certainly seems to be what that abstract suggests, it also implies that the majority of incorrect answers are also given without any reflection (61%:39%). That to me seems like the greatest area of interest.
Especially if we assume that the correct/incorrect ratio mirrors previous studies of CRT. If that is the case then that 61% could be approaching half of the total sample. (sadly we don’t have the numbers though)
Thanks for the cite though, that’s an interesting summary, be good to get hold of the full paper.
I apologise, I’m not sure what you are asking me to bet on here, I’m no gambler.
Is it along the lines of
“a random person takes the CRT question, they did no reflection on their answer what are the odds that they get the question incorrect?”
I don’t think I’m going to be more effective in trying to explain than I already have, so I’ll just say again that to me it’s obvious that the type of logic/math trap clearly exemplified by the other questions in the video (bat and ball, lilies on a pond) is not clearly exemplified by the roadway question as presented.
And I’ll let others who are better at the back and forth on things like this continue the discussion
Your reasoning was clear, I just disagree somewhat, opinions will vary on that, I think we’ve both explained why we think what we think so yes, no need for either of us to revisit it.
Watching the video one would conclude that errors get made because people fail to reflect on their first answer. Based on that one would think that if someone failed to reflect they would be more likely than average to have an incorrect answer, and someone who did reflect would be more likely than average to be correct. Reflection is important to avoiding errors.
But the opposite is true.
Those who reflected are more likely to be in the incorrect group than those who did not reflect. Those who failed to reflect are more likely to be in the correct group than those who did.
I certainly agree with that. If anything, the roadway question is closer in spirit (though still not the same as) the example @Chronos gave, where some people are apparently conditioned to approach it as the kind of math problem that is solved by taking the numbers given in the problem and performing a relatively simple calculation on them.
“proportionate” and “equivalent” does not mean the same thing, although many ppl use the terms interchangeably. 8 would be the correct answer if the task was to find the number of other crimes to make both countries’ murder and other crime rates proportionate to each other.
