Basically Kohn researches the available research and then draws conclusions from that, in addition he points out many fallacies in the claims made by proponents of “behaviorism” or whatever its called these days.
in Kohns research there is always (or nearly so anyway) a control group. in the linked study there is none, yet the researchers claim that the results of a study done on 2 boys without a control proves Kohn to be wrong.
My question is the op, is this simply confirmation bias (or something else) or is it actual sound science. I cannot see how this study proves anything, sure the boys did more math and got better at it but how can anyone claim it was the rewards that were the reason? wouldnt any kid do better who took part in a study like this where they were given special worksheets and things to do? how do we know it was the rewards and not the environment? that combined with the 2 week window (they came back 2 weeks later to see how the boys were doing) as the proof of long term positive effects of rewards just leaves a bad feeling about the validity of the work, and really scares me that people take this seriously.
Hmmm, I’ve got no particular expertise in that field, but…
The paper definitely set off my skeptical alarms. Two subjects observed over a two week period? That’s a nice little case study, but not a nice big conclusive pile of data. But the authors don’t claim much – their strongest statement is that their results are “inconsistent with warnings about use of token rewards to motivate children.” And they wrap up the paper with the usual “more experiments, please” sort of statement. I can’t really fault the authors – it’s a very limited sort of case study, but nothing in it seems like bad science. It’d be a great pilot study, to be followed by a bigger experiment…
But there’s no way a little experiment like that is enough to falsify an established body of evidence, or much of anything for that matter. It takes a lot of evidence from multiple sources to overthrow the doctrine in a particular discipline.
Not that I’m defending Kohn’s work… it could be just as bogus for all I know.
After thinking about what I wrote, I want to emphasize something. No single paper is enough to really prove a point. All scientific research, even at its best, is only provisional.
There are some very rare and astonishing papers that almost conclusively overturn a well-regarded theory. That sort of paper will earn a Nobel prize. Even with these most people will be skeptical, and only will be convinced by further papers confirming the new result. Or, they’ll stubbornly stick to their beliefs and the theory will only die out when its proponents retire.
With my (non expert!) reading of the paper by Mcginnis et al., I really can’t find any egregious errors in their work. It’s a very small sort of case study, but most scientific results are small and limited (even my own!). It’s not a conclusive result at all, certainly not conclusive enough to absolutely refute contradictory evidence from other papers. The authors may have been unconsciously trying to prove a particular idea, but that’s human nature and an occupational hazard for all scientists. Rigorous statistics and solid experimental design only ameliorate the biases that every scientists holds. In this paper, the authors readily admit how limited their findings are, and that’s part of what makes their work decent science, even though it has a very limited scope.
And, again, I have no idea of the strength of Kohn’s claims. If his conclusions come from several larger high-quality studies, then that’s more likely to be true than the claims of Mcginnis et al. with their single limited result. Are there more papers that contradict Kohn’s work?