My boss emailed the contractor and told him that he needed to address my concerns ASAP. If that doesn’t scream, “YOU HAVE MY FULL SUPPORT, MONSTRO”, I don’t know what would.
Oh, crap. Is this an older researcher? Possibly one trained before more statistical rigor was expected of biologists?
He wouldn’t be the only one out there trying to skim by without a deep understanding of biostatistics, but that’s not a good sign of his statistical savvy.
He’s not old. Maybe early 50s? I know he doesn’t know SAS or R. I once asked if he knew anything about pivot tables in Excel and he looked at me like I was crazy. But I guess I just assumed that he was kinda old school. Like my grad advisor, who used to perform Chi-Square contigency tests on scratch paper. She wasn’t a stats guru either. But she’d be able to defend her work.
I don’t know why he hired him. There are so many hungry young biologists out there who’d die for this opportunity, who’d work their asses off. It’s a shame.
Then I don’t see how you can be held responsible for the deliverables. If your boss has refused to share it, you can’t be expected to hold the contractor to it.
He hasn’t refused to share it. I’ve just never asked to see it, because the thought has never occurred to me that we had to tell this guy how to be a scientist.
If the contractor felt like I was making demands above and beyond what’s laid out in his contract, he could have said so at any time. But he has said nothing.
Because they don’t have the experience he has, even though they are trained more recently and should know how to use newer tools available to them. :rolleyes:
Said as someone who almost didn’t get her current job because they were eyeing someone who had been doing her job since before she was born, despite I having the same degree (PhD) and a specialty certification he didn’t have (which proved that my knowledge at least was sound, somewhat vast, and more recently acquired than his).
Being able to perform Chi-Square tests (in fact, knowing what those are) already puts her waaaaaay ahead of the stats-knowledge curve IME. One of my pet peeves is people whose knowledge of continuous probability functions is limited to “Gauss’ bell” (don’t even ask them about its cummulative version) and who do things such as apply it on n=3 :smack:
If this consultant is a published author, then he knows his way around the mostly universally required “methods and materials” component of many peer reviewed journal articles. And any analyst can organically track and document an activity as it progresses. A sufficiently detailed process will engage an analytic plan prescribing methods and data management before any data is collected or analyses performed.
If pressed on their “analytic plan prescribing methods and data management”, many academic biologists (even tenured ones!) will say:
"Uhh, I’m going to copy my observations from my lab notebook into an Excel File, and do some T-tests I guess. Or maybe another kind of test, I dunno? What’s a heterosid… heteroscat… heteroschedastic, anyhow?
Backups? Is it OK if I just copy that excel file to the 10-year-old external hard drive full of data from the lab?"
I’m afraid you have this guy pegged 100%.
“I know you told me to use a geometric mean, but I’m gonna use an arithmetic mean anyway cuz I just like it better. Don’t ask me why cuz I don’t know. I’m just going to delude myself into thinking they are completely interchangeable and that they point to the same inferences. Because I’m an idiot who is in way over his head.”
We still haven’t heard back from him. So I feel entitled to make fun of his simple ass.
See how you really need to do things is report a linear regression on three points that form a triangle and then argue with me when I tell you to get back in the lab. Because that’s how you did things at Caltech. Then you can be a true pro scientist.
As an academic biologist myself, who interacts with a whole lot of academic biologists, I absolutely agree with this comment.
I know I’m hopeless with statistics, so every time I need anything more complicated than a chi-squared, I run down to the statistics core facility. At least at our institution, it’s a free service and they’re fantastic about both doing the analysis and explaining it to me so that I can have confidence in the numbers, too. (And they give me method details! And I put them in the eventual paper!)
Unfortunately, the guy still has not ponied up the methods.
We’ve told him multiple times that he will not be paid until he does. And still, crickets.
I don’t know what we’re going to do.
Any way your lab can get out of the contract with this person and contract someone else?
Glad to hear your boss and your department is firmly on the right side.
On the other hand, it’s disappointing and very suspicious that the scientist can’t come up with even the most rudimentary description of statistical methods. Clearly if he had his shit together he’d be able to say, off the top of his head, “I did this specific statistical test on that data.”
If he was sloppy but still did everything mostly correctly, it should only be a matter of a few hours of digging through files or notebooks (or asking a grad student to do the same) to figure it all out…
I agree that you should try to get out of the contract if at all possible. A technical report that purports to describe statistical results without detailing the methodology is essentially fraudulent.
The guy in question is the chair of an advisory panel my agency has organized. To have someone re-do his work would be effectively kicking him off of the committee. Which I don’t have a problem doing, personally.
But it wouldn’t be without consequence. He’s gotten a lot of favorable press over the past few years because of his involvement with this study. Rightly or wrongly, he has made himself the public spokesperson of the whole production. We will not be able to fire him without people getting all up in our business to find out why.
I have no interest in ruining this guy’s reputation in the public arena. And frankly, it will be embarrassing for us to have to admit we hired an incompetent to be our expert scientist.
We also have deadlines. The report was already several months late.
I just wish I knew WTF was going on.
He can’t/won’t provide methods because the data, and the “statistics”, are probably cooked. I still stand by that hunch.
Unless he can show the stat info for at least some of what he has done for you, this may blow up in your faces because it can be said that this type of error should have been spotted long before. Cutting him loose and being embarrassed might be better than keeping him on and purposefully staying in the middle of this house of cards.
If this is stuff you want to publish in a peer-reviewed journal, they may likely demand his methodology before publishing or accepting the paper. Another reason he should give it.