If we've all become armchair epidemiologists, now more than ever we should be challenging our preconceptionsby Michael Blastland / May 13, 2020 / Leave a comment
“Two thirds of people hospitalised with Covid-19 in New York have been socially distancing.”
Maybe we’re thrown by his nugget of news, maybe we’re delighted, but we’re all paying attention:
“Is this true?’’
It is. The mental gears turn…
“OMG, how could that be happening?”
…to a terminal conclusion:
“Social distancing doesn’t work!”
Naturally, there’s a “but.” In truth, you knew there would be. Whether that “but” is welcome will depend on your prior views. If you instinctively bristle against social distancing, any “but” is awkward, muddying your newfound “proof” that it doesn’t work. If you approve of social distancing, you’ll be relieved to find that the claim isn’t all it seems.
The difficulty in a raucous environment is how to make sure the cautious “but” is heard over the OMG conclusion. If we’ve all become armchair epidemiologists now, this is the vital, but often overlooked part of our education: to learn how downright uncomfortable the armchair ought to be. If you’re not struggling, you’re doing it wrong. It’s a lesson that real epidemiology—all expertise in fact—must also continually relearn.
We want to know everything, urgently. We want the newest and latest. We want it especially if it proves us right. And—here’s the twist—so do they! Ego and hope are plentiful in science too. Questioning the evidence always takes patience. Questioning evidence that we like takes steel.
Research circles wrestle with exactly this dilemma: how to make sure the “buts” don’t get elbowed aside in the urgency to tell the world the latest finding, especially if it’s a finding they like, such as: “have you seen how effective our new drug is?”
Daniël Lakens, a statistician, is a case in point. He recently began the experiment of actually paying a red team to challenge his own work, positively inviting “buts” at every stage. As he wrote in the journal Nature earlier this week about Covid-19 research: “Finding ways to prove ourselves wrong is a scientific ideal, but it is rarely scientific practice. Openness to critiques is nowhere near as widespread as researchers like to think.” The best time for scrutiny, he adds, is before you have fallen in love with your results.
In similar vein, John Krakauer, professor of Neuroscience at Johns Hopkins University School of Medicine, says of the work in his lab: “I encourage an…