One week in October a rather alarming issue of the Economist was brought to my door, bearing the headline: “How science goes wrong.” The accompanying leader article made a rather compelling case for reform in the way that science is done. What the article failed to point out, however, is that reform is already well underway.
The Economist’s basic arguments are that science has become complacent, and is no longer self-correcting. People are publishing in a hurry, and the peer review process is no longer fit for purpose. The reality is not so simple. What’s happening is that the nature, scale and pace of science is changing more rapidly than ever before. We shouldn’t be surprised that the time-honoured procedures that have served science well for centuries have to evolve, and they are. Experimental particle physics provides a good case study.
Let’s take a look at academic publishing. The reality of life for a young researcher in a modern particle physics experiment is that time scales are long. At Cern, for example, whole cohorts of doctoral students have graduated through work undertaken on the Large Hadron Collider experiments before those experiments have even started to take data. And even now that we’ve got data, it’s not necessarily the publication record that will make one name from a collaboration of 3,000 stand out. These experiments have therefore had to find other ways of recognising excellence, and they do so by using factors such as a student’s publication record internal to the collaboration, or activity in working groups. Particle physics may be ahead of the curve here, but in many other areas of science, teams are also getting larger.
What of the peer review process? It’s not perfect, but there’s no better way to critically examine a piece of work than to have it dissected by experts. In many areas of science, post-publication peer review is complementing traditional pre-print peer review through submission of articles to pre-print servers. Everything that comes out of our big collaborations at Cern is internally peer reviewed before submission to traditional peer reviewed journals, and everything is also posted on a pre-print server, where its success is determined by post-publication peer review. That’s a high level of scrutiny.
One very important case made by the Economist is the value of a negative result, and here I could…