Filed under: Academia, Epistemology, Manifesto, Philosophy, Science | Tags: peer review, scientific method
You often hear people responding to the claims of homeopathists and the like with a request to see the peer-reviewed literature supporting their claims. The idea is that peer review is a unique characteristic of proper science that is essential to make science work. I recently read a history of peer review in science (subscription to Trends in biotechnology is unfortunately required) that throws some doubt on this. Fascinatingly, peer review didn’t really take off until 1959 with the invention of the photocopier: before then, it was just too expensive to make copies of the papers to send out for peer review. There was something like peer review before then, in that the editor would ask the opinions of friends and colleagues, but this was far from the standard we have for peer review today. A few journals would make the effort to make copies of papers and send them out for review, but it was rare.
Since science before 1959 was, in fact, extremely successful, and able to develop very well, it cannot be the case that peer review is an essential component of science. That’s not to say we should do away with it: it serves an important function today, essentially that of reducing the workload of the editor. So many papers are submitted for publication that some method for choosing which papers should and shouldn’t be published is necessary, and the reviews help the editor make that decision. What the exact function of peer review today ought to be, and how it should evolve, was the subject of an interesting debate in Nature in 2006 (no subscription needed for this one, I think).
So the people demanding peer-reviewed papers from the homeopathists should perhaps question what it is exactly that they’re doing. My guess is that they use it as an easy, but wrong way to distinguish science from pseudoscience. It makes for a good quip. The danger, though, as argued in this article, is that in fact it’s easy to cherry-pick bad ‘scientific’ papers that have passed the peer-review process. By using peer review as the ‘gold standard’ of scientific enquiry, they allow the justification of all sorts of nonsense.
This poses a problem though: if it’s not peer-review, what is it that makes science work? There have been many theories about this, most focussing on methodological aspects of scientific enquiry, for example Popper’s falsifiability criterion. I want to suggest an alternative point of view that doesn’t focus on methodology:
Science is the study of problems that can be addressed with the techniques available to us.
Techniques here should be understood to include both technology and, for example, mathematical or statistical tools. In this view then, physics is the ‘hardest’ science because the problems it looks at are the easiest. The variables in physical problems are much less interdependent, there are many fewer of them, and their interactions are much simpler than those in, for example, the study of social systems or economics. Our simple, mostly linear mathematical techniques and statistical methods based on independent variables then apply very nicely to physical problems.
Why does science work in this view then? The answer is that science is part of a social process: scientists do science, they are genuinely interested in finding out what happens, and even though individual scientists will sometimes be very wrong, will try to promote a bad theory that doesn’t have much support, or will attack a rival theory, eventually the egotistical motives involved will fade whereas the usefulness of good theories will last. This process may take a generation before change happens, but it eventually works. This process relies on there being at least some bias in favour of good theories over bad ones, even if the bias is small in comparison to the egotistic biases in favour of established theories. However, if available techniques are not good enough to introduce this bias, the good theories won’t win out over the bad ones, even in the long run. Thus, in unscientific subjects we shouldn’t be surprised to see cyclical variations in theories, where the period of the cycle is related to the duration of a scientific career. Young researchers will look at the work of the old researchers with disdain and propose radical alternatives, to which they will become attached. In turn, their theories will be rejected by those that come after them, and so on without ever stabilising.
In a way this is obvious: if there is no evidence to support or reject theories, then we cannot improve them. But it’s important to note that whether there is or is not any evidence to support or reject theories is to a large extent a function of how easy the problem being studied is. The fact that evidence in physics is so much stronger than it is in economics is hardly a reason for scorn of economists.
This point of view on why science works may help scientists look at researchers in other fields both more humbly and more reasonably. Humbly, in that they should realise that the reason their field of study is so advanced is that the problems are so easy, and reasonably, in that it should help them to see more accurately why it is that some fields make better progress than others, without getting distracted by methodology.