Reports of the death of self-correction have been greatly exaggerated.
The common narrative of self-correction is that, in the beginning, mistakes are made, such as errors in analysis, incorrect or incomplete theories, or outright fraud; yet over time, these errors are discovered, fixed, and thus science inches ever closer to truth. But many scholars have grown critical of this narrative, pointing to the ongoing replication crisis and arguing that science is not necessarily self-correcting at all. Fortunately, in response to this crisis, steps are being taken to try and fix science, with tools like pre-registration, new approaches to statistical analysis, and more. Time will tell whether these changes will have a positive effect, but there are early signs of progress.
I think this narrative is wrong. Science was self-correcting all along. More than that, imaging science as broken is misleading, because is pre-supposes ways to “fix” science, when the reality is much more complicated.
My argument for why science was always self-correcting is pretty simple. There were problems in science, and scientists (the critics) pointed them out, and now problems are being resolved. So those critics, but pointing out issues in science, proved themselves wrong.
The common narrative of self-correction implies that scientific communities naturally police themselves. I don’t think this view is wrong, but I do think it is incomplete. It isn’t that self-policing is distributed across communities, but rather that certain members of scientific communities become the police. Some, like John Ioannidis, Elizabeth Bik, Guillaume Cabanac, and others, independently adopt the role of identifying errors and issues within their scientific fields (or science as a whole), and calling attention to them. Scholars might even be rewarded for adopting this role, with more citations, fame, and respect.
Science was self-correcting all along. It just took a while for some individuals to naturally adopt the role of “police” in response to ongoing issues. They may have done so out of their own ideals, but they may also be rewarded for their efforts, in the form of new fame and access to a broad audience interested in scientific integrity. I see no reason why this process will stop, either. In the future, new issues in science will be discovered, some scholars will self-select into the “police”, and issues will be addressed.
So in a sense, the critics of self-correcting science prove themselves wrong, because it is their own actions that serve to correct science.
There are still massive problems in science. bad incentives promote flashy yet poorly-evidenced findings over rigorous research and integrity. Studies that fail to replicate are cited just as much, if not more than others. Journals are not held to account for retractions or bad science. Replications continue to fail. And yet, progress is being made, even if it is slow and uneven. Science has the capacity to identify problems and seek changes.
I think it is important to realize that science has always been self-correcting. To claim otherwise only contributes to a legitimacy crisis of science, leading both scholars and lay people to dismiss whole swathes of research out of hand. Moreover, starting from the position that science is not self-correcting may lead to policies that are counterproductive—quick schemes to “fix” science, or else measures that burden scientists, with little payoff.
Instead of worrying about the existence of self-correction, we should instead concern ourselves with more practical questions, such as how to improve the integrity of knowledge as its produced, or how to speed up and better reward self-correction. These are problems that require trade-offs, often requiring spending limited resources and taxing the time of scientists. Discussing these trade-offs, though, will contribute to a better kind of science.