This is a slightly edited (amended, essentially) version of my article published today at The Conversation.
In cases where a problem within a community is detected and collective action is required to address the problem. one needs to strike a fine line or any efforts to convince the community that action is required will fail. If one describes the problem too cautiously, the message will be ineffective in rallying the community and collective action will not happen. Describe the problem too drastically, and the community will not be convinced by the alarmist rhetoric and also fail to take action.
Here are some data, together with a computed trend that indicates what I think would be best described as a looming crisis for the scientific community: not yet urgent, but if no action is taken, the trend will threaten the entire scientific enterprise within the coming 30 years.
Currently, the number of scientific papers retracted from PubMed is a mere 0.05%, an astonishingly low rate, given the vagaries of the job and the common .05 statistical significance level in biomedicine. However, recently this rate has been rising. The rise is exponential such that, if the trend were to continue, by about 2045 almost all scientific publications will have to be retracted. Apart from high-profile fraud cases which since recently also make it into the general news media, the numbers themselves are not yet alarming. Extrapolating the current trend, we will not reach the .05-level, i.e. 5% of all published articles in a given year will be retracted, before 2033 – it will take almost twenty years for this hundredfold increase, i.e. doubling about every three years. If the current trend were to continue that far into the future, 10% would be reached about 2036, 20% in 2039, 40% in 2042, 80% in 2045 and soon thereafter, for every new publication in PubMed, at least one older one would have to be retracted.
Causes
Usually, any problem at this scale will have a number of underlying causes. Science, like any enterprise, is a complex affair. In this special case, however, a large part of these causes can be traced to the reward structure that scientists have built around them to make this enterprise work. They are related to how scientists choose to publish their results and how fellow scientists reward this publishing behavior.
Unfortunately, even with a clearly identifiable cause, in this case there is no easy fix.
History
Starting in the 1960s, the rising number of scholarly journals increased subscription costs for libraries too quickly. Institutional libraries and university faculty were faced with the problem of deciding which journals to read and subscribe, keeping their budgets in mind. One solution, it seemed, was to rank journals according to how many citations each of its publication garnered on average.
Back then only a few realized that a generation later their future colleagues would not only adjust their reading habits to journal rank, but that those reading habits will translate into new publishing habits. After all, every one wants their work to be read by as many as possible.
Today, where you publish matters more than what you publish. This behavior may vary from country to country and field to field, but it is quite widely known that tenure, promotion, grant funding, etc. all depends on where in the hierarchy of journals one’s research results have been published.
What makes it worse is this feedback loop: more citations means more attention. More attention more citations. As long as space is scarce, more manuscript submissions mean higher rejection rates, for some journals the rejection rates can be as high as 90%. Like long lines in front of hip clubs, higher rejection rates mean more validation for the few researchers that actually manage to get published in these journals and hence more citations.
Publishers
Not surprisingly, as researchers adjusted their behavior, so did publishers. The top journals did not increase space for the increased demand. Instead, they spawned ever more lower ranking offspring to reinforce the ratchet and keep the demand high.
With the arrival of the internet, publishers went through a complete reversal of their role in knowledge dissemination. Before the internet, publishers would heed their profession, investing in printing presses and delivery systems for a wide and efficient system of dissemination to the public. Today, academic publishers have become the opposite. These corporations are now “hiders”: they invest in paywalls, hiding the research from the public that paid for it.
But perhaps this is all fine and well? After all, publishers select the very best science using professional editors and need to protect their investment.
Rankings
Superficially, one may assume that such strong selection would only enable the most groundbreaking, methodologically sound research to clear such monumental hurdles. However, the data speak a different language. While indeed quantifications of ‘novelty’ and ‘importance’ have been found to correlate significantly with journal rank, the correlation is so weak, that for every publication correctly published in a high-ranking journal, there is one that should not have been published there. Conversely, such a large percentage of research findings that did not manage to get published in a top journal end up being recognized as deserving after being published in a lower journal, that the practical value of journal rank as an evaluation signal is negligible, despite the statistics. Even more damaging, however, quality-related metrics such as technical quality, methodological soundness, reproducibility and so on either failed to show any correlation with journal rank, or correlate negatively. These data support the observation of decreasing intervals between prominent scientific fraud scandals and the increasing range and scope of each case, mostly based on fraudulent work published in top journals.
Marketing
Apparently, as the pressure mounts to publish in top journals, authors race to find a way to present their work as important and groundbreaking, no matter the actual content and quality. Science today has become a rat race between authors marketing their research and professional editors struggling to see through the fluff. It is the perfect arrangement to bring our the worst in all participants: scientists fear for their livelihoods and do what is necessary to survive, with little regard to science. Publishers fear their business model and do what is necessary to survive, with little regard to science. And librarians are caught in the middle.
Multiplicators
This straightforward, evidence-based analysis of the current state of affairs suggests a slightly more alarmist suspicion. Exponential effects are typical for social phenomena, where feedback is driving much of the dynamics of the system. The already described, relatively fast feedback loop propped up by journal rank supports a second, slower loop: after a generation of increasing pressure to publish in top journals, many of the now leading scientists are excellent at marketing their research to top journals. After all, this is how they were selected. It may of course happen that these researchers are also excellent scientists, but this was often enough not the selection criterion. These scientists are now training the next generation of scientists in how to be successful in science. The anecdotal evidence suggests that – intentional or not – marketing one’s research to top journals may play a dominant role in this training. If this were indeed the case, it would help explain the exponentially increasing retraction rates in the scientific literature and constitute cause for grave concern with respect to the looming crisis in science.
Reform
Of course, laziness, vanity, ambition, desperation, hubris, self-aggrandizement, sleep deprivation or bad time management are not going to disappear any time soon. Scientists are human too. The realistic task can thus not be to altogether eliminate shoddy, unreliable science, but to reform our infrastructure which currently almost appears to be purposefully designed to exacerbate aforementioned traits and bring out the worst in scientists, thereby wasting resources and threatening lives. Scientific discoveries are unreliable enough even under the best of circumstances, given the complexity of the matter, there surely is no need to worsen the situation. Instead, we must design an infrastructure that brings out the best in scientists and aligns their incentives with that of science and the public.
Technicalities
Given the current status quo, this is technically rather straightforward: one has to remove the incentives for scientists that encourage and reward marketing at the expense of science and instead encourage and reward actions of scientists that benefit both science and scientists. Given that scholarly communication is at the heart of the scientific endeavor, this is where we should place most emphasis in our infrastructure reform. What we need is to abandon the traditional idea of journal publishing altogether, in favor of an institution-based platform that allows every tax-payer, scientist or not, to access all the literature, data and software that is being paid for out of the public purse. For the past two decades, corporate publishers have proven to have become exceedingly poor custodians of the public good that is research results. From their continued and often shamefully backstabbing and cunning resistance to change their business model to provide even simple read-only access to the scientific literature to those who paid for the research in the first place, to the current prohibition of text- and data-mining access for scientists to their own literature, corporate publishers have proven beyond the shadow of a doubt that they have forfeited every right to be perceived as partners of scientists. Instead, the evidence weighs heavy that many if not most of these corporations are now enemies of science.
Solutions
Small versions of such open access platforms exist in many university libraries today, in various forms. They need to be combined and made interoperable using common standards. In less fortunate parts of the world, where research billions are not as abundant, governments felt forced, long before us, to realize how wastefully corporate academic publishing treats tax-funds. These countries have very successfully been running a multi-national platform, SciELO, which, if implemented on a global scale and funded with only a single digit fraction of what corporate publishing costs today, would eliminate the pernicious incentives currently in place and provide full open access to the public. Recent initiatives like SHARE, the DPLA or the LPC could serve as nucleation points for such developments in the rich countries. Provided that the reputation system built into this platform is guided by the same evidence-based reasoning and tested with the same scientific rigor we commonly apply to our experiments, this kind of reform would not only come for free, it would save billions in tax-funds every year and, by virtue of its public accessibility, literally save countless lives.
It is a collective disgrace for the entire scientific community that saved money and saved lives are apparently not sufficient incentives to drive forceful and rationally designed reform right now. Instead, as in many other communities, vested, not exclusively corporate, interests are still keeping an upper hand.
This must change.
