Sep12

Scholarly societies partly to blame for post-truth age?

In: science politics • Tags: , , , ,

Reading the reactionary defense of the digital stone age in AAAS’ flagship magazine Science, I felt reminded of the now infamous “Make American Science Great Again” letter to Trump and all the other public statements by scholarly societies over the last 30 years on how this internet thing is a threat to their revenue and hence must be opposed.

The development of the internet by scholarly institutions should have been the opening bell for a broader thought process within scholarly societies of how this new technology may revolutionize scholarly discourse. After all, the purpose and mission of scholarly societies is the building and maintenance of communities and fields of study. Nobody was more aware of that than Henry Oldenburg when he founded the first scientific journal for his society in 1665, the “Philosophical Transactions of the Royal Society”. Granted, with the journal lagging the invention of movable type and the Gutenberg-type printing technology by about 200 years, Oldenburg wasn’t really taking advantage of what one would call modern technology, but his Royal Society was only founded in 1660, so he clearly reacted quickly to the needs of his members.

The same cannot be said of his successors in today’s many scholarly societies. Prioritizing revenue, it was not lost to them that technologies such as email and browsers would allow them to copy their mail and journal distributions onto a digital format, at huge savings. Judging from their public messaging since then, for the exact same financial reasons, they have refused to invest even the smallest amount of innovation or progressive thought into what broader opportunities internet technology might provide for their mission going forward. Instead, it looks as if all of that intellectual energy flowed into conserving outdated concepts and demonizing digital progress into a threat for every scholarly society’s revenue. For nearly three decades now, scholarly societies, collectively, appear to have been preoccupied with looking back, at the expense of looking forward. Not even after 2006, when the term “social media” should have provided an etymological prompt even for the dimmest of professional “society” administrators, was there a change of direction. Apparently, the best some of our societies can do these days is installing forum technology from the late 1990s and call it “community“.

Much has been written about the consequences this reactionary attitude has had on the public accessibility of the scholarship these societies have been publishing. Here, I would like to speculate on perhaps much more pernicious consequences.

Ever since the infamous “Flame Wars” in the Usenet/Newsgroups of the 1990s (and probably already before that), one aspect of online discourse had become obvious for nearly any user: online discourse can (and sometimes inevitably will) escalate rapidly into one that nobody would call ‘civil’ any more. Passionate debate is something scholars have been actively participating in and contributing to for centuries. From early rivalries of ‘gentleman’ scientists, via back-and-forth publications of journal articles, editorials or commentaries to later antagonism between authors, editors and reviewers, there have been numerous and varied occasions for learning how to pursue reasoned discourse in a productive, scholarly way. It is impossible to predict which direction online discourse would have taken, or which functionalities current social media would have implemented, had scholarly societies perceived online discourse as an opportunity for their mission rather than a threat to their revenue. With scholarly communication far from being perfect, it is also not clear whether academics ever were in any position to claim superior knowledge. However, it also seems unlikely that early, systematic and competent engagement by scholarly communities with the goal to facilitate scholarly discourse in a way that minimizes the chances of escalation and radicalization could never have changed the course of history in any conceivable way.

Looking back 30 years, it is difficult the escape the impression that civic discourse has become less civil. The road from alternative facts to online mobs to death threats and real world political violence and bloodshed has shortened significantly. Radicalization of large sections of voters and a more general drifting apart of political positions over time is not specific to the internet age, but it appears to be facilitated by social media giving marginal groups or ideas an audience they lacked before. The capability of bringing marginal ideas to a broader audience cuts both ways: ideas that modern society benefits from can be amplified just as much as those it thought it had better left in the dustbin of history. Scholarly debate is all about deciding which hypotheses and theories should be kept and pursued and which should be abandoned. Sifting facts from alternative facts and how to bring the former to prominence and the latter into obscurity is a problem that could have tackled 30 years ago, if the new internet had been a focus of thought for the societies who claim to exist to foster scholarly communication, rather than use scholars to generate revenue to pay their staff. In order to be productive, all scholarly discussions need to be as passionate as necessary to be engaging and thought-provoking, but also as civil as possible to not provoke anger and aggression. Individual personalities vary and this topic comes up in various online discussions time and again, but collectively it appears there is a fairly broad Goldilocks zone of scholarly discourse where most scholars feel welcome, comfortable and discussions are productive.

So while nobody can know how civic discourse would look today had scholarly societies leaned less to the green, I’d argue that at least now, 30 years later, would be a good time to stop, take pause and think hard if the luddite path really is the one scholarly societies want to keep marching on. Personally, I’d go even further: I blame the majority of our scholarly societies that in these past three decades they have put their revenue before their mission. This perverse prioritization has not only hampered scholarly communication in general and the accessibility of scholarly articles specifically, it has also caused the scholarly community to miss a window of opportunity where it could have had an outsize influence on civic society at large by influencing the means and rules of communication. Some may argue that this paints an exaggerated picture of an actually very limited influence of the scholarly community beyond academia, but given the role these institutions played in developing the internet, I fear such arguments may be motivated by the desire to deflect responsibility. It is my impression that the collective digital torpor of academia, especially after it had implemented the internet, leaving the reins to corporations and political activists, is partly to blame for the way things are online today. In other words, is it possible that, if scholarly societies had not been so preoccupied with defending their revenue at all costs, they could have been able to assume a central role in the development of social technologies and help shape the way they operate?

This last aspect deserves a final paragraph. The large, influential societies have traditionally replied that their revenue is important either for their mission in general, or for their ECR services, their lobbyism or any of their other activities. Looking at the political situation in the leading scientific nations of the world today and beyond, it appears that very early investment of intellectual energy into the means and rules of public discourse and how to facilitate productive reasoning and rationality, at a time when they were still in their digital infancy, would possibly have accomplished more than all the dollars, euros and pounds poured into campaigns, political lobbyism or awareness weeks combined. As such, scholarly societies have not only failed their members and scholarship writ large, they have also missed a golden opportunity to maybe help make also the world outside of academia a little bit more evidence-based.

Posted on September 12, 2022 at 16:29 1 Comment
Jul08

Off to Paris for #FENS2022 with two posters

In: own data • Tags: , , , , ,

The first conference after the Sars CoV2 pandemic! We’re headed for Paris, France tomorrow and our lab will present two posters, the work of graduate student Andreas Ehweiner and postdoc Radostina Lyutova.

Andreas has been working on the cellular and genetic mechanisms underlying operant self-learning (a form of motor learning). On his poster, he presents evidence that one crucial site of plasticity for this type of motor learning is motor neurons of the fly ventral nerve cord co-expressing both FoxP and aPKC. Interestingly, FoxP expression in the brain seems to be dispensable for self-learning and knocking-out FoxP in all neurons in the adult only seems to show an effect on learning 14 days after the knock-out. Overexpressing a constitutively active form of aPKC improves motor learning, such that these flies still learn when training has been reduced to a level where wild-type flies no longer show a significant learning score. I’m particularly happy that we now seem to have found which of the five Drosophila PKC genes is involved in operant self-learning. We had been trying to identify the right PKC for a few years now. Surprisingly, it is not the same PKC as the one that is required for self-learning in Aplysia, for instance. Andreas is writing up his thesis at the moment, so he sure would appreciate job offers. Andreas is smart, resourceful and persistent. Here is his poster (link opens A0 PDF):

Radostina studies the regulation of self-learning by other forms of learning. Operant self-learning only requires eight minutes of training (see Andreas’ poster) when no other stimuli can be learned. If such stimuli are present, self-learning requires overtraining of 16 minutes. This procedure is reminiscent of habit formation in vertebrates. Radostina found that habit formation in flies is regulated by a single cell receiving input from a prominent insect neuropil, the mushroom body. If she inhibits this cell, flies show a habit already after eight minutes, suggesting that the inhibition of self-learning is mediated via this mushroom body output neuron (MBON 02). Connectome analysis revealed that MBON 02 input is provided mainly by Kenyon cells from the little-studied lateral and dorsal accessory calyx, which, in turn, receive thermosensory and visual input, respectively. This has been a massive effort over several years, clouded by a case of sabotage, that Radostina has picked up and breathed new life into. Here is her poster:

Posted on July 8, 2022 at 09:37 Comments Off on Off to Paris for #FENS2022 with two posters
Apr01

EU: academic publishers are monopolists

In: science politics • Tags: , ,

The market power of academic publishers has been a concern for all those academic fields where publication in scholarly journals is the norm. For most non-economist researchers, the anti-trust aspects of academic publishing are likely confusing and opaque.

For instance, libraries and consortia are exempted from organizing tenders for their publication needs as each article exists only in one journal with one publisher. This is called the single or sole source exemption from procurement law and essentially means that academic publishers have monopolies on each of their articles and hence each of their journals.

At the same time, this conglomerate of monopolies is often referred to as the “publishing market“, where there is market consolidation or concentration, leading up to an “oligopoly“.

So which is it now, a market with competing providers or a conglomerate of monopolists?

The distinction lies in the perspective taken. From the perspective of libraries and readers (the demand side), neither publishers nor journals can be substituted and so the publishers appear, effectively, as monopolists. From the perspective of publishers (the supply side), they offer content across disciplines and effectively compete for the money available in the sector. However, as the libraries and readers do not have any choice but strive to achieve maximal coverage of the literature, this competition does not depend on the quality of the products. Instead, publishers devise pricing strategies trying to obtain the largest piece of the pie. The notorious “Big Deal” bundles, where publishers aim to sell increasingly large journal packages, irrespective of the use of the journals for the institution, are an example of such a pricing strategy.

In a sports analogy, one could see a sector with demand-side substitutability as a gymnastics competition, where gymnasts (companies) aim to present sets that are more difficult and contain less mistakes (high quality products) than their competitors, in order to gain points (market share) from the judges (customers). An academic publishing sector with only supply side competition is more like a hot dog eating competition, where libraries and readers would be the hot dogs.

Any analogy fails at some point, so probably more fitting are statements from experts in the field. The European Union has, of course, been very active in this area. For instance, a crucial aspect of market regulation is merger control. In a document from 2001, Dr. Atilano Jorge Padilla reports to the European Commission on “The role of supply-side substitution in the definition of the relevant market in merger control” and writes:

There is a wide consensus among competition authorities, legal experts and economists
about the need to refer to demand-side substitutability for defining relevant markets. The
same unanimity, however, does not exist in connection with supply-side substitutability. On
the contrary, there seems to be substantial controversy as to the relevance of supply-side
constraints for market definition

This is consistent with subjective experience for any customer: the lack of choice is a defining aspect of planned economies as opposed to market-based economies where consumers always choose among competing products. It thus makes no sense to call a sector where there is no supply-side substitutability a ‘market’.

Since the EU common market aims to protect consumer choice, EC market reports are an excellent source for professional analyses of the academic publishing sector. Over a number of years now, various documents have been produced that corroborate the lack of demand-side substitutability and hence the characterization of the academic publishing sector as a conglomerate of monopolies, rather than a proper market.

Already when analyzing a merger case before the EC in 2003, the verdict was clear:

From a demand-side point of view, it is rare that two different publications be viewed as perfect substitutes. There usually are differences in the coverage, comprehensiveness and content provided by two different publications. From the point of view of functional interchangeability, two different publications could hardly be regarded as substitutable by the end-users, the readers.

or

Consumers will rarely substitute one publication for another in reaction to their relative prices. In this case, a strict demand approach would lead to the definition of a multitude of relevant markets of imprecise boundaries and small dimensions.

In a document entitled “Study on the economic and technical evolution of the scientific publication markets [sic] in Europe” the authors come to similar conclusions:

substitution possibilities across journals are limited, so that publishers do have significant market power.

or

Since researchers do not see the various publishers as good substitutes and need access to all good journals, consortia only introduce a relatively weak ‘buyer-power’ counterpart to the rising concentration in the publishing market.

Also in 2015, again on the occasion of a planned merger, the EC confirmed their earlier findings that

from a demand-side point of view, it is rare that two different publications can be viewed as perfect substitutes, as there are differences in the coverage, comprehensiveness and content provided. Therefore, in terms of functional interchangeability, two different publications could hardly be regarded as substitutable by the end-users, the readers. On that basis, the Commission found that consumers will rarely substitute one publication for another following a change in their relative prices and concluded that a strict demand approach would lead to the definition of a multitude of relevant markets of imprecise boundaries and small dimensions.

or

Publications in different academic subjects are indeed not substitutable from the readers’ perspective.

Taken together, over almost 20 years now, the EU has consistently come to the conclusion that academic publishing is not a market in the sense that it does not provide for customer choice (i.e., demand-side substitutability). All of these consistent documents and the fact that publishers obviously enjoy the single/sole source exemption, serve to fully justify the term ‘monopolists’ when speaking of academic publishers.

It is therefore not surprising, if now, after so many decades of established fact, scholarly organizations such as the German Council for the Sciences and Humanities also come to the conclusion that

academic publications are a unique, non-substitutable commodity. […] a journal title can give a publisher a non-competitive market position […] Functioning, competition-driven market structures do not exist

Inasmuch as current developments may serve to, one day, establish demand-side substitutability, e.g., by transforming academic publishing from a content-based to a service-based sector in the future, it is paramount to emphasize that this must entail that the single source exemption must be dropped and be replaced by proper tender processes, as is standard operating procedure required by law for all other, digital or non-digital products and services where demand-side substitutability exists.

Posted on April 1, 2022 at 15:06 2 Comments
Mar31

Scholarship has no time to waste

In: science politics • Tags: , , , ,

Academia is under attack from two angles, which seems to suggest that we may not have decades to get our house in order.

The first and older of this two-pronged attack comes from politics. Around the world, anti-science movements seek to discredit reason and abolish science. Be it abolishing tenure, doubting the value of publicly funded science or forcing entire institutions to close, authoritarian politicians around the globe strive to stifle and subjugate academia. Creationists, anti-vaxxers, climate-deniers or anti-vivisectionists are supporting and playing this political game, flush with funds from special interests, broadening the political attack on reason. In some cases, the attacks are explicitly launched referencing unreliable science, others come from different directions. Anything that can be used against scholarship will be used. In addition to these focused attacks on science and often scientists, there is a more diffuse attack on facts, aiming to undermine public trust in institutions, with the goal to increase the effectiveness of campaigns designed to promote the personal cult around certain individuals: when facts do not matter any more in what has been called a post-factual world, opinions and feeling remain as the only currency in public debate. The consequences of such long-term efforts can be seen in neoliberal policies pushing the corporatization of universities, entangling scholars around the world in precarious working conditions, suffering from hypercompetition and avoiding anything that could be seen as risky. Increased bureaucracy imposing numbers games combine with tiny salaries and huge workloads to create the vulnerable academic, caught in the iron fists of feudal PIs and professional administrators. The degree to which academics must endure such circumstances varies dramatically between institutions and countries, but as a global phenomenon, they stifle innovation, critique and promote not only questionable research practices, but the active dissemination of “hype and hyperbole“: misinformation, breeding distrust.

A second front was opened about ten years ago now from an entirely different and mostly unanticipated direction. More than just flush with funds, but this time financed by academia herself, academic publishers started (escalated?) their own attack on science by gobbling up and developing digital surveillance technologies. To expand the sources of user data, these corporations bought digital tools covering all aspects of academic life, from literature search, data analysis, writing, citing or outreach, all the way to citation analysis for research assessment. These corporations formerly known as publishers are using their expanded digital surveillance network to accomplish two separate goals. First, a copy of the data is aggregated with private data from scholarly users and sold, either to advertisers, to law enforcement agencies not allowed to collect such intrusive data themselves, or to any authoritarian government interested in identifying potential opposition intelligentsia. The second goal is to expand the monopolies they enjoy on scholarly content, to a monopoly on all scholarly services, i.e., the mother of all vendor lock-ins. Packaging all the different tools in a single bundle and selling it to institutions akin to subscription “Big Deals”, would make it impossible for any institution buying such a package to ever switch to a different provider again. An analogy outside of academia would be a merger of Microsoft, SAP, Google and Facebook. There are two corporations so far that are standing ready to deploy such bundles, RELX (parent of Elsevier) and Holtzbrinck (SpringerNature, Digital Science). A related data analytics corporation specializing on scholarly data is Clarivate (Web of Science, ProQuest).

Both onslaughts aim to undermine independent scholarship and subjugate it for special interests, be it political or financial. To some extent, both have been quite successful already. In fact, in the use of journal rank and other citation metrics, the political and financial fronts have closed ranks and are cooperating. The worst outcome of succumbing to these attacks would be the destruction of publicly funded science. At best, loosing on both fronts would entail academia finding itself permanently strapped in neoliberal purgatory, with a vast precariate, cut-throat competition and results nobody can take seriously any more, in other words: Idiocracy.

It is starting to become clear that to defend against these attacks with their multi-faceted consequences all over academia, it will take swift and decisive counter-measures, both within academia and in cooperation with initiatives outside. Inside academia, first and foremost, open flanks must be closed to allow the enemy only the smallest attack surface possible. Productivity and impact metrics have long been known to be both flawed and counter-productive in that they tend to reward unreliable science. Because this has been known for such a long time, calls for a reform of the academic reward system are old and have recently redoubled. Such calls are, of course warranted, justified and appropriate. However, implementation of a replacement may come too late, if these calls are not supplemented with actions that operate on much smaller timescales.

With research assessment and peer-review taking place on all levels from hiring and tenure decisions, promotion, funding and renewal, nearly all scholars are involved, either as participants or as evaluators, often in rotating roles over time. This means that grassroots movements striving for a change in the reward system face the challenge of winning the hearts and minds of every single researcher on the planet. Just how big is this challenge? Citing OECD statistics, the 2018 STM report lists about 7.1 million full-time equivalent researchers globally. Given that many of these will be part time employees and sites like ResearchGate list about 17 million users, adding at least 50% to these 7 million or a total of about 11 million currently active, individual researchers is probably required for a lower bound estimate. Convincing all of them to change how they assess science is not going to happen overnight.

How could one possibly increase the speed by which academia can replace its reward system? A straightforward approach would be to take away the means that prop up the current reward system, forcing academia to come up with a replacement. Ideally, one would want to replace the technical infrastructure that maintains the current reward system and replace it with one that not just necessitates, but facilitates the creation of a new reward system. There are several ways to do this. Last year, ten experts have proposed that regulators and funding agencies generate incentives for institutions to shift funds away from the journals upon which much of the current reward system is based, and towards a modern journal replacement that facilitates the development of novel reward solutions.

Such a replacement is capable of stopping the financial onslaught on science and providing the means to guard data against corporate greed. Inasmuch as the old reward system incentivized hype, hyperbole and questionable research practices, the replacement also helps stem the flow of irreproducible science. As such, academia has the power to set these processes in motion in self-defense. If academia has the collective will remains to be seen. To defend against the broader attack on facts, such actions can ever only be necessary prerequisites that need to be complemented by other initiatives engaging the broader public.

Posted on March 31, 2022 at 17:24 1 Comment
Mar30

Why publication services must not be negotiated

In: science politics • Tags: ,

or: how journals are like 1930s Rolls Royce Phantom IIs

Recently, the “German Science and Humanities Council” (Wissenschaftsrat) has issued their “Recommendations on the Transformation of Academic Publishing: Towards Open Access“. On page 33 they write that increasing the competition between publishers is an explicit goal of current transformative agreements:

publishers become publication service providers and enter into competition with other providers

This emphasis on competition refers back to the simple fact that as content (rather than service) providers, legacy publishers currently enjoy monopolies on their content, as, e.g., the European Commission has long recognized: In at least two market analyses, one dating as far back as 2003 and one from 2015, the EC acknowledges the lack of a genuine market due to the lack of substitutability:

it is rare that two different publications can be viewed as perfect substitutes, as there are differences in the coverage, comprehensiveness and content provided. Therefore, in terms of functional interchangeability, two different publications could hardly be regarded as substitutable by the end-users, the readers. On that basis, the Commission found that consumers will rarely substitute one publication for another following a change in their relative prices

or

Publications for different academic subjects are clearly not substitutable from the reader’s point of view. Even within a given discipline, there may be little demand side substitution from the point of view of the individual academic between different publications.

As this lack of substitutability is one of the main sources of the problems associated with academic publishing today, not just the German WR, but many initiatives around the globe see increased competition among publishers as key to moving forward.

As much as these aspects appear uncontroversial to the point that one can find similar wordings in many different texts on the topic, what seems to be lacking from many of these texts is one crucial aspect of the practical consequences of competition: procurement rules demand that tenders be held for products and services above a certain value.

Traditionally, the products of publishers have been exempt from such rules by the “single/sole source exemption”: because the journals sold by the publishers constitute monopolies, there is no substitutability, no competition and hence no way to organize a tender. All of this changes, of course, once the goal is to transform academic publishing into a market with substitutability and hence, competition, as per the public statements in the documents surrounding many “transformative agreements”. Once there is substitutability and the value of the transaction exceeds a given limit, procurement rules mandate tenders.

Heeding their assessment of academic publishing as a conglomerate of monopolies if it is based on content, rather than services, the EC opted for a tender when they became a customer with academic publishing needs. The result of this tender is “Open Research Europe” a platform where EU-funded researchers publish for free. Clearly, procurement rules would demand that all institutions follow the example of the EC and stop their negotiations with publishers and hold analogous tenders instead.

One may argue that authors ought to be able to choose their publication venue and I agree, of course. there should not be any restrictions on the choice of publication venue. However, arguably, this does not necessarily entail that the public purse must reimburse authors for even their most extravagant publication choices, if reasonable substitutes exist.

An analogy may help explain the argument: In many areas of science, transportation is needed for small groups of students and faculty to do field research. According to procurement rules, a tender would be organized and the award may, for instance, go to a company that sells electric vans, seating seven passengers and a driver, and offers a ten year warranty. Many would probably agree that the cost of some tens of thousands $/€ for each van is reasonable, given the functionality of the vans and their climate-friendly propulsion. Faculty, however, claim that because these vans do not carry enough prestige, each van must instead be replaced with eight 1930s Rolls Royce Phantom II, such as this one:

Without such prestige, the faculty argue, they cannot work, risk their careers and funding. Arguments that these ancient vehicles are unreliable, unaffordable and dysfunctional are brushed away by emphasizing that their academic freedom allows them to drive whatever vehicle they want to their field work. Moreover, they argue, the price of around one million is “very attractive” because of the prestige the money buys them.

With this analogy, it becomes clear why and how tenders protect the public interest against any individual interests. In this analogy, it is likely also clear that academic freedom does not and should not trump all other considerations. In this respect, I would consider the analogy very fitting and have always argued for such a balance of public and researcher interests: academic freedom does not automatically exempt academics from procurement rules.

Therefore, ten experts advocate a ban on all negotiations with publishers and, instead, advocate policies that ensure that all publication services for public academic institutions must be awarded by tender, analogous the the example set by Open Research Europe and analogous to how all other, non-digital infrastructure contracts are awarded.

Posted on March 30, 2022 at 16:54 2 Comments
Feb04

Replacing the prestige signal

In: science politics • Tags: , ,

tl;dr: Evidence suggests that the prestige signal in our current journals is noisy, expensive and flags unreliable science. There is a lack of evidence that the supposed filter function of prestigious journals is not just a biased random selection of already self-selected input material. As such, massive improvement along several variables can be expected from a more modern implementation of the prestige signal.

Some common responses to the proposal to replace the now more than 35,000 peer-reviewed scholarly journals with more modern solutions are “Why do you want to get rid of peer review?” or “How should we know what to read without journals?”.

Both, of course, imply insufficient engagement with the proposal and its consequences. As for the first question regarding peer-review, there is currently very little evidence as to the cost-effectiveness of peer-review and most comparisons between un-reviewed ‘preprint’ manuscripts and the published articles after peer-review, show few substantial differences. Given the huge costs associated with peer-review, one would expect peer-review effectiveness to increase dramatically by modernizing the way it is organized and deployed. As this conclusion and expectation is not new, the literature is brimming with suggestions for improving peer-review, just waiting to be tested. The proposal to replace journals specifically includes a more effective way to leverage the power of peer-review.

The second question implies two implicit assumptions, namely that the prestige inherent in journal rank carries a useful signal and that this signal is actually being commonly used when making decisions about which portion of the scholarly literature to read. Let’s see if there is evidence supporting these assumptions.

There is some evidence in the literature (review of a subsection) that there indeed is a signal conveying more ‘novel’ or ‘breakthrough’ discoveries in the more prestigious journals. However, this literature also emphasizes that the signal to noise ratio is very poor: it contains many false-positives (articles that turn out to be not particularly novel or ground-breaking) and false-negatives (many novel and ground-breaking discoveries are first published in lower ranking journals). Unfortunately, because of the nature of journal publishing, the full extent to which the false-negatives contribute the noise in the signal cannot be known, as it is commonly not known which submissions have been rejected by any given journal (more about that later). Adding insult to injury, this already noisy signal is then degraded even further by the reliability of the signal being no higher (in fact, slightly lower) than in average journals. This means that more of the published articles in prestigious journals, even those that are novel or signify a break-through, turn out to be irreproducible or contain more errors than articles in average journals. Finally, this weak signal is then bought at a price that exceeds the costs of producing such a scholarly article by about a factor of ten. Thus, in conclusion, the prestige signal is noisy, the science it flags is unreliable and the moneys it draws from the scholarly community exceed the cost of just producing the articles by almost an order of magnitude.

It may thus be of little surprise that one can find evidence that this noisy signal that tends to also flag unreliable science does not seem to be used as commonly by readers as it is sometimes claimed. Unfortunately, there is no direct evidence as to how readers use the scholarly literature, in part, again, because our literature is fragmented into 35,000 different journals. Therefore, currently, we are forced to restrict ourselves to citations as a proxy for reading: one can only cite what one has also read. Using the signal of journal rank (as routinely and reliably measured by impact factor), Lozano et al. predicted subsequent citations to published articles. In other words, the authors tested the assumption that journal rank signals to readers what they should read and, consequently, cite. When looking at it historically (figure below), this predictive power of journal rank starts surprisingly low, but statistically significant, due to the authors analyzing millions of articles.

figure

As the serials crisis starts to hit in the 1960s/70s and libraries are forced to use the impact factor to cancel subscriptions to low-ranking journals, this predictive power, not surprisingly, starts to increase, if ever so slightly: you really can only cite what you can read. The advent of keyword searches and online access then abolishes this modest increase and today, the predictive power of journal rank on citations/reading appears to be as low as it ever was. Given that citations to articles in prestigious journals are often used by authors to signal the importance of their topic, even these low numbers are probably overestimating the true effect of journal rank on reading decisions. So there seems to be an effect of prestige on citations/reading, but it is very weak, indicating that either only very few readers are using it or each reader is not applying it very stringently. In other words, the low predictive value of journal rank on future citations can be considered a reflection of users realizing that the publication venue contains little useful information when making choices about what to read.

Noticing an apparent disconnect between these citation data and the fact that journal rank is partly based on citations, Pedro Beltrao pointed to a different study, where high impact factor journals, on average, tend to be enriched in papers that will be highly cited in the future, comparing the slice of 1% most highly cited papers. However, these results only superficially seem to be in contradiction to the data presented by Lozano et al. In fact, I would argue that the 1% study actually supports the interpretation that journal rank is only very rarely used as a signal of what to read or cite: because there are more “novel” or “break through” papers in these journals (as detailed above), people will cite these rare 1% papers more often. But that effect hardly changes the probability of the remaining 99% of papers to be cited. These papers are ‘regular’ papers that could have been published anywhere. This evidence thus suggests that readers decide what they cite not based on containers, but based on content: the 1% of articles in prestigious journals that are actually novel and ground-breaking are cited more, and the other 99% are cited just as much as articles in any other journal. In fact, this insight is corroborated when comparing citation distributions between journals.

All of this evidence supports the interpretation that yes, there is a signal in journal rank, but it mainly comes from a very low percentage of the papers (that then, in turn, is less reliable than average) and hardly translates to the vast majority of the papers in prestigious journals at all: the effect is almost gone when you include all papers. Therefore, arguably, the little effect that remains and that can be detected, is likely not due to the selection of the editors or reviewers, but due to the selection of submission venue by the authors: they tend to send their most novel and break-through work to the most prestigious journals. The fact that what we think is our most novel and groundbreaking work is too often also our worst work in other aspects, probably explains why the reliability of publications in prestigious journals is so low: not even the professional editors and their prestigious reviewers are capable of weeding out the unreliability.

Taken together, despite the best efforts of the professional editors and best reviewers the planet has to offer, the input material that prestigious journals have to deal with, appears to be the dominant factor for any ‘novelty’ signal in the stream of publications coming from these journals. Looking at all articles, the effect of all this expensive editorial and reviewer work amounts to probably not much more than a slightly biased random selection, dominated largely by the input and to probably only a very small degree by the filter properties. In this perspective, editors and reviewers appear helplessly overtaxed, being tasked with a job that is humanly impossible to perform correctly in the antiquated way it is organized now.

Unfortunately, given the current implementation of the prestige signal by antiquated journals, it remains impossible to access the input material and test the interpretation above. If this interpretation of the evidence were correct, much would stand to be gained by replacing the current implementation of the ‘prestige signal’ with a more modern way to organize it.

How could a more modern system support a ‘prestige signal’ that would actually deserve the moniker? Obviously, if journals were to be replaced by a modern information infrastructure, only our imagination is the limit for which filters the scholarly community may want to implement. Some general ideas may help guide that brainstorming process: If the use of such ‘prestige’ not only in career advancement and funding, but also in the defense of the current system is anything to go by, there should be massive demand for a prestige signal that was worth its price. Today, this prestige arises from selectivity based on expertise (Nature’s slogan always was “the world’s best science”). This entails an expert-based filter that selects only very few (‘the best’) out of of the roughly 3 million peer-reviewed articles being published each year. Importantly, there is no a priori need to objectively specify and determine the criteria for this filter in advance. In a scenario after all journals had been replaced by a modern infrastructure for text, data and code, such services (maybe multiple services, competing for our subscriptions?) need only record not just the articles they selected (as now) but also those they explicitly did not select in addition to the rest that wasn’t even considered. Users (or the services themselves or both) would then be able to compute track records of such services according to criteria that are important to them.

Mimicking current implementations, e.g., the number of citations could be used to determine which service selected the most highly cited articles, how many it missed and which it falsely didn’t even consider. But why stop at bare citations? A modern infrastructure allows for plenty of different markers for scholarly quality. One could just as well use a (already existing) citation typology to differentiate between different types of citations, one could count replications, media mentions, anything, really, to derive track records by which these services may be compared. Given the plentiful demand indicated by the fervent supporters of prestigious journals, services would compete with each other using their track records for the subscriptions of individual and institutional users, providing for innovation at competitive prices, just like any other service market. Such an efficient, competitive marketplace of services, however, can ever only arise, if the current monopoly journals are replaced with a system that allows for such a market to be designed. If demand was not as high as expected, but such a signal nevertheless desired by some, a smaller, more basic implementation could be arranged on a non-profit, open source basis, funded by the vast savings that replacing journals would entail. One may also opt to hold competitions for such services, awarding prizes to the service that best serves the needs of the scholarly community. The possibilities are endless – but only once the scholarly community finds a way to put itself into a position where it has any power over the implementation of its infrastructure.

Perhaps most importantly, such a modern implementation of the prestige filter offers the promise of actual consequences arising from the strategies these prestige services employ. Today, publishing articles that later turn out to be flawed has virtually no tangible consequences for the already prestigious journals, just as a less prestigious journal can hardly increase its prestige by publishing stellar work. With power comes responsibility and our proposal would allow the scholarly community to hold such services to account, a feat virtually impossible to reach today.

P.S.: Much like the ideas for replacing journals have been around for decades, the ideas for replacing prestige signals have been discussed fro a long time. There is a post from Peter Suber from 2002, and there is one of my own from 2012.

Posted on February 4, 2022 at 13:08 10 Comments
Jan11

Small changes, big effects

In: science politics • Tags: , ,

Sometimes, only small changes are required for potentially huge downstream effects. Last September, ten international experts have identified two key decisions that may help academia to break out of the decades-long lock-in with academic publishers:

  1. Regulators no longer granting academic institutions the single-source exemption for publisher negotiations
  2. Funding agencies updating their eligibility criteria for institutions to include modern infrastructures.

EU regulators long-since recognize in principle that academic publishers are monopolies, i.e., they are not substitutable, justifying the single-source exception granted to academic institutions for their negotiations with academic publishers (another such negotiation round just recently concluded in the UK). Openly contradicting this justification for the single source exemption, the EU Commission nevertheless classifies academic publishing as a market and, moreover, demonstrates with Open Research Europe, that public, competitive tenders for publishing services are possible. This now offers the opportunity for the first decision: we propose that now is the time for regulators to no longer allow academic institutions to buy their publishing services from academic publishers that do not compete with one another in such tenders. The consequences would be far-reaching, but the most immediate ones would be that the (mostly secret and NDA-protected) negotiations between institutions and publishers, which allowed prices and profits to skyrocket in the last decades, would now be a thing of the past. Another consequence is that the obvious contradiction between academic publishing as a set of recognized monopolies in procurement regulation, but as a regular market in anti-trust regulation would be resolved. After this decision, academic publishing would be an actual market that could be regulated by authorities in pretty much the same way as any other market, preventing future lock-ins and monopolies. Yet another consequence would be that competitive pricing would reduce the costs for these institutions dramatically, by nearly 90% in the long term, amounting to about US$10 billion annually world-wide.

For as long as replacing journals or canceling subscriptions have been discussed, probably a decade or more now, a recurring question has always been how to ensure that any saved funds stay within infrastructure units such as libraries, where these moneys currently are administered. In this regard, we propose that funding agencies may cooperate globally to require eligible institutions to invest in such modern infrastructure, if they strive to remain eligible to receive funding. This proposal was already considered feasible and one consequence would be that institutions would face immediate requirements for investments for which the saved funds from the then illegal publisher negotiations may be a convenient source. Another consequence would be the de-funding of journals and their replacement with a superior technology. A combined consequence of the two changes would be that the scholarly infrastructure would be back under the ownership of the scholarly community, such that any mistakes, inefficiencies or new developments could be fixed or implemented without pleading to international corporations for their generous support. These ownership rights would also afford the scholarly communities to design their own reputation and assessment systems as well as to apply the scientific method for their improvement, continuous adjustment and development.

While I personally resent the concept of other people needing to help us cleaning up the mess we have created ourselves, these decisions from regulators and funding agencies, if implemented, would go a very long way to assisting the scholarly community to break free of the exploitative, parasitic lock-in we have been struggling against for at least 30 years now. In an ideal world, we academics ought to be capable of solving this collective action problem ourselves. However, I take our less-than-stellar track record over these last three decades as evidence that we do not live in such an ideal world and outside help is now the only practically feasible way out, no matter how resentful the thought of it may be. At the same time, this same, underwhelming track record suggests that we likely won’t be getting it right on the first try. But because we would then have control over our infrastructure, we can not only fix any mistakes, we can also hire experts to help us prevent future mistakes.

P.S.: Obviously, ‘small’ is relative. ‘Small’ is meant here in relation to the massive, global consequences these two suggested changes could potentially entail. “Disallowing the single source exemption” may be spelled in a 5-word phrase, but may not be politically easy to implement, especially given massive lobbying efforts from the corporations. Likewise, while funders may already have acknowledged that updating their eligibility criteria in the described way would be feasible, they have also emphasized that it would not be easy. Thus, while the changes proposed here may look small from an academic perspective, it does not entail or imply that they are easy or effortless. Nevertheless, if the integrity and trust of scholarship around the globe is of any value, arguably, the efforts required for these changes should undoubtedly be worth it.

Posted on January 11, 2022 at 16:22 5 Comments
Dec13

Academic publishing – market or collectivization?

In: science politics • Tags: , , ,

Last week’s podium on the commodification of open science entitled “If you are not paying for the product, you are the product?” was surprisingly unanimous on the need to radically modernize academic publishing and abolish the current publishing system relying mainly on corporate publishers with monopoly status. It appeared as if the present funders, librarians, scientists and other experts essentially only argued about how and when this replacement for corporate publishers should be brought abut, not if.

It was also unexpected that this was probably the first time I was not representing the most radical position on the panel. The proposals to remove usage rights from publicly funded research papers, or to ban for-profit publishers altogether, prompted the moderator, Jan-Martin Wiarda, to ask if these were calls for an expropriation of the publishers. Julia Reda was quick to point out that the goal was not to expropriate anybody, but that the accurate technical term for what she was proposing was “collectivization”.

Such collectivization of academic publishing had been brought up several times before the panel, such as in a much-discussed piece by Jason Parry earlier this year, entitled “A world without sci-hub“:

Nationalization could not only address the problems of academic publishing and make scientific work and innovation based on it easier, it could also boost American soft power in an age of waning American geopolitical dominance. 

David Wiley has proposed that the federal government take the intellectual property of academic publishers using the power of eminent domain.

Similar opposition to a market-based approach to scholarly publishing has been voiced by experts in the UK, such as Samuel Moore, an organizer of the the Radical Open Access Collective, that states that “as part of our refusal to concede open access to the forces of conservativism and the market, the collective attempts to strengthen alliances between [various related groups]”. He writes in his eminently readable article entitled “Open Access, Plan S and ‘Radically Liberatory’ Forms of Academic Freedom“:

Of course, much of Plan S is market-centric and simply perpetuates the inequities of contemporary neoliberal capitalism.

or

Any transition to open access must be framed as part of a project to emancipate researchers from the values of the neoliberal university and its requirement to publish primarily within the marketized publishing industry.

or

Clearly, Plan S is not designed to address these issues: it is wholly embedded in the liberal political economy of European nation states. The policy cannot and will not improve or fix precarity, biblio-monoculturalism or the marginalization of minority scholarships. To the extent that the APC road is prioritized, Plan S will likely exacerbate these issues by perpetuating a market-based approach. The zero-embargo repository aspect to Plan S, on the other hand, may conform to a more emancipatory approach to publishing and knowledge production, but only if coupled with support for an alternative and diverse ecosystem of non-commercial publishing projects

From the panel discussion and the quoted experts, it appears the parasitic exploitation of scholarship by the ex-publisher monopolists has created a radical backlash aiming to prevent any kind of market to ever be allowed around scholarly works. The proponents of such a collectivization effort point to examples from other economies and markets where collectivization has become necessary. The current handful of major publishers is seen as both an economic threat and as a social and scholarly menace in that “marketized publishing” is said to be inherently unfair and inequitable.

These anti-market experts provide excellent arguments against the status quo and take hard-to-refute positions with regard to equitable scholarship, protections of academic freedom, or scholarly governance. On these issues, there can be little dispute. However, in our proposal to replace academic journals, we outline how a market-based approach can lead to the same goals, by creating an actually functioning market. It appears as if collectivization proponents confuse the current conglomerate of de facto monopolies with a market and from there argue that all markets must be dispensed with. Given the constant use of the word ‘market’ to describe the scholarly publishing industry (not the least by publishers who have a vested interest in appearing to regulators as market-based), this misunderstanding is not surprising. In fact, even the European Commission has a long history of defining this industry as a market, if only from the perspective of the publishers. It appears that this publisher propaganda and lobbying has not only successfully infected the EC, but also scholarly experts seem to be buying into the market-myth of the corporations.

Because well-regulated markets can operate effectively and fairly, scholarly publishing market regulation has been called for by EU experts, even if the market were defined from a publisher perspective. Our proposal goes far beyond such band-aid solutions to a system that is too fundamentally broken to be repaired, as the panel last week unanimously agreed. We propose a market that is characterized by substitutable providers, under the governance of the scholarly community and where the market regulation includes, e.g., open source mandates that prevent future monopolization of any aspect of the market. The world has experienced that planned economies lack innovation and cost control which is exceedingly important for a publicly funded scholarly community. Arguably, therefore, an evidence-based approach to replacing the corporate journals with modern information technology must always be a market-based approach to replace the current system which is not a market.

UPDATE:
A small, informal poll on Twitter seems to reveal that people’s appetite for radical change is larger than one may have expected:

Posted on December 13, 2021 at 12:42 5 Comments
Nov22

Prioritizing academic publishers

In: science politics • Tags: , , , ,

The debate over how publishers use the large “non-publication costs” (Fig. 1) that they incur and academic libraries, mainly, are funding has been going on for some time now. Above and beyond the cost items we discuss in our paper on publication costs, it has been established that investments in surveillance technology are also part of the publisher spending academic libraries are financing. In an informal brief generated by the German GFF, the legal expert who authored it describes a situation with clear indications that one cost item which we discuss in the paper has been particularly detrimental for the scholarly community and the tax payer: lobbyism.

non-publication costs
Fig. 1: More than half of an average scholarly article price goes towards non-publication costs. Source. The question is: what are the academic publishers spending this revenue on?

Even before Franz Inglefinger coined his eponymous rule that every scholarly article must only be published in a single journal, have scholarly journals been a collection of monopolies. Every article is only available at a single source and so there is no market and no competition. This lack of substitutability has not only been the main reason for exempting subscriptions of scholarly journals from procurement rules (“single/sole source exemption”), but of course also behind the supra-inflationary price rises to the detriment of the tax payer funding public research and teaching institutions (Fig. 2).

serials crisis
Fig. 2. Supra-inflationary price increases for academic journal subscriptions

When the late Jon Tennant and I filed our formal complaint to the European Commission in 2018, in which we detailed how scholarly journal publishing was not a market but a collection of small monopolies, we had no idea that the EC was already well aware of that fact and saw nothing wrong with it. In fact, their reply at the time surprised us, when it indicated that the EC concurred with our description of scholarly journals being collections of monopolies, but saw levers for regulation/mitigation elsewhere.

Today, I have been privy to the informal brief written by a legal expert of the GFF mentioned above. It cites two prior EC instances from 2003 and 2015 where the EC had already acknowledged the lack of a genuine market due to the lack of substitutability (the reply to our complaint is thus just one in a long list of such documents acknowledging the lack of competition in scholarly publishing). In these documents, the EC writes, for instance:

In particular, from a demand-side point of view, it is rare that two different publications can be viewed as perfect substitutes, as there are differences in the coverage, comprehensiveness and content provided. Therefore, in terms of functional interchangeability, two different publications could hardly be regarded as substitutable by the end-users, the readers. On that basis, the Commission found that consumers will rarely substitute one publication for another following a change in their relative prices

and

Publications for different academic subjects are clearly not substitutable from the reader’s point of view. Even within a given discipline, there may be little demand side substitution from the point of view of the individual academic between different publications.

and

In this case, a strict demand approach would lead to the definition of a multitude of relevant markets of imprecise boundaries and small dimensions.

“Small dimension” of course meaning that every article would constitute its own market. The term “demand-side” here is defined as both readers, i.e., academics as well as the institutions paying for the journals, i.e., libraries, while “supply-side” refers to the publishers:

The results of the market investigation in the present case confirmed the relevance of supply-side considerations for the definition of the relevant product market in the academic publishing sector. Publications in different academic subjects are indeed not substitutable from the readers’ perspective. However, many academic publishers appear to be active across most of the possible segmentations of the market, and offer publications covering several disciplines

The quote here is an example of how the EC is well aware of the conflicting interests between readers and libraries on the one hand (demand-side) and publishers (supply-side) on the other, while at the same time expressing a clear prioritization (“confirmed the relevance”) of the interests of the supply-side over the interests of the demand-side. The dysfunctionality of the current situation for readers and libraries is understood, acknowledged and dismissed by the EC as “not relevant” – very similar to the reply we received for our formal complaint. In this particular quote, a fig-leaf is offered by stating that the big publishers cover many scholarly fields, leading to each library having contracts with several publishers, giving the superficial impression that there would be several suppliers in a “supply-side” market. The sentence just prior, however, makes it clear that this is, in fact, not really a genuine market, but one that exists only on paper, solely for regulatory purposes.

The GFF legal expert expressed his assessment that courts would be unlikely to challenge these EC market definitions as the definitions are seen as the lesser of two evils: explicit confirmation of the publisher monopolies would entail the loss of the market and with it loss of regulatory power. In other words, if the publishers were not operating within a market, they would also not be subject to regulatory market oversight. The consequence would be completely unregulated mergers between the most dominant players, which is seen as worse than the status quo.

His assessment is not changed by the “single source exemption” from procurement rules: as these rules and anti-trust regulations are separate jurisdictions, the fact that spending rules define academic publishers as monopolists while according to anti-trust regulators they are not, may be a logical contradiction, but not a legal one.

Thus, there are multiple reasons why the EC chose to merely acknowledge the interests of the scholarly community and instead define markets from the perspective of publishers. At this point, it is impossible to tell how much each reason has influenced the authors of these documents. It may not even be all that important to find this out in order to inform the scholarly community on the implications from this legal assessment. As I see it, there are two main lessons to be drawn from this legal analysis:

  1. If the scholarly community strives to achieve a genuine publishing market with competition, we have to design it ourselves. Politicians or other decision-makers outside of scholarship will not perform this task for us. There is not going to be any help from outside. In the lack of a clear market alternative, regulators prefer a group of relatively small monopolists over few large or even a single monopolist. The scholarly community is the only actor who can provide this missing alternative.
  2. Among the “non-publication costs” is a sizeable chunk of money that is being used in various legislatives around this globe to convince legislators and regulatory bodies that the interests of the academic publishing corporations outweigh the interests of the tax payer. This constitutes yet another example of how the scholarly community provides funds to publishers which then get used against the interests of the scholarly community.

From these lessons follows a clear strategic way forward: if the scholarly community wants to escape from the parasitic relationship with these corporations, it needs to create a genuine market alternative such that regulators need not fear the complete loss of market regulations. Luckily, there now are ways to create such a market and it is fully within the powers of the scholarly community to create it on its own. For this market to become a market of service providers, the scholarly content needs to come back under the control of the scholarly community. Once this market exists, the scholarly community must lobby at the relevant bodies to ensure both that “demand-side” interests receive at least the same “relevance” as “supply side” interests and that the “single source exemption” from procurement rules is annulled for scholarly publishers.

Posted on November 22, 2021 at 16:29 2 Comments
Oct08

The trinity of failures

In: science politics • Tags: , , , ,
More and more experts are calling for the broken and destructive academic journal system to be replaced with modern solutions. This post summarizes why and how this task can now be accomplished. It was first published in German on the blog of journalist Jan-Martin Wiarda.

Front cover of  the now-vanished  Australasian Journal of Bone & Joint Medicine . Source: Scan from the-scientist.com website 

SOMETIMES PAPER IS USEFUL: The gritty scan above from The Scientist’s website is one of the few traces left of the Australasian Journal of Bone & Joint Medicine. The journal was part of a bundle of fake journals that the academic publisher Elsevier offered as peer-reviewed journals, but which were actually paid advertisements – including for the drug Vioxx from the pharmaceutical company Merck, which was withdrawn from the market after unreported deaths surfaced and were litigated. 

The nine fake magazines are just one example of decades of exploitation of science by academic publishers: From their attempts to either criminalize Open Access via the Research Works Act or discrediting it with PR, to fighting green open access, to going behind the backs of negotiating partners, or, currently, surveilling scientists by installing tracking technologies on their publishing platforms – as well as constructing a new Silk Road of Science Communication through collaboration with Chinese authorities. At a time when the political culture (and tribal) wars have spilled over into the field of science and when trust in the honesty and reliability of research has become more important than ever, the scholarly publication system through which science enters the public, is becoming more and more dubious.

Perhaps not entirely coincidentally, for the biggest publishers, their publications have only played a relatively minor role for about ten years now. Long-term pricing schemes have provided them with steady revenue that exceed publication costs by around a factor of ten. This windfall has been filling their war chests for numerous acquisitions that have enabled the publishers to rebrand themselves as data analytics businesses, analogous to the large Internet companies. The purchased tools now span the entire research life cycle: Databases, electronic laboratory notebooks, analysis tools, authoring systems and bibliometrics are linked with the publishing platforms to create a “live trap for researchers” from which data can be continuously extracted in order to monetize them.

Ireland’s Science Foundation, for example, has just signed Elsevier to analyze its future direction, and no one asked where the “vast array of data” that the company brought with it, actually came from.

A trinity of failures: reliability, affordability, functionality

This development leads to a trinity of failures in the academic publication infrastructures:

o The systemic pernicious incentives of publish or perish fuel the replication crisis in many disciplines: publications in the most renowned journals promote careers, but at the same time the most unreliable science is published there.

o The commercialization of Open Access turns the long-standing serials crisis into an article crisis: the costs rise inexorably if only the direction of the payments is turned from reading to publishing and the system otherwise remains under the regime of monopolists (which have long since outgrown the status of major publishers), and continues to provide profits free from competition.

o The focus of the ex-publishers on user data, of researchers on publications and of institutions on cash flow and rankings led to a functionality crisis in which some of the most basic digital functionalities remain out of reach for research objects.

We can now switch at any time

Precisely because the scientific journals are at the center of this trinity of failures, experts have been calling for a modernization of information infrastructures in science for at least 15 years. In fact, the first calls for radical reform can be traced all the way back to the the late 1990s. Several possible alternatives are now available, such that we could switch at any time now.

Out of these alternatives, let’s pick the publication platform “Open Research Europe” (ORE) as only one example. Researchers funded by the European Union publish in ORE free of charge and open access. ORE is a platform owned by the EU and not by a publisher. The EU can replace the publication service provider with a different one if it is not satisfied with the provider. This creates real competition for publication services that is impossible in the reputation-based journal system. ORE is part of the “Open Research Central” (ORC) service, in which publication platforms such as ORE are aggregated into a common literature corpus, so that all institutions can, in principle, find their place here. 

The academic publishers were already a platform economy long before the well-known Internet companies adapted this principle in order to secure their dominant market position. Also the legacy publishers abuse their unassailable position in the scientific system, which links publications with the collection of metrics of prestige. Long before services like ORC were developed, it was therefore clear that a journal replacement would have to make a clean cut with this logic of the platform economy: because it locks-in scientific communication in a similar way how WhatsApp and other messengers lock-in private communication – even though before these messengers there were already protocols and standards for e-mail that enabled the exchange of service providers. The recent Facebook downtime again reminds us all that protocols and standards are superior to platforms.

Consequently, the traditional magazines must be replaced by a decentralized, resilient, evolvable network where everything is connected by open standards and protocols, that much has been clear for at least two decades now. Such a network allows for seamless switching from one provider to another under the control of the scientific community and would allow the journal article as the only scientific output that “counts”, to fade into the background. Instead, the focus would then be on the interwoven web of text, data and code, which would provide a much better orientation function for scientific knowledge than the journals have ever been able to do. The concept behind services like ORC is aimed precisely at developing such a decentralized information infrastructure.

It can work without slaughter

Academic journals are far from such an infrastructure and are therefore either considered already dead, or one would like to ensure that they will be soon: “Slaying the journals” is the slogan of one initiative, and in view of the painful experiences with journals over the decades, this version of “eat the rich” will certainly sound attractive to a number of scholars.

Not invoking any kind of slaughter, we have recently posted our own detailed proposal for a solution that contains two novel approaches for tackling the trinity of failures:

  • First, we propose that research funders expand the existing minimum requirements for the infrastructure of recipient institutions, namely by the decentralized information infrastructures mentioned above. In that way, the journal alternatives become the staple of good (and ultimately more prestigious) scientific practice. The decades-long standstill in digital infrastructures at academic institution evinces that these modernization incentives are obviously needed in order to phase out funding of the overpriced and outdated journals and instead invest in modern technologies. 
  • Second, we suggest to establish open standards and open source norms that guarantee an efficiently functioning market and prevent further monopolies on scientific tools or output. This is elementary because so far the solutions developed from the scientific community have been bought up by the relevant players so that no alternative ecosystems can develop. Such open standards already exist and promise, like the FAIR principles, to ensure scientific quality and good practice. In order to expand, secure and enforce these standards and norms, we propose setting up a standardization body like the World Wide Web Consortium (W3C) – under the governance of the scientific community, to enable the development of open scientific infrastructures that support the entire research process.

In the long-standing discussion about an academic journal replacement, opposition has traditionally been voiced by referring to journal prestige, the guidance it offers, and how this issue could not be resolved without journals. However, this argument misses two essential aspects: First, the reputation of the journals lacks any empirical basis and therefore does not represent a valid source for evaluations. To combat the wide-spread misuse of journal prestige, initiatives have sprung up such as the San Francisco Declaration of Research Assessment, to which now both the German funder DFG and the European Research Council are now signatories. Second, we know from the data that the ex-publishers sell, that there is, of course, an endless number of quantitative and qualitative evaluation options that are based entirely on the researchers’ daily interactions with their digital research objects. However, these fine-grained data are not in the hands of science, but in the possession of the ex-publishers, who, as the chairwoman of the German Council for Information Infrastructures, Petra Gehring, recently wrote, “consider the entire intellectual cycle of publicly funded and hence free research, as their future product”. Soon, such algorithmic employment decisions may make the misuse of journal rank seem benign in comparison.

Will scholarship keep repeating the same mistakes?

The Irish example can serve to illustrate Gehring’s tough statement: “The globalized struggle for profitable markets that these new forms of value creation offer, was not recognized.” It would be downright absurd for publicly funded science to repeat the same mistake now also with research and user data, as it did with publications: first the taxpayer pays for the production and then again when science buys back its own product. If the current system were to be continued, scholars would again be at the mercy of the corporations, and neither could the funding agencies ensure that society benefits from the knowledge and the potential for added-value in the data.

The current debate about digital sovereignty makes no sense without a decisive redirection of funds away from journals and towards open infrastructures and their upgrade towards a basis for building reputation. After 30 years of stagnation, this redirection would finally put an end to the trinity of failures in the academic publication system.

I am grateful to Renke Siems who drafted the first version of the German article and who insisted on more diplomatic language than I usually use.

Posted on October 8, 2021 at 13:56 Comments Off on The trinity of failures