Apr14

Nature reviewers endorse hype

In: science politics • Tags: ,

In a paper published in Nature Neuroscience now over a year ago, the authors claimed to have found a very surprising feature, which was long thought to be a bug. In my blog post covering the hype in the paper and that in an accompanying news-type article, I wrote that today, it has become ever rarer that scientists admit to standing on the shoulders of giants, as we are not rewarded for referring to giants, but only for being giants ourselves. The blog post has triggered some email correspondence between a number of colleagues in or close to this field, at the end of which was the decision that two of us, Nicolas Stergiou and I, would contact the journal and inform them of the missing references in the two articles.

The first reply was that we ought to write a ‘letter to the editor’ instead of informally notifying the journal that there were some crucial references missing. So we sat down and wrote our letter (submitted February 10, 2015):

Dear Sir,

HHMI Investigator Michael Eisen recently wrote that we ought not only to come down hard on people who cheat but also on the far great number of people who overhype their results. He suggests that hyping should be punished just as fraud. This letter to the editor represents such an effort as we would like to report on such a case of hyping.

“Standing on the shoulders of giants” is what scientists say to acknowledge the work they are building on. It is a statement of humility and mostly accompanied by citations to the primary literature preceding the current work. In today’s competitive scientific enterprise, however, such humility appears completely misplaced. Instead, what many assume to be required in the struggle to survive is to convince everyone that they are the giant, the genius, the prodigy who is deserving of the research funds, the next position, tenure. The Nature Neuroscience article “Temporal structure of motor variability is dynamically regulated and predicts motor learning ability” by Wu et al. (doi:10.1038/nn.3616) with its accompanying news-type article “Motor variability is not noise, but grist for the learning mill” by Herzfeld and Shadmehr (doi:10.1038/nn.3633) from earlier this year clearly fall within this category. Both articles claim that the researchers have made the game-changing discovery that something long thought to be a bug in our movement system is actually a spectacular feature. It is argued that this discovery is such a huge surprise, because nobody in their right mind would have ever thought this “unwanted characteristic” to actually serve some purpose.

The problem with this line of argument is that probably most people in the field thought it should be obvious, even to be expected – and not surprising at all. Skinner is largely credited with the analogy of operant conditioning and evolution. This analogy entails that reward and punishment act on behaviors like selection is acting on mutations in evolution: an animal behaves variably and encounters a reward after it initiated a particular action. This reward will make the action now more likely to occur in the future, just as selection will make certain alleles more frequent in a population. Already in 1981, Skinner called this “Selection by Consequences“ (Science Vol. 213 no. 4507 pp. 501-504, DOI: 10.1126/science.7244649). Skinner’s analogy sparked wide interest, e.g. an entire journal issue (Behavioral and Brain Sciences 7(04), 1984), which later appeared in book form (The Selection of Behavior: The Operant Behaviorism of B. F. Skinner: Comments and Consequences. A. Charles Catania, Stevan R. Harnad, Cambridge University Press). Clearly, the idea that reinforcement selects from a variation of different behaviors is not a novel concept at all, but more than three decades old and rather prominent. This analogy cannot have escaped anybody working on any kind of operant/motor learning, except those seriously neglecting the most relevant literature. This interaction of variability and selection is a well-known and not overly complicated concept, based in evolutionary biology and psychology/neuroscience. Consequently, numerous laboratories have been studying various aspects of this interaction for a long time. Skinner’s projection was that increased behavioral variability leads to increased operant learning rates, just like increased mutations rates lead to increased rates of evolutionary change. More than a dozen years ago, Allen Neuringer showed this to be the case in rats (Psychonomic Bulletin & Review 2002, 9 (2), 250-258, doi: 10.3758/BF03196279), but there are studies in humans as well (Shea, J. B., & Morgan, R. B. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179–187). That such variability is beneficial, rather than detrimental has been shown even in situations where the variability is so high, that the acquisition rate is reduced, but post-training performance is enhanced (Schmidt RA, Bjork RA (1992): New conceptualizations of practice: Common Principles in Three Paradigms Suggest New Concepts for training. Psychological Science, 3(4): 207-217).

Wu et al. confirm both Skinner’s conjecture as well as previously published reports (some cited above) that indeed the rate of learning in operant conditioning is increased in subjects where the initial variability in the behavior is higher. However, instead of citing the wealth of earlier work, Wu et al. claim that their results were surprising: “Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning”. Herzfeld and Shadmehr were similarly stunned: “These results provide intriguing evidence that some of the motor variability commonly attributed to unwanted noise is in fact exploration in motor command space.”

We regard it as highly unlikely that none of the seven authors in total should have never heard of Skinner or the work over the last four decades by many human movement scientists that have explored the temporal structure of human movement variability and its relationship with motor learning. The work by senior scientists such as Karl Newell, Michael Turvey, Richard Schmidt, and their students published in books and hundreds of journal articles is completely ignored, just as the work by several younger mid-career scientists such as Nick Stergiou, Jeff Hausdorff, Thurmon Lockhart, Didier Dilignieres, and many others. After a thorough review of this literature the authors may realize that their results are neither new nor novel. If indeed the authors were unaware of this entire section of literature so relevant to their own research, it would be an indictment in its own right.

Hence, in formal corrections of both articles, we would expect all mentions throughout both texts of how surprising these findings were, to be replaced with references including, but not limited to the works cited above.

Yours sincerely,

In hindsight, we probably ought to have mentioned that the research of course has merit and that there are a lot of valuable and exciting results in the paper, but that it is one specific aspect that is really not surprising at all. However, we wanted to make it brief and focused just on the hype. Maybe that was a mistake. Be that as it may, after about two months of peer review, our letter was rejected. Perhaps not so surprisingly, the peer reviewers apparently were the same ones that reviewed the original manuscript (at least one of them):

This is the basis on which I recommended publication, and I do not feel it needs a corrigendum.
After all, the data indicate that such high-ranking journals do not use very rigorous peer-review, leading to the least reliable science being published there. What was quite surprising, though, was the implicit support and even endorsement of the hype:
I agree that the article’s tone is a little more breathless than strictly required, but this is the style presently in vogue
and
The letter complains about “over-hyping” about certain claims made in the paper related to previous work. I have some sympathy with the letter writers on this front. However, I also have some sympathy for the authors, who understandably were trying to emphasize the novelty of their work.
It thus appears as if at least two of the three reviewers agreed with us, that the articles were hyping the results, but they don’t see anything wrong with hype. This means that hype is not just a problem of journalists any more. Hype is not just a problem of journals and GlamMagz any more. Hype has now arrived in the middle of the scientific community with explicit endorsement from reviewers. It looks like Michael Eisen’s call for hype-punishment will go largely unheeded.
You can download the complete reviews of all reviewers here (PDF), to make sure none of the quotes above were taken out of context.
Posted on April 14, 2015 at 10:21 32 Comments
Apr09

Why this GlamMag will likely not ask for my review again

In: personal • Tags: , ,

I really loathe reviewing for GlamMagz for two main reasons. For one, it’s hard to remain neutral: publication of a paper in my field in such a journal is beneficial both for the field and for the young people who are authors on this paper. Second, the demands of some of my colleagues so often make my blood boil. At that point I’m very happy these reviews are anonymous and I really don’t want to know the names of these colleagues. Here are some of the things the reviewers wrote in this most recent round:

“the authors have not convinced me of the conceptual novelty of their findings to warrant publication in a very top-tier journal”

“appears standard for a top-tier journal”

“However, currently, this manuscript is more suited for a specialized journal”

This seems to suggest to me that these colleagues apply different standards for different journals. So when the journal sent each reviewer the reviews of the other reviewers and asked for comments, one of my comments was the following:

The intuitive notion of journal rank is a figment of our imagination, devoid of any empirical support. So-called “top-tier journals” exist in the same reality in which homeopathy cures, dowsing rods find water and astrologers correctly predict the future. Asking for additional ‘interesting’ or ‘curious to know’ experiments merely because of the venue the authors chose borders on the unethical, IMHO, as there are millions of ‘curious to know’ experiments to be potentially carried out. Reviewers ought to try and avoid multiple standards in their reviews and restrain additional experiments to the minimum required by the statements of the authors. Professional editors are being paid precisely because they can predict whether their readers will find the statements of the authors ‘interesting’ or ‘curious to know’.

(Where ‘curious to know’ was a phrase one of the reviewers used whenever they wanted some tangentially related experiment to be done)

I doubt the editors at GlamMagz are very interested in these sorts of comments. If they are, I now have a boilerplate from which to draw future comments.

Posted on April 9, 2015 at 14:33 13 Comments
Mar31

Is DIY really just for the scholarly poor?

In: news • Tags: , ,

Writing in the latest issue of Lab Times, Alex Reis portraits two sections of ‘do-it-yourself’ in the biosciences. One is the group of ‘citizen scientists’, some of which are organized in DIYbio. The other group covered is that of cash-strapped biologists who create “low-cost customized devices” “out of necessity”, instead of “heading for the nearest catalogue to find the best equipment to buy”.

I’m not so much concerned with the attitude that the catalogues apparently hold the “best” equipment – as opposed to that equipment which will make grants more expensive and hence pull in more overhead for the university and more prestige for the PI. I’m more concerned with the impression this article gives that only the scholarly poor need to resort to DIY, whereas the first-world, well-funded, top-ranked laboratories of course always buy the best equipment from the catalogues for their cutting edge, world-class science.

Instead of denigrating laboratories who try to refrain from wasting tax funds on overpriced equipment, shouldn’t one instead ask what kind of research this is, where the equipment is already being sold by for-profit companies? To look for the one thing that hasn’t been put under a microscope, yet? To sequence the one gene or genome that hasn’t been sequenced, yet? To amplify the one sequence that hasn’t seen a PCR machine, yet? To obtain a band from the one protein that hasn’t been sent through a gel, yet? To spin the one liquid that hasn’t seen the inside of a salad spinner, yet? I’m exaggerating and oversimplifying, of course, but to make a point.

Quite logically, if you look at things nobody has looked at before, there cannot exist a company that provides you with a handy machine, so you just have to build the equipment or reagent yourself. Thus, in fact, every cutting-edge science by definition has to be DIY. The super-resolution microscopes for which this year’s Nobel was awarded couldn’t be bought in a store: Betzig, Hell, Moerner and colleagues had to build them themselves. If you can buy it in a store, also by definition, someone must have looked at something like this before and you’re just following in their footsteps.

One may argue, that perhaps most, if not all breakthrough science must be DIY, simply because you cannot sell equipment that doesn’t exist.

P.S.: Obviously, this post is not meant to denigrate all my many colleagues who buy all of their equipment or reagents. This research is of course very valuable and also in our lab we mostly use equipment that was designed by others than ourselves, even if it cannot be bought, and only rarely design it ourselves. I object to silly rankings and trivial comparisons in general and I only want to point out that it is very easy to argue in exactly the opposite way to counter the impression that this article is giving.

Posted on March 31, 2015 at 13:41 20 Comments
Mar26

Watching a paradigm shift in neuroscience

In: science • Tags: , , , , ,

ResearchBlogging.orgWhen I finished my PhD 15 years ago, the neurosciences defined the main function of brains in terms of processing input to compute output: “brain function is ultimately best understood in terms of input/output transformations and how they are produced” wrote Mike Mauk in 2000 (DOI: 10.1038/76606). Since then, a lot of things have been discovered that make this stimulus-response concept untenable and potentially based largely on laboratory artifacts.

For instance, it was discovered that the likely ancestral state of behavioral organization is one of probing the environment with ongoing, variable actions first and evaluating sensory feedback later (i.e., the inverse of stimulus response). It was found that even the most rigid and iconic of stimulus-response systems – spinal reflexes – still show rudiments of probing the environment with spontaneous, variable actions and evaluating the sensory consequences later. A recently discovered instance of a so-called ‘rare predator’ phenomenon exemplified that rigid stimulus-response coupling cannot be evolutionary stable:

Kevin Mitchell thus aptly referred to the hypothetical class of animals without unpredictability “lunch” (see his excellent article for a more verbose explanation). In humans, functional magnetic resonance imaging (fMRI) studies over the last decade and a half revealed that the human brain is far from passively waiting for stimuli, but rather constantly produces ongoing, variable activity (the so-called default mode network, DMN, in our resting state), and just shifts this activity over to other networks when we move from rest to task or switch between tasks. Tellingly, the variations in DMN activity account for a large part of our behavioral variability.

As one would expect, this dramatic shift in perspectives from input/output to output/input has led to a slew of recent publications which were not thinkable a mere 15 years ago. For instance, it was reported that rodent brains add variability to sensory input. In Aplysia, it was shown that such variability can be generated by balancing excitatory and inhibitory input, but also that individual neurons (see Fig. 4b) are capable of showing spontaneous variability in their firing patterns, even when they are isolated from the rest of the nervous system. In the most recent annual meeting of the Society for Neuroscience, where I usually only find very few presentations on ongoing activity and how it leads to variability, there now were several posters on exactly this topic, seemingly out of nowhere. The most recent publication is from an animal where the connectome is so dominated by feed-forward connections from sensory to motor neurons, that even today it would be difficult to imagine how the neurobiology underlying behavioral variability could be studied in such an animal, the nematode worm Caenorhabditis elegans: Feedback from Network States Generates Variability in a Probabilistic Olfactory Circuit.

In this paper from the laboratory of Cori Bargmann, Gordus et al. look at a circuit in the C. elegans nervous system which controls reversal behaviors (Fig. 1). The main component of the system is a neuron called AVA. When AVA is active, the animal reverses its course. Sensory input to this neuron is provided by olfactory neuron AWC. For instance, if AWC is stimulated by an attractive odorant, it stops firing, such that AVA also stops firing, making reversals less likely. Two additional neurons are involved in this circuit, AIB and RIM, and the characterization of their role in the circuit was the main result of this publication.

Fig. 1: The C. elegans reversal circuit with the number of electrical and chemical synapses between each network component

Fig. 1: The C. elegans reversal circuit with the number of electrical and chemical synapses between each network component

The first interesting observation from the circuit connectivity is that there are more connections from the sensory neuron to the AIB interneuron than to the reversal neuron AVA. This would be a major head scratcher if the main function of nervous systems were to relay sensory information to motor centers, but if sensory input merely modulates ongoing activity, even in nematodes, then this doesn’t seem so surprising any more.

Contrary to the idea that a connectome dominated by feed-forward connections from sensory to motor areas implies that it mainly computes motor output from sensory input, also the nervous system of C. elegans is best characterized by constantly changing, ongoing activity, much like all the other nervous systems previously studied in this regard. Even the small circuit studied by Gordus et al. demonstrates that:

gordus_fig2

Fig. 2: Even in the absence of sensory stimulation in immobilized animals, the activity in the reversal circuit fluctuates constantly. The Y-axis depicts fluorescence as a measure of calcium levels (i.e., activity) in the neurons.

Interestingly, the neurons exhibit a sort of binary activity state, that for the most part is either on (neuron is active) or off (neuron is inactive):

Fig.: 3: The three neurons spend most of their time either in an 'off' dtate or in an 'on' state.

Fig.: 3: The three neurons spend most of their time either in an ‘off’ state or in an ‘on’ state, as can be seen from the probability (P) of fluorescence (F).

According to this classification, there are three main states (of the eight theoretically possible) the circuit is commonly found in, mainly due to the strong correlation between neurons because of their electrical and chemical coupling: just over 60% of the time the system is in ‘all on’, roughly 20% is ‘all off’ and for the remaining 20% it is in ‘only AIB on’:

Fig. 4: The three most common network states for the reversal circuit

Fig. 4: The three most common network states for the reversal circuit

By selectively inhibiting each member of the circuit, the authors discovered that the role of AIB and RIM was to increase the variability of the reversal circuit. The input from the olfactory neuron AWC was always very precise and predictable if, e.g., an attractive odor was presented, but the activity of the reversal circuit always varied significantly and this variability was reduced if AIB or RIM were silenced. An example of how variable the response of the circuit is compared to the sensory input without any experimental manipulation of AIB or RIM can be seen in Fig. 5:

The olfactory neuron AWC responds is drastically more deterministic to an attractive odor than the components of the reversal circuit.

The olfactory neuron AWC responds is drastically more deterministic to an attractive odor than the components of the reversal circuit.

Thus, the authors make an excellent case for RIM and AIB being incorporated into the reversal circuit specifically to inject much needed variability into an otherwise maladaptively deterministic reversal circuit. Surprisingly, even though the feed-forward connections dominate the connectivity also in this little circuit, the variability provided by the feedback connections dominate an adaptive feature of the behavior, its variability. This work adds C. elegans to the elongating list of animals, whose nervous systems are organized such that ongoing activity is modulated by external stimuli. It seems, in such nervous systems, even a numerically small feedback component provides a fundamental contribution to the overall architecture. What does this mean for brains whose anatomy appears to be dominated by feedback loops?

Gordus, A., Pokala, N., Levy, S., Flavell, S., & Bargmann, C. (2015). Feedback from Network States Generates Variability in a Probabilistic Olfactory Circuit Cell DOI: 10.1016/j.cell.2015.02.018

Posted on March 26, 2015 at 12:50 100 Comments
Feb10

How not to contact faculty as student

In: I get email • Tags: ,

Over the weekend, I received the following short message from a Hotmail account:

Dr. Brembs,
Thank you for your work and insight. Students from my doctorate of psychology epistemological course have recently discussed your article entitled ‘Towards a scientific concept of free will as a biological trait‘.
Acknowledging my vastly inferior scientific skills and reputation compared to yourself, I must respectfully disagree, however, with your current world view – you could call it scientific or philosophical for all intents and purposes. I have nothing but respectful curiosity for your divergent opinion as a prominent scientist in an unequivocally lopsided debate favouring the quantitative materialistic and biological approach. Major ontological questions are still far from being answered.
What is your objective argument against an article such as the following one? I hope that, if you indeed have the time to read it and offer a reply, could set aside any potential bias against the other of the article should you already hold one:
Sincerely,
-name redacted-
-university redacted-
Ph.D. candidate
This email is full of signs that the student may not have been all that interested in any information or discourse, but instead trying to make some sort of ‘gotcha’ statement.
Acknowledging my vastly inferior scientific skills and reputation compared to yourself, I must respectfully disagree, however, with your current world view – you could call it scientific or philosophical for all intents and purposes.
Curiously, the author first states that their scientific training is lacking and then they disagree with a scientific position. One wonders if the author often disagrees with people who they admit are more competent.
In cases where I feel incompetent, I usually try to read and learn and ask for advice before I decide I’m competent enough to disagree. So either you are incompetent and ask for advice, or you are competent and disagree. You can’t have it both ways.Moreover, the use of the word ‘current’ with ‘world view’ assumes that I were to frequently change my world view, an assumption which I find highly unusual – does anybody frequently change their world view? If so, is the author’s current world view still the same as the one they held when they wrote the message?And, perhaps most importantly, I’d definitely refrain from so transparently and clumsily trying to flatter the person I’m trying to coax into a reaction – that would just make me look completely mental (pardon the pun).Already this sentence seems to indicate that the author has some catching up to do on more than just their scientific training…
I have nothing but respectful curiosity for your divergent opinion
What now? Opinion or world view? Or are these the same for the author? And is it curiosity or disagreement? Note: If you expect to be taken seriously, at least do not contradict yourself within the first few sentences of a message.
as a prominent scientist in an unequivocally lopsided debate favouring the quantitative materialistic and biological approach. Major ontological questions are still far from being answered.
Again, a clumsy attempt at flattering me, not good. I’m also torn as to what the author is trying to say here. For one, I’m not aware of any debate as to the material nature of our world (by ‘material’ I mean physical as opposed to, say, ghosts). So far, I have never met nor read nor heard of any colleague claiming they had observed ghosts interfere with their experiments. If there actually were a debate, I’m not surprised it is so unequivocally lopsided in favor of materialism – there is no evidence for ghosts. At least, I’m glad the author also finds that here are major questions to be answered – if not, as a scientist, I’d be out of a job! What would I do then?
What is your objective argument against an article such as the following one?
It appears, the author is expecting me to use subjective arguments in response to their message. So first they clumsily attempt to flatter me by saying how superior I am and now they try to insult me by expecting me to not be able to articulate an ‘objective’ argument? Way to go confirming my suspicions that there is more amiss with this author than just their scientific training. And I’m only allowed one such argument, not several?
As I see it, there are perhaps three simple rules one could take away from this:
  1. If you really want to have information, please state what you want to know or what you don’t understand, don’t beat about the bush.
  2. Don’t contradict yourself, neither by using different words for the same thing, nor by first flattering and then insulting me.
  3. If you happen to violate rules 1, 2 or both, at least use your institutional address or you might be taken for some random cook trying to troll.
Posted on February 10, 2015 at 16:40 7 Comments
Feb06

Publishers, stop torturing your reviewers!

In: science politics • Tags: , , ,

UPDATE, 10-02-2015: After a hint from a user on Twitter, I now know that it is possible to open a PDF document in several windows, one for text, one for legends and one for figures. Figures and legends occupy one virtual desktop and the text another. In this way, I can actually review on-screen, but it is one heck of a work-around and by no means convenient.

 

I cannot take it any more. This has been bugging me for more than ten years, but now I’ve finally had it. From today on, I will refuse to review any manuscripts that come in a format where I cannot easily have figures, legends and text side by side on my screen (or at least inline), like, e.g., in eLife’s Lens:

lensThe standard way of reviewing articles is to receive a PDF with all the text together first, then all the legends together below that and at the very end all the figures, one on each PDF page. This makes it virtually impossible to check the statements the authors make about their data, as one has to scroll the screen back and forth between text, legend and figure, just to find out what is displayed on the figure. Obviously, it takes ages to find the correct legend and it’s virtually impossible to find the spot where one left off to go looking for figure and legend. This needs to stop. It will stop for me. I can’t waste my time any more with such nonsense!

If you are an editor or publisher and would like me to review for your journal, make it easy and fun for me to do it and not a chore. Either develop a system like Lens, or license Lens, or ask your authors to not separate text, legends and text in their submissions, whatever! I don’t care, just stop torturing your reviewers.

Personally, I already submit my articles with figures and legends together, irrespective of what the publishers want me to do. From now on, I will not only refuse to review a manuscript where I cannot see figures, legends and text side-by-side (or inline) on my screen, I will also ignore publisher rules to separate the three in my own submissions – I don’t want the reviewers of my manuscripts to suffer more than they already have to, reviewing my work.

Posted on February 6, 2015 at 15:56 94 Comments
Feb04

Random Science Video: The value of fly research

In: random science video • Tags: ,
Drosophila: Small fly, BIG impact - Part 1 (Why the fly?)
Drosophila: Small fly, BIG impact – Part 1 (Why the fly?)

Video von YouTube laden. Dabei können personenbezogene Daten an Drittanbieter übermittelt werden. Hinweise zum Datenschutz

Posted on February 4, 2015 at 14:02 1 Comment
Jan07

Booming university administrations

In: science politics • Tags: , , ,

This is a post loosely based on an article appearing today in the German newspaper “Frankfurter Allgemeine Zeitung” by Axel Brennicke and me. The raw data for our analysis is available. Please do let us know if you find a mistake.

UPDATE, 09/01/2015: A commenter made me aware of data rows in the raw data we had overlooked before. In our analysis, we evaluated all non-scientific professional staff, e.g., also technical support or library staff. As I’m actually quite fond of the libraries and the technical support we have, I looked at the trends for the ‘pure’ permanent administration staff and found and increase of 17% from 2005 to 2012, while permanent scientist positions increased only by 0.04%. Taking only these two groups of employees, the ratio between scientists and administrators shrinks to 0.57 in 2005 and 0.64 in 2012, i.e., the average administrator has to support less than two permanently employed scientists. In my opinion, this would have been the better data to use, but my co-author is not quite as convinced. Either way, even focusing on ‘pure’ admin staff conveys essentially the same message as the full overall data, albeit perhaps less dramatically. This is precisely why I am an open science advocate: making your data open allows you to discover more and improve your science!

UPDATE, 16/03/2015: I was just made aware of very similar numbers from the USA, published in this article by Jon Marcus.

Noam Chomsky, writing about the Death of American Universities, recently reminded us that reforming universities using a corporate business model leads to several easy to understand consequences. The increase of the precariat of faculty without benefits or tenure, a growing layer of administration and bureaucracy, or the increase in student debt. In part, this well-known corporate strategy serves to increase labor servility. The student debt problem is particularly obvious in countries with tuition fees, especially in the US where a convincing argument has been made that the tuition system is nearing its breaking point. The decrease in tenured positions is also quite well documented (see e.g., an old post). So far, and perhaps as may have been expected, Chomsky was dead on with his assessment. But how about the administrations?

To my knowledge, nobody has so far checked if there really is any growth in university administration and bureaucracy, apart from everybody complaining about it (but see update above with link to similar numbers from the US). So Axel Brennicke and I decided to have a look at the numbers. In Germany, employment statistics can be obtained from the federal statistics registry, Destatis. We sampled data from 2005 (the year before the Excellence Initiative and the Higher Education Pact) and the latest year we were able to obtain, 2012.

The aim of the Excellence Initiative and the Higher Education pact was to improve the funding situation for universities to increase scientific competitiveness and to allow them to cope with the rising student numbers. Student numbers increased from 2005-2012 by 25% to 2.5 million. The number of full-time university employees in research and teaching also increased, but only by 18% to just over 142,000, i.e., not keeping up with student numbers. Looking at the type of contracts, we can confirm the US trend of an increasing precariat also here in Germany: in 2005, 50% of all full-time employees were on short-term contracts. This fraction increased by about one percentage point per year to now more than 58%.

University leadership and politicians often argue that such short-term employment benefits research as scientists constantly need to learn new skills in different places and the short-term contracts keep the work-force up-to-date and flexible. However, this argument also holds for administrations: fluctuating student numbers, a constantly changing funding landscape and the constant turnover of university employees ought to mandate a flexible and slim administration as well.

However, the numbers speak a different language. Every full-time university scientist is administrated by 1,28 employees in administration. Of these 182,255 administrative and professional employees, only 25% are on fixed-term contracts. The remaining 75% are virtually tenured: it is almost impossible to make them redundant, according to German laws and regulations for public employees.

Perhaps the two initiatives have improved on this situation? Maybe in 2005, the situation was even worse? Alas, the fraction of ‘tenured’ administrative personnel and professional staff was only 70% in 2005. In fact, for every permanent position that was created in research and teaching between 2005-2012, 3.7 such positions were newly created in the university administrations. This means that the average German scientist now supervises 7 more students than in 2005, while the burden on administrators and professional staff has only increased by 2.7 students.

These numbers show the real winners and losers of the increased cash flow in this past decade: the students and scientists lose out, while university administrations benefit the most from the billions. At this point, to support the work of 60,438 permanently employed scientists in German universities, it is apparently required to permanently employ 135,897 administrative and professional employees.

In fact, for every permanent position that was created in this time, ten fixed-term positions were created, exacerbating the already abysmal job prospects of German early-career scientists. This parallels the development in many other research nations such as the US or the UK. The situation is most dramatic for early-career scientists. The tip of this huge iceberg are increasing number of science scandals, such as the recent case of the young Haruko Obukata whose mentor committed suicide after the manipulations of his student became known, or Felisa Wolfe-Simon, who called her arsenic-less strain of bacteria “GFAJ1”: give Felisa a job.

As I remarked long ago, the trend to corporatize higher education parallels the rise in retractions. This rise in retractions is so rapid, that by about 2045 every published article will be accompanied by the retraction of another article, if the current trend continues. Back in 2010, Amy Bishop killed three of her colleagues at University of Alabama in a shooting spree after she was denied tenure because her publication rate was deemed insufficient. Scientists on non-permanent positions today need to publish many articles in journals ranking high in a hierarchy that is devoid of any empirical justification. On top of that, they need to devise and find funding for especially expensive experiments, as the higher the funding, the higher the so-called ‘overhead’ that the universities receive. Last year, professor Stefan Grimm committed suicide after his employer, Imperial College London told him he had 12 months to pull in 200,000 Pounds in research funding or be fired.

Today’s top scientists thus have been selected by their marketing competence to sell their work to hi-ranking journals and by their ability to come up with expensive experiments. If their research is actually reliable and sound, it is pure coincidence. There exist no incentives today for just doing reproducible science.

The trends are clear on all fronts: business as usual has become untenable. There needs to be reform and it has to be international and far-reaching. Tinkering a little here and there is not going to cut it. On the up side, a lot can be established by universities themselves, without outside assistance: administration funds can be siphoned into research/teaching positions, and the billions now tied up in subscriptions can be used to develop a digital infrastructure which comprises a reputation system that rewards scientists for doing reliable science, once all subscriptions have been canceled. This is technically and financially feasible and within a sufficiently short time-frame. But action has to be taken now and it has to be decisive. The downside? It has to be collective action.

P.S.: We have made the raw data on which we have written our article available for everyone to check. Mistakes happen and we would not want to end up like Reinhardt and Rogoff.

P.P.S.: Two related articles, one in English and one in German (PDF).

Posted on January 7, 2015 at 11:58 271 Comments
Oct20

In which scientists behave like rats in a Skinner box

In: science politics • Tags: , , ,

Skinner used the term “schedules of reinforcement” to describe broad categories of reward patterns which come to reliably control the behavior of his experimental animals. For instance, when he rewarded rats for pressing a lever at a given interval after the last reinforcement (i.e., fixed interval; FI), the animals would pause pressing the lever until just before the interval was over and then start pressing the lever like mad. When the number of presses are plotted cumulatively over time, this leads to a scalloped plot (black trace in Fig. 1). A similar, but clearly distinct curve can be observed when the reward comes only after a given number of lever presses (i.e., fixed ratio; FR): there, the animals stop pressing while they consume the reward and then resume repeatedly pressing the lever until the next reward is consumed (blue trace in Fig. 1). If the ratio or interval is varied, however, the animals more or less consistently press the lever at a rate given by the reward frequency (red and green traces, Fig. 1).

Fig. 1.: Different categories of response patterns to different schedules of reinforcement. VR – variable ratio; FR – fixed ratio; VI – variable interval; FI – fixed interval

In an analogous way, if you tell scientists that they need a certain number of publications to get/keep their jobs, they will double the number of articles they publish. Likewise, if you tell scientists that they need to “acquire external funding” in order to get/keep their jobs, they will increase the number of grants they write, until it’s almost all they do. This latter response class is particularly absurd and counterproductive: the number of scientists keeps increasing, at least for now. In most countries, the increase is larger than the increase in public funds, or the public funds are even decreasing. This means that the success rate of getting research funding must drop, even if every researcher would not increase the number of grants they write. However, as each scientist needs grants to keep working, the dropping success rates have the same consequence as a VR schedule with a low probability of reward (red line, Fig. 1): scientists maintain a very high level of grant writing which matches the decreasing success rate: if it’s 10%, you have to – on average – write 10 grant proposals to get one of them funded. If the success rate is down to 5%, you have to write 20 and so on. As scientists multiply and keep increasing the number of grants they write, the success rates keep dropping accordingly, making the scientists write yet more proposals. A vicious cycle if ever there was one. Skinner couldn’t have devised a more pernicious schedule himself.

That may seem absurd enough, but it is only half of the story. As research “income is crucial to the development of world-class research“, not only universities are ranked according to the volume of government funds they attract, but also (and likely consequently) the scientists within each institution are ranked according to the funds they attract. In Germany, for example, how much funds a scientist can attract, can have direct effects on their salary. Thus, if there is a cheaper and a more expensive way to do the same experiment, it is in the scientist’s own best interest to chose the more expensive experiment. As it stands today, scientists should write as many grant proposals as possible and make them as expensive as possible. In other words: the successful scientist of today wastes not only time, but also tax funds. If you factor in that in order to get your grants funded, you also must publish in the so-called ‘top-journals’ where the key factor is getting past the professional editor, the most successful scientists of this day must be great salespersons: they need to sell grant agencies their wasteful research proposals and editors their unreliable papers. If we are lucky, some of them might also be great scientists, as well.

 

Posted on October 20, 2014 at 12:14 59 Comments
Oct01

How important is the Impact Factor?

In: science politics • Tags: , ,

I recently was sent a report from a university-wide working group on the publishing habits within the Freie Universität Berlin. I don’t think this document is available online, but I think I’m not doing anything illegal if I publish some of the survey results here. The working group polled all faculty members of the university on various questions concerning scholarly publishing. One of the questions was on the importance of Thomson Reuters’ Impact Factor for the respondents. Here the results:

n= none/low middle/high
Discipline 207 43% 57%
Vetenary medicine 17  0%  100%
Biology, Chemistry, Pharmacy 31  3% 97%
Economics 17  12%  88%
Physics 10  20%  80%
Political and social studies 17  24%  76%
Didactics and psychology 14  36%  64%
Geosciences 11  36%  64%
Mathematics and computer sciences 22  50%  50%
Philosophy and humanities 25  88%  12%
Law 11  91%  9%
History and cultural studies 24  92%  8%

It is quite clear that in all disciplines but the humanities, the Impact Factor is considered to be influential. In particular the biomedical field is near unanimous in its submission under the Impact Factor, with physics, social and geosciences trailing.

Interestingly, a majority of respondents in all disciplines lament the high importance of bibliometric indicators in general and the Impact Factor in particular. This seems puzzling, given that it’s the faculty that make these numbers important, at least here in Germany.

The report emphasizes both the inconsequential (and unscientific) behavior of their own faculty as well as the detrimental consequences of the collusion between the big database companies, Thomson Reuters and Reed Elsevier, for the scientific community in general. The working group report specifically recommends the university to drop the use bibliometric indicators in internal evaluations.

Footnote: The report also emphasizes that the importance of the Open Access debate is exaggerated in the public discussion and is not represented in the faculty responses, where Open Access plays a minor role and which shows that publication traditions are fairly constant over several generations of scholars and change only little.

Posted on October 1, 2014 at 12:32 24 Comments