While our own manuscript on journal rank is almost through the peer-review process, this morning I received several messages announcing the DORA (San Francisco declaration on research Assessment), which I signed immediately. Echoing some of the sentiments we also refer to in our article, part of the declaration reads:
The Journal Impact Factor is frequently used as the primary parameter with which to compare the scientific output of individuals and institutions. The Journal Impact Factor, as calculated by Thomson Reuters, was originally created as a tool to help librarians identify journals to purchase, not as a measure of the scientific quality of research in an article. With that in mind, it is critical to understand that the Journal Impact Factor has a number of well-documented deficiencies as a tool for research assessment. These limitations include: A) citation distributions within journals are highly skewed [1–3]; B) the properties of the Journal Impact Factor are field-specific: it is a composite of multiple, highly diverse article types, including primary research papers and reviews [1, 4]; C) Journal Impact Factors can be manipulated (or “gamed”) by editorial policy [5]; and D) data used to calculate the Journal Impact Factors are neither transparent nor openly available to the public [4, 6, 7].
Of course I wholeheartedly agree with this declaration and with all the recommendations mentioned in it (which is why I signed it). However, while the recommendations are great, they merely echo what people have been editorializing about for at least a decade, some even longer. I also fail to see any clear vision as to how publication reform is supposed to happen. Finally, since our paper isn’t out, yet, (other than on arxiv) they cannot include in their statement that there is absolutely no evidence that research published in high-IF journals are in any way better than those in journals with a lower IF: there is not a single study that I know of where there is a positive correlation of anything that could be construed as a direct measure of any aspect of scientific quality with Impact Factor.
In the absence of such evidence, the appeal in the declaration to drop journal rank as an evaluation signal is just that: an appeal – it lacks evidence-based reasoning and substance, as one could simply argue that the flaws of the IF just need to be fixed.
Nevertheless, such a public and widespread declaration is justified, comes at the right time and carries the right message, so it deserves our full support, even if it may not go quite far enough in its demands for reform. Clearly, being a platform seeking the most widespread support, some compromises have to be made.
Now go and sign the declaration already, what are you waiting for?
