PEPIOPs – prolific editors who publish in their own publications

I recently reported on a method for identifying authors who contribute an unusually high proportion of papers to a specific journal. As I noted in my previous blogpost, one cannot assume any wrongdoing just on the grounds of a high publication rate, but when there is a close link between the author in question and the journal's editor, then this raises concerns about preferential treatment and integrity of the peer review process.

In running my analyses, I also found some cases where there wasn't just a close link between the most prolific author in a journal and the editor: they were one and the same person! At around the time I was unearthing these results, Elisabeth Bik tweeted to ask if anyone had examples of editors publishing in their own journals. I realised the analysis scripts I had developed for the 'percent by most prolific' analysis could be readily adapted to look at this question, and so I analysed journals in the broad domain of psychology and behavioural science from six publishers: Springer Nature, Wiley, Taylor and Francis, Sage, Elsevier and American Psychological Association (APA). I focused on those responsible for editorial decisions – typically termed Editor-in-Chief or Associate Editor. Sometimes this was hard to judge: in general, I included 'Deputy editors' with 'Editors-in-Chief' if they were very few in number. I ignored members of editorial boards.

Before reporting the findings, I thought I should consult more widely about whether people think it is appropriate for an editor to publish in their own journal.

As a starting point, I looked at guidelines that the Committee on Publication Ethics (COPE) has provided for editors  

Can editors publish in their own journal? 
While you should not be denied the ability to publish in your own journal, you must take extra precautions not to exploit your position or to create an impression of impropriety. Your journal must have a procedure for handling submissions from editors or members of the editorial board that will ensure that the peer review is handled independently of the author/editor. We also recommend that you describe the process in a commentary or similar note once the paper is published.

They link to a case report that notes how this issue can be particularly problematic if the journal is in a narrow field with few publication outlets, and the author is likely to be identifiable even if blinding is adopted.

I thought that it would be interesting to see what the broader academic community thought about this. In a Twitter poll I asked specifically about Editors-in-Chief:

The modal response was that it was OK for an Editor-in-Chief to publish in their own journal at a modest rate (once a year or less). There was general disapproval of editors publishing prolifically in their own journals – even though I emphasised that I was referring to situations where the editorial process was kept independent of the author, as recommended by COPE.

My poll question was not perfectly phrased - I did not specify the type of article: it is usual, for instance, for an editor to write editorials! But I later clarified that I meant regular articles and reviews, and I think most people interpreted it in that way.

The poll provoked some useful debate among those who approved and disapproved of editors publishing in their own journals.

Let's start with some of the reasons against this practice. I should lay my cards on the table and state that personally I am in agreement with Antonia Hamilton, who tweeted to say that as Editor in Chief of the Quarterly Journal of Experimental Psychology she would not be submitting papers from her lab to that journal during her term of office, citing concerns about Conflict of Interest. When I was Editor-in-Chief of Journal of Child Psychology and Psychiatry, I felt the same: I was concerned that my editorial colleagues would be put in a difficult position if they had to handle one of my papers, and if my papers were accepted, then it might be seen as involving a biased decision, even if it that was not the case. Chris Chambers, who was among the 34% who thought an Editor-in-Chief should never publish in their own journal, expressed concerns about breaches of confidentiality, given that the Editor-in-Chief would be able to access identities of reviewers. Others, though, questioned whether that was the case at all journals.

Turning to those who thought it was acceptable for an Editor-in-Chief to publish in their own journal, several people argued that it would be unfair, not just on the editor themselves, but also on members of their research group, if presence of an editorial co-author meant they could not submit to the journal. How much of an issue this is will depend, of course, on how big your research group is, and what other publication outlets are available.  Several people felt there was a big difference between cases where the editor was lead author, and those where they played a more minor role. The extreme case is when an editor is a member of a large research consortium led by someone else. It would seem unduly restrictive to debar a paper by a consortium of 100 researchers just because one of them was the Editor in Chief of the journal. It is worth remembering too that, while being a journal editor is a prestigious role, it is hard work, and publishers may be concerned that it becomes a seriously unattractive option if a major outlet for papers is suddenly off-limits.

Be this as it may, the impression from the Twitter poll was that three papers or more per annum starts to look excessive. In my analysis, shown here I found several journals where the Editor-in-Chief had coauthored 15 or more articles (excluding editorials/commentaries) in their own journal between 2015 and 2019.  I refer to these as PEPIOPs (prolific editors who publish in their own publications). For transparency, I've made my methods and results available, so that others can extend the analysis if they wish to do so: see https://github.com/oscci/Bibliometric_analyses. More legible versions of the tables are also available.

Table 1 shows the number of journals with a PEPIOP, according to publisher. 

Table 1: Number of journals with prolific authors. Columns AE and EIC indicate cases where Associate Editor or Editor-in-Chief published 15+ papers in the journal between 2015-2019

The table makes it clear that it is relatively rare for an Editor-in-Chief to be a PEPIOP: there were no cases for APA journals, and the highest number was 5.6% for Elsevier journals. Note that there are big differences between publishers in terms of how common it is for any author to publish 15+ papers in the journal over 5 years: this was true for around 25% of the Elsevier and Springer journals, and much less common for Sage, APA and Taylor & Francis. This probably  reflects the subject matter of the journals - prolific publication, often on multiauthored papers, is more common in biosciences than social sciences.

Individual PEPIOPs are shown in Table 2. An asterisk denotes and Editor-in-Chief who authored more articles in the journal than any other person between 2015-2019.

These numbers should be treated with caution – I did not check whether any of these editors had only recently adopted the editorial role - largely because this information is not easy to find from journal websites*. I looked at a small sample of papers from individual PEPIOPs to see if there was anything I had overlooked, but I haven't checked every article - as there a great many of them.  There was one case where this revealed misclassification: the editor of Journal of Paediatrics and Child Health wrote regular short commentaries of around 800 words summarising other papers: this seems an entirely unremarkable thing for an editor to do, but they were classified by Web of Science as 'articles' which led to him being categorised as a PEPIOP.  This illustrates how bibliometrics can be misleading.

Table 2: Editors-in-chief who published 15+ papers in own journal 2015-2019

In general I did not find any ready explanation for highly prolific outputs.  And I where I did spot checks I found only one case where there was an explanatory note of the kind COPE recommended (in the journal Maturitas) - on the contrary, in many cases there was a statement confirming no conflict of interest for any of the authors. Sometimes there were lists of conflicts relating to commercial aspects, but it was clear that authors did not regard their editorial role as posing a conflict. It was also worth mentioning there were cases where an Editor-in-Chief was senior author.

The more I have pondered this, the more I realise that the reason why I am concerned particularly by Editor-in-Chief PEPIOPs is because this is the person who is ultimately responsible for integrity of the publication process. Although publishers increasingly are alert to issues of research integrity, journal websites typically advise readers to contact the Editor-in-Chief if there is a problem. Complaints about editorial decisions, demands for retractions, or concerns about potential fraud or malpractice all come to the Editor-in-Chief. That's fine provided the Editor-in-Chief is a pillar of respectability. The problem is that not all editors are the paragons that we'd like them to be: one can adopt rather a jaundiced view after encountering editors who don't even bother to reply to expressions of concern, or adopt a secretive or dismissive attitude if plausible problems in their journal are flagged up. And there are also cases on record where an editor has abused the power of their position to enhance their own publication record. For this reason, I would strongly advise any Editor-in-Chief not to be a PEPIOP; it just looks bad, even if a robust, independent editorial process has been followed. We need to have confidence and trust in those who act as gatekeepers to journals.

My principal suggestion is that publishers could improve their record for integrity if they went beyond just endorsing COPE guidelines and started to actively check whether those guidelines are adhered to. 

*Note: 21st Aug 2020: A commenter on this blogpost has noted that Helai Huang fits this category - this editor has only been in post since 1 Jan 2020, so the period of prolific publication predated being an editor-in-chief.

Editorial integrity: Publishers on the front line

Thanks to some live tweeting by Anna Sharman (@sharmanedit), I've become aware that the 13th Conference of the European Association of Science Editors (EASE) is taking place in Strasbourg this weekend.

The topic is "Scientific integrity: editors on the front line", and the programme acknowledges Elsevier, who presumably have contributed funding for the conference.

It therefore seems timely to give a brief update of developments following three blogposts I wrote during February-March 2015, documenting some peculiar editorial behaviour at four journals: Research in Autism Spectrum Disorders (RASD: Elsevier), Research in Developmental Disabilities (RIDD: Elsevier), Developmental Neurorehabilitation (DN: Informa Healthcare) and Journal of Developmental and Physical Disabilities (JDPD: Springer).

To do the story full justice, you need to read these blogposts, but in brief, blogpost 1 described how Johnny Matson, the then editor of both RASD and RIDD had published numerous articles in his own journal, and engaged in frequent self-citation, leading to his receiving a 'highly cited' badge from Thomson Reuters. In the comments on that blogpost, another intriguing factor emerged, which was Matson's tendency to accept papers with little or no review. This was denied by Elsevier, despite clear evidence of very short acceptance lags that were incompatible with review.

Blogpost 2 was prompted by Matson defending himself against accusations of self-citation by pointing out that he published in journals that he did not edit. I checked this out and found he had numerous papers in two other journals: DN and JDPD, and that the median lag between a paper of his being submitted and accepted in DN was one day. (JDPD does not provide data on publication lags). I therefore looked at the editors of those journals, and found that they themselves were publishing remarkable numbers of papers in RASD and RIDD, again with extremely short publication lags. A trio of editors and editorial board members (Jeff Sigafoos, Giulio Lancioni and Mark O'Reilly), co-authored no less than 140 papers in RASD and RIDD between 2010 and 2014, typically with acceptance times of less than 2 weeks. Some of the papers in RIDD were not even in the topic area of developmental disabilities, but covered neurological conditions acquired in adulthood.

In blogpost 3, I turned the focus on to the publisher of RASD and RIDD, Elsevier, to query why they had not done anything about such irregular editorial practices. I did a further analysis of publication lags in RIDD, showing that they had dropped precipitately between 2008 and 2012, and that there was a small band of authors whose prolific papers were published there at amazing speed. I provided all the statistical data to support my case, including interactive spreadsheets that made it easy to determine which editors and authors had been benefiting from the slack editorial standards at these journals.

There was some interesting fall-out from all of this. The second blogpost drew fire from supporters of the editors I had "outed", accusing me of bad behaviour and threatening to complain to my university. Since everything I had said was backed by evidence, this did not concern me. I received heartfelt messages of support from people who were appalled that a particular approach to autism intervention had been promoted by this group of editors, who were in effect using their status to gain the veneer of scientific credibility for work which was not in fact peer-reviewed.  I was also contacted by several academics telling me that everyone knew this had been going on for years, but nobody had done anything; this level of passivity was surprising given that many were angry that  authors had reaped benefits from their staggeringly high publication rate, while those who were outside the charmed circle were left behind. I was urged to go further and raise my concerns with the universities employing those who were capitalising on, or engaging in, lax editorial behaviour. I do, however, have an extremely demanding job and I hoped that I had done enough by shining a light on dubious practices, and providing the full datasets that provided evidence. However, I now wonder if I should have been more pro-active.

I wrote to express my concerns to publishers of all four journals, and had my correspondence acknowledged. But then? Well, not a lot.

It's clear that Elsevier has taken some action. Indeed, my first blogpost was prompted by Michelle Dawson noting on Twitter that the editorial boards of RASD and RIDD had mysteriously disappeared from the online journals. She had previously noted Matson's pattern of mega-self-citation, and I had written directly to him, with copy to the publisher, some months previously to express concern, when I realised that I was listed as a member of the editorial board of RASD. Elsevier did not acknowledge my letter, but it is possible that the changes to the editorial boards that they had started were linked to my concerns.

The first direct response I had from Elsevier was some weeks after my final blogpost, when they explained that they were looking into the situation regarding unreviewed papers, but that this was a huge job and would take a long time. They were presumably disinclined to rely on the files that I had deposited on Open Science Framework, which show the identity and submission and acceptance data for every paper in RASD and RIDD.  They did appoint new editors and a small group of associate editors for both journals, all with good track records for integrity.

I have heard on the grapevine that they are now evaluating articles published in those journals that have been identified as not having undergone peer review; some of those approached to do these evaluations have mentioned this to me. It's rather unclear how this is going to work, given that, across the two journals, there are nearly 1000 papers where the available data indicate a lag from receipt to acceptance of under 2 weeks. I guess we should be glad that at least the publisher is taking some action, albeit at a snail's pace, but I am dubious as to whether there will be any retractions.

Meanwhile, Developmental Neurorehabilitation changed publisher around the time I was writing, and is now under the care of Taylor and Francis. I wrote to the publisher explaining my concerns and received a polite reply, but then heard no more. I note that the Editor in Chief is now Wendy Machalicek, who previously co-edited the journal with Russell Lang. Lang's doctoral advisor was Mark O'Reilly, editor of JDPD, and one of the prolific trio who featured in blogpost 2. Lang himself co-authored 24 papers in RASD and 13 in RIDD, and 35 of these 39 papers were accepted within 2 weeks of receipt. Machalicek has published 11 papers in RASD and 5 in RIDD, and 12 of these 16 papers were accepted within 2 weeks of receipt.  She also did her doctorate in O'Reilly's department, and several of her papers are co-authored with him. In an editorial last year, Lang and Machalicek announced changes to the journal, some of which seem to be prompted by a desire to make the reviewing process more rigorous under the new publisher. However, one change is of particular interest: the scope of the journal will be broadened to consider "developmental disability from a lifespan perspective; wherein, it is acknowledged that development occurs throughout a person's life and a range of injuries, diseases and other impairments can cause delayed or abnormal development at any stage of life." That will be good news for Giulio Lancioni, who was previously publishing papers on coma patients, amyotrophic lateral sclerosis, and Alzheimer's disease in RIDD. He and his collaborators – Jeff Sigafoos, Mark O'Reilly, as well as Russell Lang and Johnny Matson – are all current members of the editorial board of the journal.

It seems to be business as usual at the Springer title, Journal of Developmental and Physical Disabilities. Mark O'Reilly is still the editor, with Lang and Sigafoos as associate editors; Lancioni, Machalicek and Matson are all on the editorial board. Springer's willingness to turn a blind eye to editors playing the system becomes clear when one sees that a recent title, "Review Journal of Autism and Developmental Disorders" has as Editor-in-Chief no less a personage than Johnny Matson. And, surprise, surprise, the editorial board includes Lang, Sigafoos and Lancioni.

One of the overarching problems I uncovered when navigating my way around this situation was that there is no effective route for a whistleblower who has uncovered evidence of dubious behaviour by editors. Elsevier has developed a Publishing Ethics Resource Kit  but it is designed to help editors dealing with ethical issues that arise with authors and reviewers. The general advice if you encounter an ethical problem is to contact the editor. The Committee on Publication Ethics also issues guidance, but it is an advisory body with no powers. One would hope that publishers would act with integrity when a serious problem with an editor is revealed, but if my experience is anything to go by, they are extremely reluctant to act and will weave very large carpets to brush the problems under.