Editorial integrity: Publishers on the front line

Thanks to some live tweeting by Anna Sharman (@sharmanedit), I've become aware that the 13th Conference of the European Association of Science Editors (EASE) is taking place in Strasbourg this weekend.

The topic is "Scientific integrity: editors on the front line", and the programme acknowledges Elsevier, who presumably have contributed funding for the conference.

It therefore seems timely to give a brief update of developments following three blogposts I wrote during February-March 2015, documenting some peculiar editorial behaviour at four journals: Research in Autism Spectrum Disorders (RASD: Elsevier), Research in Developmental Disabilities (RIDD: Elsevier), Developmental Neurorehabilitation (DN: Informa Healthcare) and Journal of Developmental and Physical Disabilities (JDPD: Springer).

To do the story full justice, you need to read these blogposts, but in brief, blogpost 1 described how Johnny Matson, the then editor of both RASD and RIDD had published numerous articles in his own journal, and engaged in frequent self-citation, leading to his receiving a 'highly cited' badge from Thomson Reuters. In the comments on that blogpost, another intriguing factor emerged, which was Matson's tendency to accept papers with little or no review. This was denied by Elsevier, despite clear evidence of very short acceptance lags that were incompatible with review.

Blogpost 2 was prompted by Matson defending himself against accusations of self-citation by pointing out that he published in journals that he did not edit. I checked this out and found he had numerous papers in two other journals: DN and JDPD, and that the median lag between a paper of his being submitted and accepted in DN was one day. (JDPD does not provide data on publication lags). I therefore looked at the editors of those journals, and found that they themselves were publishing remarkable numbers of papers in RASD and RIDD, again with extremely short publication lags. A trio of editors and editorial board members (Jeff Sigafoos, Giulio Lancioni and Mark O'Reilly), co-authored no less than 140 papers in RASD and RIDD between 2010 and 2014, typically with acceptance times of less than 2 weeks. Some of the papers in RIDD were not even in the topic area of developmental disabilities, but covered neurological conditions acquired in adulthood.

In blogpost 3, I turned the focus on to the publisher of RASD and RIDD, Elsevier, to query why they had not done anything about such irregular editorial practices. I did a further analysis of publication lags in RIDD, showing that they had dropped precipitately between 2008 and 2012, and that there was a small band of authors whose prolific papers were published there at amazing speed. I provided all the statistical data to support my case, including interactive spreadsheets that made it easy to determine which editors and authors had been benefiting from the slack editorial standards at these journals.

There was some interesting fall-out from all of this. The second blogpost drew fire from supporters of the editors I had "outed", accusing me of bad behaviour and threatening to complain to my university. Since everything I had said was backed by evidence, this did not concern me. I received heartfelt messages of support from people who were appalled that a particular approach to autism intervention had been promoted by this group of editors, who were in effect using their status to gain the veneer of scientific credibility for work which was not in fact peer-reviewed.  I was also contacted by several academics telling me that everyone knew this had been going on for years, but nobody had done anything; this level of passivity was surprising given that many were angry that  authors had reaped benefits from their staggeringly high publication rate, while those who were outside the charmed circle were left behind. I was urged to go further and raise my concerns with the universities employing those who were capitalising on, or engaging in, lax editorial behaviour. I do, however, have an extremely demanding job and I hoped that I had done enough by shining a light on dubious practices, and providing the full datasets that provided evidence. However, I now wonder if I should have been more pro-active.

I wrote to express my concerns to publishers of all four journals, and had my correspondence acknowledged. But then? Well, not a lot.

It's clear that Elsevier has taken some action. Indeed, my first blogpost was prompted by Michelle Dawson noting on Twitter that the editorial boards of RASD and RIDD had mysteriously disappeared from the online journals. She had previously noted Matson's pattern of mega-self-citation, and I had written directly to him, with copy to the publisher, some months previously to express concern, when I realised that I was listed as a member of the editorial board of RASD. Elsevier did not acknowledge my letter, but it is possible that the changes to the editorial boards that they had started were linked to my concerns.

The first direct response I had from Elsevier was some weeks after my final blogpost, when they explained that they were looking into the situation regarding unreviewed papers, but that this was a huge job and would take a long time. They were presumably disinclined to rely on the files that I had deposited on Open Science Framework, which show the identity and submission and acceptance data for every paper in RASD and RIDD.  They did appoint new editors and a small group of associate editors for both journals, all with good track records for integrity.

I have heard on the grapevine that they are now evaluating articles published in those journals that have been identified as not having undergone peer review; some of those approached to do these evaluations have mentioned this to me. It's rather unclear how this is going to work, given that, across the two journals, there are nearly 1000 papers where the available data indicate a lag from receipt to acceptance of under 2 weeks. I guess we should be glad that at least the publisher is taking some action, albeit at a snail's pace, but I am dubious as to whether there will be any retractions.

Meanwhile, Developmental Neurorehabilitation changed publisher around the time I was writing, and is now under the care of Taylor and Francis. I wrote to the publisher explaining my concerns and received a polite reply, but then heard no more. I note that the Editor in Chief is now Wendy Machalicek, who previously co-edited the journal with Russell Lang. Lang's doctoral advisor was Mark O'Reilly, editor of JDPD, and one of the prolific trio who featured in blogpost 2. Lang himself co-authored 24 papers in RASD and 13 in RIDD, and 35 of these 39 papers were accepted within 2 weeks of receipt. Machalicek has published 11 papers in RASD and 5 in RIDD, and 12 of these 16 papers were accepted within 2 weeks of receipt.  She also did her doctorate in O'Reilly's department, and several of her papers are co-authored with him. In an editorial last year, Lang and Machalicek announced changes to the journal, some of which seem to be prompted by a desire to make the reviewing process more rigorous under the new publisher. However, one change is of particular interest: the scope of the journal will be broadened to consider "developmental disability from a lifespan perspective; wherein, it is acknowledged that development occurs throughout a person's life and a range of injuries, diseases and other impairments can cause delayed or abnormal development at any stage of life." That will be good news for Giulio Lancioni, who was previously publishing papers on coma patients, amyotrophic lateral sclerosis, and Alzheimer's disease in RIDD. He and his collaborators – Jeff Sigafoos, Mark O'Reilly, as well as Russell Lang and Johnny Matson – are all current members of the editorial board of the journal.

It seems to be business as usual at the Springer title, Journal of Developmental and Physical Disabilities. Mark O'Reilly is still the editor, with Lang and Sigafoos as associate editors; Lancioni, Machalicek and Matson are all on the editorial board. Springer's willingness to turn a blind eye to editors playing the system becomes clear when one sees that a recent title, "Review Journal of Autism and Developmental Disorders" has as Editor-in-Chief no less a personage than Johnny Matson. And, surprise, surprise, the editorial board includes Lang, Sigafoos and Lancioni.

One of the overarching problems I uncovered when navigating my way around this situation was that there is no effective route for a whistleblower who has uncovered evidence of dubious behaviour by editors. Elsevier has developed a Publishing Ethics Resource Kit  but it is designed to help editors dealing with ethical issues that arise with authors and reviewers. The general advice if you encounter an ethical problem is to contact the editor. The Committee on Publication Ethics also issues guidance, but it is an advisory body with no powers. One would hope that publishers would act with integrity when a serious problem with an editor is revealed, but if my experience is anything to go by, they are extremely reluctant to act and will weave very large carpets to brush the problems under.

Will Elsevier say sorry?

Elsevier, the publisher of Research in Autism Spectrum Disorders (RASD) and Research in Developmental Disabilities (RIDD) is no stranger to controversy. It became the focus of a campaign in 2012 because of its pricing strategies and restrictive practices. Elsevier responded that the prices it charged were fair because of the added value it brought to the publishing process. Among other things they claimed that: "We pay editors who build a distinguished brand that is set apart from 27,000 other journals. We identify peer reviewers."

Well, the claim of added value is exploded by the recent revelations of goings-on at RASD and RIDD as documented in my recent blogposts here and here. One of the points to emerge from a detailed scrutiny of publications data is that not only was there evidence of the editor publishing numerous papers in his own journal: in addition, he was frequently bypassing the peer review process altogether. Evidence on this point is indirect, because the published journals do not document the peer review process itself. However, it is possible to look at the lag from receipt of the paper to acceptance, which can be extracted separately for each individual paper. I have looked at this further using merged data on publication lag with information available from Web of Science to create a dataset that contains all papers published in RIDD between 2004 and 2014, accompanied by the dates for receipt and acceptance of papers.

I was able to extract this information for 2060 papers. There were 23 papers for which the date information was not available: 15 of these were authored by the editor. The median publication lag for the remainder is shown by year in Figure 1.  This shows a fascinating progression.

Figure 1. Acceptance lags in days, and % papers with revision

Between 2004-2006, it seems that RIDD was a perfectly normal journal. The lag from receipt to acceptance was 3-4 months, and nearly all papers underwent some revision during that period. Fewer than 4% of papers were accepted within 3 weeks. In 2007, the sister journal RASD was launched, and the editor started to accept papers with shorter lags and without revision. During 2008-2011, the median lag between receipt and acceptance fell to two weeks or less, and only a minority of papers underwent revision.  Subsequently, this pattern of editorial behaviour was reversed to some extent, and by 2014 we were back to a median lag of 80 days with 78% of papers undergoing revision.

Nevertheless, all was not well. In my previous post, I noted that a group of people with editorial associations with these journals, Sigafoos, O'Reilly and Lancioni, had published numerous papers in RASD and RIDD. I had analysed the acceptance lags for RASD and shown they were substantially shorter than for other authors.  Figure 2 shows part of a larger figure which can be found here; it demonstrates that remarkably short acceptance times for papers with Sigafoos as author (usually accompanied by Lancioni and O'Reilly) were also seen in RIDD. The full dataset on which these figures are based is available here*.

Figure 2: Lag from receipt to acceptance for RIDD papers 2010-2014. Black dots show papers authored by Sigafoos

It is difficult to believe that nobody at Elsevier was aware of what was going on.  In 2011, at the height of the rapid turnaround times, there was a fivefold increase from the 2004 level of submissions. Many journals have grown in size over this period, but this was massive. Furthermore, the publisher was recording the dates of receipt and acceptance with each paper: did nobody actually look at what they were publishing and think that something was odd? This was not a brief hiccup: it went on for years. Either the publisher was slumbering on the job, or they were complicit with the editor.

Most academics find publishing a paper a stressful and tedious business. How nice, then, you might think, to have a journal that speeds the process along and avoids the need to grapple with reviewer comments. I have heard from several people who tell me they published a paper in RIDD; when it was accepted without review, they were surprised, but hardly in a mood to complain about it.  So does it matter?

Well, yes. It matters because RIDD and RASD are presented to the world as peer-reviewed journals, backed up by the 'distinguished brand' of Elsevier. We live in times when there is competition for jobs and prizes, and these will go to those who have plenty of publications in peer-reviewed journals, preferably with high citations. If an editor bypasses peer review and encourages self-citation, then the quality of the work in the journal is misrepresented and some people gain unfair advantages from this. The main victims here are those who published in RASD and RIDD in good faith, thinking that acceptance in the journal was a marker of quality. They will be feeling pretty bitter about the 'added value' of Elsevier right now, as the value of their own work will be degraded by association with these journals.

It is not surprising that Elsevier wants to focus on the future rather than on the past. They are a signatory to the Committee on Publication Ethics, and the discovery that two of their journals have flouted numerous of the COPE guidelines is an embarrassment that could have commercial implications. But I'm afraid that, while I understand their position, I don't think this is good enough. Those who published good work in journals where the publisher failed in its duty of oversight deserve an acknowledgement that there were problems and an apology.

* Update 23rd March 2015: Data now added for Research in Autism Spectrum Disorders in a separate Excel file.