Sunday, 1 February 2015

Journals without editors: What is going on?



Last October, I was surprised to see a tweet from @autismcrisis (Michelle Dawson) saying "Belatedly noticed: @deevybee is on the editorial board of Johnny Matson's RASD?! Well I'm speechless. Wow."
Many of those reading this tweet were puzzled as to what this was all about, but I was aware that Michelle had earlier drawn attention to an odd feature of the journal Research in Autism Spectrum Disorders (RASD): an unusually high proportion of the papers in the journal were authored by the Editor, Johnny Matson*.
People were even more puzzled by my reaction, which was to express surprise and to thank Michelle for telling me. Surely, you might think, you'd know if you were on the Editorial Board of a journal. But, actually, that's not always the case. I had no recollection of having joined the Editorial Board, but my memory for the past is not good. I did remember having some correspondence with Johnny Matson around 2008, about a nice article he'd written concerning problems with some of the 'gold standard' diagnostic instruments (see this blogpost for more on this). I suspected that I'd agreed to serve on the Editorial Board in the course of this email exchange, but I had no details of this on file. Journals vary considerably in how far they treat their Editorial Board as emblematic, and how far they actually make use of the board in decision-making. I had barely had any interaction with Research in Autism Spectrum Disorders over the years, and had remained sufficiently unaware of my Editorial Board role to omit this from my curriculum vitae. But there I was, as Michelle had noted, listed as a member of the Editorial Board on the journal website.
I realised that I had to take action for two reasons. First, RASD was an Elsevier journal. I had been convinced by the arguments of Tim Gowers to sign the Cost of Knowledge statement, pledging not to support Elsevier, in the light of their exorbitant pricing of journals. In 2012 I had resigned from the editorial boards of other Elsevier journals. But I'd been unaware that I was on the Editorial Board of RASD, so I had not written to them. Second, I was interested in Michelle's concerns about editorial practices at the journal. I thought I had better check her claims out for myself. What I found was disturbing.  Matson had been editor of Research in Developmental Disabilities (RIDD) since the journal's inception in 1987, and he then took up editorship of the sister journal, Research in Autism Spectrum Disorders when it started in 2007. As can be seen in Figure 1, his publication rate shot up around 2007, and an analysis using Web of Science reveals that many of these papers were published in RASD or RIDD. Indeed, Matson is an author on over ten percent of the papers published in RASD since 2007.
Figure 1: Publications identified from Web of Science**



Around 2007, Matson's papers also started being highly cited as can be seen in Figure 2. 

Figure 2: Matson citations from Web of Science**
On Web of Science he currently has an H-index of 59: those of you who know about these things will recognise this as an impressive value that would usually be indicative of a scientist who does highly influential work. His University webpage proudly displays the 'Highly Cited' badge from Thompson Reuters. However, there is something unusual about Matson's citation profile: just over half of the citations are self-citations. I did some comparisons with other top scientists in the field of autism/intellectual disabilities, and it is clear that Matson is an outlier in terms of his rate of self-citation (see Figure 3).
Figure 3: Comparison of self-citation rate for autism/ID researchers***

I wrote to Matson to query my editorial board status and to explain why I wished to resign and received a curt reply, confirming that I had agreed to be on the Editorial Board, but he would remove me. Nevertheless, my name remained on the Editorial Board list of the journal website for a while. But then, a few weeks ago, there was a new development. For both RIDD and RASD, the pages on the journal website showing information about the Editors and Editorial Board disappeared. What, I wonder, is going on?

* (update 22 Mar 2015): Michelle's tweets on this topic can be accessed from the sidebar on her blogpost
**Search terms were AUTHOR: (matson j*) AND TOPIC: (autism OR intellectual)
***Here I simply selected well-known researchers whose names were distinctive enough to make a search unchallenging. Where necessary to disambiguate authors I added the same Topic search term as for Matson  

Update 5th February 2015
In the past few days there have been some new developments. 
On 2nd Feb, Alicia Wise (@wisealic) of Elsevier responded to a concerned tweet from @DavidPriceUCL to say
 Hi, David - colleagues have been speaking to community about new Editors-in-Chief; appointments will be announced shortly. 
Then on 4th Feb, Michelle Dawson (@autismcrisis) tweeted:RASD finally has (cryptic) editorial board info for March 2015 issue compare to Feb 2015 issue  
You have to download the pdfs to see the information. The Feb issue list Matson as editor and gives the full editorial board. The March issue deletes the editorial board but retains Matson as editor. We might, from Alicia Wise's tweet, have expected the opposite.

I have, of course,  no idea if these changes relate to anything in this blogpost. 

Update 7th February 2015
Michelle Dawson pointed out on Twitter that back in June 2013 she suggested that someone should look at how far the impact factor of RASD and RIDD is affected by Matson's self-citations. I doubt there'd be much effect on RIDD, where Matson's papers constitute only around 1% of articles, but I did the sums for RASD. Readers who have access to Web of Science can see the historical impact factor data here. I simply recomputed the data after searching for papers in a 2-year period with the search term NOT Matson J* as author. I had to read the N citation data from the bar plot you get from citation reports in Web of Science, so precision not guaranteed, but here's what it looks like:


P.S. I am hearing on the grapevine that people have had papers accepted in RASD and RIDD without reviewing, but nobody seems willing to say that publicly. If there are any brave souls out there who are prepared to speak out, please can you do so via Comments. Thanks. 

Update 14th February 2015

In the comments below, Michael Osuch, Publishing Director for Neuroscience & Psychology journals at Elsevier, has added some notes of clarification. He states: "Under Dr Matson’s editorship of both RIDD and RASD all accepted papers were reviewed, and papers on which Dr Matson was an author were handled by one of the former Associate Editors. Dr Matson and his team of Associate Editors stepped down at the end of 2014."
The issue of Matson's papers being handled by an Associate Editor is key. When discussing this point, some people have argued that editors should never publish in their own journal. I disagree, and think it is reasonable provided (a) it is a relatively rare occurrence and (b) the paper is handled by an Associate Editor (AE) who ensures that the paper is subjected to a rigorous review. In such cases it is crucial that the AE is someone who will not be unduly influenced by their relationship with the editor.
In this regard it is worrying to see that two people who were, until 2015, the first listed AEs on both RASD and RIDD are relatively junior with close links to Matson. Thompson Davis III is an associate professor in Matson's department with expertise in autism and anxiety disorders. To identify his publications on Web of Science I searched for AUTHOR: (Davis T*) AND TOPIC: (autism OR anxiety) AND ADDRESS: (Louisiana State University). This brought up 41 publications. Only a few of these are published in RASD or RIDD (five papers from 2007 to 2011), but he has eleven co-authored papers with Matson. Jill Fodstad's CV indicates she took the Clinical Psychology program at Louisiana State University, where it seems that Matson encouraged students to co-author papers with him. (This is evident from an analysis of his coauthors in RASD/RIDD). She is now based at Indiana State University.  55 of her 59 publications are co-authored with Matson, and 30 of them are published in RASD or RIDD. 
Good editorial practice would dictate that neither of these AEs should be assigned papers authored by Matson because the unequal power relationship with him would make it extremely difficult to give a dispassionate appraisal of the work.
Of course, I do not know which AEs did handle Matson's papers, but I would be surprised if an experienced senior editor would have accepted his manuscripts without challenging the high rate of self-citation.

Update 15th February 2015
An email from a colleague reads as follows re another Associate Editor at RASD, Jeff Sigafoos:
"I have heard of several colleagues having papers accepted extremely quickly at Developmental Neurorehabilitation.  The editor at that time was Jeff Sigafoos, but I think he has since stepped down from that position.
Anyway, I looked up Jeff Sigafoos website...look at his publication list: http://www.victoria.ac.nz/education/about/staff/publications-jeff-sigafoos
...an amazing number of papers in RIDD and RASD.  "
I also checked the submission/acceptance lag for Sigafoos' papers in RIDD. For the first 20 I found, 7 were accepted within one day and a total of 14 within one week. This does rather question the claim by Michael Osuch that Matson acted as sole referee 'in a minority of cases'.
I should add that Developmental Neurorehabilitation is not an Elsevier journal: it is published by Informa Healthcare.
Finally, you might ask whether anyone is hurt by this. I think the answer is that it is damaging to other people who published in any of these journals in good faith, and who now will have the validity of their work questioned because of inadequate peer review.

23 comments:

  1. Fascinating...out of curiosity, I looked at a recent paper in one of these journals, and the editorial information is as follows "Received 23 October 2013, Accepted 29 October 2013, Available online 21 December 2013"...yep, a six day turn around between initial submission and official acceptance.

    ReplyDelete
  2. Wow! I had a look at a few and the turnaround is certainly brisk - though this seems to apply to all papers, and not just those authored by Matson. Just saw one with other authors that had:
    Received 21 June 2011
    Received in revised form 26 June 2011
    Accepted 27 June 2011
    Available online 4 August 2011

    ReplyDelete
  3. well, if you're going to publish 50+ papers a year, you have to get 'em in and get 'em out on a weekly basis.

    ReplyDelete
  4. As the Publishing Director for Neuroscience & Psychology journals at Elsevier, I would like to clarify some issues addressed in your blog.

    Under Dr Matson’s editorship of both RIDD and RASD all accepted papers were reviewed, and papers on which Dr Matson was an author were handled by one of the former Associate Editors. Dr Matson and his team of Associate Editors stepped down at the end of 2014, and he retains a honorary position of ‘Founding Editor’, which does not include handling manuscripts.

    Since last November we have been engaging with members of the community to identify potential replacements for both titles. We welcome further engagement with the community to build the appropriate editorial teams, and would welcome expressions of interest from suitably qualified scholars.

    In the meantime, our publishing staff have been working, alongside a freelance editor, at securing at least two suitable expert referees for papers submitted. We are working to avoid a backlog and limit any potential delays for authors of those papers. Editorial decisions themselves will be made by the new editorial team when appointed shortly. As we move towards making those appointments, we will ensure that quality peer review remains our priority.

    ReplyDelete
  5. Sorry, Dr Osuch, but your statement 'Under Dr Matson’s editorship of both RIDD and RASD all accepted papers were reviewed' is simply untrue. I have been contacted by authors who were happily surprised to find their papers accepted without review. People don't like to admit to it (as it devalues their paper) but it happened, and the short gaps between receipt and acceptance of papers would be impossible in some cases unless there were no review. Has Elsevier checked the files and seen the referee reports?

    ReplyDelete
  6. Yes, we have looked at the files. In a minority of cases, Dr Matson acted as sole referee. Our focus is now on the future by bring on board new editorial teams whose priority will be to ensure that all accepted articles have a minimum of two referee reports.

    ReplyDelete
    Replies
    1. Thank you for the clarification. For further comment on the issue of Associate Editors, please see the update dated 14th Feb at the end of this blogpost.

      Delete
  7. "Dr Matson acted as sole referee"

    It doesn't feel right for an editor to act as sole referee. I wonder whether all published articles should include details of the number and names of the referees. (I think I've seen this somewhere - in Frontiers.)

    ReplyDelete
    Replies
    1. It is unusual. But the Author Guidelines of both RIDD and RASD contain the following text:

      "In order to maintain a rapid rate of review all submitted manuscripts are initially reviewed by the Editor in Chief for completeness and appropriateness to the journal's stated Aims and Scope. Manuscripts that pass the initial review will be handled by the Editor, sent out to reviewers in the field, sent to an associate editor for handling, or some combination thereof, solely at the discretion of the Editor."

      This unusual clause (it appears nowhere else, only in the RIDD and RASD guidelines) seems to allow the Editor to not just reject, but also accept, papers, all on his own.

      Delete
    2. Thanks for the clarification, Neuroskeptic. I had guessed that the editor was allowed to do this, or the publishing director probably wouldn't have mentioned it so publicly!! But it still doesn't feel right - to me, anyway!

      But what about my suggestion that all published articles should include the names of people who reviewed it? I think it would make things more transparent and perhaps encourage more rigorous reviews. It's just a suggestion, though.

      Delete
    3. I agree with it. Also, we don't whether it works or not, yet it can allow us to evaluate data more clearly. However, Frans de Waal was not only the reviewer of a paper by his PhD student, but also the author. It was stated in the paper, leading us to know does it work?

      Do we need a study to see its effect?

      Delete
    4. I am sorry. I meant to say I agree with you.

      after know, it is as to whether it works?

      Is there a edit button anywhere?

      Delete
    5. Of course, there are very good reasons for anonymity for authors AND reviewers. We all keep score = tit for tat. But it would be informative to know the reviewers' recommendations. I recently reviewed a manuscript that was rejected by 2 of the 3 reviewers (after major revisions) but published regardless. The one reviewer who recommended acceptance had nothing concrete to say about the paper (and not much at all, actually) while the other two catalogued the many problems at length. So, rather than names, something like: Reviewer 1 - Reject, Reviewer 2 - Reject, Reviewer 3 - Accept without revisions. This should be done for initial submissions and each revision. I see problems with this too but it's certainly better than using names. It's really not a good idea (particularly for younger scholars) to make enemies in the field.

      Delete
  8. This comment has been removed by the author.

    ReplyDelete
  9. He is also editor of the newly established Springer "Review Journal of Autism and Developmental Disorders"

    I wonder what Springer have to say....

    ReplyDelete
    Replies
    1. Can they just make a rule of if you are an editor of one journal, then you cannot be an editor of another journal because it may be more likely to lead to higher self-citations and higher h-index, which they don't deserve? In other words, it focuses on our reputation rather than scientific rigour, thus not helping in science in the long run. We know he is human and not some other creature, but I think we need to be quite rigorous about that.

      Delete
  10. I’d like to add my perspective based on my attempt to publish in RIDD. It may be the case the favored colleagues were published without review or expedited review, but I’m concerned that work from junior faculty in areas other than autism were not given a fair chance or a true peer-review process. Last year I submitted a manuscript that was appropriate to RIDD’s stated Aims and Scope. It also was data-based, innovative, and clinically important. When I received the initial review of the manuscript I was very surprised. There was a definite mismatch between the two reviewers’ comments and the editor’s decision. The editor did not provide any feedback or summary with his decision. I discussed this mismatch with several of my senior colleagues, and I was uniformly advised to contact the editor about my concern. Below you can see the correspondence.

    Dear Dr. Matson:
    Thank you for the update on my manuscript “RIDDXXXX Title.” The reviewers had some very good suggestions for a revision, and the tone of the review is encouraging and provides direction for revision. Given this positive tone, I was surprised by the “reject” decision, because the reviewer comments felt more consistent with a “revise and resubmit” or “major revision” decision. It feels like there is mismatch between the reviewer comments and the final decision. Can you reconsider or explain this to me a bit?

    I look forward to hearing from you.


    Response from Jonny Matson:

    Well,
    The decision by the reviewers and more importantly me as action editor was reject. I suggest you take the points that make sense ,revise and resumbit elsewehere.

    Dr. Matson


    Upon receiving this curt response from Johnny Matson (his typos are left unchanged), I realized that my work did not have a chance in “his” publication and that he made the final decision but was not willing to discuss, explain, or expand upon the decision. My manuscript was not given full consideration, likely because the topic did not interest Johnny or because I was not one of his known colleagues. Johnny Matson abused his role as editor in that he used the journal as a venue to publish his own work and that of his colleagues, while he denied access and a peer review process to those who attempted to publish quality work. The manuscript I originally submitted to RIDD has since been accepted in a peer-reviewed journal. What was lost? My time and momentum in advancing my line of research was definitely compromised. For junior faculty it is deflating to realize that senior researchers will abuse their status and that peer-reviewed journals like RIDD and RASD can be hijacked by egomaniacs. It seems that Elsevier and other publishers need to establish and improve upon the procedures that editors implement for the peer review process. I applaud Elsevier for ousting king Johnny, and I hope that they will improve upon their process and make sure there are checks and balances in place to ensure that the peer review process actually occurs.

    ReplyDelete
    Replies
    1. This is appalling. I really hope his institution are looking into his abuse of power.

      I'm glad your paper ended up somewhere more deserving :)

      Delete
    2. I had a similar inexplicable rejection experience with Johnny Matson within 4 minutes of submitting when I had checked and the paper did indeed fit the scope of the journal. One of his comments was about the quality of the design being poor. I resubmitted the manuscript to unchanged to another journal of repute in the field had a review which accepted following revisions interestingly the subsequent review made specific comments on how well the study had been designed. This reduced my faith in the journal and in his judgement. In my experience it has been known and surreptitiously discussed for many years in the field at conferences and meetings and many have simply avoided that journal as a result. I'm very glad to see he's no longer on the editorial board of the journal and may consider submitting future work there now. I dislike the existence of such people in the academic world as I think they undermine our credibility and it's difficult enough to get our work out there and into the public domain as it is. I think it's shameful and exploitative.

      Delete
  11. Elsevier is pleased to announce the Editor-in-Chief appointment for Research in Autism Spectrum Disorders, with effect from 1st March 2015.

    Dr Sebastian Gaigg, Senior Lecturer in Psychology at City University London, UK, has accepted the position of Editor-in-Chief on Research in Autism Spectrum Disorders. Dr Gaigg began his academic career as a research assistant to Professor Bowler in the Autism Research Group at City University London, investigating learning and memory processes in Autism Spectrum Disorder (ASD) on successive grants funded by the Wellcome Trust and Medical Research Council. In parallel he completed his PhD on the role of emotion related processes in ASD and subsequently took up a permanent faculty position in the Psychology Department in 2009.

    Further announcements will be made shortly regarding the appointment of an editorial team to support Dr Gaigg.

    ReplyDelete
  12. Michael, these are steps in the right direction.

    ReplyDelete
  13. In response to the appeal for authors to confess to having papers accepted without review...

    I submitted a single authored review article to the Review Journal of Autism and Developmental Disorders, where Dr Matson is Editor-in-Chief, on 14th June 2013. The paper was assigned to "the Editor(s)' the next day.

    On 7th August I received the following message:
    "We are pleased to inform you that your manuscript, "A Targeted Review of Computer-Assisted Learning for People with Autism Spectrum Disorder: Towards a Consistent Methodology", has been accepted for publication..."

    I didn't receive any reviewer comments or further information. I've just checked my online author account and can find no record of reviews or reviewers there either.

    I was disconcerted of course but thought that maybe there had been reviews, but the comments were minor, and the editors were happy to accept the original. RJADD is a new journal and was published for the first time in January 2014 I think, so I also thought maybe they were under some pressure to secure content and needed to be quick / not very picky. I'm relatively early in my career and, while I review papers of course, I'm not on any editorial boards.

    I pushed it out of my mind - we all know how hard it is to get work published and this was my first single author paper so I had no-one to consult. Plus because it is a literature review I was a bit less worried than if I had been presenting original data. Even so, I have re-read that paper about 4 times since it was published, looking for faults and wondering if it is really any good. The experience was extremely unsettling.

    ReplyDelete