Showing posts with label Scientific Reports. Show all posts
Showing posts with label Scientific Reports. Show all posts

Wednesday, 1 January 2025

Retrospective look at blog highlights of 2024: What happened next?

It's always interesting for a blogger to look back to see which posts have garnered most attention. In 2024 there were three standout items, my resignation from the Royal Society, an Open Letter about editorial failings at the journal Scientific Reports, and a guest post by René Aquarius about his experiences as a reviewer for MDPI. For each of these blogposts, it's interesting to consider not just the number of hits, but also the broader impact. Blogging is a great way to let off steam, but does it actually achieve anything?

Resignation from the Royal Society 
As I said at the time of writing the post, I don't expect my resignation to change much at the Royal Society, but I was surprised at the amount of interest in the matter. The story was picked up by national media (e.g. here and here), and even got covered in France and the Netherlands. I had anticipated I might be attacked by a swarm of trolls, but there was no more than a handful of rude emails, saying the Royal Society was better off without me. What was more remarkable was the outpouring of positive reactions. My inbox was swamped with people thanking me for taking this step - not just academics, but people from all walks of life. It's customary when elected to the Royal Society for colleagues to send congratulatory emails: the number of congratulations I received for resigning was about five times as many as I had for being elected. I felt I had struck a chord with many people who were fed up with unsuitable individuals being honoured. British people in particular were outraged that the Royal Society was honouring someone who was spreading misinformation with the apparent aim of undermining our democracy.

I've seen only one serious attempt to challenge the arguments in my blogpost - by Toby Young in the Spectator. I can imagine that the comms team at the Royal Society may not have been overjoyed at his main line of argument, which was to say that there had been many Fellows of the Royal Society with rebarbative opinions who hadn't been thrown out, so why should they start now? He described me as a left-wing Torquemada, which I rather enjoyed, except that his implication was that my objections to Musk were political. Had he read my blogpost, he'd see that politics didn't come into it - it is the lack of scientific integrity that is the issue.

As a delightful twist to the story, Toby Young, who is notorious for his foul-mouthed comments on women and disabled people, has just been appointed to the House of Lords, showing that the Royal Society is not alone among the British establishment in honouring those who represent the antithesis of their core values. I can only assume that Kemi Badenoch, who was responsible for his elevation, did it to own the libs.
This letter, cosigned by 22 scientific sleuths, noted that laughably awful papers were getting published in the Springer Nature journal Scientific Reports, which is generally regarded as a serious journal. It looked to us as if the journal was being infiltrated by paper mills who were using it as an outlet for fraudulent or low quality work. For this to happen, there need to be editors who are either turning a blind eye to such articles, or were actively working with a paper mill to accommodate them. A quick analysis of the editors listed on the journal website turned up 28 who had records on PubPeer indicative of involvement in research misconduct.

So what has happened since? We received a reply from Chris Graf, the Research Integrity Director at Springer Nature, thanking us for our letter and emphasising the extent to which the publisher was putting resources into tackling research fraud and paper mills. The article that had sparked off the Open Letter was retracted fairly promptly.

But when I rechecked the Editorial Board list today, 26 the 28 editors we'd listed were still in position - even though we'd given links to PubPeer entries that specified problematic behaviours of all of them. The two who are no longer listed are Ilyas Khan, and Achyut Shankar. I have no idea whether their disappearance from editorial roles at Scientific Reports has anything to do with the Open Letter. (Rule 1 of sleuthing: don't expect anyone in power to tell you if they have acted on information you provided).

And a final wrinkle to the story: a fellow sleuth told me about another editor linked to paper mills, Masoud Afrand, who had not been included in our list. He had actually been removed from the list of editors in March 2022, but then reappeared at some point in the summer of 2024 (as verified by the Wayback Machine). The Editor-in-Chief, Rafal Marszalek, explained this as a "clerical error". In email correspondence I pointed out to him that this was a pretty odd kind of clerical error, and that it did nothing to assuage growing concerns that there may be someone at a senior level at Scientific Reports who was actively working with paper millers.  I did not get a reply.

My experience as a reviewer for MDPI: guest post by René Aquarius
In this post, René describes the fate of a paper that he agreed to review for the Journal of Clinical Medicine, published by MDPI. The paper had numerous methodological flaws, including lack of a control group and discrepancies between the registered protocol and the final study, and René recommended rejection. What then followed was a bizarre series of exchanges with MDPI editors, who encouraged resubmission by the authors, then attempted to block René from re-reviewing the paper, after he'd agreed to do so. Eventually the paper was transferred to  another MDPI journal, Geriatrics, where he was again asked to review it. When he pointed out that the paper was largely unchanged from the original submission, it was withdrawn - but he then found it published a few weeks later in yet another MDPI journal.

This blogpost attracted a lot of comments, not least from people who had had similar experiences with MDPI. I have to say that these experiences, coupled with other evidence such as this and this make me very dubious about standards of peer review at MDPI. The impression is that hard-pressed editorial staff are expected to ensure peer review of submissions is achieved in 2-3 weeks, and they accordingly come to rely on anyone who will do a speedy job, without regard for quality. Indeed, high-quality reviews that raise difficult issues are problematic for them, because they will slow up the process. Accordingly these are sidelined or quietly forgotten about, in favour of minimalist 'reviews' such as those you can see here in reviews 1, 3 and 4.

Will anything change? Hopefully, René's post has helped raise awareness of the problematic aspects of peer review at MDPI journals. I doubt the publisher will do anything to change their peer review model unless the reputational damage from such revelations starts to hit their bottom line. Interestingly, in my last post of 2024, I noted that just before the Christmas break, Finland's Publication Forum (JUFO) downgraded the status of 187 MDPI journals in their index, a move that will disincentivize Finnish researchers from publishing there. However, a day later, MDPI announced that it had secured a national publishing agreement with ZB Med, which offered substantial discounts to authors from over 100 German Universities.

Epilogue
In each of these cases, we can see that a blogpost has raised the profile of an issue that is of relevance to academic scientists. Concrete impacts are harder to demonstrate - where changes happen, it's not clear if they are related to the blogpost, or just coincidental. And for most topics, it's a case of two steps forward and one step back, at best. But if I've learned one thing from my many years writing about such things, it's that you can't trust the people in positions of power to actually do the right thing unless they are prodded repeatedly and publicly (and sometimes not even then). So I hope to continue blogging through 2025, even though it may at times seem futile.

Happy New Year, and Illegitimi non carborundum.

Comments are moderated. Those that are in scope and from an identified person will be published after a delay. 


Wednesday, 16 October 2024

An open letter regarding Scientific Reports

16th October 2024 

to: Mr Chris Graf
Research Integrity Director, Springer Nature and Chair Elect of the World Conference on Research Integrity Foundation Governing Board.

 

Dear Mr Graf,

We are a group of sleuths and forensic meta-scientists who are concerned that Springer Nature is failing in its duty to protect the scientific literature from fraudulent and low quality work. We are aware that, as noted in the 2023 Annual Report, you are committed to maintaining research integrity. We agree with the statement: “To solve the world’s biggest challenges, we all need research that’s reliable, trustworthy and can be built on by scientists and innovators. As a leading global research publisher, we have a pivotal role to play.” It is encouraging to hear that the Springer Nature research integrity group doubled in size in 2023. Nevertheless, we have a growing sense that all is not well concerning the mega journal Scientific Reports.

Some of the work that has been published is so seriously flawed that it is not credible that it underwent any meaningful form of peer review. In other cases, when we have reported flawed papers to the editor or integrity team, the response has been inadequate. A striking example cropped up last week when a “corrected” version of an article was published in Scientific Reports. This article had been flagged up by Guillaume Cabanac as containing numerous “tortured phrases” that are indicative of fraudulent authors attempting to bypass plagiarism checks; the authors were allowed to “correct” the article by merely removing some (not all) of the tortured phrases. This led some of us to look more closely at the article. As is evident from comments on PubPeer, it turned out to be a kind of case study of all the red flags for fraud that we look for. As well as (still uncorrected) tortured phrases, it contained irrelevant content, irrelevant citations, meaningless gibberish, a nonsensical figure, and material recycled from other publications.

This is perhaps the most flagrant example, but we argue that it indicates problems with your editorial processes that are not going to be fixed by AI. The only ways an article like this can have been published are either through editorial negligence or outright malpractice. For it to be negligence would require a remarkable degree of professional incompetence from a handling editor. The possibility of malpractice, would mean there is a corrupt handling editor who bypasses the peer review process entirely or willingly appoints corrupt peer reviewers to approve the manuscript. We appreciate that some papers that we and others have reported have been retracted, but in other cases blatantly fraudulent papers can take years to be retracted or to receive any appropriate editorial action.

We have some specific suggestions for actions that Springer Nature could take to address these issues.

  1. Employ a task force of people with the necessary expertise to carry out an urgent audit of all editors of Scientific Reports. We have looked at the editors on your website, and it is clear that this is an enormous task, given that there are over 13,000 of them, and they are not listed with disambiguating information such as Orcid IDs. Even so, in a few hours, by cross-checking this list against PubPeer, it was possible to identify the 28 cases listed below, covering a range of disciplines, and all, in our view, with pretty clear-cut evidence of problems. Four are members of the Editorial Board. We stress, this is just the low-hanging fruit which was fairly easy to detect.
  2. The list of problematic articles appended below or tabulated on the Problematic Paper Screener might provide an alternative route to identify editors who should never have been given a gatekeeping role in academic publishing. As well as checking the papers we list below, we recommend that all other articles accepted by the same editors should be scrutinised.
  3. Detection of problematic articles and editors could be helped by requiring open peer review for all journals, and ensuring that the name and Orcid ID of the handling editor is included with the published meta-data for all articles.
We hope these suggestions will be helpful in ensuring that research published in Scientific Reports is reliable and trustworthy.

Yours sincerely

Dorothy Bishop
Guillaume Cabanac
François-Xavier Coudert
René Aquarius
Nick Wise
Lonni Besançon
Simon A.J. Kimber
Anna Abalkina
Rickard Carlsson
Samuel J Westwood
Patricia Murray
Nicholas J. L. Brown
Smut Clyde
Leonid Schneider
Ian Hussey
Tu Duong
Gustav Nilsonne
Jamie Cummins
Alexander Magazinov
Elisabeth Bik
Mu Yang
Corrado Viotti
Sholto David


 

Appendices

1. Some examples of editors with concerning PubPeer entries

Editorial board Ghulam Md Ashraf
Editorial board Eun Bo Shim
Editorial board Ajay Goel
Editorial board Rasoul Kowsar

AGEING Vittorio Calabrese
AGRICULTURE Sudip Mitra
ANALYTICAL CHEMISTRY Syed Ghulam Musharraf
CELL BIOLOGY Gabriella Dobrowolny
CHEMICAL ENGINEERING Enas Taha Sayed
CIVIL ENGINEERING Manoj Khandelwal
CLINICAL ONCOLOGY Marcello Maggiolini
COMPUTATIONAL SCIENCE Praveen Kumar Reddy Maddikunta
DRUG DISCOVERY Salvatore Cuzzocrea
ENDOCRINOLOGY Sihem Boudina
ENVIRONMENTAL ENGINEERING Rama Rao Karri
ENVIRONMENTAL SCIENCE Mayeen Uddin Khandaker
GASTROENTEROLOGY AND HEPATOLOGY Sharon DeMorrow
IMMUNOLOGY Marcin Wysoczynski
INFECTIOUS DISEASES Fatah Kashanchi
MATHEMATICAL PHYSICS Ilyas Khan
MICROBIOLOGY Massimiliano Galdiero
NETWORKS AND COMPLEX SYSTEMS Achyut Shankar
NEUROLOGY Yvan Torrente
RESPIRATORY MEDICINE Soni Savai Pullamsetti
STRUCTURAL AND MOLECULAR BIOLOGY Stefania Galdiero
https://pubpeer.com/publications/42901FD2901EC917E3EE54B8DBD749#4 (authors claim a correction is underway, but none published for 2 years)
https://pubpeer.com/publications/01FE09F1127DF0598985987677A101 (part of a list of many flagged papers from this author group. Corrected rather than retracted)
https://pubpeer.com/publications/69EDBAECD50F31B051ECECCD1DF346 (notified on 31-3-2023 about this paper, no action so far)
https://pubpeer.com/publications/F8A1AD2B165888A06C18B28C860E7B. EiC contacted Nov. 22 with authorship concerns, responded that he would investigate. No action taken so far.
https://pubpeer.com/publications/286F83F9553D29F82CD4281309A1E4. Has had EoC for authorship irregularities since July 22, no action taken since.
https://pubpeer.com/publications/5BEDDDA9CF92B9CDDD2AB1AA796271 (blatantly nonsensical paper reported to publisher in June 2024; no action as yet)
https://pubpeer.com/publications/37B87CAC48DE4BC98AD40E00330143 (various corrections since 2022, and in Feb 2023 readers were told “conclusions of this article are being considered by the Editors. A further editorial response will follow the resolution of these issues”. 19 Months later we are still waiting.)


3. Some examples of journal-level reports posted on PubPeer

Scientific Reports

other Springer Nature journals:

Chemosphere