Tomatoes roaming the fields and canaries in the coalmine: another embarrassing paper for MDPI

 

Many publishers are getting nervous about infiltration by paper mills, who can torpedo a journal's reputation when they succeed in publishing papers that are obvious nonsense. In a recent Open Letter, a group of sleuths drew attention to an example in Scientific Reports, published by Springer Nature.

After the Open Letter was published, the paper that instigated our concern was promptly retracted by the journal, but as far as I can tell, not much else has changed. The point about a paper like this is that it is so blatantly bad that it cannot have been through any kind of serious editorial scrutiny or peer review. It acts as a canary in the coalmine: if gobbledegook is published in your journal, it's an indicator that you need to look very carefully at your editorial processes, and act immediately to remove editors who let this stuff in. Sadly, I haven't yet seen much evidence of that happening at Scientific Reports.

This post, however, concerns another publisher, MDPI, who have regularly featured on my blog, and not in a good way. Last month, I commented on the strange state of affairs whereby Finland had downgraded its classification of 187 MDPI journals because of evidence of "minimum time spend for editorial work and quality assessment", at the same time that German universities had secured a national publishing agreement with MDPI. The story I have to tell here may confirm Finland's judgement, and give Germany pause for thought. It concerns this article: Abbas, R., Amran, G. A., Hussain, I., & Ma, S. (2022). A Soft Computing View for the Scientific Categorization of Vegetable Supply Chain Issues. Logistics, 6(3), https://doi.org/10.3390/logistics6030039 

As with the Springer Nature example, the first indication of problems came via the Problematic Paper Screener, the excellent system that checks articles for various red flags, including "tortured phrases". These provide an indicator that a paper has probably been plagiarised but then passed through a process that substitutes synonyms for main words, with the aim of evading plagiarism detection software. So, as noted on PubPeer, in this case we have "fluffy logic" for fuzzy logic, and "unaided ML" for unsupervised machine learning. 

However, example sentences in which tortured phrases were embedded indicated a deeper problem. Most of the text is incomprehensible, and things start to get seriously weird when the authors get on to tomatoes. We are told: 

.... the third creation framework considered for the creation phase is tomatoes. This creation framework is devoted to developing homegrown creatures brought up in rural settings to create vegetables. This can bring domesticated tomatoes likewise to broad or serious frameworks. Broad frameworks include creatures wandering meadows (ordinarily under the oversight of a herder). Differently, serious tomatoes are situated in shut foundations and are outfitted with ICT innovation, which empowers creatures to be observed continuously. Inside these creation frameworks, the most run-of-the-mill issues we run over are meadow observing [75], creature government assistance [76], creature conduct following [77], and tomato creation forecast and enhancement [78,79], as displayed in Figure 3. 

According to a VSC point of view, the formal meanings of these issues are recorded beneath. 

• Field checking: This issue is connected with the exact recognizable proof of meadow inventories to separate between the most reasonable sorts for tomatoes purposes. 

• Tomato government assistance: This is centered around the example arrangement of the dehydration way of behaving in brushing creatures for investigations of creature nourishment, development, and well-being. 

• Tomato growth checking: This depends on the utilization of conduct investigations to recognize early indications of medical problems and advance early negotiation. 

A clue to the origin of this material comes from the cited references, which are about pigs and cattle. Anonymous PubPeer commenter Nerita vitiensis found that a substantial part of the text was adapted from a previous work by different authors, but with the topics of "livestock and fish" changed to "tomatoes and cruciferous vegetables". This explains the description of tomatoes as "creatures" under the oversight of a herder. 

The authors of this piece seem seriously out of their depth, as evidenced by the bland comments apparently written by Chat GPT that they provided on PubPeer. 

Now, one very good thing about MDPI is that it generally identifies the academic editor who handled a paper, and it sometimes also makes public the reviewer reports. This should mean that when a major foul-up like this occurs, it should be possible to identify and purge those responsible for accepting the work. 

The academic editors who accepted this article are Xue-Ming Yuan, who is currently soliciting papers for a special issue in the MDPI journal Mathematics, and Anrong Xue.

The MDPI website shows reports from three named reviewers. 

The first reviewer, Edyta Kardas, was concerned about the use of first-person language, and punctuation, but not apparently about statements about animated tomatoes. She reviewed 8 papers for MDPI journals in 2024.

The second reviewer, Alejandro Vega-Muñoz focused solely on the structure of the article, but apparently did not look at the content. He has edited two special issues for other MDPI journals. 

 The third reviewer, Francesco Barreca, attempted a synopsis of the article (which I could not understand) and then had just two suggestions:

"The work is well done but I have some remarks:

• The figures should be review, the dimension are variable

 • Moderate English changes are required"

The "moderate English changes" were unspecified. Barreca has a track record of editing a special issue of another MDPI journal.

 
Last week, I contacted Publication Ethics at MDPI to draw their attention to this article, noting the dereliction of duty by reviewers and editors, and suggesting that as well as retracting the paper, they should remove the editors and peer reviewers from their database. They replied to say: 

"We confirm that the Editorial Office is investigating the concerns related to this paper following the guidelines of the Committee on Publication Ethics https://publicationethics.org/ of which we are a member and our policy https://www.mdpi.com/ethics#_bookmark29.

We would like to inform you that this case is a priority for us, and we are actively working to resolve it. We will update you on the outcome of this investigation as soon as possible."

I await developments with interest. It is widely recognised that COPE guidelines are not well-suited for dealing with this kind of situation: they make the default assumption that authors should be consulted to give their perspective when criticisms are raised - a reasonable assumption in many cases, but not when there is such blatant evidence of fakery.

The most serious case of infestation of a publisher by nonsense occurred in 2022-3, when the publisher Hindawi (owned by Wiley) was targeted by paper mills who, among other things, generated numerous papers that I labelled as AI gobbledegook sandwiches. Eventually, the publisher withdrew literally thousands of papers and closed the Hindawi brand, after complaints by shareholders started impacting profits.

Like many of the sleuths who track down paper mills, I have become cynical about the commitment to research integrity that is claimed by many publishers, including MDPI. But I do believe they will act when it is in their interests to do so. As the amount of nonsense and disinformation in the scientific literature increases, I think we'll enter a new phase where trustworthiness of journal contents will start to have much higher value. If you want to be taken seriously as a peer-reviewed journal, you just cannot continue to pump out articles accompanied by superficial verbiage from "peer reviewers" that makes no real contact with the subject matter. Publishers will need to act now to clean up their editorial boards if they want to stay in business. 

Note: Comments on this blog are moderated, so there is a delay before they appear.  Anonymous or off-topic comments are not accepted. 

Retrospective look at blog highlights of 2024: What happened next?

It's always interesting for a blogger to look back to see which posts have garnered most attention. In 2024 there were three standout items, my resignation from the Royal Society, an Open Letter about editorial failings at the journal Scientific Reports, and a guest post by René Aquarius about his experiences as a reviewer for MDPI. For each of these blogposts, it's interesting to consider not just the number of hits, but also the broader impact. Blogging is a great way to let off steam, but does it actually achieve anything?

Resignation from the Royal Society 

As I said at the time of writing the post, I don't expect my resignation to change much at the Royal Society, but I was surprised at the amount of interest in the matter. The story was picked up by national media (e.g. here and here), and even got covered in France and the Netherlands. I had anticipated I might be attacked by a swarm of trolls, but there was no more than a handful of rude emails, saying the Royal Society was better off without me. What was more remarkable was the outpouring of positive reactions. My inbox was swamped with people thanking me for taking this step - not just academics, but people from all walks of life. It's customary when elected to the Royal Society for colleagues to send congratulatory emails: the number of congratulations I received for resigning was about five times as many as I had for being elected. I felt I had struck a chord with many people who were fed up with unsuitable individuals being honoured. British people in particular were outraged that the Royal Society was honouring someone who was spreading misinformation with the apparent aim of undermining our democracy.

I've seen only one serious attempt to challenge the arguments in my blogpost - by Toby Young in the Spectator. I can imagine that the comms team at the Royal Society may not have been overjoyed at his main line of argument, which was to say that there had been many Fellows of the Royal Society with rebarbative opinions who hadn't been thrown out, so why should they start now? He described me as a left-wing Torquemada, which I rather enjoyed, except that his implication was that my objections to Musk were political. Had he read my blogpost, he'd see that politics didn't come into it - it is the lack of scientific integrity that is the issue.

As a delightful twist to the story, Toby Young, who is notorious for his foul-mouthed comments on women and disabled people, has just been appointed to the House of Lords, showing that the Royal Society is not alone among the British establishment in honouring those who represent the antithesis of their core values. I can only assume that Kemi Badenoch, who was responsible for his elevation, did it to own the libs.

Open letter regarding Scientific Reports

This letter, cosigned by 22 scientific sleuths, noted that laughably awful papers were getting published in the Springer Nature journal Scientific Reports, which is generally regarded as a serious journal. It looked to us as if the journal was being infiltrated by paper mills who were using it as an outlet for fraudulent or low quality work. For this to happen, there need to be editors who are either turning a blind eye to such articles, or were actively working with a paper mill to accommodate them. A quick analysis of the editors listed on the journal website turned up 28 who had records on PubPeer indicative of involvement in research misconduct.

So what has happened since? We received a reply from Chris Graf, the Research Integrity Director at Springer Nature, thanking us for our letter and emphasising the extent to which the publisher was putting resources into tackling research fraud and paper mills. The article that had sparked off the Open Letter was retracted fairly promptly.

But when I rechecked the Editorial Board list today, 26 the 28 editors we'd listed were still in position - even though we'd given links to PubPeer entries that specified problematic behaviours of all of them. The two who are no longer listed are Ilyas Khan, and Achyut Shankar. I have no idea whether their disappearance from editorial roles at Scientific Reports has anything to do with the Open Letter. (Rule 1 of sleuthing: don't expect anyone in power to tell you if they have acted on information you provided).

And a final wrinkle to the story: a fellow sleuth told me about another editor linked to paper mills, Masoud Afrand, who had not been included in our list. He had actually been removed from the list of editors in March 2022, but then reappeared at some point in the summer of 2024 (as verified by the Wayback Machine). The Editor-in-Chief, Rafal Marszalek, explained this as a "clerical error". In email correspondence I pointed out to him that this was a pretty odd kind of clerical error, and that it did nothing to assuage growing concerns that there may be someone at a senior level at Scientific Reports who was actively working with paper millers.  I did not get a reply.

My experience as a reviewer for MDPI: guest post by René Aquarius

In this post, René describes the fate of a paper that he agreed to review for the Journal of Clinical Medicine, published by MDPI. The paper had numerous methodological flaws, including lack of a control group and discrepancies between the registered protocol and the final study, and René recommended rejection. What then followed was a bizarre series of exchanges with MDPI editors, who encouraged resubmission by the authors, then attempted to block René from re-reviewing the paper, after he'd agreed to do so. Eventually the paper was transferred to  another MDPI journal, Geriatrics, where he was again asked to review it. When he pointed out that the paper was largely unchanged from the original submission, it was withdrawn - but he then found it published a few weeks later in yet another MDPI journal.

This blogpost attracted a lot of comments, not least from people who had had similar experiences with MDPI. I have to say that these experiences, coupled with other evidence such as this and this make me very dubious about standards of peer review at MDPI. The impression is that hard-pressed editorial staff are expected to ensure peer review of submissions is achieved in 2-3 weeks, and they accordingly come to rely on anyone who will do a speedy job, without regard for quality. Indeed, high-quality reviews that raise difficult issues are problematic for them, because they will slow up the process. Accordingly these are sidelined or quietly forgotten about, in favour of minimalist 'reviews' such as those you can see here in reviews 1, 3 and 4.

Will anything change? Hopefully, René's post has helped raise awareness of the problematic aspects of peer review at MDPI journals. I doubt the publisher will do anything to change their peer review model unless the reputational damage from such revelations starts to hit their bottom line. Interestingly, in my last post of 2024, I noted that just before the Christmas break, Finland's Publication Forum (JUFO) downgraded the status of 187 MDPI journals in their index, a move that will disincentivize Finnish researchers from publishing there. However, a day later, MDPI announced that it had secured a national publishing agreement with ZB Med, which offered substantial discounts to authors from over 100 German Universities.

Epilogue

In each of these cases, we can see that a blogpost has raised the profile of an issue that is of relevance to academic scientists. Concrete impacts are harder to demonstrate - where changes happen, it's not clear if they are related to the blogpost, or just coincidental. And for most topics, it's a case of two steps forward and one step back, at best. But if I've learned one thing from my many years writing about such things, it's that you can't trust the people in positions of power to actually do the right thing unless they are prodded repeatedly and publicly (and sometimes not even then). So I hope to continue blogging through 2025, even though it may at times seem futile.

Happy New Year, and Illegitimi non carborundum.

Comments are moderated. Those that are in scope and from an identified person will be published after a delay.