How paper mills go straight (without going clean): a shift to dual-use services

Guest post by Matt Spick

                           Photo: Parsadanov/Shutterstock.com

From relative obscurity, paper mills have recently moved into the spotlight of academic attention. These organisations - which sell manuscripts or citations to authors to enhance scholarly metrics - are growing so rapidly that many fields are being overwhelmed. The overall proportion of paper mill outputs was estimated at 1.5–2% of all scientific papers published in 2022, but a recent AI-screening cancer study estimated that around 10% of recent cancer manuscripts in some venues may be paper mill products, and in data-intensive fields paper mill outputs can now outnumber legitimate publications. The increased attention has also been driven by the integrity community highlighting unethical behaviours, notably in the FoSci Report 2026, and has resulted in initiatives such as United2Act. This is in addition to publishers and third parties setting up a growing number of integrity checking systems, whether for citation anomalies, duplicated images, or tortured phrases. But this is an adversarial process, and backward-looking checks will inevitably miss what happens next.

One option for paper mills is to stop selling unethical products, and shift to dual-use services instead. The dual-use business model has a long history. During the US prohibition era, manufacturing moonshine came with severe risks. To avoid these risks, and transfer both criminal intentions (mens rea) and criminal action (actus reus) to the customers, the California Vineyardist Association created a front organisation, Fruit Industries Ltd, to sell Vine-Glo. This innovative product consisted of a block of concentrated grape juice, easily dissolved in water to create a refreshing and entirely legal fruit-based drink. The grape juice block also included an explicit warning to purchasers that “After dissolving the brick in a gallon of water, do not place the liquid in a jug away in the cupboard for twenty days, because then it would turn into wine.” And of course there are other examples of unethical actors shifting into legitimate business, whether in laundry services or waste management businesses. Such operations are hard to police, precisely because they overlap with entirely legal commercial interests.

The paper mill equivalents of Vine-Glo might include proprietary software tools to data dredge and p-hack large open datasets, such as the Global Burden of Disease Study or NHANES. Such tools could be used legitimately, but if videos were created explaining how to mass produce p-hacked papers, this would be claimed as “beyond the developers’ control”. Training courses might emerge that offer primers on scientific writing to complete beginners, starting with an icebreaker at 9:00 am on day one and concluding with submission to a PubMed indexed journal at 5:00 pm on day two. Naturally, acceptance would not be guaranteed, and the trainers would stress that this was purely a training service, not a ‘pay for authorship’ model. Proving that these practices were linked to specific examples of problematic authorship would be impossible at the individual paper level.

The lack of action (sadly, there is no such thing as the science police) is especially frustrating in an industry that is slow to respond even in clear cases of retractions being needed, due to a conservative culture around accusations of wrongdoing, which - partly for good reasons - prefers to tolerate higher levels of unethical outputs rather than inadvertently criticising or punishing scientists. At the same time, the surge in publication volumes (measured in the tens of thousands across biomedical and life sciences research) makes it impossible to conclude that no problematic behaviour is occurring.

Mills turning to dual-use products would also compromise large parts of the integrity community's current toolkit: detection based on image duplication, tortured phrases and citation anomalies would largely fail, because the outputs would represent genuine analyses of real data with real (if trivial) results. In turn, this is likely to lead to prevention having to move upstream, to pre-registration before data release and gatekeeping of access.

Will service mills offering software and training replace the existing paper mills? In practice, both models are likely to continue to exist, if only because hiring, promotion and graduation decisions are all influenced by (or completely dependent upon) publication counts. As long as the traditional model remains profitable (with first-author slots advertised at up to USD 5,600, more than sufficient to grease the wheels through editor bribery elsewhere in the publication chain) and the underlying demand continues, there will be no incentive to leave money on the table. Conventional paper mills may be able to use LLMs and agentic platforms to improve the quality of their outputs (as a different form of adversarial adaptation), making them harder to detect and preserving their value proposition for unethical authors who simply want to purchase authorship. Many potential customers are likely to find the scientific publishing process too arcane, and will be happy to continue to buy an end-to-end service. Service mills seem likely to grow in importance, however, for authors who are more risk averse. At the same time - for expert users - agentic AI platforms for science may trivialise secondary outputs such as reviews and open data analyses, providing a third route to ‘enhancing’ an author’s scholarly record. 

This will present a challenge to the wider scientific community, which has only begun to wake up to the problem of paper mills as manuscript factories. In reality the mills are diversifying their offerings, creating the illusion of ‘going straight’ through dual-use products, and it seems inevitable that some expert users will be able to disintermediate the paper mills completely through agentic AI. We need to recognise that all these types of output can dilute the scholarly record, decreasing the signal to noise ratio and delaying the translation of literature into real impact, to the disadvantage of both direct users and also society as a whole.

Note: Comments are welcome, but are moderated, so there may be a delay before they appear. In general, comments are accepted if on-topic and non-anonymous.

 

 

An analysis of PubPeer comments on highly-cited retracted articles

PubPeer is sometimes discussed as if it is some kind of cesspit where people smear honest scientists with specious allegations of fraud. I'm always taken aback when I hear this, since it is totally at odds with my experience. When I conducted an analysis of PubPeer comments concerning papers from UK universities published over a two-year period, I found that all 345 of them conformed to PubPeer's guidelines, which require comments to contain only "Facts, logic and publicly verifiable information". There were examples where another commenter, sometimes an author, rebutted a comment convincingly. In other cases, the discussion concerned highly technical aspects of research, where even experts may disagree. Clearly, PubPeer comments are not infallible evidence of problems, but in my experience, they are strictly moderated and often draw attention to serious errors in published work.

The Problematic Paper Screener (PPS) is a beautiful resource that is ideal to investigate PubPeer's impact. It not only collates information on articles that are annulled (an umbrella term coined to encompass retractions, removals, or withdrawals), but it also cross-references this information with PubPeer, so you can see which articles have comments. Furthermore, it provides the citation count of each article, based on Dimensions.  

The PPS lists over 134,000 annulled papers; I wanted to see what proportion of retractions/withdrawals were preceded by a PubPeer comment. To make the task tractable, I focused on articles that had at least 100 citations, and which were annulled between 2021 and 2025. This gave a total of 800 articles, covering all scientific disciplines. It was necessary to read the PubPeer comments for each of these, because many comments occur after retraction, and serve solely to record the retraction on PubPeer. Accordingly, I coded each paper in terms of whether the first PubPeer comment preceded or followed the annulment.  

Flowchart of analysis of PPS annulled papers

 

I had anticipated that around 10-20% of these annulled articles would have associated PubPeer comments; this proved to be a considerable underestimate. In fact, 58% of highly-cited papers that were annulled between 2021-2025 had prior PubPeer comments. Funnily enough, shortly after I'd started this analysis, I saw this comment on Slack by Achal Agrawal: "I was wondering if there is any study on what percentage of retractions happen thanks to sleuths. I have a feeling that at least around 50% of the retractions happen thanks to the work of 10 sleuths." Achal's estimate of the percentage of flagged papers was much closer than mine. But what about the number of sleuths who were responsible?

It's not possible to give more than a rough estimate of the contribution of individual commenters. Many of them use pseudonyms (some people even use a different pseudonym for each post they submit), and combinations of individuals often contributed comments on a single article. Some of the PubPeer comments had been submitted in early years, when they were just labelled as "Unregistered submission" or "Peer 1" etc., so any estimate will be imperfect. The best I could do was to focus just on the first comment for each article, excluding any comments occurring after a retraction. Of those who had stable names or pseudonyms, the 10 most prolific commenters had commented on between 9 and 50 articles, accounting for 27% of all retractions in this sample. Although this is a lower proportion than Achal's estimate, it's an impressive number, especially when you bear in mind that there were many comments from unknown contributors, and the analysis focused only on articles with at least 100 citations.

Of course, the naysayers may reply and say that this just goes to show that the sleuths who comment on articles are effective in causing retractions, not that they are accurate. To that I can only reply that publishers/journals are very reluctant to retract articles: they may regard it as reputationally damaging, and be concerned about litigation from disgruntled authors. In addition, they have to go through due process and it takes up a lot of resources to make the necessary checks and modify the publication record. They don't do it lightly, and often don't do it at all, despite clear evidence of serious error in an article (see, e.g.  Grey et al, 2025). 

If an article is going to be retracted, it is better that it is done sooner rather than later. Monitoring PubPeer would be a good way of cleaning up a polluted literature - in the interests of all of us. Any publisher can do that for free: just ask an employee of the integrity department to check new PubPeer posts every day—about 40 minutes and you’re done. PubPeer also provides publishers with a convenient dashboard to facilitate this essential monitoring task.

It would be interesting to extend the analysis to less highly-cited papers, but this would be a huge exercise, particularly since this would include many paper-milled articles from mass retractions. I hope that my selective analysis will at least demonstrate that those who comment on problematic articles on PubPeer should be taken seriously. 

 

Post-script: 7 February 2026

One of the commentators with numbered comments below has complained that I am censoring criticism, and has revealed their identity on LinkedIn as Ryan James Jessup, JD/MPA.  My bad - I usually paste a statement at the end of a blogpost explaining that Comments are moderated so there can be a delay, but I accept nonanonymous comments that are polite and on topic.  Jessup didn't take up my offer of incorporating his arguments in a section at the end of the blog, so I have accepted them and you can read them in the Comments.

I actually agree with a lot of what he says, but some points I disagree with, so here are my thoughts.

Points 1-2. He starts by stating the piece confuses correlation with causation.  On reflection I think he's right. The word "role" in the title is misleading, and I have accordingly changed the title of the post from "The role of PubPeer in retractions of highly-cited articles" to "An analysis of PubPeer comments on highly-cited articles".

3.  He argues that selection of highly-cited papers was done to fudge the result because these papers are most likely to be noticed and commented on.  The  actual reason for selecting these papers was to focus on outputs that had had some influence; many people assume PubPeer commentators just focus on the low-hanging fruit from papermills, which nobody is going to read anyhow. There is nothing to stop Jessup or anyone else doing his own analysis using another filter to see if these results generalise to less highly-cited articles. It involves just a few hours of rather tedious coding. Maybe sample a random 800 articles?  

4. He argues that "annulled" papers covers various categories.  I am glad to be able to clarify that in the sample of 800 papers that I analysed, all were retractions.

5. He disagrees that my opinion of whether PubPeer comments were factual and accurate has any value, and that they could be defamatory or otherwise falsely imply misconduct.  From my experience, I reckon it would be difficult to get such material past PubPeer moderators, but if he can provide some examples, that would be helpful.  

6. He says the coding method is subjective "They read comments and decide whether the first comment preceded or followed annulment".  The dates are provided for the retraction notice in the PPS, so this is just a matter of checking if the PubPeer comment (also dated) appeared before or after that date.

7. Re the identification of "top 10 sleuths".  I noted the limitations inherent in the data, so I am not sure what Jessup is complaining of here. The fact remains that a small number of individuals have been very effective in identifying issues in highly-cited articles prior to their retraction.

8.  Jessup argues that I'm saying that “journals don’t retract lightly, therefore PubPeer must be right”.  The first part of that argument has ample evidence. If he is aware of cases where PubPeer comments have indeed led to inappropriate retractions, then he should name them.

9-11. I do actually have some understanding of how retraction processes work in journals, but my concern is the failure of many journals/publishers to initiate the first step in the process.  I think we're in agreement that the current system for retracting articles from journals is broken. We also agree that PubPeer comments should be regarded as tips. My suggestion is simply that if publishers have a useful free source of tips, they should use it. A few of them do, but many don't seem motivated to be proactive because it just creates more work.

The prolific PubPeer commenters that I know would love it if the platform could be used primarily for civilised academic debate, as was the original intention. Unfortunately, science can't wait until the broken system is repaired; we do need to clean up a polluted literature. I would add that the idea that those who comment on PubPeer are doing it for the glory is laughable. The main reaction is to be ignored at best and abused at worst. They are unusual people who are obsessive about the need to have a reliable scientific literature.

 

 

 

 

New publishing models will only work if authors embrace them

Complaints about the broken academic publishing system have been around for years and are getting louder. A common theme is that with the rise of open access publishing, commercial publishers have grasped the opportunity to grow their profits from article-processing-charges (APCs). Whereas in the past, journals competed to be the most highly respected outlet, now they compete to publish on the grounds of speed and quantity of publications (see e.g. Timmis et al, 2025).

In response, various new initiatives have arisen. My focus here is on the F1000 publishing model, which was adopted by the Wellcome Trust in 2016 for their journal Wellcome Open Research. In this model, the author deposits an article on the platform (in effect as a preprint), which then is updated to a published version if two positive peer reviews are obtained. The only editorial input is from office staff who check that the submission meets basic criteria and that the selected peer reviewers are appropriate. This system could quickly get overwhelmed with low quality submissions, but aims to avoid that by restricting submissions to authors funded by the Wellcome Trust, which agrees to pay their APCs. I've always been a fan of open access, and have published several papers in Wellcome Open Research, encouraged by the prospect of straightforward and free open access publication.

A check on the Dimensions platform shows that Wellcome Open Research has grown in popularity over the years, and now dominates outputs from Wellcome-funded researchers.

Figure 1. Plot of the top 5 journals, publications funded by Wellcome Trust between 2016-2025.

My funding by Wellcome Trust came to an end and in 2016 I took up an ERC Advanced Grant. Towards the end of that grant, in March 2021, the European Commission (EC) announced that they were setting up a new journal, Open Research Europe that adopted the same F1000 model and offered free open access publication for EC-funded researchers. In contrast to Wellcome Open Research, Open Research Europe has not been enthusiastically embraced. A search on Dimensions showed that a large proportion of EC-funded research is published open access with for-profit publishers (see Figure 2). Open Research Europe is not shown because the number of publications is relatively small: 213, 251, 290, 374 for the years 2021 to 2024 respectively. It rates 25th among journals used by EC researchers, whose favourite publisher appears to be MDPI.

Figure 2. Plot of the top 5 journals, publications funded by EC between 2021-2025.

This raises two questions: who is paying the APCs, and why don't researchers publish on their funder's platform, which offers them free open access?

Of course, Open Research Europe is relatively young, and its uptake may have been influenced by its launch coinciding with emergence from lockdown. It's possible that communications from the EC encouraging grantholders to publish there haven't been sufficient to raise awareness of this option. I'd be curious if any readers who have EC funding could comment on barriers to uptake. Meanwhile, between 2021-2025 around US$973 million* in APCs has gone into the coffers of publishers - money that could have been used to fund researchers in other ways. On a rough estimate, around US$197 million of this has been paid to the most popular publisher, MDPI.

*To obtain this estimate, I searched Dimensions.ai using search terms Funder = European Commission (EC), Publication Type = Article, and year range from 2021-2025, and then used Analytical views to generate a table of source titles. For the first 30 titles, I manually checked the APC and used a currency converter to convert to $US. For the remaining titles, I estimated the APC as equivalent to the average for the first 30 titles. I coded MDPI as publisher for titles I recognised, but I did not carefully check each one. A csv file with the relevant data can be found here: https://osf.io/rcxd3.

Reference

Timmis, K., et al. (2025). Journals operating predatory practices are systematically eroding the science ethos: A gate and code strategy to minimise their operating space and restore research best practice. Microbial Biotechnology, 18(6), e70180. https://doi.org/10.1111/1751-7915.70180 

 

Comments are moderated to prevent spam, but relevant, on-topic comments are welcome. 

What is going on in Hindawi special issues?

A guest blogpost by Nick Wise 

 http://www.eng.cam.ac.uk/profiles/nhw24

The Hindawi journal Wireless Communications and Mobile Computing is booming. Until a few years ago they published 100-200 papers a year, however they published 269 papers in 2019, 368 in 2020 and 1,212 in 2021. So far in 2022 they have published 2,429. This growth has been achieved primarily by the creation of special issues, which makes sense. It would be nearly impossible for a journal to increase its publication rate by an order of magnitude in 2 years without outsourcing the massive increase in workload to guest editors.

Recent special issues include ‘Machine Learning Enabled Signal Processing Techniques for Large Scale 5G and 5G Networks’ (182 articles), ‘Explorations in Pattern Recognition and Computer Vision for Industry 4.0’ (244) and ‘Fusion of Big Data Analytics, Machine Learning and Optimization Algorithms for Internet of Things’ (204). Each of these special issues contains as many papers as the journal published in a year until recently. They also contain many papers that are flagged on Pubpeer for irrelevant citations, tortured phrases and surprising choices of corresponding email addresses.

However, I am going to focus on one special issue that is still open for submissions, and so far contains a modest 62 papers: ‘AI-Driven Wireless Energy Harvesting in Massive IoT for 5G and Beyond’, edited by Hamurabi Gamboa Rosales, Danijela Milosevic and Dijana Capeska Bogatinoska. Given the title of the special issue, it is perhaps surprising that only two of the articles contain ‘wireless’ in the title and none contain ‘energy’. The authors of the other papers (or whoever submitted them) appear to have realised that as long as they included the buzzwords ‘AI’, ‘IoT’ (Internet of Things) or ‘5G’ in the title, the paper could be about anything at all. Hence, the special issue contains titles such as:

• Analysis Model of the Guiding Role of National Sportsmanship on the Consumer Market of Table Tennis and Related IoT Applications 
• Evaluation Method of the Metacognitive Ability of Chinese Reading Teaching for Junior Middle School Students Based on Dijkstra Algorithm and IoT Applications 
• The Construction of Shared Wisdom Teaching Practice through IoT Based on the Perspective of Industry-Education Integration
  

Of the 62 papers, 60 give Hamurabi Gamboa Rosales as the academic editor and 2 give Danijela Milosevic. Why is the distribution of labour so lopsided? One can imagine an arrangement where the lead editor does the admin of waving through irrelevant papers and the other 2 guest editors get to say that they’ve guest-edited a special issue on their CV.

Of course, in addition to boosting publication numbers for the authors and providing CV points for the guest editors, every paper in the special issue has a references section. Each reference gives someone a citation, another academic brownie point on which careers can be built. An anonymous Pubpeer sleuth has trawled through the references section of every paper in this special issue and found that Malik Bader Alazzam of Amman Arab University in Jordan has been cited 139 times across the 62 papers. The chance that the authors of almost every article would independently decide to cite the same person seems small.

The most intriguing fact about the papers in the special issue however, is that only 4 authors give corresponding email addresses that match their affiliation. These 4 include the only 3 papers with non-Chinese authors. Of the other 58, 1 uses an email address from Guangzhou University, 6 use email addresses from Changzhou University, and 51 use email addresses from Ma’anshan University. All of the Ma’anshan addresses are of the form 1940XXXX@masu.edu.cn and many are nearly sequential, suggesting that someone somewhere purchased a block of sequential email addresses (you do not need to be at Ma’anshan University to have an @masu email address). The screenshot below shows a sample (the full dataset is linked here).

A subset of the titles from the special issue with their corresponding email addresses, all of the form 1940XXXX@masu.edu.cn

The use and form of the email addresses suggests that all of these papers are the work of a paper mill. It is hard to imagine otherwise how 51 different authors could submit papers to the same special issue using the same institutional email domain and format. Indeed, before 2022 only 2 papers had ever used @masu.edu.cn as a corresponding address according to Dimensions. It is equally hard to imagine how Hamurabi Gamboa Rosales is unaware. How can you not notice that, of the 19 papers you receive for your special issue on the 12th of July, 18 use the same email domain that doesn’t match their affiliation? This may also explain why Hamurabi has dealt with almost all the papers himself. This special issue should be closed for submissions and an investigation begun.

Stepping back from this special issue, this is not an isolated problem. There are at least 40 other papers published in Wireless Communications and Mobile Computing with corresponding emails from Ma’anshan, and Dimensions finds there are 46 in Computational Intelligence and Neuroscience, 38 in Computational and Mathematical Methods in Medicine and 30 in Mobile Information Systems, all published in 2022 and all in Hindawi journals. What are the chances that 18404032@masu.edu.cn is used in a special issue in Computational Intelligence and Neuroscience, 18404038@masu.edu.cn in Disease Markers and 18404041@masu.edu.cn in Wireless Communications and Mobile Computing?

Finally, masu.edu.cn is only one example of a commonly used email domain that doesn’t match the author’s affiliation. It is conceivable that the entire growth in publications of Wireless Communications and Mobile Computing, Computational Intelligence and Neuroscience (163 articles in 2020, 3,079 in 2022) and Computational and Mathematical Methods in Medicine (225 in 2020, 1,488 in 2022) is from paper mills publishing in corrupted special issues.

Nick Wise

*All numbers accurate as of the 12th October 2022.

We need to talk about editors

Editoris spivia

The role of journal editor is powerful: you decide what is accepted or rejected for publication. Given that publications count as an academic currency – indeed in some institutions they are literally fungible – a key requirement for editors is that they are people of the utmost integrity. Unfortunately, there are few mechanisms in place to ensure editors are honest – and indeed there is mounting evidence that many are not. I argue here that we can no longer take editorial honesty for granted, and systems need to change to weed out dodgy editors if academic publishing is to survive as a useful way of advancing science. In particular, the phenomenon of paper mills has shone a spotlight on editorial malpractice.

Questionable editorial practices

Back in 2010, I described a taxonomy of journal editors based on my own experience as an author over the years. Some were negligent, others were lordly, and others were paragons – the kind of editor we all want, who is motivated solely by a desire for academic excellence, who uses fair criteria to select which papers are published, who aims to help an author improve their work, and provides feedback in a timely and considerate fashion. My categorisation omitted another variety of editor that I have sadly become acquainted with in the intervening years: the spiv. The spiv has limited interest in academic excellence: he or she sees the role of editor as an opportunity for self-advancement. This usually involves promoting the careers of self or friends by facilitating publication of their papers, often with minimal reviewing, and in some cases may go as far as working hand in glove with paper mills to receive financial rewards for placing fraudulent papers.

When I first discovered a publication ring that involved journal editors scratching one another’s backs, in the form of rapid publication of each other’s papers, I assumed this was a rare phenomenon. After I blogged about this, one of the central editors was replaced, but others remained in post. 

I subsequently found journals where the editor-in-chief authored an unusually high percentage of the articles published in the journal. I drew these to the attention of integrity advisors of the publishers that were involved, but did not get the impression that they regarded this as particularly problematic or were going to take any action about it. Interestingly, there was one editor, George Marcoulides, who featured twice in a list of editors who authored at least 15 articles in their own journal over a five year period. Further evidence that he equates his editorial role with omnipotence came when his name cropped up in connection with a scandal where a reviewer, Fiona Fidler, complained after she found her positive report on a paper had been modified by the editor to justify rejecting the paper: see this Twitter thread for details. It appears that the publishers regard this as acceptable: Marcoulides is still editor-in-chief at the Sage journal Educational and Psychological Measurement, and at Taylor and Francis’ Structural Equation Modeling, though his rate of publishing in both journals has declined since 2019; maybe someone had a word with him to explain that publishing most of your papers in a journal you edit is not a good look.

Scanff et al (2021) did a much bigger investigation of what they termed “self-promotion journals” - those that seemed to be treated as the personal fiefdom of editors, who would use the journal as an outlet for their own work. This followed on from a study by Locher et al (2021), which found editors who were ready to accept papers by a favoured group of colleagues with relatively little scrutiny. This had serious consequences when low-quality studies relating to the Covid-19 pandemic appeared in the literature and subsequently influenced clinical decisions. Editorial laxness appears in this case to have done real harm to public health.

So, it's doubtful that all editors are paragons. And this is hardly surprising: doing a good job as editor is hard and often thankless work. On the positive side, an editor may obtain kudos for being granted an influential academic role, but often there is little or no financial reimbursement for the many hours that must be dedicated to reading and evaluating papers, assigning reviewers, and dealing with fallout from authors who react poorly to having their papers rejected. Even if an editor starts off well, they may over time start to think “What’s in this for me?” and decide to exploit the opportunities for self-advancement offered by the position. The problem is that there seems little pressure to keep them on the straight and narrow; it's like when a police chief is corrupt. Nobody is there to hold them to account. 

Paper mills

Many people are shocked when they read about the phenomenon of academic paper mills – defined in a recent report by the Committee on Publication Ethics (COPE) and the Association of Scientific, Tehcnical and Medical Publishers (STM) as “the process by which manufactured manuscripts are submitted to a journal for a fee on behalf of researchers with the purpose of providing an easy publication for them, or to offer authorship for sale.” The report stated that “the submission of suspected fake research papers, also often associated with fake authorship, is growing and threatens to overwhelm the editorial processes of a significant number of journals.” It concluded with a raft of recommendations to tackle the problem from different fronts: changing the incentives adopted by institutions, investment in tools to detect paper mill publications, education of editors and reviewers to make them aware of paper mills, introduction of protocols to impede paper mills succeeding, and speeding up the process of retraction by publishers.

However, no attention was given to the possibility that journal editors may contribute to the problem: there is talk of “educating” them to be more aware of paper mills, but this is not going to be effective if the editor is complicit with the paper mill, or so disengaged from editing as to not care about them. 

It’s important to realise that not all paper mill papers are the same. Many generate outputs that look plausible. As Byrne and Labbé (2017) noted, in biomedical genetic studies, fake papers are generated from a template that is based on a legitimate paper, and just vary in terms of the specific genetic sequence and/or phenotype that is studied. There are so many genetic sequences and phenotypes, that the number of possible combinations of these is immense. In such cases, a diligent editor may get tricked into accepting a fake paper, because the signs of fakery are not obvious and aren’t detected by reviewers. But at the other extreme, some products of paper mills are clearly fabricated. The most striking examples are those that contain what Guillaume Cabanac and colleagues term “tortured phrases”. These appear to be generated by taking segments of genuine articles and running them through an AI app that will use a thesaurus to alter words, with the goal of evading plagiarism detection software. In other cases, the starting point appears to be text from an essay mill. The results are often bizarre and so incomprehensible that one only needs read a few sentences to know that something is very wrong. Here’s an example from Elsevier’s International Journal of Biological Macromolecules, which those without access can pay $31.50 for (see analysis on Pubpeer, here).

"Wound recuperating camwood a chance to be postponed due to the antibacterial reliance of microorganisms concerning illustration an outcome about the infection, wounds are unable to mend appropriately, furthermore, take off disfiguring scares [150]. Chitin and its derivatives go about as simulated skin matrixes that are skilled should push a fast dermal redesign after constantly utilized for blaze treatments, chitosan may be wanton toward endogenous enzymes this may be a fundamental preference as evacuating those wound dressing camwood foundation trauma of the wounds and harm [151]. Chitin and its derivatives would make a perfect gas dressing. Likewise, they dampen the wound interface, are penetrability will oxygen, furthermore, permit vaporous exchange, go about as a boundary with microorganisms, and are fit about eliminating abundance secretions"

And here’s the start of an Abstract from a Springer Nature collection called Modern Approaches in Machine Learning and Cognitive Science (see here for some of the tortured phrases that led to detection of this article). The article can be yours for £19.95:

“Asthma disease are the scatters, gives that influence the lungs, the organs that let us to inhale and it’s the principal visit disease overall particularly in India. During this work, the matter of lung maladies simply like the trouble experienced while arranging the sickness in radiography are frequently illuminated. There are various procedures found in writing for recognition of asthma infection identification. A few agents have contributed their realities for Asthma illness expectation. The need for distinguishing asthma illness at a beginning period is very fundamental and is an exuberant research territory inside the field of clinical picture preparing. For this, we’ve survey numerous relapse models, k-implies bunching, various leveled calculation, characterizations and profound learning methods to search out best classifier for lung illness identification. These papers generally settlement about winning carcinoma discovery methods that are reachable inside the writing.”

These examples are so peculiar that even a layperson could detect the problem. In more technical fields, the fake paper may look superficially normal, but is easy to spot by anyone who knows the area, and who recognises that the term “signal to noise” does not mean “flag to commotion”, or that while there is such a thing as a “Swiss albino mouse” there is no such thing as a “Swiss pale-skinned person mouse”. These errors are not explicable as failures of translation by someone who does not speak good English. They would be detected by any reviewer with expertise in the field. Another characteristic of paper mill outputs, featured in this recent blogpost, are fake papers that combine tables and figures from different publications in nonsensical contexts.

Sleuths who are interested in unmasking paper mills have developed automated methods for identifying such papers, and the number is both depressing and astounding. As we have seen, though some of these outputs appear in obscure sources, many crop up in journals or edited collections that are handled by the big scientific publishing houses, such as Springer Nature, Elsevier and Wiley. When sleuths find these cases, they report the problems on the website PubPeer, and this typically raises an incredulous response as to how on earth did this material get published. It’s a very good question, and the answer has to be that somehow an editor let this material through. As explained in the COPE&STM report, sometimes a nefarious individual from a paper mill persuades a journal to publish a “special issue” and the unwitting journal is then hijacked and turned into a vehicle for publishing fraudulent work. If the special issue editor poses as a reputable scientist, using a fake email address that looks similar to the real thing, this can be hard to spot.

But in other cases, we see clearcut instances of paper mill outputs that have apparently been approved by a regular journal editor. In a recent preprint, Anna Abalkina and I describe finding putative paper mill outputs in a well-established Wiley journal, the Journal of Community Psychology. Anna identified six papers in the journal in the course of a much larger investigation of papers that came from faked email addresses. For five of them the peer review and editorial correspondence was available on Publons. The papers,  from addresses in Russia or Kazakhstan, were of very low quality and frequently opaque. I had to read and re-read to work out what the paper was about, and still ended up uncertain. The reviewers, however, suggested only minor corrections. They used remarkably similar language to one another, giving the impression that the peer review process had been compromised. Yet the Editor-in-Chief, Michael B. Blank, accepted the papers after minor revisions, with a letter concluding: “Thank you for your fine contribution”. 

There are two hypotheses to consider when a journal publishes incomprehensible or trivial material: either the editor was not doing their job of scrutinising material in the journal, or they were in cahoots with a paper mill. I wondered whether the editor was what I have previously termed an automaton – one who just delegates all the work to a secretary. After all, authors are asked to recommend reviewers, so all that is needed is for someone to send out automated requests to review, and then keep going until there are sufficient recommendations to either accept or reject. If that were the case, then maybe the journal would accept a paper by us. Accordingly, we submitted our manuscript about paper mills to the Journal of Community Psychology. But it was desk rejected by the Editor in Chief with a terse comment: “This a weak paper based on a cursory review of six publications”. So we can reject hypothesis 1 – that the editor is an automaton. But that leaves hypothesis 2 – that the editor does read papers submitted to his journal, and had accepted the previous paper mill outputs in full knowledge of their content. This raises more questions than it answers. In particular, why would he risk his personal reputation and that of his journal by behaving that way? But perhaps rather than dwelling on that question, we should think positively about how journals might protect themselves in future from attacks by paper mills.

A call for action

My proposal is that, in addition to the useful suggestions from the COPE&STM report, we need additional steps to ensure that those with editorial responsibility are legitimate and are doing their job. Here are some preliminary suggestions:

1. Appointment to the post of editor should be made in open competition among academics who meet specified criteria.
2. It should be transparent who is responsible for final sign-off for each article that is published in the journal.
3. Journals where a single editor makes the bulk of editorial decisions should be discouraged. (N.B. I looked at the 20 most recent papers in Journal of Community Psychology that featured on Publons and all had been processed by Michael B. Blank).
   
4. There should be an editorial board consisting of reputable people from a wide range of institutional backgrounds, who share the editorial load, and meet regularly to consider how the journal is progressing and to discuss journal business.
5.  Editors should be warned about the dangers of special issues and should not delegate responsibility for signing off on any papers appearing in a special issue.
6. Editors should be required to follow COPE guidelines about publishing in their own journal, and publishers should scrutinise the journal annually to check whether the recommended procedures were followed.
7. Any editor who allows gibberish to be published in their journal should be relieved of their editorial position immediately.
   

Many journals run by academic societies already adopt procedures similar to these. Particular problems arise when publishers start up new journals to fill a perceived gap in the market, and there is no oversight by academics with expertise in the area. The COPE&STM report has illustrated how dangerous that can be – both for scientific progress and for the reputation of publishers.

Of course, when one looks at this list of requirements, one may start to wonder why anyone would want to be an editor. Typically there is little financial renumeration, and the work is often done in a person’s “spare time”. So maybe we need to rethink how that works, so that paragons with a genuine ability and interest in editing are rewarded more adequately for the important work they do.

P.S. Comment moderation is enabled for this blog to prevent it being overwhelmed by spam, but I welcome comments, and will check for these in the weeks following the post, and admit those that are on topic. 

 

Comment by Jennifer Byrne, 9th Sept 2022 

(this comment by email, as Blogger seems to eat comments by Jennifer for some reason, while letting through weird spammy things!).

This is a fantastic list of suggestions to improve the important contributions of journal editors. I would add that journal editors should be appointed for defined time periods, and their contributions regularly reviewed. If for any reason it becomes apparent that an editor is not in a position to actively contribute to the journal, they should be asked to step aside. In my experience, editorial boards can include numerous inactive editors. These can provide the appearance of a large, active and diverse editorial board, when in practice, the editorial work may be conducted by a much smaller group, or even one person. Journals cannot be run successfully without a strong editorial team, but such teams require time and resources to establish and maintain.

Editorial integrity: Publishers on the front line

Thanks to some live tweeting by Anna Sharman (@sharmanedit), I've become aware that the 13th Conference of the European Association of Science Editors (EASE) is taking place in Strasbourg this weekend.

The topic is "Scientific integrity: editors on the front line", and the programme acknowledges Elsevier, who presumably have contributed funding for the conference.

It therefore seems timely to give a brief update of developments following three blogposts I wrote during February-March 2015, documenting some peculiar editorial behaviour at four journals: Research in Autism Spectrum Disorders (RASD: Elsevier), Research in Developmental Disabilities (RIDD: Elsevier), Developmental Neurorehabilitation (DN: Informa Healthcare) and Journal of Developmental and Physical Disabilities (JDPD: Springer).

To do the story full justice, you need to read these blogposts, but in brief, blogpost 1 described how Johnny Matson, the then editor of both RASD and RIDD had published numerous articles in his own journal, and engaged in frequent self-citation, leading to his receiving a 'highly cited' badge from Thomson Reuters. In the comments on that blogpost, another intriguing factor emerged, which was Matson's tendency to accept papers with little or no review. This was denied by Elsevier, despite clear evidence of very short acceptance lags that were incompatible with review.

Blogpost 2 was prompted by Matson defending himself against accusations of self-citation by pointing out that he published in journals that he did not edit. I checked this out and found he had numerous papers in two other journals: DN and JDPD, and that the median lag between a paper of his being submitted and accepted in DN was one day. (JDPD does not provide data on publication lags). I therefore looked at the editors of those journals, and found that they themselves were publishing remarkable numbers of papers in RASD and RIDD, again with extremely short publication lags. A trio of editors and editorial board members (Jeff Sigafoos, Giulio Lancioni and Mark O'Reilly), co-authored no less than 140 papers in RASD and RIDD between 2010 and 2014, typically with acceptance times of less than 2 weeks. Some of the papers in RIDD were not even in the topic area of developmental disabilities, but covered neurological conditions acquired in adulthood.

In blogpost 3, I turned the focus on to the publisher of RASD and RIDD, Elsevier, to query why they had not done anything about such irregular editorial practices. I did a further analysis of publication lags in RIDD, showing that they had dropped precipitately between 2008 and 2012, and that there was a small band of authors whose prolific papers were published there at amazing speed. I provided all the statistical data to support my case, including interactive spreadsheets that made it easy to determine which editors and authors had been benefiting from the slack editorial standards at these journals.

There was some interesting fall-out from all of this. The second blogpost drew fire from supporters of the editors I had "outed", accusing me of bad behaviour and threatening to complain to my university. Since everything I had said was backed by evidence, this did not concern me. I received heartfelt messages of support from people who were appalled that a particular approach to autism intervention had been promoted by this group of editors, who were in effect using their status to gain the veneer of scientific credibility for work which was not in fact peer-reviewed.  I was also contacted by several academics telling me that everyone knew this had been going on for years, but nobody had done anything; this level of passivity was surprising given that many were angry that  authors had reaped benefits from their staggeringly high publication rate, while those who were outside the charmed circle were left behind. I was urged to go further and raise my concerns with the universities employing those who were capitalising on, or engaging in, lax editorial behaviour. I do, however, have an extremely demanding job and I hoped that I had done enough by shining a light on dubious practices, and providing the full datasets that provided evidence. However, I now wonder if I should have been more pro-active.

I wrote to express my concerns to publishers of all four journals, and had my correspondence acknowledged. But then? Well, not a lot.

It's clear that Elsevier has taken some action. Indeed, my first blogpost was prompted by Michelle Dawson noting on Twitter that the editorial boards of RASD and RIDD had mysteriously disappeared from the online journals. She had previously noted Matson's pattern of mega-self-citation, and I had written directly to him, with copy to the publisher, some months previously to express concern, when I realised that I was listed as a member of the editorial board of RASD. Elsevier did not acknowledge my letter, but it is possible that the changes to the editorial boards that they had started were linked to my concerns.

The first direct response I had from Elsevier was some weeks after my final blogpost, when they explained that they were looking into the situation regarding unreviewed papers, but that this was a huge job and would take a long time. They were presumably disinclined to rely on the files that I had deposited on Open Science Framework, which show the identity and submission and acceptance data for every paper in RASD and RIDD.  They did appoint new editors and a small group of associate editors for both journals, all with good track records for integrity.

I have heard on the grapevine that they are now evaluating articles published in those journals that have been identified as not having undergone peer review; some of those approached to do these evaluations have mentioned this to me. It's rather unclear how this is going to work, given that, across the two journals, there are nearly 1000 papers where the available data indicate a lag from receipt to acceptance of under 2 weeks. I guess we should be glad that at least the publisher is taking some action, albeit at a snail's pace, but I am dubious as to whether there will be any retractions.

Meanwhile, Developmental Neurorehabilitation changed publisher around the time I was writing, and is now under the care of Taylor and Francis. I wrote to the publisher explaining my concerns and received a polite reply, but then heard no more. I note that the Editor in Chief is now Wendy Machalicek, who previously co-edited the journal with Russell Lang. Lang's doctoral advisor was Mark O'Reilly, editor of JDPD, and one of the prolific trio who featured in blogpost 2. Lang himself co-authored 24 papers in RASD and 13 in RIDD, and 35 of these 39 papers were accepted within 2 weeks of receipt. Machalicek has published 11 papers in RASD and 5 in RIDD, and 12 of these 16 papers were accepted within 2 weeks of receipt.  She also did her doctorate in O'Reilly's department, and several of her papers are co-authored with him. In an editorial last year, Lang and Machalicek announced changes to the journal, some of which seem to be prompted by a desire to make the reviewing process more rigorous under the new publisher. However, one change is of particular interest: the scope of the journal will be broadened to consider "developmental disability from a lifespan perspective; wherein, it is acknowledged that development occurs throughout a person's life and a range of injuries, diseases and other impairments can cause delayed or abnormal development at any stage of life." That will be good news for Giulio Lancioni, who was previously publishing papers on coma patients, amyotrophic lateral sclerosis, and Alzheimer's disease in RIDD. He and his collaborators – Jeff Sigafoos, Mark O'Reilly, as well as Russell Lang and Johnny Matson – are all current members of the editorial board of the journal.

It seems to be business as usual at the Springer title, Journal of Developmental and Physical Disabilities. Mark O'Reilly is still the editor, with Lang and Sigafoos as associate editors; Lancioni, Machalicek and Matson are all on the editorial board. Springer's willingness to turn a blind eye to editors playing the system becomes clear when one sees that a recent title, "Review Journal of Autism and Developmental Disorders" has as Editor-in-Chief no less a personage than Johnny Matson. And, surprise, surprise, the editorial board includes Lang, Sigafoos and Lancioni.

One of the overarching problems I uncovered when navigating my way around this situation was that there is no effective route for a whistleblower who has uncovered evidence of dubious behaviour by editors. Elsevier has developed a Publishing Ethics Resource Kit  but it is designed to help editors dealing with ethical issues that arise with authors and reviewers. The general advice if you encounter an ethical problem is to contact the editor. The Committee on Publication Ethics also issues guidance, but it is an advisory body with no powers. One would hope that publishers would act with integrity when a serious problem with an editor is revealed, but if my experience is anything to go by, they are extremely reluctant to act and will weave very large carpets to brush the problems under.