I was recently asked to review a paper that piqued my curiosity. From the abstract it sounded as if it was a secondary analysis of a dataset that I had deposited in a repository. But the described data didn't quite match what was in the repository.
As I usually do, I said I'd only review it if data and code were available. The authors complied: I was pointed at the repository where my data were stored, and when the editor requested it on my behalf, a script in Python code arrived.
As I had suspected, the whole thing was fabricated. The analysis would have required linking data from two separate repositories - but the linkage information was not publicly available. No mention was made of the repository where the bulk of the supposedly analysed data were held. And the python script didn't clarify how the data were processed, and didn't even run with dummy code.
So far, so depressingly predictable. But, despite all these issues, I was shocked at how plausible the paper appeared to be.
In the past, I've been able to detect fraudulent stuff because it is either incomprehensible, formulaic, full of tortured phrases, had results that were far too good to be true, or described methods that were so implausible as to defy the laws of probability. Of course, I have no idea how often I've missed stuff that has slipped through the net; particularly in recent months, I've reviewed papers that have aroused deep suspicions but without sufficient evidence to justify calling it out as fraud.
This latest case, though, seemed in a different league. The paper was not formulaic - it was in a rather specialised area, and purported to ask an original and interesting question. One of the things that made it seem believable was that the results were rather messy, raising more questions than answers. I wonder if I'd have identified the fraud if it hadn't concerned my own datasets.
I assume this is one of the fruits of all those data centres that are popping up everywhere - they're facilitating the generation of more plausible fakes. So we're going to need yet more ways of defending the literature against them. I have strong doubts as to whether the world of academic publishing is ready for this, and I'm not confident that any kind of AI-screening will cope with the more sophisticated kinds of AI-fraud. I cling to the faint hope that human experts still might be able to trap some of it, but I'm feeling more and more like a member of the Light Brigade, galloping into the Valley of Death-of-Research-Credibility.
For what it's worth, I'll offer my thoughts on things reviewers can look out for, but to be honest, the frauds are getting so good that these methods now seem inadequate.
1. The first rule is to assume that what you are reading might be fraudulent. Alas, the age is long gone when it was safe to assume that a new manuscript was trustworthy. In 2021, Richard Smith, former editor of the BMJ, suggested we should assume health research is fraudulent until proved otherwise, and there's no reason to suppose that other fields are not similarly affected.
2. Are data and code available? As noted above, I won't review unless they are, because it's just too easy to make stuff up. In the longer term, I worry that AI can generate entire papers with a replication package, but I hope that requiring these may at least slow down the fraudsters. I strongly recommend that all journals reporting empirical research require a replication package, as has become standard in the field of economics (see e.g. here and here). Of course, actually checking data and code can take up a lot of time, which is the one thing most reviewers don't have.
3. Do the cited references exist and are they cited appropriately? Hallucinated references are widely recognised as a hallmark of AI-generated material; the paper I refer to above didn't have any of those. But it did have a lot of "misaligned" citations - the kind of thing described in this post by Jill Walker Rettberg. She noted that as well as citing sources that were at best tangentially related to the referring text, there would be an absence of citations to the key work in the area. Checking this stuff is yet another tedious addition to the reviewer's workload.
4. Does the article make sense? Failure here could involve lack of coherent structure, use of nonsensical "tortured phrases", and/or headings that follow the Introduction/Methods/Research and Discussion (IMRaD) structure seemingly inserted at random points in the text (see Matusz et al 2025). This kind of thing is highly diagnostic if you find it, but I suspect it is becoming less common as the fraudsters up their game.
5. Consider whether it is plausible that the researchers did what they said they did within the stated time frame and with the stated resources. Some examples where this was a red flag are described here. This may be hard to judge if no time frame is given but for certain kinds of research, such as clinical trials, this should be stated. This is also a key check in InspectSR - see points 2.4 and 2.5 here. Even so, implausible research can be hard to detect unless the reviewer has conducted similar studies themselves.
6. Are the researchers and their affiliations legitimate? We need to be particularly careful here, because by considering provenance of research, we risk moving back to a time when research was trusted only if it came from people with some kind of track record, and/or who were personally known to the reviewer or editor. Research publishing should not be a walled garden that keeps out newcomers. But equally, if we take the walls down, we need to be vigilant to ensure the garden is not overwhelmed by weeds. As with point 5, plausibility is also relevant here: if two researchers with no track record from different countries claim to have collaborated on a technically complex analysis without any funding, I'd want to look at whether their CVs explained how they had acquired the skills to do this.
Perhaps the most depressing thing of all is that just when we need to be mustering competent researchers who can undertake the careful and time-consuming work of reviewing new submissions, academic publishers are pursuing rapid growth, which will provide ever more opportunities for fraudsters to find an outlet for their work. They don't seem to appreciate (or some would say they don't care) that there's a shortage of competent and motivated people to serve as editors and reviewers in the existing system, let alone in a vastly expanded one. I don't think this will end well.
Posts
