IEEE Has a Pseudoscience Problem

Guest post by Solal Pirelli

The IEEE, full name Institute of Electrical and Electronics Engineers, is one of the main scientific publishers in domains related to its name. Many IEEE venues, such as ICSE in software engineering and IROS in robotics, are “top” venues that publish important research. While these are conferences and not journals, computer science and related fields are unusual in that conferences are typically the more prestigious option.

But as I’ve covered before in the case of another big computer science publisher, world-class research can coexist with world-class nonsense. Many not-so-top IEEE venues publish “AI gobbledegook sandwiches”, pointless papers that apply standard machine learning or artificial intelligence to basic data sets resulting in vague predictions supposedly improving on ill-defined baselines.

Unfortunately, bad science published by IEEE isn’t limited to boring applications of boring algorithms to boring data. In this blog post, I’ll present IEEE-published pseudoscience of various kinds, show how this correlates with other problems, and discuss why publishers don’t do enough about it. 

All kinds of quackery 

The IEEE has published numerous new “methods” to help providers or users of pseudoscientific disciplines. Ayurveda is enhanced with a “preprocessing framework” to detect diabetes, a neural network to classify herbs, and even an AI assistant. Astrology is automated with a machine learning model. Myers-Briggs personality type testing is granted another neural network.

Some IEEE papers are at the very fringe of pseudoscience, unconventional even by quack standards. A symposium on antennas and propagation published three papers by the same author on “scientific traditional Chinese medicine”, a variant based on electromagnetism and 5G with “supernatural potential” (see here, here and here). An Indian conference on electronics published four papers (here, here, here and here) by the same first author on “electro-homeopathy”, the brainchild of a 19th century Italian count that an Indian high court called “nothing but quackery” a decade before these papers were published.

Of course, no list of pseudoscience would be complete without perpetual motion. That’s right, the IEEE has published two papers on perpetual motion in 2017 and 2022! How these were not desk-rejected is anyone’s guess.

Even work that is not pseudoscientific in itself can propagate harmful or downright absurd stereotypes. Consider what IEEE-published and supposedly peer-reviewed papers have to say about autism: 

• “Children with autism require constant care because you never know what will trigger them” (source) 
• "If symptoms of autism are detected early, children with autism usually return to normal development after effective medical intervention" (source) 
• “A baby born with autism spectrum disorder may have a lower-than-average heart rate. Complete blockage of the heart at birth is rare. Abnormal heart rate leads to heart block. So, there is a high chance of the child's death due to permanent heart blockage at any time.” (source)

Why it matters

One may think that such papers won’t cause harm because they’re unlikely to be read, since they are mostly in unknown venues and unrelated to the IEEE’s domain. While I personally disagree since I believe publishing pseudoscience risks breaking the public’s trust in legitimate research, let me provide a more objective argument. Pseudoscience in papers is heavily correlated with other problematic practices that are more difficult to detect automatically. This makes searching for pseudoscience an effective way to find problematic venues, complementary to existing techniques.

The preprocessing framework to detect diabetes with Ayurveda? In a conference that accepts papers on the same day they are submitted, somehow speeding up the weeks or months usually necessary for proper peer review.

The neural network that classifies Ayurvedic herbs? In a conference that plagiarized its peer review policy from Elsevier’s “Transport Policy” journal. Look for fragments of this policy in your favorite search engine and you’ll find a surprising number of venues that have done so, seemingly without noticing the references to Transport Policy.

The four papers on electro-homeopathy? In a conference that published a mathematical “algorithm” amounting to high school mathematics. While exact definitions of “novelty” vary, no one could credibly claim that this paper is novel enough for a scientific conference.

The 2017 paper on perpetual motion? In a conference that didn’t notice an entirely plagiarized section in that paper, ironically from a source explaining why perpetual motion is impossible. How this is compatible with IEEE’s policy of checking all content for plagiarism is unclear.

The paper claiming “you never know what will trigger” autistic children? In a conference supposedly happening in a London office building, whose four IEEE-published editions only feature one paper from a European university among a sea of India-based authors. Did the authors of this conference’s papers really travel to the other side of the globe to present in a place not designed for presentations?

The neural network for Myers-Briggs? In a conference chaired by a professor whose Russian university is under sanctions from the US, the EU, Ukraine, and even Switzerland!

Action is rare 

The expected process here would be to report this nonsense to the publisher, who would investigate, quickly conclude these papers should never have been published, lose faith in the peer review process that led to their acceptance, and issue retractions. Barring extremely strong evidence from conference chairs that some cases were truly one-off exceptions, such retractions would cover entire editions of conferences.

This happens… sometimes. The IEEE has retracted papers before, such as this one after “only” five months. They have also retracted entire venues, such as this one totaling 400 papers, four years after it was reported.

 
But the IEEE frequently does not react at all to reports. Guillaume Cabanac, who specializes in scientific fraud detection, has repeatedly and publicly called them out. For instance, he’s reported telltale signs of ChatGPT as in this paper that includes “Regenerate Response” in the middle of text and this paper that includes “I am unable to […] due to the fact I am an AI language model”. He’s also reported “tortured phrases”, attempts at avoiding plagiarism detection that instead create nonsense such as “parcel misfortune” instead of “packet loss” in computer networking, in sometimes large concentrations. Cabanac and other sleuths have published “proceedings-level reports” on PubPeer, such as this one, when entire IEEE conferences have problems. None of the examples in this paragraph have led to any public reaction from the IEEE.

The IEEE occasionally issues “expressions of concern”, such one as for this paper over a year after concrete evidence of plagiarism was publicly reported. But expressions of concerns are not retractions. In mid-2023, Retraction Watch noted that hundreds of IEEE papers reported by Guillaume Cabanac and Harvard lecturer Kendra Albert were still up for sale. A year and a half later, that remains the case.

One case noted above is particularly noteworthy in terms of both reputation and IEEE awareness: The “scientific TCM” papers were published in the 2022 and 2023 editions of the “International Symposium on Antennas and Propagation”, a 6-decade-old conference whose 2024 edition boasted the IEEE President as a keynote speaker. Clearly, the IEEE is aware of the venue and its papers. What’s the point in “reporting” them?

Processes are inadequate 

The scale of publishers’ actions is nowhere near the scale of the problem. Creating a new conference or journal does not require that much time if the peer reviewing process is fake. As long as the average time it takes a publisher to retract a venue is higher than the time it takes to create a new venue, there won’t be meaningful progress.

Current publisher processes are designed to correct honest mistakes, not to fight malice. The time it takes to contact authors, wait for their response, wait for them to find original data, and so on is worth it when a single paper has a problem that can be explained by human error. But any such process is a waste of time when a paper contains blatant pseudoscience, has obviously been plagiarized, or uses terminology so bizarre no reviewer could have understood it.

To give an example of scale, here’s a collision of pseudoscience and tortured phrases. The paper on an AI assistant for ayurveda mentioned earlier is in the “2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT)”. Guillaume Cabanac’s Problematic Paper Screener currently lists 185 cases of tortured phrases manually confirmed by Cabanac himself, with another 160 pending assessment. These include “herbal language” instead of natural language, “system getting to know” instead of machine learning, “give-up-to-give-up” instead of end-to-end, and “0.33-celebration” instead of third-party.  

Individually contacting and waiting for hundreds of authors just in case they can explain why their paper talks about 0.33-celebrations isn’t going to cut it. Neither is individually contacting and waiting for dozens of conference editors just in case they can explain why their peer review process didn’t spot this nonsense. 

What can we do?

Given the incentives and processes at play, it’s not surprising to see the IEEE or any other big publisher publish pseudoscience. The authors of the papers mentioned in this post probably didn’t do anything illegal, except maybe for occasional plagiarism of copyrighted content, but nobody has the time and money to sue for such boring violations. This gives publishers a double excuse: they’re not publishing anything illegal, and retractions without a solid legal basis could backfire.

The scientific community needs to ban the “incompetence” defense from authors and stop associating with publishers that can’t be bothered to act quickly enough.  

Authors who publish obvious nonsense should not get a chance to explain themselves or “correct” their paper. 

Publishers make enough money from processing and selling articles. They can defend themselves from occasional lawsuits by angry authors, and they can hire scientific integrity specialists.

When I say “scientific integrity specialist”, that can unfortunately be as simple as “person looking for specific keywords in Google Scholar”. It’s what I did to find pseudoscience, and you can do that too. Report these on PubPeer, directly to publishers, or both. You can also go to the Problematic Paper Screener’s page listing articles that have not been manually assessed yet, and follow the instructions.

Finally, remember that most scientists have no idea this is going on. You can help by publicly calling out problematic papers and lack of action. Ask candidates for governance boards in more democratic publishers like the IEEE what they plan to do about fraud. Discourage institutions, especially public ones in democratic countries, from making blanket deals with publishers.

Seven reasons for keeping Elon Musk as a Fellow of the Royal Society

 

Last November, I wrote a blogpost explaining why I had resigned as a Fellow of the Royal Society (FRS). In brief, over the summer a group of 74 FRSs asked the RS Council to consider revoking Musk's FRS on the grounds that he had attempted to interfere with British politics by spreading disinformation with the aim of stirring up unrest. The Council took advice, considered carefully and concluded that nothing should be done. I discussed this decision with senior figures in the RS and presented them with additional evidence of Musk's unsuitability for the honour of FRS. I was told that they could take another look, but they had to abide by the procedures laid out in the Statutes, and this could take some time. Last week I heard that the Council had met again to discuss this issue and come to the same conclusion: nothing should be done.  

It's hard to evaluate their reasoning, as no details have been given. Colleagues of mine who remain as FRSs occasionally report snippets of information they have heard from members of Council, but these have been inconsistent and unreliable. But meanwhile, I've had many members of the public contact me about this issue. The majority are strongly supportive of my position; even those who were on the fence leapt off it after the infamous "Roman" salute. Musk's involvement in the dissolution of US academic institutions, achieved by suppressing some of their activities and starving them of funds, has further solidified opinion against him. But few FRSs have spoken out. Such information that I've gleaned has come from informal contacts, where I've heard seven different arguments against the expulsion of Elon Musk.  Given that I've previously laid out a set of reasons in favour of Musk's expulsion,  in the interests of balance,  I present here the counter arguments.  

1. Musk should not have his FRS withdrawn because he does important and innovative scientific research, and is a role model for scientists worldwide. 

In fact, I include this one for completeness, but I've not encountered a single person who has made this case.  It seems generally accepted that he is the antithesis of the scholarly ideal set out by Jacob Bronowski (2008: Science and Human Values) - if you turn all the negative statements to positives, and positives to negatives, this is a fair description of Musk: 

... they do not make wild claims, they do not cheat, they do not try to persuade at any cost, they appeal neither to prejudice nor to authority, they are often frank about their ignorance, their disputes are fairly decorous, they do not confuse what is being argued with race, politics, sex or age, they listen patiently to the young and to the old who both know everything. These are the general virtues of scholarship, and they are particularly the virtues of science.  

Instead, I've seen a weary procession of arguments that all begin with someone saying "Of course, Musk is terrible but we shouldn't expel him because...."  

2. The Royal Society can't be political.  

This argument has some merit. On the one hand, the Royal Society is a registered charity, and on the other hand it offers science policy advice to government. As this blogpost from the Charity Commission makes clear: 

 Charities are required to be independent and cannot have political purposes, and this is important for public trust in charities. As such, charities must never stray into party politics – they must never promote, or be seen to promote, a political party or candidate. As trustees and charity leaders you must protect your charity’s reputation and not allow your organisation to be used as a vehicle for the expression of the party-political views of any individual trustee, employee, political party, or candidate.

There are two reasons why I think the expulsion of Musk is entirely compatible with the Charity Commission rules.  

First, there are plenty of reasons to object to Musk's FRS that have nothing to do with politics. He has repeatedly used his social media platform to attack Anthony Fauci, a respected scientist and Foreign Member of the Royal Society. This is not usual academic discourse: it is calling for Fauci to be prosecuted, and blaming him for medical crimes. Fauci needs personal security protection because there is credible threat to his life, stirred up by various factions who disapproved of his role in the Covid-19 pandemic - including Musk, who loathed having restrictions on movement imposed and is now arguing that Fauci's research somehow caused the pandemic. It is easy to find examples of Musk's attacks on Fauci by Googling for "Musk Fauci" - e.g. this piece in the New York Post. 

Other reasons are provided in my blogpost: lack of appropriate regulatory approval for Neuralink, and spreading of disinformation about climate science and vaccinations. 

Second, the fact that Musk also does overtly political things shouldn't make people timid about disapproving of his other actions. And most of Musk's political activities are part and parcel of his disinformation campaigns, and can be objected to on the grounds that they are dishonest. 

In sum, if the Royal Society were to campaign against the Reform Party, that would be improper. But to dissociate themselves from the spreading of disinformation seems to me to be a valid and desirable activity, compatible with their charitable status, and their stated aims: 

The Society ... is a registered charity, undertaking a range of activities that provide public benefit either directly or indirectly. As a national academy, it represents the UK and collaborates with international partners to advocate for science and its benefits.  

3. The Royal Society has other odious people as Fellows. 

I was amused to see this argument trotted out in a couple of pieces in the media - first in the Spectator by Toby Young (someone whose own ennoblement has raised eyebrows), and then by Jawad Iqbal in the Times. Both pieces noted that James Watson remained an FRS despite his abhorrent views on eugenics. Iqbal also noted that Prince Andrew had been made a Royal Fellow in 2013. This was perhaps an unwise example to use, given that Prince Andrew's name quietly disappeared from the list of Fellows in 2022, after he was encouraged to resign from a number of honours.

I haven't heard this case made by many FRSs, and maybe that's because they can see what a grubby argument it is. But there is more appetite for a related point:  

4. Expelling Musk could set a dangerous precedent.

As it happens, some people who contacted me after my resignation have taken the opportunity to tell me about other dodgy FRSs - not because they adopt argument 3, but on the contrary, because they think that if the Royal Society were to start taking its Code of Conduct seriously, there are others who should also be looked at. One can see that this line of argument might generate a degree of nervousness among Council members.  

5. Musk's supporters might say bad things about the Royal Society if he were expelled.  

There's three versions of this: (a) they'd say the RS was political (see point 2); (b) they'd say the RS was 'woke', and (c) they'd say RS was 'elitist'.  

These arguments remind me all too sadly of the decline in the state of debate in British politics. It seems to be accepted as a defensible line of argument these days to warn against doing something that you know to be right because someone else might, either mischievously or sincerely, misattribute your motives for doing so. The solution is to state clearly what your reasons are, and not get derailed by name-calling. If the Royal Society is really worried about reputational damage, then they should realise that being designated as 'woke' or 'elitist' by your opponents is far less of an insult than being described as cowardly by your friends.  

6. There could be bad consequences for science and scientists if Musk were expelled. 

This argument sounds suspiciously like a case for appeasement of a bully. History has taught us that appeasement does not end well. Indeed, even in the past weeks, we've seen various academic organisations and institutions scrabbling to obey directives from the Trump regime to remove all mention of diversity, equality and inclusion from their documents and websites: it has not saved them from the depredations of DOGE. Philip Ball has written in Chemistry World about the tendency for institutions to show "anticipatory obedience" and the importance of resistance.  

A more specific version of this argument maintains that it could make matters worse for Anthony Fauci if the Royal Society were to expel Musk, particularly if his attacks on Fauci were cited as a reason. Sadly enough, it seems to me we have here what scientists call a floor effect - i.e. the situation for Fauci is so bad - with serious threats on his life and his personal security protection now removed - that it's not clear it could get any worse. Showing him some solidarity may not achieve much, but it would confirm that the Royal Society is prepared to stand up to bullies and support those who do deserve the accolade of being honoured.  

7. There could be bad consequences for the Royal Society if Musk were expelled.  

The main bad consequence that goes beyond name-calling (see 5) would be if Musk decided to mount a legal challenge to his expulsion. No doubt the legal counsel that the Royal Society has employed will have judged how likely this is to happen, and how likely it could be successful if it did happen. Nobody wants to get embroiled in legal battles, which can be expensive and arduous. My personal view is that the Royal Society would have a stronger defence against legal action if it polled the whole Fellowship and the result turned out in favour of expelling Musk. I suggested that the Fellowship should be consulted last Summer but was told that was not in line with the Statutes. (I should add that I'm not confident that the Fellowship would vote to expel Musk - many of them seem swayed by arguments 2-6, but with every day that passes, his malign influence on science and society increases, and so I think it's possible he might be voted out).  

Personally, I think the Royal Society should take the risk of a legal challenge. They are a wealthy organisation, and they represent the voice of scientists in the UK. Our fellow scientists in the USA are now under a level of pressure that even the most pessimistic of us had not anticipated. It is hard for individual scientists to resist. But the Royal Society has the clout and the resource to weather the storm. If they would take a stand, this would show solidarity with our friends across the pond, by confirming that the Brits aren't going to honour someone who is playing a major role in dismantling scientific research in the USA.