Can systematic reviews help clean up science?

 

The systematic review was not turning out as Lorna had expected

Why do people take the risk of publishing fraudulent papers, when it is easy to detect the fraud? One answer is that they don’t expect to be caught. A consequence of the growth in systematic reviews is that this assumption may no longer be safe. 

In June I participated in a symposium organised by the LMU Open Science Center in Munich entitled “How paper mills publish fake science industrial-style – is there really a problem and how does it work?” The presentations are available here. I focused on the weird phenomenon of papers containing “tortured phrases”, briefly reviewed here. For a fuller account see here. These are fakes that are easy to detect, because, in the course of trying to circumvent plagiarism detection software, they change words, with often unintentionally hilarious consequences. For instance, “breast cancer” becomes “bosom peril” and “random value” becomes “irregular esteem”. Most of these papers make no sense at all – they may include recycled figures from other papers. They are typically highly technical and so to someone without expertise in the area they may seem valid, but anyone familiar with the area will realise that someone who writes “flag to commotion” instead of “signal to noise” is a hoaxer. 

Speakers at the symposium drew attention to other kinds of paper mill whose output is less conspicuously weird. Jennifer Byrne documented industrial-scale research fraud in papers on single gene analyses that were created by templates, and which purported to provide data on under-studied genes in human cancer models. Even an expert in the field may be hoodwinked by these. I addressed the question of “does it matter?” For the nonsense papers generated using tortured phrases, it could be argued that it doesn’t, because nobody will try to build on that research. But there are still victims: authors of these fraudulent papers may outcompete other, honest scientists for jobs and promotion, journals and publishers will suffer reputational damage, and public trust in science is harmed. But what intrigued me was that the authors of these papers may also be regarded as victims, because they will have on public record a paper that is evidently fraudulent. It seems that either they are unaware of just how crazy the paper appears, or that they assume nobody will read it anyway. 

The latter assumption may have been true a couple of decades ago, but with the growth of systematic reviews, researchers are scrutinizing many papers that previously would have been ignored. I was chatting with John Loadsman, who in his role as editor of Anaesthesia and Intensive Care has uncovered numerous cases of fraud. He observed that many paper mill outputs never get read because, just on the basis of the title or abstract, they appear trivial or uninteresting. However, when you do a systematic review, you are supposed to read everything relevant to the research question, and evaluate it, so these odd papers may come to light. 

I’ve previously blogged about the importance of systematic reviews for avoiding cherrypicking of the literature. Of course, evaluation of papers is often done poorly or not at all, in which case the fraudulent papers just pollute the literature when added to a meta-analysis. But I’m intrigued at the idea that systematic reviews might also serve the purpose of putting the spotlight on dodgy science in general, and fraudsters in particular, by forcing us to read things thoroughly. I therefore asked Twitter for examples – I asked specifically about meta-analysis but the responses covered systematic reviews more broadly, and were wide-ranging both in the types of issue that were uncovered and the subject areas. 

Twitter did not disappoint: I received numerous examples – more than I can include here. Much of what was described did not sound like the work of paper mills, but did include fraudulent data manipulation, plagiarism, duplication of data in different papers, and analytic errors. Here are some examples: 

Paper mills and template papers

Jennifer Byrne noted how she became aware of paper mills when looking for studies of a particular gene she was interested in, which was generally under-researched. Two things raised her suspicions: a sudden spike in studies of the gene, plus series of papers that had the same structure, as if constructed from a template. Subsequently, with Cyril Labbé, who developed an automated Seek & Blastn tool to assess nucleotide sequences, she found numerous errors in the reagents and specification of genetic sequences of these repetitive papers, and it became clear that they were fraudulent. 

An example of a systematic review that discovered a startling level of inadequate and possibly fraudulent research was focused on the effect of tranexamic acid on post-partum haemorrhage: out of 26 reports, eight had sections of identical or very similar text, despite apparently coming from different trials. This is similar to what has been described for papers from paper mills, which are constructed from a template. And, as might be expected for a paper mill output, there were also numerous statistical and methodological errors, and some cases without ethical approval. (Thanks to @jd_wilko for pointing me to this example). 

Plagiarism 

Back in 2006, Iain Chalmers, who is generally ahead of his time, noted that systematic reviews could root out cases of plagiarism, citing the example of Asim Kurjak, whose paper on epidural analgesia in labour was heavily plagiarised. 

Data duplication 

Meta-analysis can throw up cases where the same study is reported in two or more papers, with no indication that this is the same data. Although this might seem like a minor problem compared with fraud, it can be serious, because if the duplication is missed in a meta-analysis, that study will be given more weight than it should have. Ioana Cristea noted that such ‘zombie papers’ have cropped up in a meta-analysis she is currently analysing. 

Tampering with peer review 

When a paper considered for a meta-analysis seems dubious, it raises the question of whether proper peer review procedures were followed. It helps if the journal adopts open peer review. Robin N. Kok reported a paper where the same person was listed as an author and a peer reviewer. This was eventually retracted.  

Data seem too good to be true 

This piece in Science tells the story of Qian Zhang, who published a series of studies on impact of cartoon violence in children which on the one hand had remarkably large samples of children all at the same age, and on the other hand had similar samples across apparently different studies.  Because of their enormous size, Zhang’s papers distorted any meta-analysis they were included in. 

Aaron Charlton cited another case, where serious anomalies were picked up in a study on marketing in the course of a meta-analysis. The paper was ultimately retracted 3 years after the concerns were raised, after defensive responses from some of the authors, challenging the meta-analysts. 

This case flagged by Neil O’Connell is especially useful, as it documents a range of methods used to evaluate suspect research. The dodgy work was first flagged up in a meta-analysis of cognitive behaviour therapy for chronic pain.  Three papers with the same lead author, M. Monticone, obtained results that were discrepant with the rest of the literature, with much bigger effect sizes. The meta-analysts then looked at other trials by the same team and found that there was a 6-fold difference between the lower confidence interval of the Monticone studies and the upper confidence interval of all others combined. The paper also reports email exchanges with Dr Monticone that may be of interest to readers. 

Poor methodology 

Fiona Ramage told me that in the course of doing a preclinical systematic review and meta-analysis of nutritional neuroscience, she encountered numerous errors of basic methodology and statistics, e.g. dozens of papers where error bars were presented without indicating if they show SE or SD; studies claiming differences between groups without a direct statistical comparison. This is more likely to be due to ignorance or honest error than to malpractice, but it needs to be flagged up so that the literature is not polluted by erroneous data.

What are the consequences?

Of course, the potential of systematic reviews to detect bad science is only realised if the dodgy papers are indeed weeded out of the literature, and people who commit scientific fraud are fired. Journals and publishers have started to respond to paper mills, but, as Ivan Oransky has commented, this is a game of Whac-a-Mole, and "the process of retracting a paper remains comically clumsy, slow and opaque”. 

I was surprised that even when confronted with an obvious case of a paper that had both numerous tortured phrases and plagiarism, the response from the publisher was slow – e.g. this comically worded example is still not retracted, even though the publisher’s research integrity office acknowledged my email expressing concern over 2 months ago.  But 2 months is nothing. Guillaume Cabanac recently tweeted about a "barn door" case of plagiarism that has just been retracted 20 years after it was first flagged up.  When I discuss the slow responses to concerns with publishers, they invariably say that they are being kept very busy with a huge volume of material from paper mills. To which I answer, you are making immense profits, so perhaps some could be channeled into employing more people to tackle this problem. As I am fond of pointing out, I regard a publisher who leaves seriously problematic studies in the literature as analogous to a restauranteur that serves poisoned food to customers. 

Publishers may be responsible for correcting the scientific record, but it is institutional employers who need to deal with those who commit malpractice. Many institutions don’t seem to take fraud seriously. This point was made back in 2006 by Iain Chalmers, who described the lenient treatment of Asim Kurjak, and argued for public naming and shaming of those who are found guilty of scientific misconduct. Unfortunately, there’s not much evidence that his advice has been heeded. Consider this recent example of a director of a primate reseach lab who admitted fraud, but is still in post. (Here the fraud was highlighted by a whistleblower rather than a systematic review, but this illustrates the difficulty of tackling fraud when there are only minor consequences for fraudsters). 

Could a move towards "slow science" help? In the humanities, literary scholars pride themselves on “close reading” of texts. In science, we are often so focused on speed and concision, that we tend to lose the ability to focus deeply on a text, especially if it is boring. The practice of doing a systematic review should in principle develop better skills in evaluation of individual papers, and in so doing help cleanse the literature from papers that should never have got published in the first place. John Loadsman has suggested we should not only read papers carefully, but should recalibrate ourselves to have a very high “index of suspicion” rather than embracing the default assumption that everyone is honest. 

P.S. 

Many thanks to everyone who sent in examples. Sorry I could not include everything. Please feel free to add other examples or reactions in the Comments – these tend to get overwhelmed with adverts for penis enlargement or (ironically) essay mills, and so are moderated, but I do check them and relevant comments will eventually appear.

PPS. Florian Naudet sent a couple of relevant links that readers might enjoy: 

Fascinating article by Fanelli et al who looked at how inclusion of retracted papers affected meta-analyses: https://www.tandfonline.com/doi/full/10.1080/08989621.2021.1947810  

And this piece by Lawrence et al shows the dangers of meta-analyses when there is insufficient scrutiny of the papers that are included: https://www.nature.com/articles/s41591-021-01535-y  

Also, Joseph Lee tweeted about this paper about inclusion of papers from predatory publications in meta-analyses: https://jmla.pitt.edu/ojs/jmla/article/view/491 

PPPS. 11th August 2022

A couple of days after posting this, I received a copy of "Systematic Reviews in Health Research" edited by Egger, Higgins and Davey Smith. Needless to say, the first thing I did was to look up "fraud" in the index. Although there are only a couple of pages on this, the examples are striking. 

First, a study by Nowbar et al (2014) on bone marrow stem cells for heart disease found that in a review of 133 reports, over 600 discrepancies were found, and the number of discrepancies increased with the reported effect size. There's a trail of comments on Pubpeer relating to some of the sources, e.g. https://pubpeer.com/publications/B346354468C121A468D30FDA0E295E.

Another example concerns the use of beta-blockers during surgery. A series of studies from one centre (the DECREASE trials) showing good evidence of effectiveness was investigated and found to be inadequate, with missing data and failure to follow research protocols. When these studies were omitted from a meta-analysis, the conclusion was that, far from receiving benefit from beta-blockers, patients in the treatment group were more likely to die (Bouri et al, 2014). 

 PPPPS, 18th August 2022

This comment by Jennifer Byrne was blocked by Blogger - possibly because it contained weblinks.

Anyhow, here is what she said:

I agree, reading both widely and deeply can help to identify problematic papers, and an ideal time for this to happen is when authors are writing either narrative or systematic reviews. Here's another two examples where Prof Carlo Galli and colleagues identified similar papers that may have been based on templates: https://www.mdpi.com/2304-6775/7/4/67, https://link.springer.com/article/10.1007/s11192-022-04434-2 

 

Contagion of the political system

 

Citizens of the UK have in recent weeks watched in amazement as the current candidates for leadership of the Conservative party debate their policies. Whoever wins will replace Boris Johnson as Prime Minister, with the decision made by a few thousand members of the Conservative Party. All options were bad, and we are now down to the last two: Liz Truss and Rishi Sunak.

 

For those of us who are not Conservatives, and for many who are, there was immense joy at the ousting of Boris Johnson. The man seemed like a limpet, impossible to dislodge. Every week brought a new scandal that would have been more than sufficient to lead to resignation 10 years ago, yet he hung on and on. Many people thought that, after a vote of no confidence in his leadership, he would step down so that a caretaker PM could run the country while the debate over his successor took place, but the limpet is still clinging on. He’s not doing much running of the country, but that’s normal, and perhaps for the best. He’s much better at running parties than leading the Conservative party.

 

I have to say I had not expected much from Truss and Sunak, but even my low expectations have not been met. The country is facing immense challenges, from climate change, from coronavirus, and from escalating energy prices. These are barely mentioned: instead the focus is on reducing taxes, with the candidates now competing for just how much tax they can cut. As far as I can see, these policies will do nothing to help the poorest in society, whose benefits will shrink to pay for tax cuts; the richer you are the more tax you pay and so this is a rich person’s policy.

 

What has surprised me is just how ill-informed the two candidates are. The strategy seems to be to pick a niche topic of interest to Conservative voters, make up a new policy overnight and announce it the next day. So we have Rishi Sunak proposing that the solution to the crisis in the NHS is to charge people who miss doctor’s appointments. Has he thought this through? Think of the paperwork. Think of the debt collectors tasked with collecting £10 from a person with dementia. Think of the cost of all of this.  And on Education, his idea is to reintroduce selective (grammar) schools: presumably because he thinks that our regular schools are inadequate to educate intelligent children.

 

On Education, Liz Truss is even worse. Her idea is that all children who score top marks in their final year school examinations should get an interview to go to Oxford or Cambridge University. This is such a crazy idea that others have written at length to point out its flaws (e.g. this cogent analysis by Sam Freedman). Suffice it to say that it has a similar vibe to the Sunak grammar schools plan: it implies that only two British universities have any value. Conservatives do seem obsessed with creating divisions between haves and have-nots, but only if they can ensure their children are among the haves.

 

Another confused statement from Truss is that, as far as Scotland goes, she plans to ignore Nicola Sturgeon, the First Minister of Scotland and leader of the Scottish National Party. At a time when relationships between Scotland and England are particularly fraught, this insensitive statement is reminiscent of the gaffes of Boris Johnson.

 

Oh, and yesterday she also announced – and then quickly U-turned – an idea that would limit the pay of public sector workers in the North of England, because it was cheaper to live there.

 

What I find so odd about both Sunak and Truss is that they keep scoring own goals. Nobody requires them to keep coming up with new policies in niche areas.  Why don’t they just hold on to their original positions, and if asked about anything else, just agree to ‘look at’ it when in power? Johnson was always very good at promising to ‘look at’ things: when he’s not being a limpet, he’s a basilisk. The more you probe Sunak and Truss, the more their shallowness and lack of expertise show through. They’d do well to keep schtum. Or, better still, show some indication that they could, for instance, get a grip on the crisis in the NHS.

 

What all this demonstrates is how an incompetent and self-promoting leader causes damage far beyond their own term. Johnson’s cabinet was selected purely on one criterion: loyalty to him. The first requirement was to “believe in Brexit” – reminiscent of the historical wars between Protestants and Catholics, where the first thing you ask of a candidate is what their religion is. Among Conservative politicians, it seems that an accusation of not really being a Brexiteer is the worst thing you can say about a candidate. Indeed, that is exactly the charge that her opponents level against Truss, who made cogent arguments for remaining in the EU before the referendum. Like a Protestant required to recant their beliefs or face the flames, she is now reduced to defending Brexit in the strongest possible terms, saying that “predictions of doom have not come true”, as farmers, fishermen, and exporters go out of business, academics leave in droves, and holidaymakers sit in queues at Dover.

 

It's known that Johnson does not want to give up the top job. I’m starting to wonder if behind all of this is a cunning plan. The people he’s appointed to cabinet are so incompetent that maybe he hopes that, when confronted with a choice between them, the Conservative Party will decide that he looks better than either of them.

 

 

 

 

A proposal for data-sharing that discourages p-hacking

Open data is a great way of helping give confidence in the reproducibility of research findings. Although we are still a long way from having adequate implementation of data-sharing in psychology journals (see, for example, this commentary by Kathy Rastle, editor of Journal of Memory and Language), things are moving in the right direction, with an increasing number of journals and funders requiring sharing of data and code. But there is a downside, and I've been thinking about it this week, as we've just published a big paper on language lateralisation, where all the code and data are available on Open Science Framework. 

One problem is p-hacking. If you put a large and complex dataset in the public domain, anyone can download it and then run multiple unconstrained analyses until they find something, which is then retrospectively fitted to a plausible-sounding hypothesis. The potential to generate non-replicable false positives by such a process is extremely high - far higher than many scientists recognise. I illustrated this with a fictitious example here. 

Another problem is self-imposed publication bias: the researcher runs a whole set of analyses to test promising theories, but forgets about them as soon as they turn up a null result. With both of these processes in operation, data sharing becomes a poisoned chalice: instead of increasing scientific progress by encouraging novel analyses of existing data, it just means more unreliable dross is deposited in the literature. So how can we prevent this? 

In this Commentary paper, I noted several solutions. One is to require anyone accessing the data to submit a protocol which specifies the hypotheses and the analyses that will be used to test them. In effect this amounts to preregistration of secondary data analysis. This is the method used for some big epidemiological and medical databases. But it is cumbersome and also costly - you need the funding to support additional infrastructure for gatekeeping and registration. For many psychology projects, this is not going to be feasible. 

A simpler solution would be to split the data into two halves - those doing secondary data analysis only have access to part A, which allows them to do exploratory analyses, after which they can then see if any findings hold up in part B. Statistical power will be reduced by this approach, but with large datasets it may be high enough to detect effects of interest.  I wonder if it would be relatively easy to incorporate this option into Open Science Framework: i.e. someone who commits a preregistration of a secondary data analysis on the basis of exploratory analysis of half a dataset then receives a code that unlocks the second half of the data (the hold-out sample). A rough outline of how this might work is shown in Figure 1.

Figure 1: A possible flowchart for secondary data analysis on a platform such as OSF

An alternative that has been discussed by MacCoun and Perlmutter is blind analysis - "temporarily and judiciously removing data labels and altering data values to fight bias and error". The idea is that you can explore a dataset and run a planned analysis on it, but it won't be possible for the results to affect your analysis, because the data have been changed, so you won't know what is correct. A variant of this approach would be multiple datasets with shuffled data in all but one of them. The shuffling would be similar to what is done in permutation analysis - so there might be ten versions of the dataset deposited, with only one having the original unshuffled data. Those downloading the data would not know whether or not they had the correct version - only after they had decided on an analysis plan, would they be told which dataset it should be run on. 

I don't know if these methods would work, but I think they have potential for keeping people honest in secondary data analysis, while minimising bureaucracy and cost. On a platform such as Open Science Framework it is already possible to create a time-stamped preregistration of an analysis plan. I assume that within OSF there is already a log that indicates who has downloaded a dataset. So someone who wanted to do things right and just download one dataset (either a random half, or one of a set of shuffled datasets) would just need to have a mechanism that allowed them to gain access to the full, correct data after they had preregistered an analysis, similar to that outlined above.

These methods are not foolproof. Two researchers could collude - or one researcher could adopt multiple personas - so that they get to see the correct data as person X and then start a new process as person B, when they can preregister an analysis where results are already known. But my sense is that there are many honest researchers who would welcome this approach - precisely because it would keep them honest. Many of us enjoy exploring datasets, but it is all too easy to fool yourself into thinking that you've turned up something exciting when it is really just a fluke arising in the course of excessive data-mining. 

Like a lot of my blogposts, this is just a brain dump of an idea that is not fully thought through. I hope by sharing it, I will encourage people to come up with criticisms that I haven't thought of, or alternatives that might work better. Comments on the blog are moderated to prevent spam, but please do not be deterred - I will post any that are on topic. 

P.S. 5th July 2022 

Florian Naudet drew my attention to this v relevant paper: 

Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022). Protecting against researcher bias in secondary data analysis: Challenges and potential solutions. European Journal of Epidemiology, 37(1), 1–10. https://doi.org/10.1007/s10654-021-00839-0

Book Review. Fiona Fox: Beyond the Hype

If you're a scientist reading this, you may well think, as I used to, that running a Science Media Centre (SMC) would be a worthy but rather dull existence. Surely, it's just a case of getting scientists to explain things clearly in non-technical language to journalists. The fact that the SMC was created in part as a response to the damaging debacle of the MMR scandal might suggest that it would be a straightforward job of providing journalists with input from experts rather than mavericks, and helping them distinguish between the two. 

I now know it's not like that, after being on the Science Media Centre's panel of experts for many years, and having also served on their advisory committee for a few of them. The reality is described in this book by SMC's Director, Fiona Fox, and it's riveting stuff.

In part this is because no science story is simple. People will disagree about the quality of the science, the meaning of the results, and the practical implications. Topics such as climate change, chronic fatigue syndrome/ME and therapeutic cloning elicit highly charged responses from those who are affected by the science. More recently, we have found that when a pandemic descends upon the world, some of the bitterest disagreements are not between scientists and the media, but between well-respected, expert scientists. The idea that scientists can hand down tablets of stone inscribed with the truth to the media is a fiction that is clearly exposed in this book.

Essentially, the SMC might be seen as acting like a therapist in the midst of a seriously dysfunctional family where everyone misunderstands everyone else, and everyone wants different things out of life. On the one hand we have the scientists. They get frustrated because they feel they should be able to make exciting new discoveries, with the media then helping communicate these to the world. Instead, they complain that the media has two responses: either they're not interested in the science, or they want to sensationalise it. If you find a mild impact of grains on sexual behaviour in rats, you'll find it translated into the headline 'Cornflakes make you impotent'.

On the other hand, we have the media. They want a good story, but find that the scientists are reluctant to talk to them, or want total control of how the story is presented. In the worst case, scientists are prima donnas who want days or weeks to prepare for a media interview and will then shower the journalist with detailed information that is incomprehensible, irrelevant, or both. When the public desperately needs a clear, simple message, the scientists will refuse to deliver it, hedging every statement.

Fox has worked over the years to challenge these stereotypes: journalists do want a good story, but the serious science journalists want a true story, and are glad of the opportunity to pose questions directly to scientists. And many scientists do a fantastic job of explaining their subject matter to a non-specialist audience. In the varied chapters of the book, Fox is an irrepressible optimist, who keeps coming back to the importance of having scientists communicating directly with the media. Her optimism is not founded in ignorance: she knows exactly how messy and complicated science can be. But she persists in believing that more good is done by communicating what we know, warts and all, rather than pretending that uncertainties and disagreements do not exist.

The role of the SMC is, however, complicated by further factions. The dramatis personae includes two other groups. First, there are science press officers, who are appointed by institutions to help scientists promote their work, and then there are government officials and civil servants, who are concerned with policy implications of science.

In her penultimate chapter, Fox bemoans the fact that the traditional press officer - passionate about science and viewing themselves as "purveyors of truth and accuracy" - is a dying breed. There remain notable exceptions, but all too often science communication has become conflated with a public relations role: pushing a corporate message, defending the institutional reputation, and even using scientific discoveries as a marketing tool. Fox notes a 2014 survey of exaggerated science reports in the media that concluded: "Exaggeration in news is strongly associated with exaggeration in press releases." I had been one of those scientists who thought the media were mostly to blame for over-hyped science reporting, but this study showed that journalists are often recycling exaggerated accounts handed to them by those speaking for the scientists.

But the problems posed by scientists, journalists and press officers are trivial compared to the obstacles created by those involved in policy. They want to use science when convenient, but also want to exert control over which aspects of science policy gets talked about. Scientists working for government-funded organisations are often muzzled, with explicit instructions not to talk to the media. One can see that this cautious approach, attempting to control the message and keep things simple, puts many civil servants and government scientists on a collision course with Fox, whose view is: "Explaining preliminary and contradictory science is messy: that should not be seen as a failure of communications".

A refreshing aspect of Fox's account is that she does not brush aside the occasions when the SMC - or she personally - may have handled a situation badly. Of course, it's easy to point the finger of blame when something does go horribly wrong, and Fox has come under fire on many occasions. Rather than being defensive, she accepts that things might have been done differently, while at the same time explaining the logic of the decisions that were taken. This is in line with my memories of meetings of the SMC advisory committee, where there were frequent post mortems - "this is how we handled it; this is how it turned out; should we have done it differently?" - with frank discussions from the committee members. When you are working in contentious areas where things are bound to blow up every now and again, this is a sensible strategy that helps the organisation learn and develop. I'm glad that after 20 years, the ethos of the SMC is still very much on the side of open, transparent communication between scientists and the media.  

Fox, Fiona (2022) Beyond the Hype: The Inside Story of Science's Biggest Media Controversies. London: Elliott and Thompson Ltd.

Some quick thoughts on academic boycotts of Russia

 

 

In response to the dramatic developments in Ukraine, several instances of academic boycotts of Russia have been proposed or implemented.  These are a few of the initiatives I have heard about on Twitter:

 

European Federation of Academies of Sciences and Humanities (ALLEA) has suspended the Russian Academy of Sciences and the National Academy of Sciencesof Belarus

 

The British Psychological Society has supported a call from Ukrainian psychologists for the Russian Psychological Society to be suspended from the European Federation of Psychologists Associations (EFPA)

 

The Australian National University suspended all institutional ties with Russian institutions.

 

Yesterday, Chris Chambers, who is a journal editor, asked people's views on whether academic journals should be considering contributions from authors based in Russia.

 

Responses were pretty polarised. I have strong views against all these kinds of academic boycott, which have not really been changed by the numerous, well-argued points made by those responding to Chris, and so I thought I'd set out my position here.

 

I should start by saying that the disagreement here is not about the rights and wrongs of the war in Ukraine. I take as a basic premise that the invasion of Ukraine is an abominable atrocity, that, as its instigator, Vladimir Putin is guilty of war crimes, and that we all should do all we can to stop the war and ensure Putin is brought to justice.

 

At a national level, sanctions are imposed by governments to put pressure on Russia that, on the one hand, serves a symbolic function of demonstrating disapproval, and on the other hand serves a practical function of weakening the Russian economy, in the hope this might help bring the invasion to an end without starting World War 3.  They are not undertaken lightly, as it is recognised that many innocent people who play no part in the invasion will be adversely affected - both Russians who protest the invasion and people in the countries who are applying the sanctions, who may have to pay higher prices, or suffer shortages of goods.

 

Other organisations, notably sports organisations, have taken the line of moving major sporting events away from Russia. I fully support such moves, because they will have a financial impact on Russia, as well as denying Putin an opportunity to act as a host, which is a role that carries considerable prestige as well as opportunities for positive publicity. I also agree with banning Russian athletes from sporting events where they are explicitly representing their country - in effect, including these athletes creates the impression that Russia is a normal member of the sporting community. So these sanctions affecting sports have a very visible symbolic role as well as a potential financial one.  That makes the cost-benefit ratio of the actions seem worthwhile, despite the unfairness to individual athletes.

 

I see academic interactions differently in terms of the cost-benefit ratio.  I don't think there'd be any financial impact to Russia of Chris or other editors refusing to consider papers by Russian authors, and the symbolic impact is likely to be pretty small. I doubt most people would notice that such a sanction had occurred, and, even if Putin was aware of it, I can't see him getting particularly upset about it.  In saying this, I'm not arguing that academics are 'special' - just that the practical impact of adopting sanctions against them would be close to zero, except for the damage to the sanctioned individuals. Add to this the fact that academics are typically among the early casualties of dictators, who dislike them for their tendency to criticise, and I think the case for this kind of academic sanction is very weak.  

 

In the responses to Chris, there are many people, on both sides, putting forward 'what about' arguments. So, for instance, what about a small Russian trader, innocent of any war crimes, whose livelihood is blighted by sanctions that affect her business? For me the question is whether such sanctions have an effect that is worthwhile - in terms of its visibility and economic impact -  given that all sanctions will trap the deserving as well as the undeserving in their net?  So this needs a response on a case-by-case basis, rather than treating it as a generic principle to be adopted in all sectors.

 

Another 'what about' issue that is raised concerns countries other than Russia. Should we start to sanction academics from any government that behaves inhumanely? It would be quite a long list, and would be difficult to draw the line.  It's worth noting that on the same principle, we could well find ourselves sanctioned by academics from other countries who regard the UK's colonial past and/or escapades in the Middle East worthy of censure.  The only reason we don't have to worry about that is because we happen to have considerable power over the machinery of academic publishing.

 

So my answer here again is that in general, academics should only consider sanctions as an extreme resort, given that they are a blunt instrument that can have serious unintended consequences. The economic sanctions currently imposed on Russia by governments are unusually effective because they have a big effect on many sectors, and we are seeing a co-ordinated activity of many nations, as well as nations prepared to take a financial hit themselves.  That's not the case for academic sanctions.

 

There are many other ways we can express our support for Ukraine. Moves are already afoot among learned societies to help Ukrainian academics who have had to leave the country, and also Russian academics who have put themselves in harm's way by speaking out against the invasion. Personally, I think the most useful thing I can do is to donate money to that cause as well as to other causes that will strengthen Ukraine's position in the conflict, rather than calling for symbolic gestures.

 

I'm prepared to listen to counter-arguments. It's clear this is one of those issues that is pretty divisive among people of good will who are usually on the same side of political debate.  To help cope with a tsunami of spam, comments on this blog are moderated, but I will monitor them and publish those that contribute (respectfully) to this discussion.

 

 

Why I am not engaging with the Reading Wars

I’m not sure whether to feel flattered or irritated by Jeff Bowers’ latest blogpost, where he expresses his disappointment in my failure to support his critique of research on phonics intervention.

The short explanation, as I already conveyed to Jeff, is that I'm busy with other stuff. I’m aware that there’s a pretty intemperate debate going on at present about the value of phonics in teaching children to read, with various people writing lengthy critiques of one another’s positions. So clearly it’s a complex and murky situation. The last thing this needs is for someone else, me, who hasn’t looked in depth at the relevant research, to weigh in with an opinion.

As background: I am a (soon-to-be retired) researcher who has never worked on reading intervention, though I do work on topics tangentially related to this. I'm currently finishing off a big project on cerebral lateralization for language. I’m also involved in a couple of other projects unrelated to reading. I am a  hands-on researcher, so am heavily involved in designing, analysing and writing up research. I currently work a 4 day week, and these three projects, plus regular reviewing, giving talks, etc, take up all of my time.

One of the fun parts of my job is that I am my own boss and, as far as time allows, I can pursue odd topics that take my fancy. I sometimes put my head above the parapet and engage in, or even initiate, academic disputes about research. These consume much time and emotional energy, so I choose carefully which battles I want to get involved in. In particular, I think hard about what can be gained - I do get passionate when something is wrong, especially where someone is behaving badly and deliberately distorting science for some nefarious ends. In such cases I will get engaged to try and put things right.

The reading wars don't look like that to me. They actually look rather like the equally heated debates going on among epidemiologists and public health experts around Covid – another topic on which I don’t’ speak out because I lack expertise. The most I’m prepared to offer is a general suggestion about perspective that might be helpful in achieving some truth and reconciliation.

In both Covid and reading wars there are arguments about research evidence and arguments about policy. Typically, there isn't really good research on which to base important decisions, but we don’t have the luxury of waiting for such studies to be done. Doing nothing is in itself a choice, for which, in the case of an epidemic, we know the consequences are likely to be bad. So it’s all about making the best decisions we can, given the patchy evidence that we have – a point nicely made by Trisha Greenhalgh in the context of COVID-19. 

The current debates have highlighted the shortage of well-designed studies to show which teaching method is most effective for which children at which age. As with medical interventions, studies also need to evaluate potential costs and harms – something which was neglected in the past, but which is relevant given that some of the disagreements are about whether particular approaches can be counterproductive.

But, as with Covid, policy decisions have to be pragmatic. This was the remit of the 2006 Rose Report, which attempted to synthesise evidence and expert opinion on reading intervention and came out recommending use of synthetic phonics. Overturning the status quo to radically change how reading is taught would only be reasonable if there was some other intervention approach for which there was better evidence of effectiveness – so much so that the costs of making a change would be justified. Maybe there is, but it hasn’t come up on my radar. 

My apologies to all of those engaged in current discussions whose contributions I have ignored. I hope this blogpost will suffice to explain why.

Boris Johnson leads the world ....in the number of false facts he can squeeze into a session of PMQs

 

The first weeks of 2022 have had quite enough drama to last us the whole year. Prime Minister’s Questions (PMQs) on Wednesdays has developed a must-watch quality that was previously reserved for the final of the Great British Bake-off. I find myself studying the oratorical approach adopted by Boris Johnson. Previously, I analysed his strategies for (usually not) answering questions in a session of a Select Committee. In PMQs the style is different. It's a gladiatorial context where the goal is to earn howls of approval for trouncing your opponent.

Last week we had the unusual sight of a remorseful Boris Johnson, trying to convince the nation that he was truly, truly sorry for allowing parties to take place on his watch, during a period when we had extremely severe lockdown rules that were preventing people seeing dying relatives or attending funerals. It was one of those apologies where you sensed he was sorry at having been found out, rather than for having done something wrong. But he clearly hoped this would be enough to settle the matter so we could put it behind us – at least until a whitewashing report is published. He had not reckoned with the Machiavellian cunning of his ex-advisor Dominic Cummings, who is drip-feeding damaging information to the media. This started a new story at the weekend, after Cummings contradicted Johnson’s denial that he knew about a party. So I wondered if we would have Remorse, the Sequel at yesterday’s PMQs. 

Johnson, however, adopted the very strategy that he had previously learned from Cummings – brazen it out. So we had the bizarre sight of a Prime Minister who has lost confidence of his own party because they no longer trust what he says, doubling down and telling yet more lies to convince everyone that this is an Absolutely Splendid Government, and he is a the best chap to lead them. (My thanks to Hansard for this transcript of the debate). 

Here is a selection of some of the more questionable claims (there is more, but I have a day job!):

 “We are building 40 new hospitals.” Cf this analysis by FullFact.  One might add that the biggest problem faced by the NHS is lack of staff, rather than buildings.

“We have also been cutting crime by 10% and putting 11,000 more police officers out on the streets” – I guess this is an improvement on previous claims of 20,000 more police officers, but it’s been debunked as misleading, given that this is reinstating police that were previously cut.  

“We now have the fastest-growing economy in the G7 and GDP is now back up above pre-pandemic levels.” Here, Google just led me to a Government report from the previous day which stated: “In Q3 2021, UK GDP grew by 1.1% compared with the previous quarter (Q2 2021), while GDP rose by 2.2% in the Eurozone and 0.6% in the US. Compared to the pre-pandemic peak, UK GDP in Q3 2021 was 1.5% lower, compared with 1.1% lower in Germany and 0.3% lower in the Eurozone. In the US GDP was 1.4% higher. 

“Thanks to the efforts of people across Whitehall, this country is now capable of producing 80% of our own PPE.” I couldn’t find recent data that might be the basis for this claim, but talking up their record on PPE as a source of pride seems a rash move by a Government whose shambolic handling of this issue was documented in a report by the Public Accounts Committee in Feb 2021:  “...its failure to be transparent about decisions, publish contracts in a timely manner or maintain proper records of key decisions left it open to accusations of poor value for money, conflicts of interest and preferential treatment of some suppliers, and undermines public trust in government procurement and the use of taxpayers’ money. Between February and July 2020, the Department of Health and Social Care spent over 12 billion on 32 billion items of PPE. We are concerned that the Department has so far identified items worth hundreds of millions of pounds which are unusable for their intended purpose, putting the efficient use of taxpayers’ money at further risk.”

As Daisy Cooper noted on Twitter, Johnson also repeated the claim that the vaccine rollout would not have been possible except for Brexit – a claim that has been debunked more than once 

Then there was this extraordinary claim: “I will tell you what this Government have been doing to look after the people of this country throughout this pandemic and beyond. We have been cutting the cost of living and helping them with the living wage. We have been cutting taxes for people on low pay. We have been increasing payments for people suffering the costs of fuel- “ People don’t need to consult Google to challenge this rosy vision of life in the UK. We can simply take use of foodbanks as an indicator of how many people are living in serious poverty. We had barely heard of foodbanks before this Government came into power – now more and more people depend on them just to exist. Not only is the cost of living going up, Jack Monroe noted that the way it is computed underestimates the impact on the poorest 

No, Prime Minister, you can’t talk your way out of this one. People are worried about what the future holds, but they are not going to reject the evidence of their eyes and ears

Response to Philip Ball's critique of scientific advisors

This week, Philip Ball wrote a piece in the New Statesman headlined: “Muted and deferential, the UK’s scientists have failed the pandemic test”. I regard Ball as a thoughtful and knowledgeable science writer, and I usually agree with him, but this piece made me uneasy. I expressed disagreement on Twitter and Ball asked for my reasons, so here is a brief response.

I should start by saying that, while I try to keep myself informed about the science of coronavirus, I have no background in infectious diseases, epidemiology or public health. I follow a lot of genuine experts on Twitter, and what has been striking is the lack of agreement between them on many policy issues. Initially there was uncertainty about the nature of the disease – it seems WHO held out for a long time against the view that the virus was airborne, a view that I think is now pretty universally accepted. This led to early mis-steps with a focus on handwashing and surface-cleaning rather than masks and indoor air quality. On other matters, such as vaccination of children, disagreements still remain. And when we move to consider potential non-pharmaceutical public health measures, there is a wide range of views – do lock-downs do more harm than good? Should we restrict foreign travel? How should a test-and-trace system be organised? Although people who know what they are talking about sometimes take strong and passionate positions on these issues, there is variation from country to country in how they are addressed, and my impression is the outcomes do not always support what we might have expected.

I’m not saying it doesn't matter what we do – on the contrary, I think it matters enormously, as the health of the world depends on getting it right. But it’s much harder to know what is right when the experts themselves are uncertain. So scientists will lay out scenarios A, B and C, and estimate a range of possible outcomes for each one, rather than agreeing that A is the only valid course of action. I suspect everyone would be happier if it was very clear what we had to do – even if it was painful – but instead both scientists and politicians have to weigh up pros and cons, knowing that they won’t be forgiven for a wrong decision. On top of everything else there is time pressure: if we wait for further research, it may be too late. 

The decisions that governments have to make are complex and high-stakes, and the fact that we have a Prime Minister and front bench who were selected solely because of their commitment to Brexit does not inspire confidence that they will select the right policies. But should scientific advisors be doing more, as Philip Ball suggests, to challenge government?

 It seems to me there are four different scenarios, each of which requires a different response.

1. Bad science/fake news

I agree that if a government minister were to state openly something that was scientific misinformation, then it is the job of senior advisors to challenge them publicly. I was among the first to criticise Deborah Birx early in 2020 when she said of Trump “He’s been so attentive to the scientific literature and the details and the data…. I think his ability to analyze and integrate data that comes out of his long history in business has really been a real benefit during these discussions about medical issues.”  See also here. There were also times when I thought Anthony Fauci should have just stood up and said ‘No, No, No!’ rather than sit in the same room while Trump spouted dangerous nonsense. However, bad though the situation in the UK is, I don’t think we have had anything like that situation. Certainly there are politicians and others who have tried to discredit scientific advisors as “scare-mongering”, including some who see a pandemic as a nice opportunity for a bit of disaster capitalism. But I haven’t seen government ministers making scientific claims about coronavirus that are so wildly wrong that no serious scientist could support them.

2. Policies that prove misjudged in hindsight 

Most people would agree that in the early days of the pandemic, the wrong policies were adopted, with an emphasis on handwashing and cleaning surfaces, and a severe lockdown imposed too late. Jeremy Farrar, who was on SAGE at the time, described these events in his book with Ajana Ahuja, Spike, and it is clear that he, and other SAGE members, regret that they did not consider more options, or argue for more rapid action. To that extent, the criticism may be valid, but my impression is that the advisors had significant input into the policies, which at the time they thought was the least bad course of action. There were other scientists who spoke out strongly in criticism of lock-downs, but it’s not clear that this helped the situation. I agree, it would be helpful for everyone who was involved in those decisions to reflect now about what they got wrong, but it would have been hard to do so at the time - not because they were "muted and deferential" but because they had significant input into the policies.

3. Ongoing policies 

We became accustomed during the past two years of seeing government advisers alongside the PM or another government minister at press briefings when a policy is announced. Ball says they should “openly criticise bad policy” – this would be fine if everyone could agree what good and bad policy were. The policy we get will typically be the least bad option, formed by trying to integrate scientific and political considerations. Nobody will be entirely happy with it – the question is not whether it has flaws, but whether there are better options that everyone could agree on. Once it is decided, it is the job of the scientific advisers to ensure it is clearly communicated and explained.

For instance, Ball says Chris Whitty was being timid in December 2021 when he suggested that we “prioritise the social interactions that really matter” at a press briefing when Johnson said “we’re not cancelling people’s parties”. This seems to me to be an unfair charge: the policy is at this point agreed. Is Whitty supposed to stand up at the podium and say “I think we should cancel parties.”?  I heard the December briefing and thought Whitty’s contribution provided useful clarification, in emphasising the importance of not dropping one’s guard, and restricting contact to a small circle. 

4. Cases where policies are not followed 

Ball also notes cases where rules or guidance were not followed: Dominic Cummings’ trip to Barnard Castle, MPs not wearing masks, or drinks at no. 10. He feels scientific advisors are “appeasing” government when they refuse to comment on such matters. I disagree. Their role is to help ensure that policies are based on the best scientific evidence. Where there is failure to adhere to policies, it’s the job of the police, politicians and the media to hold people to account. 

Government advisers and Independent SAGE

Concerns about lack of transparency around SAGE led to the formation of Independent SAGE. I regard this as a very useful development, that serves much of the function that Ball sees as missing in regular SAGE. They provide a wealth of information about different aspects of pandemic management, and can express their views robustly, and suggest alternatives to current policies. Perhaps one way forward would be to formalise the need for a body separate from scientific advisors that is quite independent from government.  But we do still need a group of scientists who work directly with government to provide scientific input into rapid decisions about what is needed now. 

I should make clear that I totally agree with Ball’s criticisms of the government – my sense is that most of his piece is aimed at highlighting how dishonest and incompetent they have been in responding to coronavirus. But it strikes me as unfair to extend the criticism to Whitty, Vallance and members of SAGE who have tried to steer us through the pandemic as best they can, even though that means working with this dysfunctional government. My view is that we would be in a far worse position without their input.

Linking responsibility for climate refugees to emissions

 

I followed the COP summit in Glasgow with interest, particularly via Channel 4 News, which is good at reporting minority voices. There were incredibly powerful speeches by people whose homes are going to be uninhabitable if we exceed 1.5 degrees warming - and indeed by some for whom extreme weather has already caused famine. For examples see speeches by Elizabeth Wathuti, youth climate activist from Kenya, Mia Mottley, Prime Minister of Barbados, and the interview with Shauna Aminath, minister of environment for the Maldives, and Seve Paeniu, Tuvalu’s minister for finance and climate change. 

The plan at COP seemed to focus on decelerating warming in order to, as far as possible, prevent this happening. But it's pretty clear that, even in the unlikely event that we hold warming to 1.5 degrees, there will be enormous numbers of people who will become climate refugees. People don't want to be forced out of their homes. It makes far more sense to adopt strategies that will avoid that than to plan for mass migrations. But sadly, human beings have a habit of not believing the worse until it is upon them. So I think we're going to have mass migrations and I'm worried nobody seems to be planning for that.

The reaction of many wealthy countries to refugees has been to keep them out at all costs and hope they go away. People fleeing their homelands are regarded like toxic waste: a problem to be exported. That pretty much sums up the attitude of the current UK government, who have at various times suggested that asylum seekers be sent to Ascension Island, confined on ship hulks  and, most recently, and to the surprise of the Albanians, sent to Albania. But this problem is not going to go away. We will see increasing numbers of people roaming around desperately trying to find somewhere to live. Quite apart from the humanitarian disaster for the refugees themselves, this will create social unrest and conflict for those in wealthy countries unless it is planned for. Germany has shown that it is possible to integrate large numbers of refugees, and that they can contribute positively to a society if given support.

So, as well as anticipating the impacts of global heating on the physical environment, we need to be planning for mass migrations of people who have lost their homeland. We will need a global strategy, where everyone plays their part. And it should be those countries who are are doing least to curb global emissions who should step up to play their part in taking in refugees. 

So I suggest that the next COP summit should aim for a more direct linkage between emissions generated and responsibility for refugees: the more you emit, the more you have to commit to giving citizenship to those who have been rendered homeless by climate change. Note I am saying citizenship, not temporary accommodation in a tent in the desert (or a ship hulk for that matter).

Heads of state who genuinely don't believe that things are that bad, and that it's okay to continue to generate emissions, will find this an easy commitment to make, as it would only kick in when a homeland has become uninhabitable; it should be possible to devise a formal criterion for this: e.g. the place is under water, or experiencing temperatures of 40 degrees or more over a sustained period.

Take, for example, Australia, whose PM, Scott Morrison, tries to blag his way out of a crisis and thinks it's fine to continue mining and exporting fossil fuels. It's ironic that Australia's solution to refugees has previously been to send them off to be imprisoned on a Pacific Island. That sort of solution may no longer be available in future, as Nauru succumbs to rising sea levels.

Given the major role that Australia has played in causing global warming, wouldn't it make sense if at the next COP it was required to step up to its responsibilities and undertake to award citizenship to Pacific Islanders whose homes will be submerged in a few decades? The greater the emissions, the higher the quota of people. And who knows, if it were put to them like that, they might even start taking the phasing out of fossil fuels more seriously?

Universities vs Elsevier: who has the upper hand?

 

The academic publisher Elsevier is currently negotiating a deal with UK universities. In Oxford, as in other universities, there have been extensive discussions about the proposed deal; the goals are to reduce costs to sustainable levels and to provide full and immediate open access to UK research. I have a nasty feeling that we could end up in the situation of those at the COP summit: with a deal that where the publishers feel they are giving away a huge amount, while the consumers are still unsatisfied. 

In the print era, publishers already had a large profit margin.  I was a journal editor when electronic publishing first came in, and I remember discussions with the publisher - who was clearly very nervous about how this might damage their bottom line. With some clever business practices, such as bundling, they managed to keep going, making more rather than less money.

Other new developments - the advance of metrics and requirements for open access publishing - also pose challenges to them, but their response, like any savvy business, is to take control of these new developments and find a way to profit from them as well.

So should we go along with this? The discussions have largely centred around money, and this is a real concern. Publishers charge massively inflated subscription charges, taking as profit money that our libraries could otherwise put to good use. But the problems go way beyond that, including blocking open access to authors' own work, and giving poor quality and even fraudulent material a veneer of respectability. The publisher currently has considerable power over what gets published but takes little responsibility when things go wrong.

But we also have power, and I think it's time we started to wield it. I have two proposals: one radical and the other less so, both of which go beyond what is currently being considered in the JISC discussions.

The radical option: move publishing in-house  

In an ideal world, we would not have any deals with for-profit publishers. Universities would take control of academic publishing. A model which I have found works well is the Wellcome Trust journal Wellcome Open Research (WOR). I served on its Editorial Advisory Board in the first years of its operation and I have published eight papers in WOR. 

Quality control is maintained by requiring that work published in WOR has been Wellcome-Trust funded. The platform is paid for by the Wellcome Trust, and all material is Open Access, with no charge to authors.

Universities could use this as a model for developing their own platforms for publishing work of their researchers. As with Wellcome Trust, researchers would be encouraged rather than required to publish there. Setting up and maintaining the platform would cost money, but this might be covered by savings from stepping back from deals with big publishers. This is not a perfect solution, of course; we would still want access to past material published by traditional publishers. But I feel this option should be given serious consideration.

The less radical option. An agreement with Elsevier that reflects what we want and need 

If we are to retain a working relationship with Elsevier, then we need to ensure that we are getting value for money, and that they are delivering what we want.

One way in which they fail to do this is through the copyright agreements that they require authors to sign. I agree with Sally Rumsey that any publisher that does not allow an author to retain rights to their own work is problematic. Furthermore, there is concern that even if Elsevier were to say they agreed to author rights retention, they would continue to adopt practices that would mislead authors into signing away their rights.

Another issue is how publishers deal with matters related to misbehaviour by editors and authors. We have a number of serious problems with the academic publishing system that have been developing for years, and have been thrown into stark relief by the pandemic. Publications in journals are a kind of academic currency, with potentially huge rewards in terms of promotion, pay and tenure of academic staff. Unfortunately, this means that some people will go to great lengths to procure publications, and this can include fraudulent means. A reputable publisher should recognise and indeed anticipate such problems and take steps to deal with them. Elsevier has been notably poor in doing so. I will give a handful of examples.

Clearly erroneous/fraudulent work remains in the literature In recent years, a massive problem has been discovered with so-called paper mills - a form of industrialised cheating that involves generation of literally thousands of fake papers.  This phenomenon was discussed at a meeting this summer, the Computational Research Integrity Conference (CRI-Conf), which brought together some of the scientists who uncovered paper mills and representatives of publishers (including Elsevier) and university integrity officers. A repeated complaint of the 'data sleuths' (scientists who uncovered the paper mills) was that when the problems were drawn to the attention of the editors or publishers, typically nothing was done.  Some of these papers are highly cited and have medical implications - see, for instance work by Jennifer Byrne and Cyril Labbé on cancer biology, or by Elisabeth Bik on image manipulations.

To its credit, Elsevier was recently cited as acting promptly when a cache of nonsense papers in special issues was uncovered, but one still has to ask how these papers, some of which were complete gibberish, originally got through the supposedly rigorous peer-review process. For another recent example, see here.

Part of the problem seems to be that there is very little oversight of editors.  I first became aware of Elsevier's lax attitude to research integrity when I discovered a ring of editors. There was overwhelming evidence that a set of editors were publishing one another's papers in their journals, with publication lags so brief as to indicate there had not been peer review. Emails to Elsevier received no reply, and so I started reporting the events on my blog. This created such publicity that Elsevier then posted a reply, but initially they denied that any papers had been published without peer review. Eventually, as the evidence mounted up, they were forced to acknowledge the problem, and claimed to be investigating it, but at no point did they communicate with me, and their investigation went on for years and has never led to any consequences. One of the offending editors, Johnny Matson, quietly disappeared without comment and is now editing another journal for another publisher. The unreviewed papers, many of which are concerned with intervention for children with developmental disorders, remain in the literature. 

More recently I did a bibliometric analysis of editors of psychology journals who publish prolifically in their own journals, and broke down the results by publisher. Elsevier had the highest number of editors-in-chief in this category,

A related case in another field is that of Didier Raoult, who was senior author on a paper advocating hydroxychloroquine as a treatment for Covid-19 that appeared in 2020 in an Elsevier journal for which he was Editor-in-Chief. It has subsequently been shown to be riddled with methodological errors, but, although Elsevier claimed to be investigating it in April 2020, it has not been retracted.

At the CRI-Conf publishers said that they were doing their best, but the problem of fake and fraudulent practices was so huge they could not be expected to tackle all of it. Yet we know publishers make enormous profits, so why aren't they putting more effort into checking the integrity of the work they publish - and dealing promptly with the problems that are drawn to their attention? Elsevier is not the only large publisher to be affected, but if it wants to be seen as a leader in the field, it should recognise that integrity of published material is of supreme importance - especially in clinical fields where people's health and wellbeing can be affected (as in the cases of Raoult and Matson noted above).

If UK Universities do strike a deal with Elsevier, then they should take the opportunity to include in any future contract conditions requirements that: 

(a) Elsevier allows authors the retention of the rights to their publications and  

(b) it provides annually an open report on the number of integrity issues that have been raised about its publications and how these have been dealt with. 

This would go some way toward ensuring that we get value for money, and are not simply adding to the profits of a company whose underlying ethos is not aligned with promoting scientific quality.

Universities have historically been extremely meek in accepting the terms and conditions imposed by big publishers, who have in turn treated academics as a cash cow. We on the one hand produce for free the product that they sell, and then pay to publish and read it. They cannot survive without the research created in Universities: we should start recognising that we need not bow down to their demands.

DISCLAIMER: I am writing in a personal capacity and not as a representative of the University of Oxford.

14/11/21

P.S. Some further examples of integrity problems with Elsevier have been pointed out to me on Twitter:

Geoff Frampton (@Geoff_Frampton) noted 'Elsevier had the largest market share among the publishers of retracted Covid-19 articles in our study' 

Patricia Murray (@PMurray_65) tweeted: Elsevier have failed to retract this 2008 Lancet paper that is known to be fraudulent and poses a risk to patients: https://thelancet.com/journals/lancet/article/PIIS0140-6736(08)61598-6/references… The paper stated the so-called 'bioengineered' airway had "a normal appearance and mechanical properties at 4 months." 

It is particularly shocking that the publisher is still making this article available (at a cost of $31) given that the author had a prison sentence for faked research, and conducted unethical experiments on seriously ill patients: see this report. 

PPS - it gets worse. There have been strenuous attempts to get the journal to retract the paper, to no avail. This has made the national news in the UK.

 

Comments on this blog are welcome, but may only appear after a delay as they are moderated to avoid spam.