When there are no consequences for misconduct: Parallels between politics and science

 

Gustave Doré: Illustration for Paradise Lost

(Updated 17 Dec 2022) 

As children, we grow up with stories of the battle between good and evil, but good ultimately triumphs. In adulthood, we know things can be more complicated: bad people can get into positions of power and make everyone suffer.  And yet, we tell ourselves, we have a strong legal framework, there are checks and balances, and a political system aspires to be free and fair.

 

During the last decade, I started for the first time to have serious doubts about those assumptions. In both the UK and the US, the same pattern is seen repeatedly: the media report on a scandal involving the government or a public figure, there is a brief period of public outrage, but then things continue as before.

 

In the UK we have become accustomed to politicians lying to Parliament and failing to correct the record, to bullying by senior politicians, and to safety regulations being ignored.  The current scandal is a case of disaster capitalism where government cronies made vast fortunes from the Covid pandemic by gaining contracts for personal protective equipment – which was not only provided at inflated prices, but then could not be used as it was substandard.

 

These are all shocking stories, but even more shocking is the lack of any serious consequences for those who are guilty. In the past, politicians would have resigned for minor peccadilloes, with pressure from the Prime Minister if need be. During Boris Johnson’s premiership, however, the Prime Minister was part of the problem. 

 

During the Trump presidency in the US, Sarah Kendzior wrote about “saviour syndrome”  - the belief people had that someone would come along and put things right. As she noted: “Mr. Trump has openly committed crimes and even confessed to crimes: What is at stake is whether anyone would hold him accountable.” And, sadly, the answer has been no.

 

No consequences for scientific fraud

So what has this got to do with science?  Well, I get the same sinking feeling that there is a major problem, everyone can see there's a problem, but nobody is going to rescue us. Researchers who engage in obvious malpractice repeatedly get away with no consequences.  This has been a recurring theme from those who have exposed academic papermills (Byrne et al., 2021) and/or reported manipulation of figures in journal articles (Bik et al., 2016).  For instance, when Bik was interviewed by Nature, she noted that 60-70% of the 800 papers she had reported to journals had not been dealt with within 5 years. That matches my more limited experience; if one points out academic malpractice to publishers or institutions, there is often no reply. Those who do reply typically say they will investigate, but then you hear no more.

 

At a recent symposium on Research Integrity at Liverpool Medical Institution*, David Sanders (Purdue University) told of repeated experiences of being given the brush-off by journals and institutions when reporting suspect papers. For instance, he reported an article that had simply recycled a table from a previous paper on a different topic. The response was “We will look into it”. “What”, said David incredulously, “is there to look into?”. This is the concern – that there can be blatant evidence of malpractice within a paper, yet the complainant is ignored. In this case, nothing happened. There are honorable exceptions, but it seems shocking that serious and obvious errors in work are not dealt with in a prompt and professional manner.

 

At the same seminar, there was a searing presentation by Peter Wilmshurst, whose experiences of exposing medical fraud by powerful individuals and organisations have led him to be the subject of numerous libel complaints.  Here are a few details of two of the cases he presented:

 

Paolo Macchiarini:  Convicted in 2022 of causing bodily harm with an experimental transplant of a synthetic windpipe that he performed between 2011-2012.  Wilmshurst noted that the descriptions of the experimental surgery in journals were incorrect. For a summary see this BMJ article.   A 2008 paper by Macchiarini and colleagues is still published in the Lancet, despite demands for it to be retracted. 

 

Don Poldermans: An eminent cardiologist who conducted a series of studies on perioperative betablockers, leading them to be recommended in guidelines from the European Society of Cardiology,  whose task force he chaired. A meta-analysis challenged that conclusion, showing mortality increased; an investigation found that work by Poldermans had serious integrity problems, and he was fired. Nevertheless, the papers have not been retracted. Wilmshurst estimated that thousands of deaths would have resulted from physicians following the guidelines recommending betablockers.

 

The week before the Liverpool meeting, there was a session on Correcting the Record at AIMOS2022.  The four speakers, John Loadsman (anaesthesiology), Ben Mol (Obstetrics and Gynecology), Lisa Parker (Oncology) and Jana Christopher (image integrity) covered the topic from a range of different angles, but in every single talk, the message came through loud and clear: it’s not enough to flag up cases of fraud – you have to then get someone to act on them, and that is far more difficult than it should be.

 

And then on the same day as the Liverpool meeting, Le Monde ran a piece about a researcher whose body of work contained numerous problems: the same graphs were used across different articles that purported to show different experiments, and other figures had signs of manipulation.  There was an investigation by the institution and by the funder, Centre National de la Recherche Scientifique (CNRS), which concluded that there had been several breaches of scientific integrity. However, it seems that the recommendation was simply that the papers should be “corrected”.

 

Why is scientific fraud not taken seriously?

There are several factors that conspire to get scientific fraud brushed under the carpet.

1.     Accusations of fraud may be unfounded. In science, as in politics, there may be individuals or organisations who target people unfairly – either for personal reasons, or because they don’t like their message. Furthermore, everyone makes mistakes and it would be dangerous to vilify researchers for honest errors. So it is vital to do due diligence and establish the facts. In practice, however, this typically means giving the accused the benefit of the doubt, even when the evidence of misconduct is strong.  While it is not always easy to demonstrate intent, there are many cases, such as those noted above, where a pattern of repeated transgressions is evident in published papers – and yet nothing is done.  

2.     Conflict of interest. Institutions may be reluctant to accept that someone is fraudulent if that person occupies a high-ranking role in the organisation, especially if they bring in grant income. Worries about reputational risk also create conflict of interest. The Printeger project is a set of case studies of individual research misconduct cases, which illustrates just how inconsistently these are handled in different countries, especially with regard to transparency vs confidentiality of process. It concluded “The reflex of research organisations to immediately contain and preferably minimise misconduct
cases is remarkable”.

3.     Passing the buck. Publishers may be reluctant to retract papers unless there is an institutional finding of misconduct, even if there is clear evidence that the published work is wrong. I discussed this here.  My view is that leaving flawed research in the public record is analogous to a store selling poisoned cookies to customers – you have a responsibility to correct the record as soon as possible when the evidence is clear to avoid harm to consumers. Funders might be expected to also play a role in correcting the record when research they have funded is shown to be flawed. Where public money is concerned, funders surely have a moral responsibility to ensure it is not wasted on fraudulent or sloppy research. Yet in her introduction to the Liverpool seminar, Patricia Murray noted that the new UK Committee on Research Integrity (CORI) does not regard investigation of research misconduct as within its purview.  

4.     Concerns about litigation. Organisations often have concerns that they will be sued if they make investigations of misconduct public, even if they are confident that misconduct occurred. These concerns are justified, as can be seen from the lawsuits that most of the sleuths who spoke at AIMOS and Liverpool have been subjected to.  My impression is that, provided there is clear evidence of misconduct, the fraudsters typically lose libel actions, but I’d be interested in more information on that point.

 

 

Consequences when misconduct goes unpunished

 

The lack of consequences for misconduct has many corrosive impacts on society. 

 

1.     Political and scientific institutions can only operate properly if there is trust. If lack of integrity is seen to be rewarded, this erodes public confidence. 

 

2.     People depend on us getting things right. We are confronting major challenges to health and to our environment. If we can’t trust researchers to be honest, then we all suffer as scientific progress stalls.  Over-hyped findings that make it into the literature can lead subsequent generations of researchers to waste time pursuing false leads.  Ultimately, people are harmed if we don’t fix fraud.

 

3.     Misconduct leads to waste of resources. It is depressing to think of all the research that could have been supported by the funds that have been spent on fraudulent studies.

 

4.     People engage in misconduct because in a competitive system, it brings them personal benefits, in terms of prestige, tenure, power and salary. If the fraudsters are not tackled, they end up in positions of power, where they will perpetuate a corrupt system; it is not in their interests to promote those who might challenge them.

 

5.     The new generation entering the profession will become cynical if they see that one needs to behave corruptly in order to succeed. They are left with the stark choice of joining in the corruption or leaving the field.

 

 

What can be done?

 

There’s no single solution, but I think there are several actions that are needed to help clean up the mess.

 

1.     Appreciate the scale of the problem.

When fraud is talked about in scientific circles, you typically get the response that “fraud is rare” and “science is self-correcting”.  A hole has been blown in the first assumption by the emergence of industrial-scale fraud in the form of academic paper-mills . The large publishers are now worried enough about this to be taking concerted action to detect papermill activity, and some of them have engaged in mass retractions of fraudulent work (see, e.g. the case of IEEE retractions here). Yet, I have documented on PubPeer numerous new papermill articles in Hindawi special issues appearing since September of this year, when the publisher announced it would be engaging in retraction of 500 papers. It’s as if the publisher is trying to clean up with a mop while a fire-hose is spewing out fraudulent content.  This kind of fraud is different from that reported by Wilmshurst, but it illustrates just how slow the business of correcting the scientific record can be – even when the evidence for fraud is unambiguous. 

Publishers trying to mop up papermill outputs

 

Yes, self-correction will ultimately happen in science, when people find they cannot replicate the flawed research on which they try to build. But the time-scale for such self-correction is often far longer than it needs to be.  We have to understand just how much waste of time and money is caused by reliance on a passive, natural evolution of self-correction, rather than a more proactive system to root out fraud.  

 

2.     Full transparency

There’s been a fair bit of debate about open data, and now it is recognised that we also need open code (scripts to generate figures etc.) to properly evaluate results. I would go further, though, and say we also need open peer review. This need not mean that the peer reviewer is identified, but just that their report is available for others to read. I have found open peer reviews very useful in identifying papermill products.

 

3.     Develop shared standards

Organisations such as the Committee on Publication Ethics (COPE) give recommendations for editors about how to respond when an accusation of misconduct occurs.  Although this looks like a start in specifying standards to which reputable journals should adhere, several speakers at the AIMOS meeting suggested that COPE guidelines were not suited for dealing with papermills and could actually delay and obfuscate investigations. Furthermore, COPE has no regulatory power and publishers are under no obligation to follow the guidelines (even if they state they will do so).

 

4.     National bodies for promoting scientific integrity

The Printeger project (cited above) noted that “A typical reaction of a research organisation facing unfamiliar research misconduct without appropriate procedures is to set up ad hoc investigative committees, usually consisting of in-house senior researchers…. Generally, this does not go well.”

In response to some high-profile cases that did not go well, some countries have set up national bodies for promoting scientific integrity. These are growing in number, but those who report cases to them often complain that they are not much help when fraud is discovered – sometimes this is because they lack the funding to defend a legal challenge. But, as with shared standards, this is at least a start, and they may help gather data on the scale and nature of the problem.  

 

5.     Transparent discussion of breaches of research integrity

Perhaps the most effective way of persuading institutions, publishers and funders to act is by publicising when they have failed to respond adequately to complaints.  David Sanders described a case where journals and institutions took no action despite multiple examples of image manipulation and plagiarism from one lab.  He only got a response when the case was featured in the New York Times.

Nevertheless, as the Printeger project noted, relying on the media to highlight fraud is far from ideal – there can a tendency to sensationalise and simplify the story, with potential for disproportionate damage to both accused and whistleblowers. If we had trustworthy and official channels to report suspected research misconduct, then whistleblowers would be less likely to seek publicity through other means.

 

6.     Protect whistleblowers

In her introduction to the Liverpool Research Integrity seminar, Patricia Murray noted the lack of consistency in institutional guidelines on research integrity. In some cases, the approach to whistleblowers seemed hostile, with the guidelines emphasising that they would be guilty of misconduct if they were found to have made frivolous, vexatious and/or malicious allegations. This, of course, is fair enough, but it needs to be countered by recommendations that allow for whistleblowers who are none of these things, who are doing the institution a service by casting light on serious problems. Indeed, Prof Murray noted that in her institution, failure to report an incident that gives reasonable suspicion of research misconduct is itself regarded as misconduct.  At present, whistleblowers are often treated as nuisances or cranks who need to be shut down. As was evident from the cases of both Sanders and Wilmshurst, they are at risk of litigation, and careers may be put in jeopardy if they challenge senior figures.

 

7.     Changing the incentive structure in science

It’s well-appreciated that if you really want to stop a problem, you should understand what causes it and stop it at source. People do fraudulent research because the potential benefits are large and the costs seem negligible.  We can change that balance by, on the one hand having serious and public sanctions for those who commit fraud, and on the other hand, rewarding scientists who emphasise integrity, transparency and accuracy in their work, rather than those that get flashy, eyecatching results.

 

I'm developing my ideas on this topic and I welcome thoughts on these suggestions. Comments are moderated and so do not appear immediately, but I will post any that are on topic and constructive.  

Update 17th December 2022  

Jennifer Byrne suggested one further recommendation, as follows:

To change the incentive structure in scientific publishing. Journals are presently rewarded for publishing, as publishing drives both income (through subscriptions and/or open access charges) and the journal impact factor. In contrast, journals and publishers do not earn income and are not otherwise rewarded for correcting the literature that they publish. This means that the (seemingly rare) journals that work hard to correct, flag and retract erroneous papers are rewarded identically to journals that appear to do very little. Proactive journals appear to represent a minority, but while there are no incentives for journals to take a proactive approach to published errors and misinformation, it should not be surprising that few journals join their efforts. Until publication and correction are recognized as two sides of the same coin, and valued as such, it seems inevitable that we will see a continued drive towards publishing more and correcting very little, or continuing to value publication quantity over quality.

 

Bibliography 

I'll also add here additional resources. I'm certainly not the first to have made the points in this post, and it may be useful to have other articles gathered together in one place.  

 

Besançon, L., Bik, E., Heathers, J., & Meyerowitz-Katz, G. (2022). Correction of scientific literature: Too little, too late! PLOS Biology, 20(3), e3001572. https://doi.org/10.1371/journal.pbio.3001572   

 

Byrne, J. A., Park, Y., Richardson, R. A. K., Pathmendra, P., Sun, M., & Stoeger, T. (2022). Protection of the human gene research literature from contract cheating organizations known as research paper mills. Nucleic Acids Research, gkac1139. https://doi.org/10.1093/nar/gkac1139 

 

Christian, K., Larkins, J., & Doran, M. R. (2022). The Australian academic STEMM workplace post-COVID: a picture of disarray. BioRxiv. https://doi.org/10.1101/2022.12.06.519378 

 

Lévy, R. (2022, December 15). Is it somebody else’s problem to correct the scientific literature? Rapha-z-Lab. https://raphazlab.wordpress.com/2022/12/15/is-it-somebody-elses-problem-to-correct-the-scientific-literature/ 

 

Research misconduct: Theory & Pratico – For Better Science. (n.d.). Retrieved 17 December 2022, from https://forbetterscience.com/2022/08/31/research-misconduct-theory-pratico/   

Star marine ecologist committed misconduct, university says. (n.d.). Retrieved 17 December 2022, from https://www.science.org/content/article/star-marine-ecologist-committed-misconduct-university-says  

Additions on 18th December: Yet more relevant stuff coming to my attention! 

 

Naudet, Florian (2022) Lecture: Busting two zombie trials in a post-COVID world.   

 

Wilmshurst, Peter (2022) Blog: Has COPE membership become a way for unprincipled journals to buy a fake badge of integrity?

 *Addition on 20th December

Liverpool Medical Institution seminar on Research Integrity: The introduction by Patricia Murray, talk by Peter Wilmshurt, and Q&A are now available on Youtube.

And finally.... 

A couple of sobering thoughts:

 

Alexander Trevelyan on Twitter noted  a great quote from the anonymous @mumumouse (author of Research misconduct blogpost above): “To imagine what it’s like to be a whistleblower in the science community, imagine you are trying to report a Ponzi scheme, but instead of receiving help you are told, nonchalantly, to call Bernie Madoff, if you wish." 

 

Peter Wilmshurst started his talk by relaying a conversation with Patricia Murray in the run-up to his talk. He said he planned to talk about the 3 Fs, fabrication, falsification and honesty.

 To which Patricia replied, “There is no F in honesty”. 

(This may take a few moments to appreciate).

 

 

A desire for clickbait can hinder an academic journal's reputation

 

On 28th September, I woke up to look at Twitter and find Pete Etchells fulminating about a piece in the Guardian.  

It was particularly galling for him to read a piece that implied research studies had shown that voice-responsive devices were harming children’s development when he and Amy Orben had provided comments to the Science Media Centre that were available to the journalist. They both noted that: 

a) This was a Viewpoint piece, not new research 

b) Most of the evidence it provided consisted of anecdotes from newspaper articles

I agreed with Pete’s criticism of the Guardian, but having read the original Viewpoint in the Archives of Disease in Childhood, I had another question, namely, why on earth was a reputable paediatrics journal doing a press release on a flimsy opinion piece written by two junior medics with no track record in the area? 

So I wrote to the Editor with my concerns, as follows: 

Dear Dr Brown 

Viewpoint: Effects of smart voice control devices on children: current challenges and future perspectives doi 10.1136/archdischild-2022-323888 Journal: Archives of Disease in Childhood  

I am writing to enquire why this Viewpoint was sent out to the media under embargo as if it was a substantial piece of new research. I can understand that you might want to publish less formal opinion pieces from time to time, but what I cannot understand is the way this was done to attract maximum publicity by the media. 

The two people who commented about it for the Science Media Centre both noted this was an opinion piece with no new evidence, relying mainly on media reports. 

https://www.sciencemediacentre.org/expert-reaction-to-an-opinion-piece-on-voice-controlled-devices-and-child-development/ 

Unfortunately, despite this warning, it has been picked up by the mainstream media, where it is presented as ‘new research’, which will no doubt give parents of young children something new to worry about. 

I checked out the authors, and found these details: 

https://orcid.org/0000-0003-4881-8293 

https://www.researchgate.net/profile/Ananya-Arora-3 

These confirm that neither has a strong research track record, or any evidence of expertise in the topic of the Viewpoint. I can only assume that ADC is desperate for publicity at any cost, regardless of scientific evidence or impact on the public. 

As an Honorary Fellow of the Royal College of Paediatrics and Child Health, and someone who has previously published in ADC, I am very disappointed to see the journal sink so low. 

Yesterday I got a reply that did nothing to address my concerns. Here’s what the editor, Nick Brown*, said (in italic), with my reactions added: 

Thank you for making contact . My response reflects the thoughts of both the BMJ media and publication departments  

Given my reactions, below, this is more worrying than reassuring. It would be preferable to have heard that there had been some debate as to the wisdom of promoting this article to the press. 

It is a key role of a scientific journal to raise awareness of, and stimulate debate on, live and emerging issues. Voice control devices are becoming increasingly common and their impact on children's development is a legitimate topic of discussion.  

I have no quarrel with the idea that impact of voice control devices on children is a legitimate topic for the journal. But I wonder about how far its role is ‘raising awareness of, and stimulating debate’ when the topic is one on which we have very little evidence. A scientific journal might be expected to provide a balanced account of evidence, whereas the Viewpoint presented one side of the ‘debate’, mainly using anecdotes. I doubt it would have been published if it had concluded that there was no negative impact of voice control devices.  

Opinion pieces are part of a very wide range of content that is selected for press release from among BMJ's portfolio of journals. They are subject to internal review in line with BMJ journals´overall editorial policy: the process (intentionally) doesn't discriminate against authors who don't have a strong research track record in a particular field  

I’ve been checking up on how frequently ADC promotes an article for press release. This information can be obtained here. This year, they have published 219 papers, of which three other articles have merited a press release: an analysis of survey data on weight loss (July), a research definition of Long Covid in children (February) and a data-based analysis of promotional claims about baby food (February). Many of the papers that were not press-released are highly topical and of general interest – a quick scan found papers on vaping, monkey pox, transgender adolescents, unaccompanied minors as asylum seekers, as well as many papers relating to Covid. It’s frankly baffling why a weakly evidenced viewpoint on a topic with little evidence was selected as meriting special treatment with a press release. 

As for the press release pathway itself, all potential pieces are sent out under embargo, irrespective of article type. This maximises the chances of balanced coverage: an embargo period enables journalists to contact the authors with any queries and to contact other relevant parties for comment. 

My wording may have been clumsy here and led to misunderstanding. My concern was more with the fact that the paper was press-released, which is, as established above, highly unusual, rather than with the embargo.  

The press release clearly stated (3 times) this article was a viewpoint and not new research, and that it hadn't been externally peer reviewed. We also always include a direct URL link to the article in question in our press releases so that journalists can read the content in full for themselves. 

I agree that the press release included these details, and indeed, had journalists consulted the Science Media Centre’s commentaries, the lack of peer review and data would have been evident. But nevertheless, it’s well-known that (a) journalists seldom read original sources, and (b) some of the less reputable newspapers are looking for  clickbait, so why provide them with the opportunity for sensationalising journal content?

While we do all we can to ensure that journalists cover our content responsibly, we aren't responsible for the manner in which they choose to do so. 

I agree that part of the blame for the media coverage lies with journalists. But I think the journal must bear some responsibility for the media uptake of the article. It’s a reasonable assumption that if a reputable journal issues a press release, it’s because the article in question is important and provides novel information from recognised experts in the field. It is unfortunate that that assumption was not justified in that case. 

I just checked to see how far the media interest in the story had developed. The Guardian, confronted with criticism, changed the lede to say “Researchers suggest”, rather than “New research says”, but the genie was well out of the bottle by that time. The paper has an Altmetric ‘attention’ score of 1577, and been picked up by 209 news outlets. There’s no indication that the article has “stimulated debate”. Rather it has been interpreted as providing a warning about a new danger facing children. The headlines, which can be found here, are variants of: 

 “Alexa and Siri make children rude” 

“Siri, Alexa and Google Home could hinder children’s social and cognitive development” 

“Voice-control devices may have an impact on children’s social, emotional development: Study” 

“According to a study, voice-controlled electronic aides can impair children’s development” 

“Experts warn that AI assistants affect children’s social development” 

“Experts warn AI assistants affect social growth of children” 

“Why Alexa and Siri may damage kids’ social and emotional development” 

“Voice assistants harmful for your child’s development, claims study” 

“Alexa, Siri, and Other Voice Assistants could negatively rewire your child’s brain” 

“Experts warn using Alexa and Siri may be bad for children” 

“Parents issued stark warning over kids using Amazon’s Alexa” 

“Are Alexa and Siri making our children DUMB?” 

“Use of voice-controlled devices ‘might have long-term consequences for children’” 

And most alarmingly, from the Sun: 

“Urgent Amazon Alexa warning for ALL parents as new danger revealed” 

Maybe the journal’s press office regards that as a success. I think it’s a disaster for the journal’s reputation as a serious academic journal. 

 

*Not the sleuth Nick Brown. Another one. 

Book Review. Fiona Fox: Beyond the Hype

If you're a scientist reading this, you may well think, as I used to, that running a Science Media Centre (SMC) would be a worthy but rather dull existence. Surely, it's just a case of getting scientists to explain things clearly in non-technical language to journalists. The fact that the SMC was created in part as a response to the damaging debacle of the MMR scandal might suggest that it would be a straightforward job of providing journalists with input from experts rather than mavericks, and helping them distinguish between the two. 

I now know it's not like that, after being on the Science Media Centre's panel of experts for many years, and having also served on their advisory committee for a few of them. The reality is described in this book by SMC's Director, Fiona Fox, and it's riveting stuff.

In part this is because no science story is simple. People will disagree about the quality of the science, the meaning of the results, and the practical implications. Topics such as climate change, chronic fatigue syndrome/ME and therapeutic cloning elicit highly charged responses from those who are affected by the science. More recently, we have found that when a pandemic descends upon the world, some of the bitterest disagreements are not between scientists and the media, but between well-respected, expert scientists. The idea that scientists can hand down tablets of stone inscribed with the truth to the media is a fiction that is clearly exposed in this book.

Essentially, the SMC might be seen as acting like a therapist in the midst of a seriously dysfunctional family where everyone misunderstands everyone else, and everyone wants different things out of life. On the one hand we have the scientists. They get frustrated because they feel they should be able to make exciting new discoveries, with the media then helping communicate these to the world. Instead, they complain that the media has two responses: either they're not interested in the science, or they want to sensationalise it. If you find a mild impact of grains on sexual behaviour in rats, you'll find it translated into the headline 'Cornflakes make you impotent'.

On the other hand, we have the media. They want a good story, but find that the scientists are reluctant to talk to them, or want total control of how the story is presented. In the worst case, scientists are prima donnas who want days or weeks to prepare for a media interview and will then shower the journalist with detailed information that is incomprehensible, irrelevant, or both. When the public desperately needs a clear, simple message, the scientists will refuse to deliver it, hedging every statement.

Fox has worked over the years to challenge these stereotypes: journalists do want a good story, but the serious science journalists want a true story, and are glad of the opportunity to pose questions directly to scientists. And many scientists do a fantastic job of explaining their subject matter to a non-specialist audience. In the varied chapters of the book, Fox is an irrepressible optimist, who keeps coming back to the importance of having scientists communicating directly with the media. Her optimism is not founded in ignorance: she knows exactly how messy and complicated science can be. But she persists in believing that more good is done by communicating what we know, warts and all, rather than pretending that uncertainties and disagreements do not exist.

The role of the SMC is, however, complicated by further factions. The dramatis personae includes two other groups. First, there are science press officers, who are appointed by institutions to help scientists promote their work, and then there are government officials and civil servants, who are concerned with policy implications of science.

In her penultimate chapter, Fox bemoans the fact that the traditional press officer - passionate about science and viewing themselves as "purveyors of truth and accuracy" - is a dying breed. There remain notable exceptions, but all too often science communication has become conflated with a public relations role: pushing a corporate message, defending the institutional reputation, and even using scientific discoveries as a marketing tool. Fox notes a 2014 survey of exaggerated science reports in the media that concluded: "Exaggeration in news is strongly associated with exaggeration in press releases." I had been one of those scientists who thought the media were mostly to blame for over-hyped science reporting, but this study showed that journalists are often recycling exaggerated accounts handed to them by those speaking for the scientists.

But the problems posed by scientists, journalists and press officers are trivial compared to the obstacles created by those involved in policy. They want to use science when convenient, but also want to exert control over which aspects of science policy gets talked about. Scientists working for government-funded organisations are often muzzled, with explicit instructions not to talk to the media. One can see that this cautious approach, attempting to control the message and keep things simple, puts many civil servants and government scientists on a collision course with Fox, whose view is: "Explaining preliminary and contradictory science is messy: that should not be seen as a failure of communications".

A refreshing aspect of Fox's account is that she does not brush aside the occasions when the SMC - or she personally - may have handled a situation badly. Of course, it's easy to point the finger of blame when something does go horribly wrong, and Fox has come under fire on many occasions. Rather than being defensive, she accepts that things might have been done differently, while at the same time explaining the logic of the decisions that were taken. This is in line with my memories of meetings of the SMC advisory committee, where there were frequent post mortems - "this is how we handled it; this is how it turned out; should we have done it differently?" - with frank discussions from the committee members. When you are working in contentious areas where things are bound to blow up every now and again, this is a sensible strategy that helps the organisation learn and develop. I'm glad that after 20 years, the ethos of the SMC is still very much on the side of open, transparent communication between scientists and the media.  

Fox, Fiona (2022) Beyond the Hype: The Inside Story of Science's Biggest Media Controversies. London: Elliott and Thompson Ltd.

Time to ditch relative risk in media reports

The Winton Centre for Risk and Evidence Communication at the University of Cambridge has done some sterling work in developing guidelines for communicating risk to the general public. In a short video,  David Spiegelhalter explains how relative risk can be misleading when the baseline for a condition is not reported. For instance, he noted that many women stopped taking a contraceptive pill after hearing media reports that it was associated with a doubling in the rate of thrombo-embolism. In terms of absolute risk the increase sounds much less alarming, going from 1 in 7000 to 2 in 7000. 

One can understand how those who aren't scientifically trained can get this wrong. But we might hope that,  in a pandemic, where public understanding of risk is so crucial, particular care would be taken to be realistic without being alarmist. It was, therefore, depressing to see a report on Channel 4 news last night where two scientists clearly explained the evidence on Covid variants in terms of absolute risk, impeccably reflecting the Winton Centre's advice, only to have the reporters translate the numbers into relative risk. I have transcribed the relevant sections: 

0:44 Reporter: "The latest evidence from the government's advisers is that this new variant is more deadly. And this is what it means:"

Patrick Vallance: "If you took somebody in their sixties, a man in their sixties, the average risk is that for 1000 people who got infected, roughly ten would be expected to unfortunately die with the virus. With the new variant, with 1000 people infected, roughly 13 or 14 people might be expected to die." 

Reporter: "That’s a thirty to forty per cent increase in mortality." 

5:15 Reporter (Krishnan Guru-Murthy): "But this is a high percent of increase, isn't it. Thirty to forty percent increase of mortality, on a relatively small number." 

Neil Ferguson: "Yes. For a 60-year-old at the current time, there's about a one in a hundred risk of dying. So that means 10 in 1000 people who get the infection are likely to die, despite improvements in treatment. And this new variant might take that up to 13 or 14 in a 1000." 

The reporters are likely to stoke anxiety when they translate clear communication by the scientists into something that sounds a lot more scary. I hope this is not their intention: Channel 4 is one of the few news outlets that I regularly watch and in general I find it well-researched and informative. I would urge the reporters to watch the Winton Centre video, which in less than 5 minutes makes a clear, compelling case for dropping relative risk altogether in media reports. 

 

This blogpost has been corrected to remove the name of Anja Popp as first reporter. She confirmed was not in this segment.  My apologies.

On the need for responsible reporting of research to the media

This was one of the first tweets I saw when I woke up this morning :

In response, a parent of two girls with autism tweeted "gutted to read this. B's statement has been final for 1 yr but no therapy has been done. we're still waiting."

I was really angry. A parent who is waiting for therapy for a child has many reasons to be upset. But the study described on the BBC Website did NOT identify a 'critical window'. It was not about autism and not about intervention.

I was aware of the study because I'd been asked by the Science Media Centre to comment on an embargoed version a couple of days ago.

These requests for commentary on embargoed papers always occur very late in the day, which makes it difficult to give a thorough appraisal. But I felt I'd got the gist: the researchers had recruited 108 children aged between 1 and 6 years and done scans to look at the development of white matter in the brain. They also gave children a well-known test of cognitive development, the Mullen scales, which assesses language, visual and fine motor skills. It's not clear where the children came from, but their scores on the Mullen scales were pretty average, and as far as I can tell, none of them had any developmental disorders.

The researchers were particularly interested in lateralisation: the tendency to have more white matter on one side of the brain than the other. Left-sided lateralisation of white matter in some brain regions is well-established in adults but there's been debate as to whether this is something that develops early in life, or whether it is present from birth. In the introduction, the authors state that this lateralisation is strongly heritable, but although that's often claimed, the evidence doesn't support it (Bishop, 2013). A preponderance of white matter in the left hemisphere is of interest because in most people, the left side of the brain is strongly involved in language processing.

The authors estimated lateralisation in numerous regions of the left and right brain using a measure termed the myelin water fraction. Myelin is a fatty sheath that develops around the axons of cells in the brain, leading to improved efficiency of neural transmission. Myelination is a well-established phenomenon in brain development.

The main findings I took away from the paper were (a) myelin is asymmetrically distributed in the brains of young children, with many regions showing greater myelin density in the left than the right; (b) although the amount of myelin increases with age, the extent of lateralisation is stable from 1 to 6 years. This is an important finding.

The authors, however, put most focus on another aspect of the study: the relationship between myelin lateralisation and language level. Overall, there was no relationship with asymmetry of a temporal-occipital region that overlapped with the arcuate fasciculus, a fibre tract important for language that previously had given rather inconsistent results (see Bishop, 2013). However, looking at a total of eight brain regions and four cognitive measures, they found two regions where leftward asymmetry was related to language or visual measures, and one where rightward asymmetry was related to expressive and receptive language.

Their primary emphasis, however, was on another finding, that there were interactions between age and lateralisation, so that, for instance, left-sided lateralisation of myelin in a region encompassing caudate/thalamus and frontal cortex only became correlated with language level in older children. I found it hard to know how much confidence to place in this result: the authors stated that they corrected for multiple comparisons using false discovery rate, but if, as seems the case, they looked at both main effects and interaction terms in 32 statistical analyses, then some of these findings could be chance.

Be that as it may, it is an odd result. Remember that this was a cross-sectional study and that on no index was there an age effect on lateralisation. So it does not show that changes in language ability - which are substantial over this age range - are driven by changes in lateralisation of myelin. So what do the authors say? Well, in the paper, they conclude "The data presented here are cross sectional, longitudinal analysis will allow us to confirm these findings; however, the changing interaction between ability and myelin may be mediated by progressive functional specialization in these connected cortical regions, which itself is partly mediated by environmental influences" (p. 16175). But this is pure speculation: they have not measured functional specialisation, and, as they appear to recognise, without longitudinal data, it is premature to interpret their results as indicating change with age.

If you've followed me so far, you may be wondering when I'm going to get on to the bit about intervention for autism and critical periods. Well, there's no data in this paper on that topic. So why did the BBC publish an account of the paper likely to cause dismay and alarm in parents of children with language and communication problems? The answer is because King's College London put out a press release about this study that contained at least as much speculation as fact. We are told that the study "reveals a particular window, from 2 years to the age of 4, during which environmental influence on language development may be greatest." It doesn't do anything of the kind. They say: "the findings help explain why, in a bilingual environment, very young typically developing children are better capable of becoming fluent in both languages; and why interventions for neurodevelopmental disorders where language is impaired, such as autism, may be much more successful if implemented at a very young age. " Poppycock.

A few months ago the same press office put out a similarly misleading press release about another study, quoting the principal researcher as stating: “Now we understand that this is how we learn new words, our concern is that children will have less vocabulary as much of their interaction is via screen, text and email rather than using their external prosthetic memory. This research reinforces the need for us to maintain the oral tradition of talking to our children.” As I noted elsewhere, the study was not about children, computers or word learning.

I can see that there is a problem for researchers doing studies of structural brain development. It can be hard to excite the general public about the results unless you talk about potential implications. It is frankly irresponsible, though, to go so far beyond your data that the headline is based on the speculation rather than the findings.

I am tired of researchers trying to make their studies relevant by dragging in potential applications to autism, schizophrenia, or dyslexia, when they haven't done any research on clinical groups. They need to remember that there are real people out there whose everyday life is affected by these conditions, and that neither they nor the media can easily discriminate what a study actually found from speculations about its implications. It is the duty of researchers and press officers to be crystal clear about that distinction to avoid causing confusion and distress.

POSTSCRIPT
11/10/13: Dr O'Muircheartaigh has commented below to absolve the KCL Press Office of any responsibility for the content of their press release. I apologise for assuming that they were involved in decisions about how to publicise this research and have reworded parts of this blogpost to remove that implication.


References 

Bishop, D. V. M. (2013). Cerebral asymmetry and language development: Cause, correlate, or consequence? Science, 340 (6138) DOI: 10.1126/science.1230531

O'Muircheartaigh, J., Dean, D. C., Dirks, H., Waskiewicz, N., Lehman, K., Jerskey, B. A., & Deoni, S. C. L. (2013). Interactions between white matter asymmetry and language during neurodevelopment. Journal of Neuroscience, 33(41), 16170-16177. doi: 10.1523/jneurosci.1463-13.2013

 

Schizophrenia and child abuse in the media

A couple of weeks ago, the Observer printed a debate headlined “Do we need to change the way we are thinking about mental illness?” I read it with interest, as I happen to think that we do need to change, and that the new Diagnostic and Statistical Manual of the American Psychiatric Association (DSM5) has numerous problems.

The discussion was opened by Simon Wessely, a member of the Royal College of Psychiatrists, who responded No. He didn’t exactly defend the DSM5, but he disagreed with the criticism that it reduces psychiatry to biology. The Yes response was by Oliver James, an author and clinical psychologist, who attacked the medical model of mental illness, noting the importance of experience, especially childhood experience, in causing psychiatric symptoms. I happen to take a middle way here; there’s ample evidence of biological risk factors for many forms of mental illness, but in our contemporary quest for biomarkers, the role of experience is often sidelined. The idea that you might be depressed because bad things have happened to you goes unmentioned in much contemporary research on affective disorders, for instance.

But, rather than getting into that debate, I want to make a more general point about evidence. In his statement, Oliver James came out with some statistics that surprised me. In particular, he said:

13 studies find that more than half of schizophrenics suffered childhood abuse. Another review of 23 studies shows that schizophrenics are at least three times more likely to have been abused than non-schizophrenics. It is becoming apparent that abuse is the major cause of psychoses.

The frustrating thing about this claim is that no sources were given. I don't work in this area, so I thought I’d see if I could track down the articles cited by James. My initial attempt was based on a hasty search of Web of Knowledge on the morning that the article appeared. I described the results of my searches on a blogpost that day, but a commentator pointed out that I'd limited myself to looking at the link between schizophrenia and sexual abuse, whereas James had been referring to childhood abuse in general. I realised that a fair and proper appraisal of his claims should look at this broader category, and accordingly I removed the original post until I could find time to do a more thorough job.

Accurate assessment of child abuse is difficult because it is often hidden away and many cases may be missed. Retrospective accounts of abuse are notoriously hard to validate: false memories can be induced, but true memories may be suppressed. All those writing in this field note the problems of getting accurate data, and the wide variations in rates of abuse reported in the general population, depending on how it is defined. For instance, Fryers and Brugha (2013) noted that for the general population, estimated rates of child physical abuse have ranged between 10% to 31% in males and 6% to 40% of females, and child sexual abuse from 3% to 29% in males and 7% to 36% in females.

Fryers and Brugha focused on prospective, longitudinal studies, taking evidence from over 200 studies. They concluded that “most abuses were associated statistically with almost all classes of disorder (psychosis being largely an exception)”, p 26, and “schizophrenia and closely related syndromes have not generally been much associated with previous child abuse but the picture is not simple.” P.27. They noted that one Australian cohort study found an increase in schizophrenic disorders in children who had been sexually abused, though this was an unusual finding in the context of the research literature as a whole.

Other recent meta-analyses have included case-control studies, where participants are recruited in adulthood, and histories of participants with schizophrenia are compared with those of a control group. Matheson et al (2012) summarised seven studies involving a comparison between patients with schizophrenia and non-psychiatric controls, but definitions of adversity varied widely. In some studies, 'adversity' extended beyond abuse, though physical, sexual and emotional abuse predominated in the definitions. Overall, this review gave results similar to those reported by James, with an adversity rate of 58% in the schizophrenia group and 27% in the controls. This high rate in those with schizophrenia depends, however, on one large study that included adversity factors going beyond abuse, such as having a parent with nervous or emotional problems, or a lot of conflict and tension in the household. In this same study, 91% of controls compared with 71% of those with schizophrenia described their childhood as "happy". If this study is excluded, rates of adversity in those with schizophrenia fall to 28% compared to 8% in controls – still a notable and statistically reliable effect, but with less dramatic absolute rates of adversity than those cited by James.

A larger meta-analysis including a total of 36 studies was conducted by Varese et al (2012), who obtained similar results:  a higher rate of childhood adversities in those who develop psychosis, with an odds ratio estimated at 2.78 (95% CI = 2.34-3.31). Significant associations of similar magnitude were found for all types of adversity other than parental death. The odds ratio is not, however, the same as a risk ratio, so should not be interpreted as indicating that those with schizophrenia are three times more likely to have suffered abuse. I don’t want to downplay the importance of the association, which is nevertheless striking, and supports the authors' conclusion that clinicians should routinely inquire about adverse events in childhood when seeing patients with psychiatric conditions.

Overall, the research literature confirms a reliable association between childhood adversity, including abuse, and schizophrenia in adulthood. The conclusion drawn by James, however, that “abuse is the major cause of psychoses” is not endorsed by any of the academic authors of the reviews I looked at. The complexity of causation in the field of neuroscience and mental health is a topic I hope to return to in a later blogpost, but for the time being, I would recommend another review of this literature by Sideli et al (2012), which discusses possible explanations for links between adversity and psychosis. Most researchers familiar with this area would endorse this quote by Fryers and Brugha (2013), reflecting on our state of knowledge in this area:

From all this work an understanding has emerged of the 'cause' of serious mental illness as complex, varied and multi-factorial, encompassing elements of genetic constitution, childhood experience, characteristics of personality, significant life events, the quality of relationships, economic and social situation, life-style choices such as alcohol and other drugs, and aging. Some of these factors have been elucidated to the point of representing acknowledged risk factors for specific forms of mental illness or mental illness in general, such as familial genes, relative poverty, major trauma, excessive alcohol consumption, extreme negative life-events, poor education, and long-term unemployment.

These may all be experienced in childhood and we do not need research to tell us that poverty, inadequate education and life events such as loss of a parent or displacement as a refugee by war, or trauma such as child sex abuse are bad. Nor should it need evidence of later consequences such as mental illness to argue for the prevention of such situations and experiences. The strongest argument is in terms of human rights. However, the issues are not generally given a high priority and people may think them exaggerated or assume that these things are just part of human life and children get over them anyway. But we should not be willing to accept these as inevitably part of human life, but fight for a better life for our children – and hope thereby for a better life for adults and the whole community.

But to come back to the impetus for the current blogpost, the point I’d really like to make is that if the Observer wants to run articles like this, where scientific evidence is cited, the editor should ask for sources for the evidence, and should provide these with the article. As noted by Prof Michael O'Donovan in a letter to the Observer, Oliver James is "unknown in the scientific community as a researcher into the origins of psychosis". This does not make his opinions worthless, but if he wants to argue his case from the scientific evidence, then we need to know what evidence he is using, just as we would expect for any reputable scientist making such claims. Most readers don't have the resources or skills to trawl through research databases trying to establish whether the evidence is accurately reported or cherry-picked. As O'Donovan points out, confident assertions about childhood causes of schizophrenia can only cause distress to families affected by this condition, and a responsible newspaper should take care to ensure that such claims have a verifiable basis.

References

Fryers, T., & Brugha, T. (2013). Childhood determinants of adult psychiatric disorder. Clinical Practice & Epidemiology in Mental Health, 9 (1), 1-50 DOI: 10.2174/1745017901309010001

Matheson, S. L., Shepherd, A. M., Pinchbeck, R. M., Laurens, K. R., & Carr, V. J. (2013). Childhood adversity in schizophrenia: a systematic meta-analysis. Psychological Medicine, 43(2), 225-238. doi: 10.1017/s0033291712000785

Sideli, L., Mule, A., La Barbera, D., & Murray, R. M. (2012). Do child abuse and maltreatment increase risk of schizophrenia? Psychiatry Investigation, 9, 87-99. doi.org/10.4306/pi.2012.9.2.87

Varese, F., Smeets, F., Drukker, M., Lieverse, R., Lataster, T., Viechtbauer, W., Read, J., van Os, J., & Bentall, R. (2012). Childhood adversities increase the risk of psychosis: A meta-analysis of patient-control, prospective- and cross-sectional cohort studies Schizophrenia Bulletin, 38 (4), 661-671 DOI: 10.1093/schbul/sbs050

Note: Thanks to the anonymous reviewer who noted the problem with my initial analysis.