Should I stay or should I go? When debate with opponents should be avoided

Suppose you are invited to speak at a conference where some of the other speakers have views very different from yours. What do you do? My guess is that most academics would say you should accept. After all, we progress by evaluating claims and counterclaims, and robust debate is the lifeblood of scientific research. I'm going to argue here that there are exceptions and explain why I think responsible scientists should avoid a meeting called "Fixing Science: Practical Solutions for the Irreproducibility Crisis".

To understand this reaction, it helps to have read Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming by Eric Conway and Naomi Oreskes (reviewed here). The synopsis from the book blurb is as follows:

The U.S. scientific community has long led the world in research on such areas as public health, environmental science, and issues affecting quality of life. Our scientists have produced landmark studies on the dangers of DDT, tobacco smoke, acid rain, and global warming. But at the same time, a small yet potent subset of this community leads the world in vehement denial of these dangers.

Merchants of Doubt tells the story of how a loose-knit group of high-level scientists and scientific advisers, with deep connections in politics and industry, ran effective campaigns to mislead the public and deny well-established scientific knowledge over four decades. Remarkably, the same individuals surface repeatedly - some of the same figures who have claimed that the science of global warming is "not settled" denied the truth of studies linking smoking to lung cancer, coal smoke to acid rain, and CFCs to the ozone hole. "Doubt is our product," wrote one tobacco executive. These 'experts' supplied it.

Uncertainty about science that threatens big businesses has been promoted by think tanks such as the Heartland Institute and Cato Institute, which receive substantial funding from those vested interests. The Fixing Science meeting has a clear overlap with those players.

The meeting first came to my attention when a mini Twitterstorm erupted after James Heathers tweeted:

Everyone's familiar with the manel, right? The all-male panel? Well, here's a whole new level for you.
Presenting: THE MANFERENCE.

I found myself wondering whether the lack of women was deliberate – maybe the organisers think you need a Y chromosome to do science – or whether it just showed they were tone-deaf to current social norms.

Once I twigged that this was organised by the National Association of Scholars, then everything fell into place. You can find the background to this organisation in their annual report here.

As several commentators have pointed out, they use the acronym NAS, which just happens to be the same as the highly respectable National Academy of Sciences: to avoid confusion here I will refer to them as NatAsSchols. The impression from their website and publications is that they are aligned with a neoliberal viewpoint and are opposed to attempts to increase diversity of race or gender in Universities.

So why is this organisation, whose mission is focused on issues such as preserving free speech and counteracting left-wing bias in Universities, running a meeting on fixing problems in science?

The NatAsSchols explains their interest in this topic as follows:

In April we published The Irreproducibility Crisis, a report on the modern scientific crisis of reproducibility—the failure of a shocking amount of scientific research to discover true results because of slipshod use of statistics, groupthink, and flawed research techniques. We launched the report at the Rayburn House Office Building in Washington, DC; it was introduced by Representative Lamar Smith, the Chairman of the House Committee on Science, Space, and Technology. This project signals our increasing commitment to address the academy’s flawed science as well as its abandonment of Western civilization and the liberal arts. We are following up The Irreproducibility Crisis with the investigation of four government agencies, including the Environmental Protection Agency. We are determined to find out just how badly irreproducible science has distorted government policy.

This makes it clear that the agenda is fundamentally a political one, designed to support the Trump administration's dismantling of environmental protections.

In 2018, Naomi Oreskes, author of Merchants of Doubt, wrote in Nature about a new 'Transparency Rule' proposed by the Environmental Protection Agency:

There is a crisis in US science, but it is not the one claimed by advocates for the rule. The crisis is the attempt to discredit scientific findings that threaten powerful corporate interests. The EPA is following a pattern that I and others have documented in regard to tobacco smoke, pollution, climate, and more. One tactic exploits the idea of scientific uncertainty to imply there is no scientific consensus. Another, seen in the latest efforts, insinuates that relevant research might be flawed. To add insult to injury, those using these tactics claim to be defending science.

February's meeting is in the same mould. The format of the meeting is cleverly constructed. The conference will be introduced and summed up by David J. Theroux (Founder, President and Chief Executive Officer of the Independent Institute and Publisher of The Independent Review) and Peter Wood, (President, NatAsSchols). Neither man has any scientific background. Theroux delighted the Heartland Institute last summer when he promoted the idea, recently publicised by Donald Trump, that wind turbines are responsible for killing numerous birds (to see this lampooned, click here)

Wood was an anthropologist who has been Provost at a small religious school, The King’s College in New York City (2005-2007), before moving to NatAsSchols. He has, as far as I can tell, no peer-reviewed publications, but he has written pieces deriding climate concerns, e.g. "the fantasies of global warming catastrophe are a kind of substitute religion, replete with a salvation doctrine, rituals of expiation, and a collection of demons to be cast out."

Another presenter is David Randall, who is Director of Research at NatAsSchols, policy advisor to the Heartland Institute and first author of the report on "The Irreproducibility of Modern Science". He is an unusual person to be authoring an authoritative report on the state of science. Web of Science turned up seven publications by him, all in politics journals, and none with any citations. His background is in history, library studies and fiction writing.

A rather puzzling choice of speaker is Richard K. Vedder, Distinguished Emeritus Professor of Economics, Ohio University and senior fellow at The Independent Institute, a think-tank founded by David Theroux. I could not find much evidence that he has shown any prior interest in science. He is Founding Director of the Center for College Affordability and Productivity in Washington, D.C and policy advisor to the Heartland Institute.

But there are also some accredited scientists on the programme, who can be divided into two camps. First, we have a set of five speakers who are aligned with NatAsSchols and/or the Heartland Institute and who have unconventional views on subjects such as climate change, pollution and gay relationships:

Elliott D. Bloom is Professor Emeritus at the Kavli Institute for Particle Astrophysics and Cosmology in the Stanford Linear Accelerator Laboratory (SLAC) and a Fellow of the American Physical Society. He has an entry on the Independent Institute website which states: "He was a member of the SLAC team with Jerome I. Friedman, Henry W. Kendall and Richard E. Taylor who received the 1990 Nobel Prize in Physics." I thought this meant he was a Nobel Laureate, but he's not listed as one. Nevertheless, he has a strong publication record in Physics. He has co-authored a presentation on "Global Warming: Fact or Fiction?", which concludes that the sun, rather than CO2 is the principal driver of climate change.

Anastasios Tsonis is Emeritus Distinguished Professor, Department of Mathematical Sciences, Atmospheric Sciences Group, University of Wisconsin Milwaukee; and Adjunct Research Scientist, Hydrologic Research Center, San Diego, California. He has worked on mathematical models of atmospheric processes and has a strong set of publications. He is a member of the academic advisory council of the Global Warming Policy Forum, a think tank founded by Nigel Lawson to combat policies designed to mitigate climate change.

Patrick J. Michaels, Senior Fellow, Competitive Enterprise Institute has a Wikipedia entry that states that "he was a senior fellow in environmental studies at the Cato Institute until Spring 2019. Until 2007 he was research professor of environmental sciences at the University of Virginia, where he had worked from 1980." Michaels also has an entry in the Website of the Heartland Institute 

Louis Anthony Cox is Professor, Department of Biostatistics and Informatics, University of Colorado and President of Cox Associates, a Denver-based applied research company specializing in quantitative risk analysis, causal modeling, advanced analytics, and operations research. He has a long list of publications. A Google search turns up an article in the Los Angeles Times which states:

The Trump administration’s reliance on industry-funded environmental specialists is again coming under fire, this time by researchers who say that Louis Anthony 'Tony' Cox Jr., who leads a key Environmental Protection Agency advisory board on air pollution, is a 'fringe' scientist and ideologue pushing policies detrimental to public health.

They refer to this paper in Science, which stated that Cox ignored consensus viewpoints on the effects of smog and particulate pollution. His work has also been criticised for its conflict with corporate interests.

Mark Regnerus, Professor, Sociology Department, University of Texas at Austin has a Wikipedia page which notes the controversy around his research on the adverse impact of a child having a parent who has been involved in a same-sex relationship. The research is funded by the Witherspoon Institute, a conservative think tank. Regnerus also contributed to an amicus brief in opposition to same-sex marriage. A sympathetic account of the controversy was published by the NatAsSchols .

Of the remaining 11 speakers, as far as I can see only one, Barry Smith (University at Buffalo), has any formal links with NatAsSchols. With a few exceptions they are psychologists/philosophers/statisticians with a specific interest in scientific reproducibility. Interest in this topic has been growing exponentially over the past 10 years, and, in general, those engaged in research in this area do so with the aim of improving scientific transparency and practice. However, they run the risk that their agenda can be weaponised to cast doubt on any particular part of scientific research that is politically or commercially inconvenient.

They will serve perfectly as foils to the five speakers whose minority views on climate/pollution/sexuality will not have to face questioning by anyone with deep expertise in those areas. I have no doubt that the reproducibility experts will have lively debate among themselves as to how the irreproducibility crisis should be fixed, in the process achieving the useful (to the organisers) goal of emphasising just what an unreliable and uncertain business science is.

Should they agree to speak at this meeting? As @briandavidearp, remarked on Twitter: "I'm wary of deliberately failing to engage/interact w/ people or organizations on the basis that they have diff moral or political commitments than me. That way balkanization and polarization lies."

That's an answer I would have agreed with a few years ago – after all, isn't that what academic life is all about? We should not just sit in our own bubble; rather we should engage respectfully with those who have different views. But this is really not about regular scientific debate. It's about weaponising the reproducibility debate to bolster the message that everything in science is uncertain – which is very convenient for those who wish to promote fringe ideas.

My view is that many of the speakers at this meeting are being played. On the one hand their presence on the programme may encourage other to agree to participate, and give false reassurance to attendees that this is a regular conference. And on the other, they will find that their arguments are scooped up by the Merchants of Doubt and used to argue that science is so uncertain that we should not accept the consensus view. We cannot be sure whether anthropogenic climate change is exaggerated, whether pollution is not really harmful, and whether gay relationships are damaging. Those who are concerned to see such ideas promoted without any debate between experts in those areas may wish to reconsider whether this meeting is really about 'Fixing Science', or whether it is rather about 'Fitting up Scientists'.

P.S.
15th January 2020
I thank Lee Jussim for engaging in the comments below. I can see that, given his understanding of the situation, it would make sense to take part in the meeting. But his understanding is different from mine, and I want to add this PS to clarify what I'm saying.

It was perhaps a mistake for me to note the neoliberal affiliations of NatAsSchols, as this appears to have given Lee the impression that my objections to the Fixing Science meeting is based on disapproval of talking to those with right-wing beliefs. I am myself on the left politically, but I agree with Brian Earp that, insofar as it is possible to do so in good faith, we should engage with those with differing views. If the meeting consisted solely of experts in philosophy of sciences/methods/metascience, with different political persuasions, I would not be warning people off – quite the contrary.

Indeed, the people I identified as belonging to the second group of speakers would seem to be exactly such a group. As I noted, I doubt that they will come to a consensus about how to 'Fix Science', but a good mix of views and perspectives is represented. No doubt Lee and others will discuss an issue that is of particular interest to me, which is how our social and cognitive biases affect how we evaluate evidence (see Bishop, 2020). Such biases are not specifically associated with left- or right-leaning politics – they affect all of us.

The problem I have with the meeting is not that the organisers are right-wing, but rather that their organisation's goals are linked to issues around higher education, and they have no credentials in science, yet they fervently advocate minority views about such topics as climate change.  Consider how bizarre it would be if, for instance, the Psychonomics Society declared that it planned to hold a meeting on 'Fixing Politics'. The NatAsScholars just doesn't have credibility in the area of scientific pratices. Alas, what they do have instead are links with funders whose vast wealth is used to attack science that threatens their vested interests. In this respect, I think the argument that 'the left-wingers are just as bad' breaks down.

But, I reiterate, the main point is not whether NatAsSchlols is left- or right-wing. It's the weird structuring of the meeting, which juxtaposes a set of experts in the 'reproducibility crisis' with a set of individuals who promote scientific views that are far from mainstream. The fact that the topics are ones that are supported by the Heartland Institute is telling, but the same strategy could in principle be used with any fringe view. Suppose you were a sceptic about evolution or vaccination, or a believer in pre-cognition. You know your arguments would not survive scrutiny by experts familiar with evidence in the area, so you don't invite those (and to be fair, it's unlikely that they'd come anyway, as there are diminishing returns in engaging with those whose minds are fixed). But what you can do is to cast doubt on all scientific evidence by inviting along those who are questioning the solidity and credibility of current scientific practices. That's what is happening here.

The general strategy has been in use for years, as documented by Conway and Oreskes, and applied to diverse topics such as tobacco dangers and acid rain, as well as climate change. The Merchants of Doubt love it when scientists themselves disagree about the nature of evidence, because it gives them a get-out-of-jail-free card.

I'm firmly of the belief we should not shove problems with science under the carpet: we need to understand the nature and extent of such problems in order to fix them. But it is a mistake to engage with those who want to exploit the presence of uncertainty to give credibility to their fringe views.

Bishop, D. V. M. (2020). The psychology of experimental psychologists: Overcoming cognitive constraints to improve research. The 45th Sir Frederic Bartlett Lecture Quarterly Journal of Experimental Psychology, 73(1), 1-19. doi:10.1177/1747021819886519

Research funders need to embrace slow science

Uta Frith courted controversy earlier this year when she published an opinion piece in which she advocated for Slow Science, including the radical suggestion that researchers should be limited in the number of papers they publish each year. This idea has been mooted before, but has never taken root: the famous Chaos in the Brickyard paper by Bernard Forscher dates back to 1963, and David Colquhoun has suggested restricting the number of publications by scientists as a solution more than once on his blog (here and here).

Over the past couple of weeks I've been thinking further about this, because I've been doing some bibliometric searches. This was in part prompted by the need to correct and clarify an analysis I had written up in 2010, about the amount of research on different neurodevelopmental disorders. I had previously noted the remarkable amount of research on autism and ADHD compared to other behaviourally-defined conditions. A check of recent databases shows no slowing in the rate of research. A search for publications with autism or autistic in the title yielded 2251 papers published in 2010; in 2019, this has risen to 6562. We can roughly halve this number if we restrict attention to the Web of Science Core database and search only for articles (not reviews, editorials, conference proceedings etc). This gives 1135 articles published in 2010 and 3075 published in 2019. That's around 8 papers every day for 2019.

We're spending huge amounts to generate these outputs. I looked at NIH Reporter, which provides a simple interface where you can enter search terms to identify grants funded by the National Institutes of Health. For the fiscal year 2018-2019 there were 1709 projects with the keyword 'autism or autistic', with a total spend of $890 million. And of course, NIH is not the only source of research funding.

Within the field of developmental neuropsychology, autism is the most extreme example of research expansion, but if we look at adult disorders, this level of research activity is by no means unique. My searches found that this year there were 6 papers published every day on schizophrenia, 15 per day on depression, and 11 per day on Alzheimer's disease.

These are serious and common conditions and it is right that we fund research into them – if we could improve understanding and reduce their negative impacts, it would make a dramatic difference to many lives. The problem is information overload. Nobody, however nerdy and committed, could possibly keep abreast of the literature. And we're not just getting more information, the information is also more complex. I reckon I could probably understand the majority of papers on autism that were published when I started out in research years ago. That proportion has gone down and down with time, as methods get ever more complex. So we're spending increasing amounts of money to produce more and more research that is less and less comprehensible. Something has to give, and I like the proposal that we should all slow down.

But is it possible? If you want to get your hands on research funding, you need to demonstrate that you're likely to make good use of it. Publication track record provides objective evidence that you can do credible research, so researchers are focused on publishing papers. And they typically have a short time-frame in which to demonstrate productivity.

A key message from Uta's piece is that we need to stop confusing quantity with quality. When this topic has been discussed on social media, I've noted that many ECRs take the view that when you come to apply for grants or jobs, a large number of publications is seen as a good thing, and therefore Slow Science would damage the prospects of ECRs. That is not my experience. It's possible that there are differences in practice between different countries and subject areas, but in the UK the emphasis is much more on quality than quantity of publications, so a strategy of focusing on quality rather than quantity would be advantageous. Indeed, most of our major funders use proposal forms that ask applicants to list their N top publications, rather than a complete CV. This will disadvantage anyone who has sliced their oeuvre into lots of little papers, rather than writing a few substantial pieces. Similarly, in the UK Research Excellence Framework, researchers from an institution are required to submit their outputs, but there is a limited number that can be submitted – a restriction that was introduced many years ago to incentivise a focus on quality rather than quantity.

The same people who are outraged at reducing the number of publications often rail against the stress of working in the current system – and rightly so. After all, at some point in the research cycle, at least one person has to devote serious thought to the design, analysis and write-up. Each of these stages inevitably takes far longer than we anticipate – and then there is time needed to respond to reviewers. The quality and impact of research can be enhanced by pre-registration and making scripts and data open, but extra time needs to be budgeted for this. Indeed, lack of time is a common reason cited for not doing open science. Researchers who feel that to succeed they have to write numerous papers every year are bound to cut corners, and then burn out from stress. It makes far more sense to work slowly and carefully to produce a realistic number of strong papers that have been carefully planned, implemented and written up.

It's clear that science and scientists would benefit if we take things more slowly, but the major barrier is a lack of researcher confidence. Those who allocate funds to research have a vested interest in ensuring they get the best return from their investment – not a tsunami of papers that overwhelm us, but a smaller number of reports of high-quality, replicable and credible findings. If things are to change we need funders to do more to communicate to researchers that they will be evaluated on quality rather than quantity of outputs.