Wednesday, 3 August 2022

Contagion of the political system


Citizens of the UK have in recent weeks watched in amazement as the current candidates for leadership of the Conservative party debate their policies. Whoever wins will replace Boris Johnson as Prime Minister, with the decision made by a few thousand members of the Conservative Party. All options were bad, and we are now down to the last two: Liz Truss and Rishi Sunak.


For those of us who are not Conservatives, and for many who are, there was immense joy at the ousting of Boris Johnson. The man seemed like a limpet, impossible to dislodge. Every week brought a new scandal that would have been more than sufficient to lead to resignation 10 years ago, yet he hung on and on. Many people thought that, after a vote of no confidence in his leadership, he would step down so that a caretaker PM could run the country while the debate over his successor took place, but the limpet is still clinging on. He’s not doing much running of the country, but that’s normal, and perhaps for the best. He’s much better at running parties than leading the Conservative party.


I have to say I had not expected much from Truss and Sunak, but even my low expectations have not been met. The country is facing immense challenges, from climate change, from coronavirus, and from escalating energy prices. These are barely mentioned: instead the focus is on reducing taxes, with the candidates now competing for just how much tax they can cut. As far as I can see, these policies will do nothing to help the poorest in society, whose benefits will shrink to pay for tax cuts; the richer you are the more tax you pay and so this is a rich person’s policy.


What has surprised me is just how ill-informed the two candidates are. The strategy seems to be to pick a niche topic of interest to Conservative voters, make up a new policy overnight and announce it the next day. So we have Rishi Sunak proposing that the solution to the crisis in the NHS is to charge people who miss doctor’s appointments. Has he thought this through? Think of the paperwork. Think of the debt collectors tasked with collecting £10 from a person with dementia. Think of the cost of all of this.  And on Education, his idea is to reintroduce selective (grammar) schools: presumably because he thinks that our regular schools are inadequate to educate intelligent children.


On Education, Liz Truss is even worse. Her idea is that all children who score top marks in their final year school examinations should get an interview to go to Oxford or Cambridge University. This is such a crazy idea that others have written at length to point out its flaws (e.g. this cogent analysis by Sam Freedman). Suffice it to say that it has a similar vibe to the Sunak grammar schools plan: it implies that only two British universities have any value. Conservatives do seem obsessed with creating divisions between haves and have-nots, but only if they can ensure their children are among the haves.


Another confused statement from Truss is that, as far as Scotland goes, she plans to ignore Nicola Sturgeon, the First Minister of Scotland and leader of the Scottish National Party. At a time when relationships between Scotland and England are particularly fraught, this insensitive statement is reminiscent of the gaffes of Boris Johnson.


Oh, and yesterday she also announced – and then quickly U-turned – an idea that would limit the pay of public sector workers in the North of England, because it was cheaper to live there.


What I find so odd about both Sunak and Truss is that they keep scoring own goals. Nobody requires them to keep coming up with new policies in niche areas.  Why don’t they just hold on to their original positions, and if asked about anything else, just agree to ‘look at’ it when in power? Johnson was always very good at promising to ‘look at’ things: when he’s not being a limpet, he’s a basilisk. The more you probe Sunak and Truss, the more their shallowness and lack of expertise show through. They’d do well to keep schtum. Or, better still, show some indication that they could, for instance, get a grip on the crisis in the NHS.


What all this demonstrates is how an incompetent and self-promoting leader causes damage far beyond their own term. Johnson’s cabinet was selected purely on one criterion: loyalty to him. The first requirement was to “believe in Brexit” – reminiscent of the historical wars between Protestants and Catholics, where the first thing you ask of a candidate is what their religion is. Among Conservative politicians, it seems that an accusation of not really being a Brexiteer is the worst thing you can say about a candidate. Indeed, that is exactly the charge that her opponents level against Truss, who made cogent arguments for remaining in the EU before the referendum. Like a Protestant required to recant their beliefs or face the flames, she is now reduced to defending Brexit in the strongest possible terms, saying that “predictions of doom have not come true”, as farmers, fishermen, and exporters go out of business, academics leave in droves, and holidaymakers sit in queues at Dover.


It's known that Johnson does not want to give up the top job. I’m starting to wonder if behind all of this is a cunning plan. The people he’s appointed to cabinet are so incompetent that maybe he hopes that, when confronted with a choice between them, the Conservative Party will decide that he looks better than either of them.





Wednesday, 29 June 2022

A proposal for data-sharing that discourages p-hacking

Open data is a great way of helping give confidence in the reproducibility of research findings. Although we are still a long way from having adequate implementation of data-sharing in psychology journals (see, for example, this commentary by Kathy Rastle, editor of Journal of Memory and Language), things are moving in the right direction, with an increasing number of journals and funders requiring sharing of data and code. But there is a downside, and I've been thinking about it this week, as we've just published a big paper on language lateralisation, where all the code and data are available on Open Science Framework. 

One problem is p-hacking. If you put a large and complex dataset in the public domain, anyone can download it and then run multiple unconstrained analyses until they find something, which is then retrospectively fitted to a plausible-sounding hypothesis. The potential to generate non-replicable false positives by such a process is extremely high - far higher than many scientists recognise. I illustrated this with a fictitious example here

Another problem is self-imposed publication bias: the researcher runs a whole set of analyses to test promising theories, but forgets about them as soon as they turn up a null result. With both of these processes in operation, data sharing becomes a poisoned chalice: instead of increasing scientific progress by encouraging novel analyses of existing data, it just means more unreliable dross is deposited in the literature. So how can we prevent this? 

In this Commentary paper, I noted several solutions. One is to require anyone accessing the data to submit a protocol which specifies the hypotheses and the analyses that will be used to test them. In effect this amounts to preregistration of secondary data analysis. This is the method used for some big epidemiological and medical databases. But it is cumbersome and also costly - you need the funding to support additional infrastructure for gatekeeping and registration. For many psychology projects, this is not going to be feasible. 

A simpler solution would be to split the data into two halves - those doing secondary data analysis only have access to part A, which allows them to do exploratory analyses, after which they can then see if any findings hold up in part B. Statistical power will be reduced by this approach, but with large datasets it may be high enough to detect effects of interest.  I wonder if it would be relatively easy to incorporate this option into Open Science Framework: i.e. someone who commits a preregistration of a secondary data analysis on the basis of exploratory analysis of half a dataset then receives a code that unlocks the second half of the data (the hold-out sample). A rough outline of how this might work is shown in Figure 1.

Figure 1: A possible flowchart for secondary data analysis on a platform such as OSF

An alternative that has been discussed by MacCoun and Perlmutter is blind analysis - "temporarily and judiciously removing data labels and altering data values to fight bias and error". The idea is that you can explore a dataset and run a planned analysis on it, but it won't be possible for the results to affect your analysis, because the data have been changed, so you won't know what is correct. A variant of this approach would be multiple datasets with shuffled data in all but one of them. The shuffling would be similar to what is done in permutation analysis - so there might be ten versions of the dataset deposited, with only one having the original unshuffled data. Those downloading the data would not know whether or not they had the correct version - only after they had decided on an analysis plan, would they be told which dataset it should be run on. 

I don't know if these methods would work, but I think they have potential for keeping people honest in secondary data analysis, while minimising bureaucracy and cost. On a platform such as Open Science Framework it is already possible to create a time-stamped preregistration of an analysis plan. I assume that within OSF there is already a log that indicates who has downloaded a dataset. So someone who wanted to do things right and just download one dataset (either a random half, or one of a set of shuffled datasets) would just need to have a mechanism that allowed them to gain access to the full, correct data after they had preregistered an analysis, similar to that outlined above.

These methods are not foolproof. Two researchers could collude - or one researcher could adopt multiple personas - so that they get to see the correct data as person X and then start a new process as person B, when they can preregister an analysis where results are already known. But my sense is that there are many honest researchers who would welcome this approach - precisely because it would keep them honest. Many of us enjoy exploring datasets, but it is all too easy to fool yourself into thinking that you've turned up something exciting when it is really just a fluke arising in the course of excessive data-mining. 

Like a lot of my blogposts, this is just a brain dump of an idea that is not fully thought through. I hope by sharing it, I will encourage people to come up with criticisms that I haven't thought of, or alternatives that might work better. Comments on the blog are moderated to prevent spam, but please do not be deterred - I will post any that are on topic. 

P.S. 5th July 2022 

Florian Naudet drew my attention to this v relevant paper: 

Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022). Protecting against researcher bias in secondary data analysis: Challenges and potential solutions. European Journal of Epidemiology, 37(1), 1–10.

Saturday, 30 April 2022

Bishopblog catalogue (updated 30 April 2022)


Those of you who follow this blog may have noticed a lack of thematic coherence. I write about whatever is exercising my mind at the time, which can range from technical aspects of statistics to the design of bathroom taps. I decided it might be helpful to introduce a bit of order into this chaotic melange, so here is a catalogue of posts by topic.

Language impairment, dyslexia and related disorders
The common childhood disorders that have been left out in the cold (1 Dec 2010) What's in a name? (18 Dec 2010) Neuroprognosis in dyslexia (22 Dec 2010) Where commercial and clinical interests collide: Auditory processing disorder (6 Mar 2011) Auditory processing disorder (30 Mar 2011) Special educational needs: will they be met by the Green paper proposals? (9 Apr 2011) Is poor parenting really to blame for children's school problems? (3 Jun 2011) Early intervention: what's not to like? (1 Sep 2011) Lies, damned lies and spin (15 Oct 2011) A message to the world (31 Oct 2011) Vitamins, genes and language (13 Nov 2011) Neuroscientific interventions for dyslexia: red flags (24 Feb 2012) Phonics screening: sense and sensibility (3 Apr 2012) What Chomsky doesn't get about child language (3 Sept 2012) Data from the phonics screen (1 Oct 2012) Auditory processing disorder: schisms and skirmishes (27 Oct 2012) High-impact journals (Action video games and dyslexia: critique) (10 Mar 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013) Raising awareness of language learning impairments (26 Sep 2013) Good and bad news on the phonics screen (5 Oct 2013) What is educational neuroscience? (25 Jan 2014) Parent talk and child language (17 Feb 2014) My thoughts on the dyslexia debate (20 Mar 2014) Labels for unexplained language difficulties in children (23 Aug 2014) International reading comparisons: Is England really do so poorly? (14 Sep 2014) Our early assessments of schoolchildren are misleading and damaging (4 May 2015) Opportunity cost: a new red flag for evaluating interventions (30 Aug 2015) The STEP Physical Literacy programme: have we been here before? (2 Jul 2017) Prisons, developmental language disorder, and base rates (3 Nov 2017) Reproducibility and phonics: necessary but not sufficient (27 Nov 2017) Developmental language disorder: the need for a clinically relevant definition (9 Jun 2018) Changing terminology for children's language disorders (23 Feb 2020) Developmental Language Disorder (DLD) in relaton to DSM5 (29 Feb 2020) Why I am not engaging with the Reading Wars (30 Jan 2022)

Autism diagnosis in cultural context (16 May 2011) Are our ‘gold standard’ autism diagnostic instruments fit for purpose? (30 May 2011) How common is autism? (7 Jun 2011) Autism and hypersystematising parents (21 Jun 2011) An open letter to Baroness Susan Greenfield (4 Aug 2011) Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011) Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012) The ‘autism epidemic’ and diagnostic substitution (4 Jun 2012) How wishful thinking is damaging Peta's cause (9 June 2014) NeuroPointDX's blood test for Autism Spectrum Disorder ( 12 Jan 2019)

Developmental disorders/paediatrics
The hidden cost of neglected tropical diseases (25 Nov 2010) The National Children's Study: a view from across the pond (25 Jun 2011) The kids are all right in daycare (14 Sep 2011) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Changing the landscape of psychiatric research (11 May 2014) The sinister side of French psychoanalysis revealed (15 Oct 2019)

Where does the myth of a gene for things like intelligence come from? (9 Sep 2010) Genes for optimism, dyslexia and obesity and other mythical beasts (10 Sep 2010) The X and Y of sex differences (11 May 2011) Review of How Genes Influence Behaviour (5 Jun 2011) Getting genetic effect sizes in perspective (20 Apr 2012) Moderate drinking in pregnancy: toxic or benign? (21 Nov 2012) Genes, brains and lateralisation (22 Dec 2012) Genetic variation and neuroimaging (11 Jan 2013) Have we become slower and dumber? (15 May 2013) Overhyped genetic findings: the case of dyslexia (16 Jun 2013) Incomprehensibility of much neurogenetics research ( 1 Oct 2016) A common misunderstanding of natural selection (8 Jan 2017) Sample selection in genetic studies: impact of restricted range (23 Apr 2017) Pre-registration or replication: the need for new standards in neurogenetic studies (1 Oct 2017) Review of 'Innate' by Kevin Mitchell ( 15 Apr 2019) Why eugenics is wrong (18 Feb 2020)

Neuroprognosis in dyslexia (22 Dec 2010) Brain scans show that… (11 Jun 2011)  Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012) Neuronal migration in language learning impairments (2 May 2012) Sharing of MRI datasets (6 May 2012) Genetic variation and neuroimaging (1 Jan 2013) The arcuate fasciculus and word learning (11 Aug 2013) Changing children's brains (17 Aug 2013) What is educational neuroscience? ( 25 Jan 2014) Changing the landscape of psychiatric research (11 May 2014) Incomprehensibility of much neurogenetics research ( 1 Oct 2016)

Accentuate the negative (26 Oct 2011) Novelty, interest and replicability (19 Jan 2012) High-impact journals: where newsworthiness trumps methodology (10 Mar 2013) Who's afraid of open data? (15 Nov 2015) Blogging as post-publication peer review (21 Mar 2013) Research fraud: More scrutiny by administrators is not the answer (17 Jun 2013) Pressures against cumulative research (9 Jan 2014) Why does so much research go unpublished? (12 Jan 2014) Replication and reputation: Whose career matters? (29 Aug 2014) Open code: note just data and publications (6 Dec 2015) Why researchers need to understand poker ( 26 Jan 2016) Reproducibility crisis in psychology ( 5 Mar 2016) Further benefit of registered reports ( 22 Mar 2016) Would paying by results improve reproducibility? ( 7 May 2016) Serendipitous findings in psychology ( 29 May 2016) Thoughts on the Statcheck project ( 3 Sep 2016) When is a replication not a replication? (16 Dec 2016) Reproducible practices are the future for early career researchers (1 May 2017) Which neuroimaging measures are useful for individual differences research? (28 May 2017) Prospecting for kryptonite: the value of null results (17 Jun 2017) Pre-registration or replication: the need for new standards in neurogenetic studies (1 Oct 2017) Citing the research literature: the distorting lens of memory (17 Oct 2017) Reproducibility and phonics: necessary but not sufficient (27 Nov 2017) Improving reproducibility: the future is with the young (9 Feb 2018) Sowing seeds of doubt: how Gilbert et al's critique of the reproducibility project has played out (27 May 2018) Preprint publication as karaoke ( 26 Jun 2018) Standing on the shoulders of giants, or slithering around on jellyfish: Why reviews need to be systematic ( 20 Jul 2018) Matlab vs open source: costs and benefits to scientists and society ( 20 Aug 2018) Responding to the replication crisis: reflections on Metascience 2019 (15 Sep 2019) Manipulated images: hiding in plain sight (13 May 2020) Frogs or termites: gunshot or cumulative science? ( 6 Jun 2020) Open data: We know what's needed - now let's make it happen (27 Mar 2021)  

Book review: biography of Richard Doll (5 Jun 2010) Book review: the Invisible Gorilla (30 Jun 2010) The difference between p < .05 and a screening test (23 Jul 2010) Three ways to improve cognitive test scores without intervention (14 Aug 2010) A short nerdy post about the use of percentiles (13 Apr 2011) The joys of inventing data (5 Oct 2011) Getting genetic effect sizes in perspective (20 Apr 2012) Causal models of developmental disorders: the perils of correlational data (24 Jun 2012) Data from the phonics screen (1 Oct 2012)Moderate drinking in pregnancy: toxic or benign? (1 Nov 2012) Flaky chocolate and the New England Journal of Medicine (13 Nov 2012) Interpreting unexpected significant results (7 June 2013) Data analysis: Ten tips I wish I'd known earlier (18 Apr 2014) Data sharing: exciting but scary (26 May 2014) Percentages, quasi-statistics and bad arguments (21 July 2014) Why I still use Excel ( 1 Sep 2016) Sample selection in genetic studies: impact of restricted range (23 Apr 2017) Prospecting for kryptonite: the value of null results (17 Jun 2017) Prisons, developmental language disorder, and base rates (3 Nov 2017) How Analysis of Variance Works (20 Nov 2017) ANOVA, t-tests and regression: different ways of showing the same thing (24 Nov 2017) Using simulations to understand the importance of sample size (21 Dec 2017) Using simulations to understand p-values (26 Dec 2017) One big study or two small studies? ( 12 Jul 2018) Time to ditch relative risk in media reports (23 Jan 2020)

Journalism/science communication
Orwellian prize for scientific misrepresentation (1 Jun 2010) Journalists and the 'scientific breakthrough' (13 Jun 2010) Science journal editors: a taxonomy (28 Sep 2010) Orwellian prize for journalistic misrepresentation: an update (29 Jan 2011) Academic publishing: why isn't psychology like physics? (26 Feb 2011) Scientific communication: the Comment option (25 May 2011)  Publishers, psychological tests and greed (30 Dec 2011) Time for academics to withdraw free labour (7 Jan 2012) 2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012) Time for neuroimaging (and PNAS) to clean up its act (5 Mar 2012) Communicating science in the age of the internet (13 Jul 2012) How to bury your academic writing (26 Aug 2012) High-impact journals: where newsworthiness trumps methodology (10 Mar 2013)  A short rant about numbered journal references (5 Apr 2013) Schizophrenia and child abuse in the media (26 May 2013) Why we need pre-registration (6 Jul 2013) On the need for responsible reporting of research (10 Oct 2013) A New Year's letter to academic publishers (4 Jan 2014) Journals without editors: What is going on? (1 Feb 2015) Editors behaving badly? (24 Feb 2015) Will Elsevier say sorry? (21 Mar 2015) How long does a scientific paper need to be? (20 Apr 2015) Will traditional science journals disappear? (17 May 2015) My collapse of confidence in Frontiers journals (7 Jun 2015) Publishing replication failures (11 Jul 2015) Psychology research: hopeless case or pioneering field? (28 Aug 2015) Desperate marketing from J. Neuroscience ( 18 Feb 2016) Editorial integrity: publishers on the front line ( 11 Jun 2016) When scientific communication is a one-way street (13 Dec 2016) Breaking the ice with buxom grapefruits: Pratiques de publication and predatory publishing (25 Jul 2017) Should editors edit reviewers? ( 26 Aug 2018) Corrigendum: a word you may hope never to encounter (3 Aug 2019) Percent by most prolific author score and editorial bias (12 Jul 2020) PEPIOPs – prolific editors who publish in their own publications (16 Aug 2020) Faux peer-reviewed journals: a threat to research integrity (6 Dec 2020) Time to ditch relative risk in media reports (23 Jan 2020) Time for publishers to consider the rights of readers as well as authors (13 Mar 2021) Universities vs Elsevier: who has the upper hand? (14 Nov 2021) Book Review. Fiona Fox: Beyond the Hype (12 Apr 2022)

Social Media
A gentle introduction to Twitter for the apprehensive academic (14 Jun 2011) Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011) Will I still be tweeting in 2013? (2 Jan 2012) Blogging in the service of science (10 Mar 2012) Blogging as post-publication peer review (21 Mar 2013) The impact of blogging on reputation ( 27 Dec 2013) WeSpeechies: A meeting point on Twitter (12 Apr 2014) Email overload ( 12 Apr 2016) How to survive on Twitter - a simple rule to reduce stress (13 May 2018)

Academic life
An exciting day in the life of a scientist (24 Jun 2010) How our current reward structures have distorted and damaged science (6 Aug 2010) The challenge for science: speech by Colin Blakemore (14 Oct 2010) When ethics regulations have unethical consequences (14 Dec 2010) A day working from home (23 Dec 2010) Should we ration research grant applications? (8 Jan 2011) The one hour lecture (11 Mar 2011) The expansion of research regulators (20 Mar 2011) Should we ever fight lies with lies? (19 Jun 2011) How to survive in psychological research (13 Jul 2011) So you want to be a research assistant? (25 Aug 2011) NHS research ethics procedures: a modern-day Circumlocution Office (18 Dec 2011) The REF: a monster that sucks time and money from academic institutions (20 Mar 2012) The ultimate email auto-response (12 Apr 2012) Well, this should be easy…. (21 May 2012) Journal impact factors and REF2014 (19 Jan 2013)  An alternative to REF2014 (26 Jan 2013) Postgraduate education: time for a rethink (9 Feb 2013)  Ten things that can sink a grant proposal (19 Mar 2013)Blogging as post-publication peer review (21 Mar 2013) The academic backlog (9 May 2013)  Discussion meeting vs conference: in praise of slower science (21 Jun 2013) Why we need pre-registration (6 Jul 2013) Evaluate, evaluate, evaluate (12 Sep 2013) High time to revise the PhD thesis format (9 Oct 2013) The Matthew effect and REF2014 (15 Oct 2013) The University as big business: the case of King's College London (18 June 2014) Should vice-chancellors earn more than the prime minister? (12 July 2014)  Some thoughts on use of metrics in university research assessment (12 Oct 2014) Tuition fees must be high on the agenda before the next election (22 Oct 2014) Blaming universities for our nation's woes (24 Oct 2014) Staff satisfaction is as important as student satisfaction (13 Nov 2014) Metricophobia among academics (28 Nov 2014) Why evaluating scientists by grant income is stupid (8 Dec 2014) Dividing up the pie in relation to REF2014 (18 Dec 2014)  Shaky foundations of the TEF (7 Dec 2015) A lamentable performance by Jo Johnson (12 Dec 2015) More misrepresentation in the Green Paper (17 Dec 2015) The Green Paper’s level playing field risks becoming a morass (24 Dec 2015) NSS and teaching excellence: wrong measure, wrongly analysed (4 Jan 2016) Lack of clarity of purpose in REF and TEF ( 2 Mar 2016) Who wants the TEF? ( 24 May 2016) Cost benefit analysis of the TEF ( 17 Jul 2016)  Alternative providers and alternative medicine ( 6 Aug 2016) We know what's best for you: politicians vs. experts (17 Feb 2017) Advice for early career researchers re job applications: Work 'in preparation' (5 Mar 2017) Should research funding be allocated at random? (7 Apr 2018) Power, responsibility and role models in academia (3 May 2018) My response to the EPA's 'Strengthening Transparency in Regulatory Science' (9 May 2018) More haste less speed in calls for grant proposals ( 11 Aug 2018) Has the Society for Neuroscience lost its way? ( 24 Oct 2018) The Paper-in-a-Day Approach ( 9 Feb 2019) Benchmarking in the TEF: Something doesn't add up ( 3 Mar 2019) The Do It Yourself conference ( 26 May 2019) A call for funders to ban institutions that use grant capture targets (20 Jul 2019) Research funders need to embrace slow science (1 Jan 2020) Should I stay or should I go: When debate with opponents should be avoided (12 Jan 2020) Stemming the flood of illegal external examiners (9 Feb 2020) What can scientists do in an emergency shutdown? (11 Mar 2020) Stepping back a level: Stress management for academics in the pandemic (2 May 2020)
TEF in the time of pandemic (27 Jul 2020) University staff cuts under the cover of a pandemic: the cases of Liverpool and Leicester (3 Mar 2021) Some quick thoughts on academic boycotts of Russia (6 Mar 2022)

Celebrity scientists/quackery
Three ways to improve cognitive test scores without intervention (14 Aug 2010) What does it take to become a Fellow of the RSM? (24 Jul 2011) An open letter to Baroness Susan Greenfield (4 Aug 2011) Susan Greenfield and autistic spectrum disorder: was she misrepresented? (12 Aug 2011) How to become a celebrity scientific expert (12 Sep 2011) The kids are all right in daycare (14 Sep 2011)  The weird world of US ethics regulation (25 Nov 2011) Pioneering treatment or quackery? How to decide (4 Dec 2011) Psychoanalytic treatment for autism: Interviews with French analysts (23 Jan 2012) Neuroscientific interventions for dyslexia: red flags (24 Feb 2012) Why most scientists don't take Susan Greenfield seriously (26 Sept 2014) NeuroPointDX's blood test for Autism Spectrum Disorder ( 12 Jan 2019)

Academic mobbing in cyberspace (30 May 2010) What works for women: some useful links (12 Jan 2011) The burqua ban: what's a liberal response (21 Apr 2011) C'mon sisters! Speak out! (28 Mar 2012) Psychology: where are all the men? (5 Nov 2012) Should Rennard be reinstated? (1 June 2014) How the media spun the Tim Hunt story (24 Jun 2015)

Politics and Religion
Lies, damned lies and spin (15 Oct 2011) A letter to Nick Clegg from an ex liberal democrat (11 Mar 2012) BBC's 'extensive coverage' of the NHS bill (9 Apr 2012) Schoolgirls' health put at risk by Catholic view on vaccination (30 Jun 2012) A letter to Boris Johnson (30 Nov 2013) How the government spins a crisis (floods) (1 Jan 2014) The alt-right guide to fielding conference questions (18 Feb 2017) We know what's best for you: politicians vs. experts (17 Feb 2017) Barely a good word for Donald Trump in Houses of Parliament (23 Feb 2017) Do you really want another referendum? Be careful what you wish for (12 Jan 2018) My response to the EPA's 'Strengthening Transparency in Regulatory Science' (9 May 2018) What is driving Theresa May? ( 27 Mar 2019) A day out at 10 Downing St (10 Aug 2019) Voting in the EU referendum: Ignorance, deceit and folly ( 8 Sep 2019) Harry Potter and the Beast of Brexit (20 Oct 2019) Attempting to communicate with the BBC (8 May 2020) Boris bingo: strategies for (not) answering questions (29 May 2020) Linking responsibility for climate refugees to emissions (23 Nov 2021) Response to Philip Ball's critique of scientific advisors (16 Jan 2022) Boris Johnson leads the world the number of false facts he can squeeze into a session of PMQs (20 Jan 2022) Some quick thoughts on academic boycotts of Russia (6 Mar 2022)

Humour and miscellaneous Orwellian prize for scientific misrepresentation (1 Jun 2010) An exciting day in the life of a scientist (24 Jun 2010) Science journal editors: a taxonomy (28 Sep 2010) Parasites, pangolins and peer review (26 Nov 2010) A day working from home (23 Dec 2010) The one hour lecture (11 Mar 2011) The expansion of research regulators (20 Mar 2011) Scientific communication: the Comment option (25 May 2011) How to survive in psychological research (13 Jul 2011) Your Twitter Profile: The Importance of Not Being Earnest (19 Nov 2011) 2011 Orwellian Prize for Journalistic Misrepresentation (29 Jan 2012) The ultimate email auto-response (12 Apr 2012) Well, this should be easy…. (21 May 2012) The bewildering bathroom challenge (19 Jul 2012) Are Starbucks hiding their profits on the planet Vulcan? (15 Nov 2012) Forget the Tower of Hanoi (11 Apr 2013) How do you communicate with a communications company? ( 30 Mar 2014) Noah: A film review from 32,000 ft (28 July 2014) The rationalist spa (11 Sep 2015) Talking about tax: weasel words ( 19 Apr 2016) Controversial statues: remove or revise? (22 Dec 2016) The alt-right guide to fielding conference questions (18 Feb 2017) My most popular posts of 2016 (2 Jan 2017) An index of neighbourhood advantage from English postcode data ( 15 Sep 2018) Working memories: A brief review of Alan Baddeley's memoir ( 13 Oct 2018)

Tuesday, 12 April 2022

Book Review. Fiona Fox: Beyond the Hype

If you're a scientist reading this, you may well think, as I used to, that running a Science Media Centre (SMC) would be a worthy but rather dull existence. Surely, it's just a case of getting scientists to explain things clearly in non-technical language to journalists. The fact that the SMC was created in part as a response to the damaging debacle of the MMR scandal might suggest that it would be a straightforward job of providing journalists with input from experts rather than mavericks, and helping them distinguish between the two. 

I now know it's not like that, after being on the Science Media Centre's panel of experts for many years, and having also served on their advisory committee for a few of them. The reality is described in this book by SMC's Director, Fiona Fox, and it's riveting stuff.

In part this is because no science story is simple. People will disagree about the quality of the science, the meaning of the results, and the practical implications. Topics such as climate change, chronic fatigue syndrome/ME and therapeutic cloning elicit highly charged responses from those who are affected by the science. More recently, we have found that when a pandemic descends upon the world, some of the bitterest disagreements are not between scientists and the media, but between well-respected, expert scientists. The idea that scientists can hand down tablets of stone inscribed with the truth to the media is a fiction that is clearly exposed in this book.

Essentially, the SMC might be seen as acting like a therapist in the midst of a seriously dysfunctional family where everyone misunderstands everyone else, and everyone wants different things out of life. On the one hand we have the scientists. They get frustrated because they feel they should be able to make exciting new discoveries, with the media then helping communicate these to the world. Instead, they complain that the media has two responses: either they're not interested in the science, or they want to sensationalise it. If you find a mild impact of grains on sexual behaviour in rats, you'll find it translated into the headline 'Cornflakes make you impotent'.

On the other hand, we have the media. They want a good story, but find that the scientists are reluctant to talk to them, or want total control of how the story is presented. In the worst case, scientists are prima donnas who want days or weeks to prepare for a media interview and will then shower the journalist with detailed information that is incomprehensible, irrelevant, or both. When the public desperately needs a clear, simple message, the scientists will refuse to deliver it, hedging every statement.

Fox has worked over the years to challenge these stereotypes: journalists do want a good story, but the serious science journalists want a true story, and are glad of the opportunity to pose questions directly to scientists. And many scientists do a fantastic job of explaining their subject matter to a non-specialist audience. In the varied chapters of the book, Fox is an irrepressible optimist, who keeps coming back to the importance of having scientists communicating directly with the media. Her optimism is not founded in ignorance: she knows exactly how messy and complicated science can be. But she persists in believing that more good is done by communicating what we know, warts and all, rather than pretending that uncertainties and disagreements do not exist.

The role of the SMC is, however, complicated by further factions. The dramatis personae includes two other groups. First, there are science press officers, who are appointed by institutions to help scientists promote their work, and then there are government officials and civil servants, who are concerned with policy implications of science.

In her penultimate chapter, Fox bemoans the fact that the traditional press officer - passionate about science and viewing themselves as "purveyors of truth and accuracy" - is a dying breed. There remain notable exceptions, but all too often science communication has become conflated with a public relations role: pushing a corporate message, defending the institutional reputation, and even using scientific discoveries as a marketing tool. Fox notes a 2014 survey of exaggerated science reports in the media that concluded: "Exaggeration in news is strongly associated with exaggeration in press releases." I had been one of those scientists who thought the media were mostly to blame for over-hyped science reporting, but this study showed that journalists are often recycling exaggerated accounts handed to them by those speaking for the scientists.

But the problems posed by scientists, journalists and press officers are trivial compared to the obstacles created by those involved in policy. They want to use science when convenient, but also want to exert control over which aspects of science policy gets talked about. Scientists working for government-funded organisations are often muzzled, with explicit instructions not to talk to the media. One can see that this cautious approach, attempting to control the message and keep things simple, puts many civil servants and government scientists on a collision course with Fox, whose view is: "Explaining preliminary and contradictory science is messy: that should not be seen as a failure of communications".

A refreshing aspect of Fox's account is that she does not brush aside the occasions when the SMC - or she personally - may have handled a situation badly. Of course, it's easy to point the finger of blame when something does go horribly wrong, and Fox has come under fire on many occasions. Rather than being defensive, she accepts that things might have been done differently, while at the same time explaining the logic of the decisions that were taken. This is in line with my memories of meetings of the SMC advisory committee, where there were frequent post mortems - "this is how we handled it; this is how it turned out; should we have done it differently?" - with frank discussions from the committee members. When you are working in contentious areas where things are bound to blow up every now and again, this is a sensible strategy that helps the organisation learn and develop. I'm glad that after 20 years, the ethos of the SMC is still very much on the side of open, transparent communication between scientists and the media.  

Fox, Fiona (2022) Beyond the Hype: The Inside Story of Science's Biggest Media Controversies. London: Elliott and Thompson Ltd.

Sunday, 6 March 2022

Some quick thoughts on academic boycotts of Russia



In response to the dramatic developments in Ukraine, several instances of academic boycotts of Russia have been proposed or implemented.  These are a few of the initiatives I have heard about on Twitter:


European Federation of Academies of Sciences and Humanities (ALLEA) has suspended the Russian Academy of Sciences and the National Academy of Sciencesof Belarus


The British Psychological Society has supported a call from Ukrainian psychologists for the Russian Psychological Society to be suspended from the European Federation of Psychologists Associations (EFPA)


The Australian National University suspended all institutional ties with Russian institutions.


Yesterday, Chris Chambers, who is a journal editor, asked people's views on whether academic journals should be considering contributions from authors based in Russia.


Responses were pretty polarised. I have strong views against all these kinds of academic boycott, which have not really been changed by the numerous, well-argued points made by those responding to Chris, and so I thought I'd set out my position here.


I should start by saying that the disagreement here is not about the rights and wrongs of the war in Ukraine. I take as a basic premise that the invasion of Ukraine is an abominable atrocity, that, as its instigator, Vladimir Putin is guilty of war crimes, and that we all should do all we can to stop the war and ensure Putin is brought to justice.


At a national level, sanctions are imposed by governments to put pressure on Russia that, on the one hand, serves a symbolic function of demonstrating disapproval, and on the other hand serves a practical function of weakening the Russian economy, in the hope this might help bring the invasion to an end without starting World War 3.  They are not undertaken lightly, as it is recognised that many innocent people who play no part in the invasion will be adversely affected - both Russians who protest the invasion and people in the countries who are applying the sanctions, who may have to pay higher prices, or suffer shortages of goods.


Other organisations, notably sports organisations, have taken the line of moving major sporting events away from Russia. I fully support such moves, because they will have a financial impact on Russia, as well as denying Putin an opportunity to act as a host, which is a role that carries considerable prestige as well as opportunities for positive publicity. I also agree with banning Russian athletes from sporting events where they are explicitly representing their country - in effect, including these athletes creates the impression that Russia is a normal member of the sporting community. So these sanctions affecting sports have a very visible symbolic role as well as a potential financial one.  That makes the cost-benefit ratio of the actions seem worthwhile, despite the unfairness to individual athletes.


I see academic interactions differently in terms of the cost-benefit ratio.  I don't think there'd be any financial impact to Russia of Chris or other editors refusing to consider papers by Russian authors, and the symbolic impact is likely to be pretty small. I doubt most people would notice that such a sanction had occurred, and, even if Putin was aware of it, I can't see him getting particularly upset about it.  In saying this, I'm not arguing that academics are 'special' - just that the practical impact of adopting sanctions against them would be close to zero, except for the damage to the sanctioned individuals. Add to this the fact that academics are typically among the early casualties of dictators, who dislike them for their tendency to criticise, and I think the case for this kind of academic sanction is very weak.  


In the responses to Chris, there are many people, on both sides, putting forward 'what about' arguments. So, for instance, what about a small Russian trader, innocent of any war crimes, whose livelihood is blighted by sanctions that affect her business? For me the question is whether such sanctions have an effect that is worthwhile - in terms of its visibility and economic impact -  given that all sanctions will trap the deserving as well as the undeserving in their net?  So this needs a response on a case-by-case basis, rather than treating it as a generic principle to be adopted in all sectors.


Another 'what about' issue that is raised concerns countries other than Russia. Should we start to sanction academics from any government that behaves inhumanely? It would be quite a long list, and would be difficult to draw the line.  It's worth noting that on the same principle, we could well find ourselves sanctioned by academics from other countries who regard the UK's colonial past and/or escapades in the Middle East worthy of censure.  The only reason we don't have to worry about that is because we happen to have considerable power over the machinery of academic publishing.


So my answer here again is that in general, academics should only consider sanctions as an extreme resort, given that they are a blunt instrument that can have serious unintended consequences. The economic sanctions currently imposed on Russia by governments are unusually effective because they have a big effect on many sectors, and we are seeing a co-ordinated activity of many nations, as well as nations prepared to take a financial hit themselves.  That's not the case for academic sanctions.


There are many other ways we can express our support for Ukraine. Moves are already afoot among learned societies to help Ukrainian academics who have had to leave the country, and also Russian academics who have put themselves in harm's way by speaking out against the invasion. Personally, I think the most useful thing I can do is to donate money to that cause as well as to other causes that will strengthen Ukraine's position in the conflict, rather than calling for symbolic gestures.


I'm prepared to listen to counter-arguments. It's clear this is one of those issues that is pretty divisive among people of good will who are usually on the same side of political debate.  To help cope with a tsunami of spam, comments on this blog are moderated, but I will monitor them and publish those that contribute (respectfully) to this discussion.



Sunday, 30 January 2022

Why I am not engaging with the Reading Wars

I’m not sure whether to feel flattered or irritated by Jeff Bowers’ latest blogpost, where he expresses his disappointment in my failure to support his critique of research on phonics intervention.

The short explanation, as I already conveyed to Jeff, is that I'm busy with other stuff. I’m aware that there’s a pretty intemperate debate going on at present about the value of phonics in teaching children to read, with various people writing lengthy critiques of one another’s positions. So clearly it’s a complex and murky situation. The last thing this needs is for someone else, me, who hasn’t looked in depth at the relevant research, to weigh in with an opinion.

As background: I am a (soon-to-be retired) researcher who has never worked on reading intervention, though I do work on topics tangentially related to this. I'm currently finishing off a big project on cerebral lateralization for language. I’m also involved in a couple of other projects unrelated to reading. I am a  hands-on researcher, so am heavily involved in designing, analysing and writing up research. I currently work a 4 day week, and these three projects, plus regular reviewing, giving talks, etc, take up all of my time.

One of the fun parts of my job is that I am my own boss and, as far as time allows, I can pursue odd topics that take my fancy. I sometimes put my head above the parapet and engage in, or even initiate, academic disputes about research. These consume much time and emotional energy, so I choose carefully which battles I want to get involved in. In particular, I think hard about what can be gained - I do get passionate when something is wrong, especially where someone is behaving badly and deliberately distorting science for some nefarious ends. In such cases I will get engaged to try and put things right.

The reading wars don't look like that to me. They actually look rather like the equally heated debates going on among epidemiologists and public health experts around Covid – another topic on which I don’t’ speak out because I lack expertise. The most I’m prepared to offer is a general suggestion about perspective that might be helpful in achieving some truth and reconciliation.

In both Covid and reading wars there are arguments about research evidence and arguments about policy. Typically, there isn't really good research on which to base important decisions, but we don’t have the luxury of waiting for such studies to be done. Doing nothing is in itself a choice, for which, in the case of an epidemic, we know the consequences are likely to be bad. So it’s all about making the best decisions we can, given the patchy evidence that we have – a point nicely made by Trisha Greenhalgh in the context of COVID-19. 

The current debates have highlighted the shortage of well-designed studies to show which teaching method is most effective for which children at which age. As with medical interventions, studies also need to evaluate potential costs and harms – something which was neglected in the past, but which is relevant given that some of the disagreements are about whether particular approaches can be counterproductive.

But, as with Covid, policy decisions have to be pragmatic. This was the remit of the 2006 Rose Report, which attempted to synthesise evidence and expert opinion on reading intervention and came out recommending use of synthetic phonics. Overturning the status quo to radically change how reading is taught would only be reasonable if there was some other intervention approach for which there was better evidence of effectiveness – so much so that the costs of making a change would be justified. Maybe there is, but it hasn’t come up on my radar. 

My apologies to all of those engaged in current discussions whose contributions I have ignored. I hope this blogpost will suffice to explain why.

Thursday, 20 January 2022

Boris Johnson leads the world the number of false facts he can squeeze into a session of PMQs


The first weeks of 2022 have had quite enough drama to last us the whole year. Prime Minister’s Questions (PMQs) on Wednesdays has developed a must-watch quality that was previously reserved for the final of the Great British Bake-off. I find myself studying the oratorical approach adopted by Boris Johnson. Previously, I analysed his strategies for (usually not) answering questions in a session of a Select Committee. In PMQs the style is different. It's a gladiatorial context where the goal is to earn howls of approval for trouncing your opponent.

Last week we had the unusual sight of a remorseful Boris Johnson, trying to convince the nation that he was truly, truly sorry for allowing parties to take place on his watch, during a period when we had extremely severe lockdown rules that were preventing people seeing dying relatives or attending funerals. It was one of those apologies where you sensed he was sorry at having been found out, rather than for having done something wrong. But he clearly hoped this would be enough to settle the matter so we could put it behind us – at least until a whitewashing report is published. He had not reckoned with the Machiavellian cunning of his ex-advisor Dominic Cummings, who is drip-feeding damaging information to the media. This started a new story at the weekend, after Cummings contradicted Johnson’s denial that he knew about a party. So I wondered if we would have Remorse, the Sequel at yesterday’s PMQs. 

Johnson, however, adopted the very strategy that he had previously learned from Cummings – brazen it out. So we had the bizarre sight of a Prime Minister who has lost confidence of his own party because they no longer trust what he says, doubling down and telling yet more lies to convince everyone that this is an Absolutely Splendid Government, and he is a the best chap to lead them. (My thanks to Hansard for this transcript of the debate). 

Here is a selection of some of the more questionable claims (there is more, but I have a day job!):

 “We are building 40 new hospitals.” Cf this analysis by FullFact.  One might add that the biggest problem faced by the NHS is lack of staff, rather than buildings.
We have also been cutting crime by 10% and putting 11,000 more police officers out on the streets” – I guess this is an improvement on previous claims of 20,000 more police officers, but it’s been debunked as misleading, given that this is reinstating police that were previously cut.  
We now have the fastest-growing economy in the G7 and GDP is now back up above pre-pandemic levels.” Here, Google just led me to a Government report from the previous day which stated: “In Q3 2021, UK GDP grew by 1.1% compared with the previous quarter (Q2 2021), while GDP rose by 2.2% in the Eurozone and 0.6% in the US. Compared to the pre-pandemic peak, UK GDP in Q3 2021 was 1.5% lower, compared with 1.1% lower in Germany and 0.3% lower in the Eurozone. In the US GDP was 1.4% higher. 
Thanks to the efforts of people across Whitehall, this country is now capable of producing 80% of our own PPE.” I couldn’t find recent data that might be the basis for this claim, but talking up their record on PPE as a source of pride seems a rash move by a Government whose shambolic handling of this issue was documented in a report by the Public Accounts Committee in Feb 2021:  “...its failure to be transparent about decisions, publish contracts in a timely manner or maintain proper records of key decisions left it open to accusations of poor value for money, conflicts of interest and preferential treatment of some suppliers, and undermines public trust in government procurement and the use of taxpayers’ money. Between February and July 2020, the Department of Health and Social Care spent over 12 billion on 32 billion items of PPE. We are concerned that the Department has so far identified items worth hundreds of millions of pounds which are unusable for their intended purpose, putting the efficient use of taxpayers’ money at further risk.”
As Daisy Cooper noted on Twitter, Johnson also repeated the claim that the vaccine rollout would not have been possible except for Brexit – a claim that has been debunked more than once 
Then there was this extraordinary claim: “I will tell you what this Government have been doing to look after the people of this country throughout this pandemic and beyond. We have been cutting the cost of living and helping them with the living wage. We have been cutting taxes for people on low pay. We have been increasing payments for people suffering the costs of fuel- “ People don’t need to consult Google to challenge this rosy vision of life in the UK. We can simply take use of foodbanks as an indicator of how many people are living in serious poverty. We had barely heard of foodbanks before this Government came into power – now more and more people depend on them just to exist. Not only is the cost of living going up, Jack Monroe noted that the way it is computed underestimates the impact on the poorest 
No, Prime Minister, you can’t talk your way out of this one. People are worried about what the future holds, but they are not going to reject the evidence of their eyes and ears