A few days ago, I saw this post on LinkedIn:
How does the gut microbiome shape early brain development? That’s what FORM, a new $50 million programme from Wellcome Leap, aims to answer. Critically, it wants to identify the role of the microbiome in autism and other neurological disorders. Applicants from universities, companies and non-profits are invited to submit project proposals by 14 November.
The full programme announcement for FORM (Foundations of a Resilient Microbiome) that you can download here hops around citing various references that indicate the microbiome is important for early development and can be influenced by factors such as antibiotics. So far, so uncontroversial. But then the topic of autism is introduced.
First we hear that autism diagnoses have increased. Then, a section devoid of references states:
Many have attributed this increase to expanded surveillance, broadening of diagnostic categories (to include milder autism-related difficulties), or increased public awareness. While all are true, the significance of the increase suggests other rising risk factors may also be contributing.
We then are told that this can't be due to genes because they are pretty stable in populations, and so we seem led remorselessly to the conclusion that it must be an environmental factor, and what better culprit could there be than the microbiome.
If I was going to predicate a $50 million research program on that premise, I'd do a bit more research into those studies on the increase in diagnosis. Diagnostic criteria have changed radically, so children who would have in the past had other diagnoses, or no diagnosis, are now encompassed within autism. Furthermore, there is wider understanding of autism, and a diagnosis can bring with it educational support, which can be a reason why parents will seek a diagnosis. Here's a simple explainer that I wrote in 2012, and subsequent studies by Lundström et al (2015), Cardinal et al (2016) and Zeidan et al (2022).
The fragments of supportive evidence that are provided for the autism/microbiome link seem cherrypicked and are not impressive. At least one claim seems just plain wrong: "Babies exposed to antibiotics in the first 6 months of life may be twice as likely to develop ASD as those exposed later." The cited paper by Azad et al (2016) doesn't mention autism and I when I searched for another source, I found a solid-looking study claiming no association. Other cited results are the kinds of findings that you get if you test for so many associations that some are bound to come up by chance. The handful of animal model studies that are cited have been criticised on methodological grounds.
Things go more seriously off the rails when specific quantitative goals are set for the program:
Autism currently affects about 3.2% of children. To identify what proportion of these cases may be attributable to gut microbiome dysfunction, we will need objective biomarkers that can detect the dysfunction with high accuracy (balanced accuracy >90%). Establishing this will require a large cohort — likely more than 15,000 children — to ensure statistical power. With that sample size, we can reliably estimate whether microbiome dysfunction accounts for as much as 50% of ASD cases (around 1.6% of all children) or as little as 10% (about 0.3%).
Three things about this:
- In a footnote it is noted that "Severe autism with an established genetic origin (about 10–20%) and mild/ moderate ASD (~40%) fall outside the scope of this program" - these estimates don't seem to take that into account. And it's not clear if the databases that will be used for the analysis actually allow one to distinguish autism subtypes.
- You can't establish causality from observational data. As has been shown by Yap et al (2021), the microbiome is influenced by specific dietary preferences of autistic children.
- It is assumed that the lower bound is that microbiome dysfunction explains 10% of cases. This shows remarkable commitment to a causal hypothesis that has no solid evidence: a realistic lower bound would be zero.
As regards the plans in "Thrust 2" to have a diagnostic set of biomarkers that will predict severe autism-related difficulties, there are so many issues here, that I recommend reading previous blogposts I wrote on this topic, here, and here. In brief, screening is only effective if there is a strong association between biomarkers and outcomes and if the biomarker measures are stable. Even if those conditions are met, if the base rate of the condition (autism) is low, you will be overwhelmed with false positives.
I was surprised that a reputable funding body was associated with this program, so I wanted to find out more about Wellcome LEAP. They are a U.S.-based non-profit organization founded by the Wellcome Trust that:
builds bold, unconventional programs, and funds them at scale. Programs optimized to deliver breakthroughs in human health over 5 – 10 years and demonstrate seemingly impossible results on seemingly impossible timelines.
No doubt I'm too conventional, but I'm nervous of "seemingly impossible" things. If they are claimed, I am suspicious, especially if this occurs on "seemingly impossible timelines".
Reading on, I can see lots of things to like about LEAP. The idea is to cut time spent in bureaucratic processes of setting up grants, and to bring together networks of researchers from different institutions and different disciplines who can work together to solve problems that involve large and complex datasets, and check generalisability of findings. This kind of international collaborative approach that involves diverse populations is a definite bonus of LEAP.
The worrying bit was the emphasis on speed - especially since this was coupled with an expectation that the results would be commercialised.
This is clarified here:
Wellcome Leap anticipates that it will normally further our mission (and the organization’s mission) to commercialize the results of Wellcome Leap-funded research. If we determine that a Performer’s organization is not making appropriate efforts to further commercialization, either itself or through a third party (e.g., a licensee), we have the option to request a meeting and a remedial commercialization plan to address the issue.
Given that I have in the past written in favour of slow science, it's perhaps not surprising that this funding model doesn't appeal to me. The thing is that not all delays in scientific progress are down to bureaucracy or timidity. A major obstacle to progress is time wasted trying to build on prior research findings that prove to be illusory.
Science should be cumulative, which means we should be able to proceed with confidence and assume that published research is robust. The likelihood of this being the case is low if we are dealing with complex multidimensional systems, where the temptation is to just hunt around until we find something that looks exciting, embellish it with a few statistical credentials and claim we have a novel result. As Chin (2025) has argued, when put under pressure to deliver speedy results, scientists may be forced into a position of hyping their findings, cutting corners, and reporting only favourable results.
I was frankly dismayed to read that in her prior role as Program Director for the Wellcome Leap “First 1000 Days” (1kD) initiative, the Program Director of FORM:
led the delivery of multiple new product opportunities to improve cognitive development in the first 1000 days of a child’s life — including a breakthrough microbiome-directed diagnostic and therapeutic now positioned for commercialization
The 1kD initiative was funded in summer of 2021. So it seems that in four years a microbiome-directed "diagnostic and therapeutic" has been developed and validated. I'm afraid that without solid published evidence, this looks like NeuroPointDX all over again.
Wellcome LEAP programs are focused around "What If" questions. My question is "What if there is no association between autism and microbiome dysfunction?" All three "thrusts" of this program depend on there being an association. My suggestion is to set aside 1% of the funds for this program for pre-registered replications of the studies that are cited as foundational for the research. I know the idea is to fund high-risk, unconventional research, but a lot of time and money could be saved by checking that the foundations are solid before building an edifice on this premise.
Comments on this blog are moderated so may take time to appear. In general, anonymous comments are not approved.
Posts
