In my previous blogpost, I discussed why many scientists can't take seriously the lofty ideals expressed in Trump's plans for Gold Standard Science, even though the basic principles seem excellent. Given the current Republican administration's catastrophic track record in undermining US science, their demands for high standards ring hollow. It appears that demands for Gold Standard Science will be weaponised against types of science they don't like.
Today I provide evidence from the opposite side of the fence. If the Trump administration was really serious about Gold Standard Science, then they should lead by example and showcase projects that are impeccably rigorous, open and transparent.
And what better place to start than with research on autism, which has been an obsession of Robert F Kennedy Jr (RFK), the 26th United States Secretary of Health and Human Services? The first inkling of what was to come was in April this year when the media reported that he had planned to find the cause of autism by September. Then, on May 27th, NIH announced a new Autism Data Science Initiative, which appears to relate to JFK's plans.
I watched with interest a video that was released on 10th June explaining the background of the project. The NIH staff presenting the video gave an evidence-based and balanced account of what is known about the etiology of autism, which was described as complex and multifactorial, involving different types of genetic causes which may interact with environmental factors. In considering the increase in autism diagnosis over time, it was noted that changes in diagnostic criteria, and the need for a diagnosis for access to services were implicated. They implicitly accepted, however, the notion that these factors were insufficient to explain the increase, and so we needed research on other factors.
They then explained how to apply for a share of the $50 million allocated to the initiative, which seemed designed to encourage machine learning approaches to data mining of autism-relevant datasets "to explore the contribution of genetic and non-genetic factors to the causes of autism and or to identify patterns associated with intervention outcomes and the use of services for autism."
A startling feature was that submissions had to be in by 27th June, one month after the scheme was announced, and 17 days after the instructional video was posted. The earliest start date was 1st September, perhaps prompted by RFK's idea of having autism etiology done and dusted by then.
You might wonder how this works with NIH's review process: the answer is that it doesn't. The funding mechanism is an Other Transaction: "an assistance mechanism that is not a grant, contract, or cooperative agreement" and the proposal does not undergo traditional NIH review, but instead is subject to "an objective scientific review. This ensures the assessment of scientific or technical merit of applications by individuals with knowledge and expertise equivalent to that of the individuals whose applications of support they are reviewing". Though, so far, it seems that the top US autism researchers have not been consulted on this scheme. Furthermore, the phrase "Autism Data Science Initiative" isn't mentioned in the massive report from the Committee on Appropriations that was published on July 31st. So this has a decidedly ad hoc feel to it.
This hurried process does not seem an optimal way to foster Gold Standard Science, which requires thought and care to go into research plans. There seem two possible explanations for this rushed approach. Either those who devised the scheme are so ignorant that they don't understand how long it takes to develop a strong research proposal, or they really don't want anyone to apply to the scheme other than specific cronies who will do their bidding.
I had thought we might have to wait until the end of September when the successful grants are announced to see who the lucky grant recipients would be. From this video clip of a recent Cabinet meeting, however, it seems that the research has already been done, results are in and will be announced next month!
Donald Trump and Jay Bhattacharya can talk about Gold Standard Science as much as they like: scientists can see for themselves this travesty of research process, which indicates that, when it comes to their own studies, those in power will know the answer before the data is in. No transparency, no pre-registration, no open data and code, no communication of error and uncertainty, no skepticism of findings or attempt to falsify hypotheses, and no impartial peer review. Truly this is Tinsel Standard Science.
Posts
No comments:
Post a Comment