A new rule that purports to 'Strengthen Transparency in Regulatory Science' has now been proposed - ironically, at a time when the EPA is being accused of a culture of secrecy regarding its own inner workings. Anyone can comment on the rule here: I have done so, but my comment appears to be in moderation, so I am posting it here.
Dear Mr Pruitt
re: Regulatory
Science- Docket ID No. EPA-HQ-OA-2018-0259
The proposed rule,
‘Strengthening transparency in regulatory science’ brings together two strands
of contemporary scientific activity. On the one hand, there is a trend to make
policy more evidence-based and transparent. On the other hand, there has, over the
past decade, been growing awareness of problems with how science is being done,
leading to research that is not always reproducible (the same results achieved
by re-analysis of the data) or replicable (similar results when an experiment
is repeated). The proposed rule by the Environmental Protection Agency (EPA)
brings these two strands together by proposing that policy should only be based
on research that has openly available public data. While this may on the
surface sound like a progressive way of integrating these two strands, it rests
on an over-simplified view of how science works and has considerable potential
for doing harm.
I am writing in a
personal capacity, as someone at the forefront of moves to improve
reproducibility and replication of science in the UK. I chaired a symposium at
the Academy of Medical Sciences on this topic in 2015; this was jointly
organised with UK major funders: Wellcome Trust, Medical Research Council and
Biotechnology and Biological Science Research Council (https://acmedsci.ac.uk/policy/policy-projects/reproducibility-and-reliability-of-biomedical-research).
I am involved in training early career researchers in methods to improve
reproducibility, and I am a co-author of Munafò, M. R et al (2017). A manifesto for reproducible science.
Nature Human Behavior, 1(1: 0021). doi:10.1038/s41562-016-0021. I would welcome
any move by the US Government that would strengthen research by encouraging
adoption of methods to improve science, including making analysis scripts and data
open when this is not in conflict with legal/ethical issues. Unfortunately,
this proposal will not do that. Instead, it will weaken science by drawing a
line in the sand that effectively disregards scientific discoveries prior to
the first part of the 21st century when the importance of open data started to
be increasingly recognised.
The proposal ignores a
key point about how scientific research works: the time scale. Most studies
that would be relevant to EPA take years to do, and even longer to filter
through to affect policy. Consequences for people and the planet that are
relevant to environmental protection are often not immediately obvious: if they
were, we would not need research. Recognition, for instance, of the dangers of
asbestos, took years because the impacts on health were not immediate. Work
demonstrating the connection occurred many years ago and I doubt that the data
are anywhere openly available, yet the EPA’s proposed rule would imply that it
could be disregarded. Similarly, I doubt there is open data demonstrating the
impact of lead in paint or exhaust fumes, or of pesticides such as DDT: does
this mean that manufacturers would be free to reintroduce these?
A second point is that
scientific advances never depend on a single study: having open scripts and
data is one way of improving our ability to check findings, but it is a
relatively recent development, and it is certainly not the only way to validate
science. The growth of knowledge has always depended on converging evidence
from different sources, replications by different scientists and theoretical
understanding of mechanisms. Scientific facts become established when the
evidence is overwhelming. The EPA proposal would throw out the mass of
accumulated scientific evidence from the past, when open practices were not
customary – and indeed often not practical before computers for big data were
available.
Contemporary
scientific research is far from perfect, but the solution is not to ignore it,
but to take steps to improve it and to educate policy-makers in how to identify
strong science; government needs advisors who have scientific expertise and no
conflict of interest, who can integrate existing evidence with policy
implications. The ‘Strengthening Transparency’ proposal is short-sighted and
dangerous and appears to have been developed by people with little
understanding of science. It puts citizens at risk of significant damage – both
to health and prosperity - and it will
make the US look scientifically illiterate to the rest of the world.
Yours sincerely
D. V. M. Bishop
FMedSci, FBA, FRS
Posts
Excellent comment.
ReplyDeleteUnfortunately I think the Scott Pruitt's intent is to remove as much valid research from the decision process as is possible. I think he and others would like it to be as completely fact-free as possible.
As you point out, much of the data for older research will be unavailable. Other research may have used propritary data under a non-release agreement.
In some cases, confidentiality laws, say in the EU may make it illegal to supply the data, especially to another government.
The anti-science crowd are cunning. Incredibly short-sighted but cunning.
I think the comment makes the assumption that the EPA can only make decisions based on high quality science, this isn't the case. The idea is to make the decision making process more transparent, which would not in any way stop the implementation of rules to control potential threats. In the use of evidence the proposals use the phrase "Where available and Appropriate". I think considering the way in which the EPA claimed that their decisions were based on evidence, which was kept secret with spurious justifications and then lost made these changes inevitable. He has also got rid of people who had financial interests in the decisions. Both of these things are consistent with the principles of scientific decision making. There is nothing to stop the use of past research if the reasoning behind its use is explicit.
ReplyDelete