The National Children’s Study: a view from across the pond

The National Institutes of Health, the major government funding agency for medical research in the US, is in trouble. Although a threat of a serious funding cut was largely averted, funding for 2011 is nevertheless $260 million less than for 2010. It is predicted that only 1 in 6 grant applications will be funded this year, the lowest level yet. Such low funding levels are not only disappointing for researchers with good ideas, they also are inefficient, because scientists end up spending more time writing unsuccessful grant proposals than in doing research.

So it gets interesting to see what does get funded. NIH has a great website where you can find out how they spend their money. I came across it when doing a study on the amount of funding going to research on different neurodevelopmental disorders. I revisited the site recently to look in more detail at the kinds of study that get funded in this general area, and I had a bit of a surprise. There were huge sums of money going to something called the National Children’s Study. I added them up and they came to around $500 million.

More Googling revealed that the National Children’s Study is a longitudinal cohort study that plans to follow 100,000 children from the prenatal period to 21 years of age,  to investigate environmental and genetic influences on health and development. There’s no doubt that big longitudinal cohort studies have provided invaluable data in the past; they can provide samples that are large enough to detect small effects, and if the sample is representative of the population, they can be used to estimate prevalence of different conditions. They are essentially correlational, though, and therefore less conclusive than experimental studies for establishing cause and effect. 

The cost of such a large-scale enterprise is bound to be high, but just how high is acceptable? The National Children’s Study website indicated that the spend to date was actually higher than the figure I had from the NIH source - $608 million, with the bulk of the expenditure, $553 million, since 2007 when the Implementation phase of the project began. And it is clear that the future spend will be higher still, as the cost of following up children and assessing their development is taken into account. The total cost was estimated in 2004 to be $2.7 billion. 

The background to the study is clearly explained in an article by Nigel Paneth. During the Clinton presidency The Children's Act of 2000 was passed, instructing the NIH and other federal agencies to “. . . plan, develop, and implement a prospective cohort study, from birth to adulthood, to . . . incorporate behavioral, emotional, educational, and contextual consequences to enable a complete assessment of the physical, chemical, biologic, and psychosocial environmental influences on children's well-being . . . .” As Paneth drily pointed out, this directive was not accompanied by any funding during the 7 years of the Bush administration, so efforts were expended instead on planning and piloting the study, in anticipation of future funding. However, too much planning can be a bad thing, and “as a ship lying in port too long becomes encrusted with barnacles, so the National Children's Study became weighted down with a myriad of measures and instruments” (Paneth, 2010). But then, salvation. Under the Obama administration, the study returned to the political agenda, and $200 million per annum was allocated for 2009-2010. But problems remain. In an article entitled “Saving the National Children’s Study”, David Savitz and Roberta Ness argued that the study is in trouble. There has been debate about the best methods of recruitment of participants, and about the organizational structure of the project. “There are multiple advisory groups representing the lead federal agencies, the extramural research community; an executive committee consisting of selected Center investigators; and oversight by the National Children’s Study Program Office, the Director of NICHD, and the Director of NIH—yet how the major decisions affecting the future of the National Children’s Study will be made remains unclear.” (Savitz & Ness, 2010). Despite the huge sums of money allocated to the project, there is still uncertainty about methods. “To decide where the study should be done and who is capable of doing it, one has to know what the study will entail. The lack of a clear protocol has led to an extended, expensive, and not-fully informative pilot phase.” (ibid).

Of course you can’t know in advance exactly how longitudinal cohort data will be used: one of the benefits of previous cohort studies has been that they provide a resource to allow investigators to test their ideas against existing data, rather than having to collect a new dataset. For example, in the UK, the Avon Longitudinal Study of Parents and Children, which started in the 1990s, is now being used to investigate hypotheses about genetic and environmental influences on behaviour.  When the project started, it wasn't possible to anticipate the developments in genetics, but the prescience of the researchers in gathering DNA samples mean that the data are invaluable, especially since a wealth of environmental and psychological measures are also available. Nevertheless, even allowing for such uncertainty the vagueness of the Hypotheses section of the National Children’s Study website, is worrying:

“Working hypotheses developed by the multiple teams of scientists have been summarized as an efficient and dynamic reference of the current questions to be addressed by the Study. The current list of hypotheses continues to evolve. It is expected that some hypotheses still being refined will be added and, over the long course of the Study, new questions will emerge and be added to the Study and some of the hypotheses included may become outdated and discarded. However, there is consensus among scientists planning the Study that as a group, the hypotheses alone can neither convey the true breadth of the Study nor completely guide the planning and design of the Study. To further define the full scope and topics of the Study, priority outcomes along with priority exposures are identified as health areas for the Study. The priority exposures and outcomes serve as an organizing framework for the Study hypotheses.” This sounds awfully like “We’re going to gather loads of data and hope that it pans out.”

Well, you might say, any nationally-based study of 100,000 families is bound to be complicated and expensive. But, as Savitz and Ness point out, there are examples of European studies that are comparable in size and scope, yet do not appear to have got so mired in difficulties. They draw attention to the Danish National Birth Cohort and the Mothers and Babies study in Norway, each of which is studying over 100,000 children enrolled in utero. I have not yet been able to find figures for the Danish study*, but the Norwegian study costed around $45 million over the 10 year recruitment period. Funding was announced this year for a new British Birth Cohort study that will follow 90,000 children from birth at a cost of £33.5 million. I’m sure that all these European cohort studies will ultimately cost far more than initially planned - such studies have a tendency to gobble up funds as the children grow older and the difficulties of maintaining a follow-up increase. The articles written on the Danish and Norwegian studies emphasise the logistic and ethical difficulties of doing studies of this kind, and the uncertainty about the future. But even so, the costs of the National Children’s Study seem set to exceed those of European studies by an order of magnitude.

I’m struck, as Savitz and Ness were, by the organizational differences between the European and US studies. Commenting on the Scandinavian cohort studies, they point out “These studies, each led by a very small group of investigators, took an early and consistent approach to their specifications and made tough decisions about what aspects to promote and what to curtail, based on a shared, explicit vision for the study.” This is very different from the huge committee-based structure in charge of the National Children’s Study. Committee-based decision-making even extends in the US study to writing up work for publication: “The Publications Committee will identify topics, set priorities, and facilitate the preparation of primary Study publications. The Committee will define the general scope and content of primary publications and assemble Writing Teams to prepare those publications. The Committee will define the charge of these Teams, monitor their progress, and review the manuscripts to ensure adherence to the charge given to the Teams, before the manuscript is submitted for publication in a peer-reviewed journal.” As an approach to scientific communication, this approach seems designed to kill stone dead any enthusiasm or creativity in the researchers. I think if NIH wants to get good science for its money, it would be better off taking an analogy of the scientist as an enthusiastic market gardener rather than factory farmer.

But a more fundamental question is whether the outcomes of this study will justify the enormous cost, or whether the money might be better spent elsewhere. A key question is how far we need another longitudinal cohort study, or whether the existing European studies will be able to answer many of the questions that the researchers are interested in. It’s not easy doing a cohort study anywhere, but it seems particularly tricky in a country that does not have a national health service. There are many areas funded by NIH where an individual investigator could do useful work with a grant of around half a million dollars. The funds spent to date on the National Children’s Study could have funded over 1200 such research projects. Viewed from this side of the pond it seems that, with the best of intentions, US funding for health research is simultaneously starving out the best of its scientists while feeding a bloated monster.

* Update on 2nd July 2011. I've now got a couple of estimates from researchers involved in the Danish study and both indicate a total cost since 1994 of less than $US 20 million. That is around 1/10th the cost of the National Children's Study just for one year, 2010. The Danish researchers make the point that the study benefits from the existence of national registers that minimize costs of case-finding, and that already contain much relevant information.

References
Kaiser, J. (2004). NIH Launches Controversial Long-Term Study of 100,000 U.S. Kids Science, 306 (5703), 1883-1883 DOI: 10.1126/science.306.5703.1883
Magnus, P., Irgens, L., Haug, K., Nystad, W., Skjaerven, R., Stoltenberg, C., & , . (2006). Cohort profile: The Norwegian Mother and Child Cohort Study (MoBa) International Journal of Epidemiology, 35 (5), 1146-1150 DOI: 10.1093/ije/dyl170
Olsen, J., Melbye, M., Olsen, S., Sorensen, T., Aaby, P., Nybo Andersen, A., Taxbol, D., Hansen, K., Juhl, M., Schow, T., Sorensen, H., Andresen, J., Mortensen, E., Wind Olesen, A., & Sondergaard, C. (2001). The Danish National Birth Cohort - its background, structure and aim Scandinavian Journal of Public Health, 29 (4), 300-307 DOI: 10.1177/14034948010290040201
Paneth, N. (2010). Saving the National Childrenʼs Study From Its Saviors Epidemiology, 21 (5), 602-604 DOI: 10.1097/EDE.0b013e3181ea5f89
Savitz, D., & Ness, R. (2010). Saving the National Childrenʼs Study Epidemiology, 21 (5), 598-601 DOI: 10.1097/EDE.0b013e3181e942cc