Study Design and Execution Matter!

Posted by The Evidence Blog on August 19, 2014

Fourteen years ago, Congress passed Public Law 106-310, the Children’s Health Act of 2000, which authorized the implementation of the National Children’s Study (NCS). The intent of the NCS was to follow about 100,000 children from birth (or before birth) until they reached age 21 to document how environmental factors, such as social factors, chemical exposure, and so forth, influence child health and development. The NCS was to be led by the National Institute of Child Health and Human Development (NICHD).

Recruitment of participants began in 2009. Initially, pregnant women in 100 random counties in the United States were recruited in a door-to-door effort. But after recruiting about 4000 women, that approach proved too expensive, and the NICHD turned to a provider-based recruitment design. By 2012, having already spent $1 billion dollars, the NICHD decided to close 40 study sites that were run by academic investigators and instead use large contractors to track the families already enrolled in the pilot study. In March 2013, Congress requested that a panel from the National Research Council (NRC) and the Institute of Medicine (IOM) review the proposed new study design and make recommendations. The panel’s report was released in June 2014; you can read the full report here.

All in all, the panel made 20 recommendations to improve the NCS. The panel agreed that the study has the potential to increase what we know about the effects of environmental exposures on child health and development and supported its comprehensive nature. Nevertheless, the panel was critical of several aspects of the study design, in particular, how the NICHD planned to enroll subjects into the study. Additionally, the panel was unable to assess certain aspects of the proposed study design because the Program Office did not provide sufficiently detailed information. Even more disturbing, however, was the panel’s finding that scientific leadership of the NCS was lacking. The panel concluded that for the NCS to be scientifically grounded and implemented in a cost-effective manner, additional scientific expertise was needed; the panel recommended that a new outside scientific advisory group be created and given authority to approve the study’s design.

No matter how well intended, if a scientific study is not planned and executed properly its results—and the associated costs to conduct the study—can be called into question. The NCS is a good case in point. Following the release of the NRC/IOM report, the director of the National Institutes of Health, Francis Collins, announced that he was putting the main NCS study on hold and gathering a team of experts to assess whether the study could be conducted given current budget constraints and, if so, how to implement the panel’s recommendations.

Political support for the NCS was robust in 2000; today, more than 14 years and $1 billion dollars later, it’s unclear whether such support still exists. It’s also unclear whether the information we could gain from the NCS justifies the expense, which is expected to be billions over the next several years. While the study results may ultimately lead to improved child health and development, can the study be redesigned and executed to ensure we get the most valuable information?

Topics: Hayes Blog

Sign up to receive updates from our blog

Our latest articles

New Call-to-action