Introduction

For most, childhood is a time of healthy growth, development, and learning. However, too many children across the United States experience chronic diseases or do not achieve their potential. These conditions are often caused by modifiable environmental factors such as air pollution, chemicals, social stress, and poor nutrition that children experience during early development and may affect their health and well-being throughout life. Studies focused on child health often do not have sufficient size, breadth, or diversity to examine the major public health challenges that affect children.

The US National Institutes of Health (NIH) established the Environmental influences on Child Health Outcomes (ECHO) Program to overcome these limitations, with the goal of producing solution-oriented research that would improve the health of children for generations to come. The ECHO Program has three main components: (1) the ECHO cohorts, (2) the ECHO Institutional Development Award (IDeA) States Pediatric Clinical Trials Network (ISPCTN), and (3) the centers and cores that support them. The IDeA States program focuses on enhancing the competitiveness for research funding to improve child health services in states that have a historically low level of NIH funding and is one of several end users that the ECHO cohorts seek to inform and engage.

This article (one of a series of companion papers detailing aspects of the ECHO Program) focuses on the ECHO cohorts, which encompass 69 diverse longitudinal cohort studies of children across multiple stages of development (i.e., prenatal, infancy, early and middle childhood, adolescence) from 44 states and Puerto Rico. Investigators interested in participating were invited to apply to a funding opportunity announcement published in December 2015 (RFA-OD-16-004). Eligible cohorts included those that were initiated in pregnancy and/or the postpartum period, and had followed or were capable of following children longitudinally. A Special Emphasis Panel administered by the NIH Center for Scientific Review assigned priority scores for funding. ECHO cohorts focus on five key pediatric outcome areas that have high public health impact: pre-, peri-, and postnatal outcomes; upper and lower airway health; obesity; neurodevelopment, including mental health; and positive health, with the goal of identifying modifiable exposures before 5 years of age that are associated with these outcomes. The cohorts conceptualize exposures broadly, including components of physical, chemical, biological, social, behavioral, natural, and built environments, to maximize the potential to influence children’s lives across multiple domains of early experience. Through the cohorts, ECHO includes more than 57,000 children representing diverse racial, geographic, and socioeconomic backgrounds.

We describe the infrastructure built by the ECHO cohorts to develop a common data collection protocol, overcome the many challenges to rapidly launching this large-scale program, and ultimately generate high-impact research on child health. We summarize the multiple processes and policies implemented to bring together a diverse group of researchers and studies into a vast research program that both supports the unique contributions of each participating cohort and creates a unified, robust cohort of over 50,000 families across the nation.

ECHO Program structure and governance

The ECHO Program Director, who reports directly to the NIH Director, leads the three components of ECHO (Fig. 1). Six centers and cores provide support for all ECHO activities: a Coordinating Center (CC),1 Data Analysis Center (DAC), Person-Reported Outcomes (PRO) Core, Human Health Exposure Analysis Resource (HHEAR), Genetics Core, and a Data Coordinating and Operations Center supporting the ISPCTN. The Steering Committee (SC) is the primary governing body of ECHO, and includes the NIH program director, two co-chairs, one investigator from each center or core, and 31 principal investigators, 1 from each of the cohort awards (many of the awards included more than 1 cohort). The SC represents the interests of all ECHO components, providing strategic direction and guiding the scientific work of ECHO. ECHO also has an Executive Committee with representatives from NIH, the centers and cores, the cohorts, and ISPCTN. Its main role is to ensure that the ECHO Program fulfills its goals by proposing strategic solutions to programmatic issues for consideration and ratification by the SC. An External Scientific Board that reports directly to the NIH Director provides expert guidance to the ECHO Program, advising on strategies to ensure its long-term success. The board considers how to overcome challenges to the ECHO Program and provides feedback to the NIH ECHO Program Office and the Council of Councils, an advisory group to the NIH Director. The External Scientific Board conducts an annual review to determine whether ECHO is meeting its goals and milestones, and whether adjustments need to be made to reflect new initiatives or public health challenges.

Fig. 1: Organizational structure of the Environmental Influences on Child Health Outcomes (ECHO) program.
figure 1

NIH National Institutes of Health, PI principal investigator, IDeA States Institutional Development Award States, HHEAR Human Health Exposure Analysis Resource, PRO Person-Reported Outcomes.

Full size image

Stitching a national quilt: bringing existing cohorts together

This unprecedented effort to bring together multiple independent studies with their own identities and scientific goals presented opportunities as well as challenges for the development of a successful child health research program. Extant data and biospecimens contributed by each cohort sped scientific collaboration. Furthermore, the expertise and experience represented by ECHO’s more than 1200 investigators and study staff span multiple disciplines (pediatrics, allergy, family medicine, epidemiology, neonatology, toxicology, pulmonary, psychiatry, and psychology, among others), life stages, and data collection and assessment modalities. Throughout the early phases of the ECHO Program and beyond, work to capitalize on these strengths and mitigate challenges has been informed by a strong commitment to team science principles, engagement with a variety of stakeholders, and fostering of multidisciplinary collaborations across all aspects of the program.

Laying the foundation for team science and stakeholder engagement

Two working groups critical to core goals of the ECHO mission were formed at the outset of the ECHO cohort program. The mission of the Team Science Working Group is to maximize ECHO’s scientific excellence and productivity by fostering team building and collaboration through effective communication. In ECHO’s early phase, focused on developing the protocol and becoming a united enterprise, this group promoted team science principles (Table 1) and provided opportunities for investigators to address conflict and build trust; develop new collaborations;2 discuss new policy developments, institutional review board issues, and data sharing agreements; and consider specific aspects of protocol development.

Table 1 Principles of team science guiding the ECHO Program.
Full size table

The Stakeholder Engagement Working Group represents the interests of participants and their families, communities, the public, researchers, providers, stakeholder organizations, and local, state, and federal entities. ECHO engagement activities are guided by the following principles: (1) participant contributions to research design, conduct, oversight, and dissemination are valuable; (2) participants are embedded within families and communities that must be incorporated into stakeholder engagement frameworks; (3) investigators are supported in building internal and community capacity to enhance engagement at the local level in order to create mutual value; and (4) ECHO is transparent and trustworthy, communicating program updates and results in formats and language understandable to participants and engaging in dialog with communities where the research is conducted. During this phase, the Stakeholder Engagement Working Group focused on participant, family, and community engagement. Early activities included defining ECHO cohorts’ stakeholders, working with ISPCTN representatives to draft the aforementioned engagement principles, hosting early engagement-focused capacity-building webinars, developing a participant experience survey, creating a Participant Advisory Board, and compiling an engagement resource and best practices repository.

Operationalizing the ECHO-wide Cohort Protocol

The ECHO-wide Cohort Data Collection Protocol (hereafter “protocol”) was drafted to facilitate the creation of the ECHO-wide cohort data platform, within which researchers can conduct solution-oriented etiologic and prediction research. The platform contains harmonized existing measures as well as standardized new measures collected and shared by the cohorts. Further, extant biorepositories across the cohorts and planned biospecimen collection as outlined in the protocol will create a rich biorepository across the ECHO cohorts spanning multiple life stages starting with pregnancy. Collectively, the ECHO-cohort biorepository and data repository will serve as a national resource platform that both ECHO investigators and the wider scientific community can use to generate innovative and impactful research questions by relating a broad range of early environmental influences with child health outcomes.

The protocol development process was initiated following the ECHO Program Kickoff Meeting in November 2016. The SC charged five Outcome Working Groups, aligned with ECHO’s five key pediatric outcomes of interest, with developing broad research questions to inform the development of the protocol. In parallel, the Protocol Working Group was formed. Two co-chairs led this working group, which also included representation from cohort investigators, the PRO Core, the DAC, and the CC. The Protocol Working Group convened six Life Stage Subcommittees, each co-led by a representative from the PRO Core and a cohort principal investigator and including representatives from the DAC, the Biospecimens and Chemical Exposures Working Groups, and each of the five Outcome Working Groups.

Over the course of the next 2 years, the Protocol Working Group engaged with the Outcome Working Groups, the Life Stage Subcommittees, and the SC to identify the most relevant exposures, outcomes, covariates, mediators, and moderators to include; refine the prioritization of data elements as essential or recommended; and ensure that selected measures were valid and developmentally appropriate (Fig. 2). These efforts were facilitated by Team Science Working Group-sponsored activities. In parallel, PRO Core representatives reviewed conceptual frameworks from previous and ongoing studies of children to identify key domains, concepts, and measures, and also reviewed each funded cohort grant proposal for exemplars of additional measures and scientific questions that might be applied to the ECHO-wide cohort.

Fig. 2: The Protocol Working Group Process.
figure 2

How ECHO working groups and program components contributed to the development of the ECHO-wide Cohort Data Collection Protocol.

Full size image

Throughout this process, the Protocol Working Group navigated a push and pull between adding scientifically justified elements to enhance the breadth of the protocol and the need to reduce elements to ensure the protocol was not overly burdensome. As an example, early in 2017, the Life Stage Subcommittees along with representatives from other working groups, cores, and centers completed worksheets to populate the protocol. This process ultimately resulted in 30 worksheets (one per outcome per life stage) containing nearly 2500 rows of data. Many data elements were represented across multiple life stages and outcome areas. In an iterative process, with input from multiple components, committees, working groups, and individuals across the ECHO Program, the Protocol Working Group further refined the list of core data elements to identify 116 that the group believed balanced breadth with burden. This list ultimately resulted in protocol version 1.0.

In addition to overseeing the generation of the protocol’s contents, the Protocol Working Group finalized its structure to define data elements as either Essential or Recommended. Essential data elements and biospecimens were those that the program required all constituent cohorts to collect and share from all willing participants in each life stage that their cohort participants (pregnant women and children) enter during the period of their funding support. To allow for innovation across ECHO, the protocol also contained a diverse array of recommended data elements and biospecimens. The protocol strongly encouraged, but did not require, collection of recommended data elements. If a cohort collects or has previously collected a data element that appears in the protocol as recommended, the cohort is expected to share these data with the ECHO-wide cohort data platform.

As part of the process of balancing scope against burden, the Protocol Working Group further classified essential data elements as preferred, acceptable, or alternative. Preferred measures are those that sufficiently balance innovation with feasibility, burden with efficiency, and breadth with depth across diverse cohorts. If the collection of the preferred measure is not feasible, cohorts may instead use the acceptable measure. Compared with the preferred measure, the acceptable measure reduces participant and/or researcher burden while maintaining acceptable scientific rigor. Examples may include data abstracted from medical records and short form versions and subscales of longer instruments. The protocol may include at least one acceptable measure for each essential data element; however, for some data elements, the acceptable measure is the same as the preferred measure. Alternative measures may be included in the protocol if they are legacy cohort measures and viable alternatives to either the preferred or acceptable measure, as confirmed by evidence of harmonization potential.

While the protocol is intended to be pluripotent, allowing a wide range of research questions to be addressed, at the time the protocol was drafted there were not yet any questions that had been ratified as high priority, against which the protocol could be assessed for appropriate coverage. Therefore, the PRO Core organized a protocol “test drive” exercise, convening an interdisciplinary group of cohort investigators to propose sample high-impact questions and then examine the extent to which the protocol measures would allow them to be answered. Also, the NIH issued a Request for Input soliciting comments and suggestions on this version 1.0 protocol from a broad range of stakeholders including all ~1200 ECHO investigators, multiple non-ECHO investigators, advocacy groups, and the general public.

Protocol version 1.0 was approved by the ECHO SC and the NIH Program office in October 2018. However, work remained to fully operationalize the protocol so that it was ready for field administration. In response, the Executive Committee established the Protocol Implementation and Evaluation (PIE) Committee. PIE Committee members represented multiple ECHO components. The role of the PIE Committee was to evaluate the protocol to continually improve its implementation feasibility and to maximize ECHO’s research utility. The PIE Committee served as the central communication and oversight group for the development of forms and measurement information sheets, training plans, and feedback from early cohort adopters and participant/cohort stakeholders. The committee worked to solicit and synthesize community feedback about the protocol and to propose adjustments to the protocol and its supporting infrastructure based on this feedback (Fig. 3). The committee considered the balance between effort required of cohort staff and participants, and the data collection necessary to inform high-impact, innovative transdisciplinary research.

Fig. 3: Evaluation process for the ECHO-wide Cohort Data Collection Protocol.
figure 3

The funnel contains the domains of protocol evaluation.

Full size image

The PIE Committee addressed concerns around excessive burden to participants and study staff with a variety of approaches. In summer 2018, the CC led interviews with “early adopters,” defined as those who had newly implemented elements of the protocol into their local cohort data collection. The committee solicited continued input from across the program via a dedicated e-mail inbox. An Alternative Measures Task Force was established to review and approve proposals for legacy cohort measures that might be substituted for essential measures in the protocol. A Data Collection Form Subcommittee continued to review and revise forms for better flow.

These efforts led to an interim protocol version 1.1, and ultimately a field-ready version 1.2 was finalized in December 2018 and approved by the central institutional review board in January 2019. The first cohort began fielding protocol version 1.2 in June 2019, with the majority of cohorts fielding the protocol by the end of 2019.

Even after ratification and fielding of protocol version 1.2 across the ECHO ecosystem, the PIE Committee continued efforts to evaluate it and plan future refinements. These activities included semi-structured interviews with field staff representing 28 different cohorts, requesting early experience with protocol implementation. Some of these cohorts also participated in the Participant Experience Survey, which was drafted and fielded by the Stakeholder Engagement Working Group and asked participants to provide feedback related to barriers to participation, perception of study burden, and interest in the resulting science from participants and families of 20 fielded cohorts.

Establishing norms: ECHO policies

As part of the governance of this complex program, cohort investigators developed policies to provide guidance on program operations. Initially, the SC directed working groups to develop policies; however, as the program matured, responsibility for policy development shifted to time-limited, milestone-driven task forces, which operated with narrower scopes of work and smaller membership. All teams engaged with stakeholders during policy development and provided updates to the Executive Committee and SC periodically to share progress and obtain input from the ECHO community. Once a policy is finalized, the SC votes to approve or not, and the NIH Program Office provides final signoff.

After initial development of the first few policies, the SC recognized the need for a group to have responsibility for ensuring consistency across the growing number of policies and so established the Policy Implementation and Evaluation Committee. Through its first 4 years, the ECHO Program has developed several policies, each to address specific needs and tensions arising from this multifaceted program (Table 2).

Table 2 ECHO-wide policies.
Full size table

Promoting and disseminating ECHO science

Having fielded the protocol and established essential infrastructure and policies, the ECHO cohorts have increasingly focused efforts on achieving the primary goal of generating high-quality science to influence policies, programs, and practices related to child health. Efforts are focused on leveraging extant data and biospecimens previously collected by cohorts, and actively responding to high-priority public health challenges, such as Neonatal Opioid Withdrawal Syndrome, COVID-19, and racial and ethnic disparities.

Publications process

The Publications Committee developed a staged approach to reviewing and approving research products to maximize opportunity for collaboration, promote high-impact innovative science, and ensure productivity. Writing team leads begin by submitting (via website upload) a brief Analysis Concept that summarizes a research idea or hypothesis and is available for review, feedback, and engagement from the ECHO scientific community using discussion boards; additional investigators are welcome to join writing teams at this stage. The CC reviews Analysis Concepts and conveys comments and next steps to the writing team lead after a 2-week review period. At this stage, the CC alerts the relevant cores and centers to review the concept and potentially join the writing team. The full writing team then meets to develop an Analysis Proposal, which includes a detailed analytic plan including specific hypotheses being tested, a full list of exposures, outcomes, and covariates, and plans for the statistical analyses. The expectation is that each Analysis Proposal results in one published manuscript.

Submitted Analysis Proposals are assigned to three primary reviewers with diverse scientific expertise and one Associate Chair from within the Publications Committee. Reviewers are asked to consider a number of aspects including solution orientation, innovation, required cohort resources, and overlap with existing projects, and provide an unblinded formal, written review. A representative from the Biospecimen Working Group may also review the proposal to assess whether any proposed use of biospecimens is appropriate, and for any proposals requiring data analysis one reviewer must be from the DAC. Reviewers may suggest changes in design, scope, and authorship and provide a preliminary recommendation (approved, approved after revisions, or not approved). The Associate Chair compiles the reviews, and comments are conveyed to the writing team leader, who revises the proposal if necessary. If there are concerns with overlap between proposals, the Publication Committee favors a collaborative approach and may suggest that writing teams merge. Revised proposals and response to reviews are compiled by the Associate Chair and distributed to the SC along with Publications Committee critiques and recommendations. The SC votes to approve or not approve the proposal.

After approval of the proposal, the writing team continues efforts to engage and collaborate with as many ECHO investigators as possible. For proposals using cohort data already on the ECHO Platform, the writing team lead contacts the cohort principal investigators to invite them and others from within the cohort to join the writing team, if interested. If the proposal involves new assays of extant biospecimens, the writing team lead approaches cohorts with appropriate outcome data to garner interest in the proposal and request biospecimens for the analysis. ECHO investigators have the unique opportunity to leverage resources available from HHEAR, which allows them to conduct program-supported lab analyses of both existing and new biospecimens. Once writing teams identify eligible cohorts, they work together to coordinate transfer of biospecimens to the lab conducting the assay, whether HHEAR or otherwise. After completing assays, the lab transfers the results to the ECHO-wide cohort data platform and also back to the contributing cohort. Analysis primarily takes place within the DAC, and the writing team drafts an abstract for a scientific conference or a manuscript. The Publications Committee reviews abstracts and manuscripts prior to submission, as well as presentations for conferences. Writing team leads are asked to notify the CC when the submission is accepted or declined, or if the journal requests revisions to the manuscript. The CC announces all publications to the ECHO investigators as well as other end user stakeholders via an ECHO Newsletter.

Driving solution-oriented research

The SC created the Strategic Planning Task Force to promote the development of high-impact research in ECHO—i.e., studies that are expected to inform programs, practices, and policies. Studies that meet these criteria are termed potential “Big Wins” and are envisioned to be projects that are at the intersection of three key elements: investigator passion, feasibility, and end-user stakeholder priorities (see companion article by Romano et al.3).

Establishing the Big Win framework stimulated supporting and synergistic activities across working groups, cores, and committees to facilitate and promote Big Wins. To enhance engagement with end users at all stages of scientific discovery, the Stakeholder Engagement Working Group and Strategic Planning Task Force formed the End-user Stakeholder Subgroup to continue building capacity within the ECHO community and developing both front-end and back-end engagement opportunities with these stakeholders. Within ECHO, “end user stakeholder” is loosely defined as an individual or group who would use ECHO research to inform policies, programs, and practice, as a subset of ECHO’s broader, more inclusive list of stakeholders defined above. End user stakeholders engaged by this group have been both at the national level, such as the Executive Director of the Children’s Environmental Health Network, and the regional level, such as representatives from state departments of health and human services.

The End-user Stakeholder Subgroup assesses internal connections to end user stakeholders to ultimately leverage ECHO investigator relationships and leadership positions within these groups. The subgroup also hosts a monthly webinar series featuring both national and local/regional stakeholders with the goal of facilitating engagement and highlighting research gaps and priorities that ECHO cohort science may address.

The Team Science Working Group also turned its attention to promoting and facilitating multi- and transdisciplinary collaborations and research products that could be Big Wins. Each year, the working group sponsors several science-focused breakouts at virtual and face-to-face SC meetings, with attendees selected to maximize disciplinary diversity and area of outcome expertise to facilitate outcome-wide concepts.4 Goals of breakout sessions have included refining existing Analysis Concepts to enhance solution orientation and Big Win potential, and generating new, solution-oriented Analysis Concepts in specific scientific areas. The Team Science Working Group also implemented a series of scientific flash talks at biannual in-person meetings and in monthly SC calls. In these 5-min talks, investigators share new ECHO-wide cohort Analysis Concepts with the broader community to stimulate interest in the topic, engage additional investigators or cohorts to join, and apprise the ECHO community of new scientific developments.

Dissemination and reciprocal learning

The ECHO cohorts developed a multipronged strategy for reciprocal learning with ECHO participants and end user stakeholders, and dissemination of research findings and study information. In alignment with the ECHO Guiding Principle of engagement, several ECHO teams engaged with the Participant Advisory Board noted above. Paid as expert advisors, this small group of participant representatives provide feedback on participant-facing communications materials (e.g., public website, lay research summaries, informational video) during their development phases. These representatives also provide feedback on the policies and processes used by the Return of Results Task Force, which is charged with ensuring that ECHO cohort findings are disseminated to participant groups, policy makers, and other relevant end user stakeholders.

Addressing high-priority public health challenges

The Executive Committee and Strategic Planning Task Force as well as working groups, centers, and cores are dedicated to responding nimbly to public health challenges as they arise. With the onset of the COVID-19 pandemic, a COVID-19 Task Force developed and fielded a survey that was added to the ECHO-wide cohort protocol in May 2020; this survey captures the impact of this pandemic experience on child and family health, spanning consequences of the virus itself and the vast social and economic impacts of local and state-level efforts to curb transmission (e.g., sheltering in place, school closures, financial strain, unemployment). The Team Science Working Group developed breakout sessions during the virtual SC meeting in April 2020 to facilitate new, multidisciplinary collaborations focused on COVID-19. The ECHO Program also released a COVID-19 Notice of Special Interest for cohorts to generate innovative scientific proposals that will inform key programs, practices, and policies in the short term by studying disparities in access to remote learning and impacts on child neurodevelopmental outcomes, examining the impact of COVID-19 infection on the health of vulnerable children, and developing strategies for data collection and analysis in the era of COVID-19, for example.

While social determinants of health have been a core focus of many ECHO cohorts, members across the ECHO community joined in the public outcry in response to the violent death of George Floyd in May 2020 and many other people of color before him. These events highlighted the need for the cohorts to specifically address the child health impacts of structural racism. As the experiences of both COVID-19 and structural racism are in many ways a local phenomenon, the ECHO cohorts are uniquely poised to provide essential insights into these pressing public health challenges. ECHO investigators have formed a Diversity and Inclusion Working Group to ensure that issues of equity remain at the forefront in ECHO’s research questions, as well as in representation of both participants and investigators.

Ongoing challenges and lessons learned

Even after the many refinements that occurred during the development process, the protocol remains large, and it can be challenging for cohorts to complete all expected measures, especially during briefer life stages such as pregnancy and infancy that also include many measures of interest. The COVID pandemic added extra challenges. However, the protocol does allow for collection at more than one time point within each life stage, and includes flexibility in route of administration (in-person visits, online surveys, or telephone interviews). Investigators and study staff with preexisting interest and expertise in specific exposure and outcome areas have been asked to dedicate time and precious cohort resources to an expanded set of measures; for example, cohorts previously dedicated to the study of a specific physical health outcome (e.g., asthma, obesity) were asked to assess neurodevelopment also, and vice versa. The ECHO awards provided financial support for these expanded measures, and the process of coming together has encouraged all participating investigators to transition in both practice and mindset to become “omnibus” cohorts. Harmonization across diverse assessment strategies and tools has been challenging (see companion article by Jacobson et al.).5 For example, prior to implementing the protocol, almost every cohort assessed household income differently, using overlapping categories. To address harmonization needs, ECHO leveraged the broad expertise of its scientific community to prioritize the most critical measures that will be commonly used across multiple analyses.

Building the procedures and infrastructure we describe here occupied much of the first few years of the ECHO program. The shared process of developing communication strategies, eliciting participation, and achieving consensus across such a disparate program has ultimately led to a group identity that, while not fully complete, has laid the foundation for collaborative and impactful science.

Conclusion

The size, breadth, and diversity of ECHO creates a data platform and biorepository that allows for research questions to be addressed that no single cohort, or even group of cohorts, could answer alone. Leveraging the research platform described in this article will inform practices, programs, and policies relevant for both new and existing public health challenges, which in turn moves the ECHO Program forward to its ultimate goal of enhancing the health of children for generations to come.