Why Health-Systems Research is Needed

The fields of rare-disease research and clinical practice are rapidly changing. Technology such as lower-cost next-generation sequencing has advanced our capacity to identify the genetic basis for an increasing number of rare single-gene diseases.1 This has led to substantial investment in diagnostic research,2 which in turn provides a foundation for developing new treatments. By the year 2020, the goals of the International Rare Diseases Research Consortium (IRDiRC) are to develop the capacity to diagnose all rare diseases and to establish 200 new or repurposed rare-disease therapies.3

Realizing the full potential of new treatments will require that they be made available to patients within effective systems of care. The gap between treatment outcomes observed in highly controlled trials in selected patients and settings (i.e., treatment efficacy) and actual impacts experienced by patients in common clinical settings (i.e., treatment effectiveness)4,5 has led to a call for investment in research addressing the downstream translation of biomedical discoveries into improved population health;6,7 rare diseases are not exempt from this gap. The integration of new therapies for rare diseases into existing systems of care, however, challenges an implicit assumption in translational medicine that there is a linear continuum from bench to bedside.8 Research to evaluate the implementation of new therapies in real-world settings must respond to an evolving set of existing interventions and care pathways as well as consider the roles of multiple decision makers, including clinical providers and policy makers along with patients and families. A nonlinear and multifaceted approach to developing, evaluating, implementing, and adapting an intervention as part of a complete health-care system is described by the UK Medical Research Council in its guidance for developing and evaluating complex interventions.9 In addition to drawing from this guidance, the health-systems orientation that we argue is needed fits within the paradigm of comparative-effectiveness research, which has an explicit applied purpose: to “assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”10

Research focused on understanding and improving health systems for patients with rare diseases must accompany the rapidly progressing basic and early translational science that is currently leading to the development of promising new therapies. Herein, we describe priorities for such health-systems research in the field of rare disease. We emphasize a need to identify the appropriate outcomes, study designs, and stakeholder-engagement strategies to facilitate the successful implementation of new therapies toward improved care and health for patients and their families.

Outcomes: Defining Effectiveness

While ideal interventions contribute to the triple aim of improved medically defined outcomes, improved patient experiences and quality of life, and manageable health-system impacts (including costs),11 trade-offs between these goals are usually necessary in a rare-disease context. For example, orphan drugs are often expensive for the health-care system but may be highly valued by patients and families if they contribute to an improved quality of life.12 By contrast, severe dietary restrictions or other lifestyle modifications may be perceived as straightforward and inexpensive by health professionals and policy makers but as highly burdensome by patients and their families.13 Researchers often rely on surrogate end points, including biomarkers that they perceive as meaningful, particularly if these end points have been correlated with clinical outcomes; however, surrogate end points are less convincing for both policy decision makers and patients.14 Patient access to interventions, particularly expensive orphan drugs, often requires approval by multiple clinical and policy decision makers.15,16 Because these trade-offs are likely to be perceived differently by different stakeholders, disagreements about whether a particular intervention is effective are common.17 Such disagreements center on both the choice of outcome and the level of improvement that constitutes a clinically meaningful change.

As an example, among the most expensive drugs for rare diseases are enzyme replacement therapies (ERTs) for lysosomal storage disorders.16 ERTs for six such disorders—Fabry disease, type I Gaucher disease, Pompe disease, and three types of mucupolysaccharidoses (MPS; MPS I, MPS II, and MPS VI)—have received regulatory approval in several jurisdictions.18,19 However, there is considerable debate about providing patients with access to ERTs, particularly within publicly funded drug programs.20 An important characteristic of lysosomal storage disorders is a high degree of heterogeneity among patients with the same specific disease. This heterogeneity applies to age at symptom onset, clinical severity, speed of disease progression, and responsiveness to treatment. Policy discussions concerning funding of ERT are dominated by debates over which patients are likely to benefit and the optimal timing of treatment. It is challenging to bring the available evidence into such discussions because studies lack consistency with regard to measurement of effectiveness.21,22 In particular, studies of ERT for lysosomal storage disorders often rely on proxy measures of functioning such as the 6-minute walk test. There is uncertainty about the relevance of such measures to the priorities of patients themselves, their connection to patient-centered outcomes such as quality of life, and, in turn, how meaningful differences can be defined.21,22

Defining effectiveness for transformative therapies

The current momentum of biomedical research in rare disease is expected to result in new interventions that will dramatically improve outcomes for patients for whom current treatment options are limited. For such “transformative” interventions ( Table 1 ),23,24,25,26,27 barriers to initial implementation may be minimal. However, for many rare diseases, although transformative treatments have led to important reductions in the risk of catastrophic outcomes, previously unrecognized morbidities have also been uncovered, and longer-term outcomes, while substantially improved, have not been optimized.28,29 Reflecting the gap between efficacy and effectiveness in practice, such long-term outcomes are likely to depend on patient characteristics, cointerventions, and models of service delivery that may affect uptake and adherence.

Table 1 Examples of “transformative” and “incremental” interventions for rare diseases
Full size table

An example of an apparently transformative therapy is the drug nitisinone for the treatment of the rare inherited metabolic disorder hereditary tyrosinemia type 1 (HT1).27 Nitisinone inhibits an enzyme in the degradation pathway of tyrosine that is upstream from fumarylacetoacetate hydrolase, a deficiency of which is the cause of the disease.27 The treatment reduces production of metabolites that may be toxic, particularly succinylacetone, and in most cases prevents the liver failure that would necessitate a liver transplant. However, children with HT1 who are treated with nitisinone may remain at some risk for liver complications, including the development of hepatocellular carcinoma.27 Long-term management of the disease is complex, requiring not only medications but also substantial diet modifications, with frequent monitoring and tailoring of care.27 Adherence to the recommended low-phenylalanine and low-tyrosine diet is particularly challenging and may have an important impact on outcomes.30 There is also emerging evidence that a relatively high proportion of individuals with HT1 face learning difficulties and/or other neurocognitive challenges.27,30,31 Although longitudinal follow-up data are limited,32 these complex features highlight the need for continued research to better understand the care pathways that lead to best outcomes for those with HT1. This example also illustrates that the outcomes relied on to determine effectiveness will often shift and need to be redefined as research progresses and shorter-term goals are met.

Defining effectiveness for incremental therapies

The impact of therapies on patient outcomes for rare diseases exists on a continuum, with a majority of newly developed therapies being likely to have more modest or variable impacts relative to the transformative interventions discussed in the previous section ( Table 1 ).22,26,33,34,35 This is partly a consequence of the challenges in developing specific new therapies for diseases affecting small numbers of patients, with current efforts focused on the promising strategy of repurposing existing drugs.36 Such “incremental” interventions may be combined in ways that yield important gains in health for some or all patients with a rare disease, but achieving such gains will arguably require even greater attention to the system of care.

Many examples of incremental therapies for rare diseases exist in the field of cystic fibrosis (CF). A subset of CF patients stand to benefit from a transformative therapy that targets specific mutations ( Table 1 ).24 However, most currently available interventions for CF are life- and health-extending rather than transformative. The average life expectancy for individuals with CF has increased significantly over recent decades, but variation in outcomes among CF patients, both across and within countries, is considerable.37,38 It has been postulated that differences in outcomes may be related, in part, to differences in patient access to effective care, particularly multidisciplinary team care in specialized CF centers.37,38,39 Because treatments for CF are often complex and burdensome to patients and their families, patient adherence is also a challenge.40 This further emphasizes the importance of coordinated care, frequent outpatient consultations involving multiple disciplines, and services such as telehealth.41 It also highlights the need to consider a range of patient-oriented outcomes in recognition of the impact of interventions and associated health services on quality of life for patients and families. Agreeing on the most appropriate outcomes will require discussion among multiple stakeholders (see “Stakeholder Engagement” below).

Study Designs: Generating Evidence to Evaluate Effectiveness

To ensure the appropriate translation of new therapies into improved outcomes for patients with rare diseases, studies must be designed in a way that promotes an understanding of the effectiveness of interventions embedded in real-world systems of care. Specifically, ideal study designs will:

  • 1. Accommodate the range of therapies and health services that are received by patients as cointerventions within care pathways or models of health service delivery (e.g., the contribution of interventions and services such as diet therapy, physiotherapy, family support and counseling, telehealth, and the roles of interdisciplinary team care, specialized treatment centers, and care coordination)

  • 2. Identify long-term impacts of care, including potential adverse effects

  • 3. Determine how the wide variety of real patients and disease characteristics influences treatment effectiveness, i.e., there is a need to account for clinical heterogeneity

Randomized controlled trials (RCTs) remain the gold standard for determining the clinical efficacy of a new intervention because they are designed to have strong internal validity under ideal conditions by minimizing bias and confounding.9 Realizing the fundamentals of RCTs is difficult in rare-disease research, even for establishing efficacy; in small patient populations, randomization may not succeed in balancing potential confounders between groups and statistical power may be inadequate to detect treatment effects.42,43 Several modified trial designs have been proposed to accommodate small samples.44 However, with the exception of n-of-1 trials and crossover designs (discussed below), these modified designs remain focused on an evaluation of treatment efficacy rather than effectiveness in real-world settings ( Table 2 ). For example, both RCTs and most adapted trial designs require standardized treatment protocols within and across centers. This standardization challenges a trial’s ability to evaluate real-world care and means that the effects of cointerventions and systems of care are difficult to investigate. RCTs are also expensive; to design a trial efficiently, shorter-term outcomes, including surrogate outcomes, may be relied on as primary end points.45 This can result in misleading conclusions if the causal pathways between the intervention, surrogate outcome, and patient-oriented outcomes are not well established.42,46 Finally, traditional RCTs estimate average treatment effects in selected patients and thus cannot accommodate clinical heterogeneity.47

Table 2 Strengths and weaknesses of evaluative study designs for health-systems research regarding rare-disease interventions
Full size table

Pragmatic randomized controlled trials (PCTs) incorporate many of the study-design characteristics of traditional RCTs, including randomization of treatment arms and standardized blinded assessments of outcome. However, PCTs differ from RCTs in their primary aim; PCTs are designed to directly inform decisions about care for routine patients rather than to explain a causal effect of treatment on outcome.48,49 The hallmark features of PCTs are inclusive criteria for patient eligibility and recruitment across multiple real-world care settings. These features compromise internal validity to some extent in favor of improved generalizability ( Table 2 ).48,49 PCTs also typically use current standard care, instead of placebo, as the control arm, and allow standard care to vary across treatment settings. Cluster randomized designs are common in PCTs. In a cluster design, the treatment setting itself, rather than the patient, is the unit of randomization. Differences in patient characteristics and provision of care across settings are measured and accounted for in PCTs. However, the emphasis is typically on identifying a single summary treatment effect that is generalizable across real-world settings rather than on explicitly investigating sources of heterogeneity. Compared with traditional RCTs, PCTs may follow patients over a more extended period to evaluate longer-term impacts and identify potential adverse effects.

Both crossover and n-of-1 trial designs attempt to address the problem of small patient numbers while maintaining some of the methodological features of a traditional RCT that serve to reduce bias ( Table 2 ).44 In both n-of-1 (single participant) and crossover trials, participants receive two treatments in random order with a washout period in between, such that each participant acts as his or her own control;43,44,50 variations of these designs may incorporate more than two treatment periods, which can help to mitigate potential bias due to carryover and period effects.44 n-of-1 trials are explicitly oriented toward supporting individual treatment decisions.51 Multiple n-of-1 trials can also be combined to provide population-level effect estimates, although there are some challenges with the approach when the sample is small.44,50 Although both n-of-1 and crossover designs are susceptible to carryover and period effects, these approaches are ideal designs for investigating heterogeneity in treatment effects across participants ( Table 2 ). A drawback of n-of-1 and crossover trials is their reliance on short-term outcomes with rapid response to the intervention.

Despite a greater vulnerability to bias and confounding relative to experimental designs ( Table 2 ), observational studies have some strengths for investigating the effectiveness of new treatments embedded in systems of care.52 For example, disease-specific patient registries collect long-term follow-up information for complete cohorts of patients. Registries that provide data on natural history, outcomes, and interventions to inform future trials serve as a platform for recruiting patients to such trials.53 Similarly, practice-based-evidence networks use observational methods to evaluate care in real-world settings, capturing standardized data on existing interventions and outcomes to explore how different patterns of care impact treatment effectiveness.54,55 Such designs can be used to investigate clinical heterogeneity, evaluate long-term outcomes, and analyze interventions within systems of care ( Table 2 ). An important challenge for observational designs is the potential for the results to be influenced by residual confounding—particularly confounding by indication, whereby the influence of factors contributing to the need for an intervention are conflated with the intervention’s effectiveness.56 An additional concern with registries is the challenge in collecting high-quality and uniform data, particularly in the multicenter environment that characterizes registries for rare diseases.

It has recently been suggested that hybrid designs that incorporate aspects of both observational and experimental approaches can mitigate the challenges of each, resulting in more efficient trials.57,58,59 Specifically, Relton et al.57 have suggested the “cohort multiple randomized controlled trial” (cmRCT), in which routine patients are enrolled into a large observational cohort. Subgroups of cohort participants are randomly selected and consented to receive particular interventions, with the remainder of the cohort acting as a control group. This approach was used in a recent rare-disease protocol.60 A similar design, the “randomized registry trial,” was discussed as a new “disruptive” clinical evaluative research design58 in reference to a coronary intervention trial61 that recruited participants from an existing registry and relied on the registry itself for outcome data. Another hybrid design is a pragmatic trial that also incorporates elements of an implementation study, either by simultaneously investigating the process of implementation or by explicitly evaluating an intervention designed to promote effective implementation.59 There is great potential for these hybrid designs to address issues important to health-systems evaluation research for rare diseases ( Table 2 ). For example, internal validity is likely to be higher than in traditional observational designs owing to control of confounding by randomization. Cointerventions, long-term outcomes, and clinical heterogeneity can also be accommodated through comprehensive long-term observational data collection in real-world settings.

Finally, knowledge-synthesis approaches, such as systematic reviews that seek to evaluate and summarize evidence across studies, will be critical for making best use of available data to inform care. Methods for systematic reviews of RCTs are well established,62 and we have mentioned the potential value of combining multiple n-of-1 trials to estimate population treatment effects.50 Methods for synthesizing the results of studies that have used different study designs are more challenging; heterogeneity in study methods and participants means that meta-analyses, in which results from multiple studies are summarized quantitatively, are difficult to produce. However, if challenges to meta-analysis can be overcome through either standardization of study methods or statistical adjustment for key methodological differences, network meta-analyses may be a promising approach for evaluating multiple interventions for rare diseases. In such analyses, indirect treatment comparisons can be made—e.g., if treatment A and treatment B have each been compared with treatment C in separate trials, network meta-analysis may be used to estimate the difference in effectiveness between treatments A and B without launching a new trial that directly compares them.63

Stakeholder Engagement to Support Knowledge Exchange

Regardless of the specific study designs used to evaluate the effectiveness of rare-disease interventions, the value of these evidence-generation activities depends on the evaluation metrics they use. We have discussed the complexity of identifying appropriate outcomes. Ideally, evaluative research should incorporate outcomes that are of greatest importance with respect to treatment goals, based on a consensus among patients, clinical providers, researchers, and policy decision makers. The importance of establishing this consensus is gaining broader recognition.64,65 This reflects the understanding that good health care is patient-centered and that patient-centered care is dependent on the degree to which outcomes of importance to patients are understood and prioritized within research protocols, as well as within health-care systems.66

Efforts are being invested into the assessment of patient-oriented outcomes,67 and the priorities of rare-disease patients are increasingly being recognized through patient involvement in research.68 However, to ensure that health care for rare diseases is truly patient-centered, these patient-oriented outcomes must be taken up by rare-disease researchers, health-care providers, and policy decision makers. We suggest that this is unlikely to be fully realized without increased attention to knowledge translation activities. The rare-disease community is bolstered by its highly engaged patients, families, and support and advocacy organizations,69 suggesting that interest in collaboration is likely to be high. That said, the community is also characterized by small, dispersed professional communities of health-care providers and researchers that may present barriers to effective knowledge exchange.70 Collaborating across stakeholder groups with very different perspectives is also challenging when the evidence about treatment effectiveness is limited and subject to debate. Researchers and decision makers may be concerned about potential bias, particularly when engaging with groups with ties to private industry. Patient representatives may feel marginalized if their input is sought belatedly or if they perceive that their views are being dismissed because of their interest in promoting patient access to interventions. It is because of these very challenges that we argue that there is a need for planned, systematic knowledge exchange.

In the field of implementation science, Graham et al.71 describe the “knowledge-to-action” cycle that encompasses both evidence generation and the subsequent implementation of evidence-informed interventions into health-care systems. Consistent with guidance for developing and evaluating complex interventions,9 the process is cyclical and depends on ongoing assessment, adaptation, and application of knowledge as evidence continuously emerges.71 The translation of knowledge into effective implementation of new therapies for rare diseases thus requires interaction among stakeholders on an early and ongoing basis. We believe this exchange of ideas is likely to be most successful if it takes advantage of both formal and informal networks of rare-disease researchers, health-care providers, policy makers, and patients. Effective mechanisms to facilitate this knowledge translation work are currently lacking.

Specifically, we argue that knowledge-exchange infrastructure—built by connecting and expanding existing networks, leveraging technology, and intentionally developing culture and community72—is needed. Through such infrastructure, the priorities of patients and families as well as other stakeholders can be clarified early in the research process, interventions can be meaningfully evaluated through clinical research studies, and those that are found to be effective can be taken up, assessed, and tailored toward optimal patient-centered care. Examples of formalized knowledge-exchange infrastructures include communities of practice and multistakeholder practice-based research networks.73 These initiatives are emerging among rare-disease groups, citing in-person and virtual opportunities to meet and exchange ideas as key to their collaborative success.74,75 Investing in knowledge-exchange infrastructure thus has the potential to provide a foundation for launching a coordinated approach to evaluative research for rare diseases, helping to ensure that such research is conducted in a way that is considered meaningful to all the communities that will be charged with integrating the results into policy and practice.

Conclusion

We have argued that investment in health-systems research is important to ensure that the benefits of new, efficacious therapies for rare diseases are realized in practice through their successful implementation within effective systems of care. Such research must focus on appropriate patient-oriented outcomes, include robust study designs that can accommodate “real-world” decision priorities, and involve effective stakeholder-engagement strategies. For transformative therapies, outcomes will need to shift and be redefined in response to the need to focus on longer-term goals as catastrophic outcomes are successfully prevented. For incremental therapies, a range of patient-oriented outcomes may be most appropriate to consider, in recognition of the impact of patterns of clinical interventions and associated health services with respect to quality of life for patients and their families. To generate new evidence, we suggest that hybrid study designs integrating elements of practice-based observational research and pragmatic trials hold the most promise for addressing priorities that include minimizing bias, accounting for cointerventions, identifying long-term impacts, and considering clinical heterogeneity. Although funding additional evaluative research beyond the efficacy trials needed to bring a rare-disease therapy to market will be challenging, investing in the observational component of a hybrid study design (e.g., a clinical registry with longitudinal data collection) would facilitate multiple intervention studies, including both efficacy and effectiveness studies, for new treatments as they become available. Thus, the hybrid design may represent the most efficient use of resources over the long term. To effectively engage with stakeholders, a knowledge-exchange infrastructure is needed to foster collaboration among patients with rare diseases and their families, health-care providers, researchers, and policy decision makers. A key priority for these groups must be collaboration toward a shared understanding of the outcomes that are of most relevance to the facilitation of patient-centered care for individuals with rare diseases and their families.

Disclosure

The authors declare no conflict of interest.