Market access and coverage policies for health technologies should ideally be based on clinical trials that assess final outcomes relevant to patients, such as survival, morbidity and health-related quality of life. Nevertheless, growing pressure for faster access to innovative treatments for patients in the past two decades has led to the introduction of various regulatory initiatives intended to facilitate this (Characterizing the US FDA's approach to promoting transformative innovation.Nat. Rev. Drug Discov. 14, 740–741 (2015))1. Consequently, regulatory authorities and payers often have to base their decisions about the use of a technology on surrogate outcomes, which allow trials to be done less expensively with fewer patients in a relatively short period. For example, more than 40% of trials used as the basis for approval of new indications by the US Food and Drug Administration (FDA) between 2005 and 2012 had a primary outcome that was a surrogate end point2. Furthermore, evidence from surrogate end points may not only expedite the regulatory approval of new health technologies but also inform coverage and reimbursement decisions. Over the past decade, between 27% and 50% of submissions to the National Institute for Health and Care Excellence (NICE) in the United Kingdom, the Pharmaceutical Benefits and Advisory Committee in Australia and the Common Drug Review in Canada were based on surrogate end points3.

However, relying on surrogate end points can pose several serious problems for health-care decision makers. First, surrogate end points may not capture the combined risk–benefit profile of a health technology4. Some drugs approved on the basis of surrogate end points have subsequently been associated with serious safety problems and have had to be withdrawn from the market or have their indications substantially restricted. Even if the surrogate lies in the only causal pathway of the disease process, reliance on surrogate end points can often lead to overestimation of the magnitude of the treatment effect on the final end point5.

Second, the use of surrogate end point data requires decision makers to extrapolate beyond the observed findings in order to estimate the expected true benefits to patients and health systems. Thus, clinical superiority on a surrogate end point may not necessarily translate into benefits that are good value for money for health care systems. This can be illustrated by the example of dasatinib, which has been approved by the European Medicines Agency for the treatment of chronic myeloid leukaemia. This approval was based on a trial showing superior confirmed complete cytogenetic response by 12 months for dasatinib versus imatinib (77% versus 66%, p = 0.007)6. However, an assessment of the drug by the NICE concluded that the estimated incremental gain in survival (22.7 years versus 21.3 years) extrapolated from the observed improvement on the surrogate end point came at a patient cost in excess of €200,000 per quality-adjusted life year. As a result of this assessment, NICE did not recommend coverage of the drug6.

In order for regulatory authorities and payers to use a surrogate end point with confidence, a validation process for such end points is needed. Here, we present a three-step framework for the validation and appropriate use of surrogate end points in both licensing and coverage or reimbursement decisions (Fig. 1).

Figure 1: Framework for validation of surrogate end points: case study of antihypertensive drugs.
figure 1

The figure shows the sequence of actions to implement in a health technology assessment of a drug technology when surrogate outcomes evidence is available. After an initial scope of the decision problem, the first step requires systematic review of the evidence explaining the relationship between the surrogate outcome and the final patient relevant outcome (establish the level of evidence; green box). This evidence will then be assessed to define whether the surrogate is associated with and predictive of the final patient relevant outcome (validation; blue box). If so, a quantification of the estimated effect on the final outcome given the observed effect on the surrogate outcome in the setting of interest will be performed (quantification; red box), and could be used as input in a cost-effectiveness analysis. QALY, quality-adjusted life-year; RCT, randomized controlled trial.

PowerPoint slide

Full size image

Proposed framework

Establish the level of evidence. The first step is to consider the hierarchy of available evidence7. The biological plausibility of the relationship between the surrogate end point and final outcome is necessary but not sufficient. Evidence is considered to be 'level 2' when a strong correlation exists between the surrogate and the final end point across cohorts or at the level of the individual patient. However, individual patient correlations do not provide the highest level of evidence in order to validate surrogate measures, although they may identify good prognostic markers8. 'Level 1' evidence requires demonstration of the relationship between the treatment effect on the surrogate end point and the final outcomes, preferably across multiple randomized trials. Trial-based evidence of a final outcome is usually not available for a new health-care technology for which surrogates are used, so this evidence needs to be sourced from other trials of the same or a similar technology — for example, trials should be of drugs from the same class or, if such evidence is not is available, drugs from a different class.

Assess the strength of the association. The second step is to assess the strength of the association between the surrogate end point and the final outcome. Among several approaches to address this issue, regression-based and meta-analytic approaches dominate the field. The most reliable approach is to perform a meta-analysis using patient-level data from all randomized trials of a treatment. When patient-level data are available, two levels of association can be estimated: the association between the surrogate and the final outcome, and the association between the effect of treatment on the surrogate and the final outcome. Thresholds set to identify good surrogates can be as high as 0.8 for correlation coefficients (ρ) or 0.65 for coefficients of determination (R2), which are particularly strict rules for the acceptability of putative surrogate end points when applied in practice9.

Quantify the relationship between the surrogate and the final outcome. The final step relates to predicting and quantifying the effect on the final outcome based on the observed effect on the surrogate. A quantitative approach has been proposed that consists of estimating the surrogate threshold effect (STE), which is the magnitude of treatment effect on the surrogate that would predict a significant treatment effect on the final outcome10. This is crucial for decisions on coverage and reimbursement. Regulators usually focus on early evidence of safety and efficacy to determine if the balance of benefits and risks is positive when informing the design of registration trials, whereas reimbursement agencies usually consider long-term effectiveness or cost effectiveness. Whether decisions on market access and reimbursement are based on a formal economic evaluation or on the magnitude of the clinical benefit, the effect of the treatment on the surrogate end point needs to be large enough to predict an improvement in the final outcome of interest before the technology can be concluded to be cost effective.

To date, few empirical assessments have investigated the adequacy of evidence for specific surrogate end points or groups of surrogates, particularly in terms of reimbursement policy. One exception is oncology, for which there is a long tradition of using surrogates. With a few exceptions, such as metastatic colorectal or ovarian cancer, the strength of the associations between the surrogate and final outcomes has tended to be relatively low in studies so far.

In conclusion, surrogates can result in market access for technologies that turn out to offer no true health benefit — or even cause harm —and can result in overestimation of treatment effects (and economic value), which can lead to inappropriate decisions on coverage. However, the use of appropriately validated surrogate end points within a consistent framework provides substantial potential to speed up access to innovative technologies that offer important value for patients and healthcare systems and to improve efficiency and equity within the R&D environment.

Disclaimer

The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the agencies or organizations with which the authors are affiliated.