Introduction

Evidence-based medicine is thought to be extremely important in contemporary medicine.1 However, until recently, actual evidence with Japanese subjects has not been sufficiently produced. It is known that despite the westernized lifestyle of the Japanese population, the incidence rate of myocardial infarction remains relatively low.2 Thus, generation of scientific evidence based on data from Japanese patients is warranted. However, owing the fact that the Japanese healthcare system covers the entire population in principle, and that people have free access to almost any kind of medical institution,3 it has been rather difficult to recruit patients into clinical trials, especially into randomized, double-blind studies, in which the patients and doctors are required to be unaware of what medicines are being administered. This is the reason why many recent clinical trials conducted in Japan adopted the prospective randomized open-label blinded end-point evaluation (PROBE) design,4 in which both the patient and the doctor are aware of what medicines are being administered. However, if not designed carefully, the accuracy of the PROBE-style study results can be compromised. In this review, we would first like to discuss the designs used in various studies and then describe the design of PROBE; thereafter, we would like to provide referral to the merits and demerits of the PROBE design trials, accompanied by recent examples.

Study designs used in epidemiological studies

Epidemiology is the study of factors affecting the health and illness of a certain population. It does not usually encompass the assessment of the efficacy of drugs or medical devices. However, the principal concepts and methodology used in clinical trials have been generated in epidemiology, and understanding them is important.

Retrospective cohort studies

In retrospective cohort studies, a population set (cohort) is defined and the risks and outcomes are investigated retrospectively. This design of epidemiological studies can be adopted when there is already a database of risks and outcomes of sufficient size. With the recent evolution of information technology, patients' demographic data, laboratory data, prescription data, and morbidity and mortality data are sometimes available over the course of several years. For example, to elucidate the relationship between chronic kidney disease and mortality, a study was conducted by referring to a registry database of coronary revascularization and valve procedures, which revealed that patients having moderate to severe acute kidney injury after CABG surgery showed worse 5-year survival compared with those who having normal or near-normal renal function.5 However, not all confounding factors might be stored in the database, which limits the use of the results of such a study. If a promising result is obtained, it should be confirmed by performing a prospective randomized control study.

Retrospective case–control studies

In the case of rare diseases, standard cohort studies will entail long time periods and high costs for identifying the cause, because of the low incidence rates of the diseases. To overcome this challenge, a case–control study might be useful, in which individuals with the disease (case) are compared with those without the disease (control) and are matched with several demographic factors such as age, sex and place of dwelling. The study will retrospectively investigate the exposure to risks in both groups to identify the cause of the disease.

Prospective cohort studies

In this study design, patient background information is collected at the start of the study or when a subject is newly recruited into the cohort and followed up by collecting information on risk exposures and incidence of morbidity and mortality; thereby, the relationships between the presumptive risk factors and disease are investigated. This type of study (for example, the Framingham study6,7) has established cardiovascular risk factors such as hypertension, hyperlipidemia resulting from smoking, age and diabetes mellitus. Although it is the most scientifically accurate design, it is laborious and usually involves extremely high costs.

Study designs used in clinical trials

In the past, single-blind prospective trials were conducted. However, due to its limited advantage over open prospective trials, currently this type of trial is conducted rarely.

Double-blind, prospective, placebo-controlled trials

Double-blind, prospective, placebo-controlled trials were the standard type of clinical trials that were considered to provide the most reliable results. Numerous trials have been conducted based on this design; such studies showed the value of antihypertensive therapy8,9 or the efficacy of statins in the primary or secondary prevention of coronary heart diseases.10,11 One of the major flaws of this design is that, once the efficacy of a treatment is established, it becomes unethical to conduct a placebo-controlled study; another flaw is that it is relatively difficult to recruit patients into this type of trial. Further, it is also difficult to use this type of trials in the assessment of interventional therapies such as comparison of coronary stents or pacemakers.

Double-blind, prospective trials without placebo control

This design allows the evaluation of a new mode of treatment against an established one. Numerous studies have established the efficacy of treating hypertension8,9 and hypercholesterolemia10,11 in the management of cardiovascular diseases, the benefits of employing β-blockers12,13,14,15 or angiotensin-converting enzyme inhibitors16,17 in the management of congestive heart failure, and so on. Thus, as stated above, it is unethical to avoid using these agents under conditions in which they are proven to be effective; in such cases, this trial design is used. The disadvantage of this design is that, because an already proven treatment is used, the difference between the new one and the established one might be marginal; this usually leads to the requirement of a larger number of patients and longer duration of studies.

PROBE design

The PROBE study was designated by Dr Hansson in 1992 as an alternative to the double-blind, prospective study design.4 In this type of study, patients are allocated to different treatment regimens in a strictly random fashion. Unlike double-blind studies, the regimens are made obvious to both physicians and patients. An important aspect is that strictly defined end points are adjudicated by an independent committee that is unaware of the treatment allocation, which guarantees the unbiased comparison of therapies and evaluation of study results.

Other conditions that require a PROBE design study include cases in which the drug warfarin is administered.18,19,20 Warfarin requires strict titration, and thus cannot be used in a double-blind study. Studies that involve the use of interventional devices are also usually designed in an open-label fashion.

As shown in Table 1, the merits of the PROBE design include better patient acceptance, lower cost, and the existence of similarities between PROBE studies and regular clinical practice.

Table 1 Advantages and disadvantages of the PROBE design compared with double-blinded design
Full size table

Precautions to be taken with the PROBE design

As described in the Introduction, in Japan, it is generally difficult to conduct a randomized, controlled, double-blind study in which both the doctors and patients are unaware of the medicines being administered. Therefore, realistically, large clinical trials have to be conducted in a PROBE fashion in Japan. If PROBE studies are designed and conducted properly, the results will not be biased. One of the ways to ensure accuracy is to use only ‘hard end points’ in primary end-point assays. Hard end points are end points that can be defined solely by objective criteria; sudden death of any cause, non-fatal myocardial infarction and non-fatal stroke are examples of hard end points. Soft end points, on the other hand, are end points that may be affected by subjective judgements, such as hospitalization due to unstable angina, congestive heart failure or coronary revascularization procedures. These end points might be defined by objective criteria, but if the attending physician deems the patient requires hospitalization or can be medically controlled in an outpatient setting is, for example, at the discretion of the physician. As shown in Table 2, most cardiovascular clinical trials with PROBE designs conducted in the West18,19,21,22,23,24,25,26,27,28,29,30,31,32,33 use only hard end points. In contrast, four major Japanese PROBE-designed trials with clinical outcomes specified as primary end points used soft end points such as unstable angina, exacerbation of heart failure or coronary revascularization procedures.34,35,36,37 One of the reasons might be that the Japanese tend to have lower incidence rates of cardiovascular diseases compared with the westerners2 and thus, soft end points are required to produce statistically significant differences with a reasonable cohort size. If these end points are reported and adjudicated in an unbiased fashion, the reliability of the results will be the same as those acquired from double-blind studies. In this context, the results of the JIKEI-Heart Study35 interested the Japanese Medical Society. One reason was that it was one of the few large clinical studies successfully conducted in Japan. This study was conducted to investigate whether addition of an angiotensin receptor blocker (valsartan) to conventional treatment was effective in reducing cardiovascular events in Japanese patients with cardiovascular disease. Although the successful lowering of blood pressure was similar in both the valsartan and non-valsartan groups, it was shown that the addition of valsartan to conventional treatment prevented more cardiovascular events than conventional treatment. This superiority of valsartan to other blood pressure-lowering agents, which goes beyond its blood pressure-lowering effect, has not been shown in other clinical trials conducted in a double-blind fashion;38,39 this is one of the other reasons that the results of this trial interested many individuals. When examined in greater detail, it is obvious that the differences in the number of soft end points are the factors that mainly contributed to the favorable results for valsartan in this study (Table 3). The primary end point of this study is a composite of several pre-specified events, and its components are not clearly shown. However, the number of each component is shown as secondary end points, which is a good indication of the composition of the primary end point. In fact, hard end points such as myocardial infarction or cardiovascular mortality did not differ significantly between the two groups. As is questioned in the editorial that accompanied the article,40 it cannot be totally ruled out that there might have been underreporting of events in the valsartan group. It is also of great interest that although most studies conducted in the West produced no significant difference as regards the primary end point, results of three of the four studies conducted in Japan were significantly in favor of the study drug, the manufacturer of which was the sponsor (Table 2). These could be considered as examples showing that the results of PROBE-designed trials should be evaluated carefully, especially in cases wherein the results with soft end points are widely discrepant from those with hard end points.

Table 2 Cardiovascular clinical trials conducted in PROBE fashion and their primary end points
Full size table
Table 3 The number of end points in JIKEI HEART Study
Full size table

Conclusion

Epidemiological studies conducted in prospective cohort design have established cardiovascular risk factors such as hypertension, hyperlipidemia, smoking and age. By applying the ideas and methodologies developed in epidemiological studies, many clinical trials have been conducted to prove the benefits of various medicines.

Randomized, controlled, double-blind clinical trials report the most scientifically accurate results. However, they usually entail high costs, render the recruitment of patients more difficult and are rather discrepant from usual clinical care. The PROBE study design is a feasible alternative to double-blind studies. However, if not designed and conducted properly, it will be more susceptible to biases. Thus, studies conducted with a PROBE design using soft end points included in the primary end point require the participating physicians to adhere to the guidelines for PROBE studies more strictly.