Next Creation Changing Your Future?

This HRA guidance poses five questions [4]:

The HRA guidance is separated into four layers, providing increasing detail, as required. Layer I sets out in tabular form the questions and considerations that researchers, sponsors, peer reviewers and RECs should ask. By clicking over a question or specific words or phrases in the table in Layer I the reader can navigate to a more detailed discussion of the topic in Layer II. From Layer II the reader can navigate to more comprehensive explanations of individual components of the sample size in Layer III. Finally, Layer IV provides explanatory notes on some underlying statistical principles.

The HRA guidance is built on the following core concepts: that sound planning is critical to the outcome of an interventional clinical trial and its ethical acceptability (bad science is bad ethics); all trials must be registered; and the results should be published [5,6]. A researcher should identify the primary research question to be answered; explain why it is important to patients and health-care practitioners; show that the study design is appropriate to answer the question posed, in particular that the sample size is likely to be adequate to meet the pre-specified aims of the study; and describe the plan to disseminate the results of the study.

The definition of the research question is key to research design. All research must have a primary question, clearly stated in advance, and founded on a systematic review of what is already known[5,7-9]. Researchers who plan studies without reviewing what has been done, risk performing research for which the answer is already known or exposing participants to ineffective or an inferior treatment [8-10].

The planning of a clinical trial depends on the primary question, and researchers should clearly and simply explain in the study protocol what the trial is aiming to show, why it is worth asking and, through consultation with public and patient groups, why this is worthwhile to patients. The primary question should be consistently stated throughout the study protocol. Population, intervention, comparator and outcome (PICO) is one useful way of formulating a research question[11].

The research protocol should explain how the proposed study methodology is appropriate for the question posed, demonstrate that the design is likely to answer the research question, and why it is the best approach. The design should be underpinned by a systematic review of the existing evidence, which should be reported in the protocol [8,9]. Absence of a systematic review raises the question: what is the design based on?

Unfortunately, our and other research has found that study protocols often lack important information on study design, which hinders external review [2,3,7,12]. The HRA guidance therefore encourages researchers to explain:

Researchers should provide all the information needed to allow an independent reviewer to both reproduce the target sample size and understand the rationale for the assumptions used in the calculation. The HRA guidance provides a checklist of the information that should be reported in the study protocol for the sample size determination. Each item is linked to more detailed explanations. Specific guidance is given on reporting sample sizes based on confidence intervals, group sequential, factorial, cluster and time-to-event studies.

The sample size determination checklist requests that researchers:

Particular emphasis is placed on the need to provide a reasoned explanation of why the treatment difference and other design assumptions are plausible for the planned study, taking into account all existing data. The importance of the difference sought, i.e. why it is worthwhile to patients, should also be explained. The justification could include reference to consultations with the public and patient groups, existing literature or published studies in which the minimum clinically important difference has been empirically determined.

Researchers should be rigorous in the determination of design assumptions [1-3]. Sample size re-estimation should be considered if there is a high degree of uncertainty [13]. Manipulating the sample size calculation, sometimes called the sample size ‘game’ or ‘samba’, to produce the desired statistical power by inappropriately overestimating the treatment effect (known as ‘optimism bias’ – the unwarranted belief in the efficacy of new therapies) or underestimating the variability leads to ‘underbuilt’ studies with insufficient power and inconclusive results [14,15]. Such studies are not ethical and waste valuable resources [16-18].

Sample size determinations must be realistic [19]. If the sample size required to detect the treatment difference of interest is unfeasible, then this should be explained in the research protocol. A well-designed and implemented trial, even one with lower power (and precision), will still yield unbiased results, which can be combined with similar unbiased trials in a meta-analysis[14].

All studies should be monitored for protocol compliance, adverse effects, patient recruitment, etc. If treatment is of long duration then accumulating efficacy data should be monitored for overwhelming evidence of efficacy or harm. No study should continue to recruit patients once the main comparisons have revealed clear-cut differences [5].

The repeated significance testing of accumulating data does have statistical implications [5]. Thus, the research protocol should describe how multiple testing has been accounted for in the sample size determination [2,3,7]. If there is a great deal of uncertainty surrounding the design assumptions then researchers are asked to consider re-estimating the sample size during the course of the study.

For clinical trials, HRA has now established that a favourable REC opinion is contingent upon trial registration in publicly accessible databases, unless acceptable reasons are provided for not doing so. The REC also needs to be assured that results will be placed in the public domain. Researchers are expected to: