Experimental designs and randomized controlled trials

Introduction

There is a fundamental need to identify the causes of illness and disability if we are to deliver effective health care for individuals and the population. It is by understanding these causes that we can formulate and justify our assessments, diagnoses and interventions. Furthermore, we justify interventions in that we can point to the evidence that demonstrates interventions are causing the beneficial changes in preventing or managing disease. In health research the concept of internal validity is related to the design of research projects and the extent to which the implementations of these designs enables researchers to unambiguously identify causal relationships between interventions and outcomes.

Experiments are an important form of intervention studies. A well-designed experiment enables researchers to demonstrate how manipulating one set of variables (the independent variables) produces systematic changes in another set of variables (the outcome or dependent variables). Experimental designs aim to ensure that all variables other than the independent variable(s) are controlled; that is, that there are no uncontrolled extraneous variables or factors that might systematically influence changes in the study’s outcome variable(s). Control is most readily exercised in the sheltered environment of research settings such as laboratories and this is one of the reasons why these settings are the preferred habitats of experimental research scientists. However, much of epidemiological and clinical research takes place in field settings (e.g. communities, hospitals and clinics), where the phenomena of human health, illness and treatment are naturally located. Even in these natural settings researchers can nevertheless exercise control over extraneous variables.

In this chapter, we will focus on an experimental design referred to as the randomized controlled trial (RCT) which is an experimental research design aimed at assessing the effectiveness of a clinical intervention. Our aim is to show how RCTs are used to demonstrate that an intervention is the cause of a subsequent effect(s). RCTs are considered by many researchers and clinicians to be the premier type of evidence for the effectiveness of clinical interventions.

The specific aims of this chapter are to:

1. Examine the basic structure of experimental research designs and RCTs.

2. Consider threats to the validity of the results obtained from experiments and RCTs.

The concept of causality

The notion of causality has been a difficult and controversial topic for philosophers. As health researchers we need to take a pragmatic approach and look to demonstrate causal relationships. Three simple criteria for demonstrating causal associations are:

1. The cause must precede (occur before) the effect.

2. The cause and effect co-vary. If the cause occurs then so does the effect.

3. If the cause does not occur, then the effect does not occur.

The first criterion is quite simple. For example, if we say that injury to the person’s arm is the cause of that person’s reported pain, we of course assume that the injury was sustained prior to the onset of the pain. Clearly, if the pain had been already present, the injury would not be seen as the cause of the pain. Second, we assume that there will be ‘concomitant variation’ between the injury and the pain. The worse the injury, the more severe the pain. As the injury recovers, a decrease in the level of pain can be expected. In general, we are establishing evidence for the existence of a causal relationship between the cause and the effect.

However, observing a relationship between two events is not sufficient to demonstrate causality. For example, night follows day in a predictable, lawful fashion, but we do not say that day causes the night or vice versa. We must attempt to eliminate plausible alternative explanations or hypotheses that offer rival causal explanations for the findings. For example, pain might persist even after the injury has healed. There might be other variables operating which maintain a person’s experience of pain.

The problem is that, apart from our intervention, there are likely to be other factors that may be influencing the pain outcome. These influences are called ‘confounding extraneous variables’ or ‘bias’ and constitute a potentially serious problem when attempting to interpret the results of a trial.

Confounding and bias in research studies

The research designs that are commonly referred to as ‘before–after’ designs are often used initially to test the safety and efficacy of novel interventions. The use of the design provides a clear example of how confounding variables can threaten the internal validity (or accuracy) of the study. Confounding variables can result in ambiguity in deciding what has caused any observed changes.

As a hypothetical example, consider the introduction of an exercise program for cardiac patients, which aimed at increasing mobility, and thereby improving health. The patients were also smokers, and were strongly encouraged to give up smoking. The researchers used ‘distance walked by patients’ (in a specified time period) as an indicator of the effectiveness of the program. Figure 7.1 represents (without showing numbers) the average walking distances before and after the exercise program. There is clearly a difference between before and after (i.e. an improvement from baseline). However, was this improvement caused only by the exercise program, or by confounding extraneous variables, or a combination of both?

Figure 7.1 Results for a hypothetical ‘before–after’ study.

Consider the following plausible alternative explanations for the difference shown in Figure 7.1:

1. The improvement might have been due to natural recovery; the patients’ mobility might have improved independently of the exercise program. This is referred to as a threat to internal validity due to maturation.

2. The improvement might have been due to other factors, such as the reduction in, or cessation of, smoking among some of the patients, resulting in an average increase in walking distance. Such confounding extraneous variables are referred to as history.

3. The improvement might have been due to a placebo effect. Placebo effects (see below) are associated with patient expectations of potential benefits of the intervention. In the example outlined above, the expectations of the patients, rather than the exercise itself, might have been the actual cause of the improvement.

4. An important source of bias is observer bias which refers to researchers unknowingly influencing the results of a study by holding expectations regarding the outcome. In the present hypothetical example, the researchers may anticipate that the exercise program will improve mobility.

The above explanations illustrate the influence confounding extraneous variables. That is, each is concerned with factors that are present at the same time as the intervention and which may have produced the observed effect.

Internal validity refers to the ability of a researcher to attribute differences (e.g. Fig. 7.1) to the effect of the independent variable. Maximizing internal validity is important for quantitative research designed to demonstrate causal effects. In order to ensure internal validity, researchers must control for the effects of confounding and bias. Experimental designs are particularly well-suited for controlling for the potential influence of confounding factors. Randomized controlled trials employ experimental designs that are applied for evaluating the efficacy of interventions.