Chapter 20 Evaluation in health promotion
Overview
Evaluating health promotion is not a straightforward task. Health promotion interventions often involve different kinds of activities, a long timescale, and several partners who may each have their own objectives. Health promotion is still seen as belonging within the health services, where the dominant evaluation model is quantitative research centred on experimental trials, with randomized controlled trials (RCTs) as the preferred evaluation tool. Health promotion has had to argue its case for a more holistic evaluation strategy encompassing qualitative methodologies and taking into account contextual features.
The focus of this chapter is on evaluating health promotion interventions. Evaluation of research studies is also part of the health promoter’s role and remit, and readers are referred to Chapters 2 and 3 in our companion volume (Naidoo & Wills 2005) for a detailed discussion of this topic. This chapter considers what is meant by evaluation, the range of research methodologies used in evaluation studies, its rationale, how it is done and the role of evaluation in building the evidence base for health promotion.
Defining evaluation
Evaluation is a complex concept with many definitions that vary according to purpose, disciplinary boundaries and values. A comprehensive definition of evaluation is: ‘the systematic examination and assessment of features of a programme or other intervention in order to produce knowledge that different stakeholders can use for a variety of purposes’ (Rootman et al 2001, p. 26). The above definition is useful because it also flags up the importance of the purpose of evaluation, and the fact that there can be many different reasons to evaluate. Evaluation can provide information on the extent to which an intervention met its aims and goals, the manner in which the intervention was carried out, and the cost-effectiveness of the intervention. It is important to be clear at the outset about the purpose of evaluation as this will determine what information is gathered and how information is obtained. The value-driven purpose of evaluation distinguishes it from research (Springett 2001). Evaluation uses resources which might otherwise be used for programme planning and implementation, so a clear purpose is also necessary in order to legitimate and protect this use of resources.
Principles to guide the evaluation of health promotion interventions
The World Health Organization (WHO) has identified four principles that should be used to guide the evaluation of health promotion interventions:
Evaluation research methodologies
What would be the strengths and limitations of the methods you identify?
Why evaluate?
Evaluation uses resources that could otherwise be used to provide services. Given that services are always in demand, there needs to be a strong rationale for devoting resources to evaluation rather than service provision. New or pilot interventions warrant a rigorous evaluation because, without evidence of their effectiveness or efficiency, it is difficult to argue that they should become established work practices. Other criteria that can be used to determine if evaluation is worth the effort relate to how well it can be done. If it will be impossible to obtain cooperation from the different groups involved in the activity, it is probably not worthwhile trying to evaluate. If evaluation has not been considered at the outset but is tacked on as an afterthought, the chances are that it will be so partial and biased as to be not worth the effort.
What to evaluate?
Health promotion objectives for smoking reduction
Although all these factors relate to health, they are quite separate, and there is no necessary connection between, say, increased knowledge and behaviour change. It is therefore inappropriate to evaluate a given objective (e.g. increased physical activity) by measuring other aspects of an intervention (e.g. number of leaflets taken at a health fair or number of people reporting that they would like to exercise more). It is important to choose appropriate indicators for the stated objectives. This issue is discussed further in Chapter 19, where the log-frame model and the use of logic to select appropriate indicators are considered.