Most evaluations of evidence-based practice (EBP) do not consider the impact on economic outcomes. Some argue that financial cost should have no role in health sector decision making, that consideration of such impact is even unethical (Williams 1998) and there is ongoing debate that the impact on economic outcomes can and should be assessed only after a statistically and clinically significant impact on outcomes has been demonstrated. In this chapter I will argue that evaluations of EBP should always consider the impact on economic outcomes, that it is unethical not to do so, and that while a two-stage process is possible in theory, it is rarely achieved in practice. To do this, we first need a common understanding of what economic outcomes are and of why economists are interested in these outcomes. Then we will look at how these economic outcomes can be assessed in evaluation of EBP. Finally, I will argue that this economic evaluation is an essential part of any evaluation of EBP, that it is the ethical way to proceed, and that it needs to be part of a single approach to evaluation.

What are economic outcomes?

To most people, economics is another word for costs, so it follows that “economic outcomes” of a change in practice is another term for financial cost. To an economist however, economic outcomes of a change in practice are the benefits of that change in practice, in terms of the value attached to all of the consequences that result from that change, for all stakeholders. However, as any change in practice is the result of a choice that has been made to implement change, there is always an “opportunity cost” in terms of the possibility that changing practice has led to an increase in the inputs required for practice, compared to what happened before the change. Inputs such as staff time, equipment, office space, etc. always have some alternative use, in which they would generate benefits of value to the same or another group of people. Therefore, any change in practice which increases use of these resources has an associated opportunity cost, in terms of the lost opportunity to use these resources elsewhere.

Economists therefore cannot let themselves consider the joy of the benefits of implemented practice change without weighing these against the possible harms that result from an increase in economic costs (the financial representation of all resources invested in the practice). To make things more confusing, economic costs are different to accounting costs, as economists will want to include all resources invested in the practice, regardless of whether or not these resources actually have to be paid for (e.g., patients’ time). This act of weighing up the benefits that result from a change in practice against the additional costs of that change is called economic evaluation.

Economics jargon and where economists are coming from

As illustrated in the above section on costs and outcomes, economists working in health care (or education, environment, transport, etc.) suffer from a tendency to speak in a language only vaguely related to English. The result is a set of terms that have one meaning in lay language and another, often critically different, meaning in economics. There are now glossaries available that can be a useful translation tool when dealing with economists (Shiell et al. 2002).

Economics is a social science and is essentially the study of how people make choices when they interact with one another in society. Most of the choices involve giving up something the person has (for most of us, hours of our own time) in order to receive something else (for most of us, money in the form of a wage). We do not do this to get money, but rather to get the things that we can exchange that money for, such as housing, food, clothes, travel, holidays, and so on. Why do we do this? To an economist, human behavior is driven by the search for happiness, or “utility” in economics jargon. Economists differ on what exactly can count as utility, but in general the discipline assumes that people make choices to maximize the utility (happiness) that accrues to the individual (or the household) over a lifetime. The discipline of economics is then based on dual aspects of these choices: the efficiency with which such choices are made (do choices maximize utility?); and the equity of the distribution of resources people start with and/or of the goods and services people end up with (who has what to start with and who ends up with what?).

It is important to note here that money is not a thing of value in and of itself; it is only a means of trading what we have for what we want. The same concept applies in evaluation: money is not an outcome in itself; it is just a handy way of expressing the value of resources invested in a practice.

Efficiency, inefficiencies and the case for government intervention

In a perfect world, people would make choices that, with some degree of random error, did mean that they maximized their utility given the resources they had available. The implication would be that any attempt to interfere or intervene in people’s exchanges (their trading of what they have for what they want) would lead to suboptimal outcomes in terms of the utility of all people.

However, perfect worlds are useful for theoretical models but do not usually exist in reality. In the real world, departures from the perfect world ideal happen all the time. Take the example of buying lunch at work. The perfect world assumes that I know all the alternatives available to me and all the relevant attributes (or characteristics) of all these alternatives. That means I know all of the different foods that I could purchase, the opening times and location of each shop, and the extent to which each of the foods would satisfy my desires for hunger–satiation, nutrition, taste, texture, etc. The perfect world also assumes that all of the food alternatives have a price that reflects only the investment of land, labor, and capital (equipment) needed to produce them, nothing more. If some of the alternatives (or their ingredients) have been affected by government subsidies, their price will be artificially low and my choice could become biased from the efficient ideal. Conversely, any food shop that can keep out competition may be able to charge artificially high prices and similarly bias my choice. My own perceptions of available alternatives and of what impact they will each have on my desired outcomes (hunger–satiation, nutrition, etc.) could be biased by advertising or by my own lack of information-processing skills due to insufficient education (in turn, the impact of my prior choices and of the inequality of the distribution of initial resources, i.e., where, when, and to whom I was born).

If the choice of lunch now seems exceedingly complicated, turn now to the choice of what interventions to provide in health care. Individuals are rarely fully informed of the alternatives available to address a health-care need and of what outcomes each of these alternatives is likely to have. Indeed, individuals are often unaware of the details of their own health-care need, or even that they have a need that can be addressed by a health-care intervention. In health-care choices, individuals are often represented by a healthcare professional (nurse, midwife, family doctor, etc.) who acts as their agent and helps the individual to make an efficient decision. But agents are human too and are themselves subject to a range of incentives (costs and benefits to them resulting from different choices) that may bias their actions away from those most likely to maximize utility of the patient.

In short, it is widely accepted that there are many sectors of a country’s economy (or structure) where government can intervene to improve the outcomes for the population compared to what would happen if everyone were left to make their own decisions. Sometimes this is based on equity concerns, but largely this argument is based on efficiency concerns, that government intervention in the funding and/or provision of services can increase the utility of the people compared to the decisions that would be made without government intervention. The extent and format of government intervention varies across different countries, but usually covers at least part of the sectors relating to health care, education, community services, public transport, environment, and defence.

However, government intervention means that we need to find an alternative way to assess whether a proposed change in practice is a good or a bad idea. If government intervention is justified by the lack of knowledge of individuals about the potential relative benefits and harms of different treatment options (often a major reason in health care), then we cannot use individuals’ choices to evaluate those treatment options. If government intervention, on equity or efficiency grounds, results in individuals facing subsidized or zero costs for treatment (as for many health-care services), again we cannot use individual choice to evaluate the relative value of different treatment options, because those individuals are no longer weighing up the relative benefits of different options against their (full) relative costs. We need to move the decision to a level higher-up the decision-making structure. This is what happens in health care; we make a decision whether or not to implement change for the whole hospital (or the whole health-care system) and then that decision applies to all of the individuals treated.

Economic evaluation in health care is therefore not the individual sort of economic evaluation that we all make subconsciously every day when we buy lunch or make any other purchasing and investment decisions. The type of economic evaluation that health economists are involved in concerns groups of people, not individuals, and therefore we are looking at the benefits that are created for a group of people due to a change in practice and the change in resource use required in making that change in practice for that group. As we are looking at groups and not individuals, economic evaluation in health care needs to know the average outcome (benefit) and the average resource use (cost) and it is these average estimates that will influence decision making.

It will always be the case that some individuals within the group may experience greater benefits and lower costs than others. This means that a decision made on the basis of the group average may not be the same decision that would be made on an individual level for every individual in that group. This is one of the downsides of group-level decision making. It can always be assessed if the evidence we have can be reliably broken down to a subgroup or individual level, as we could then assess whether a practice change that is judged to be not worthwhile for the whole group may still be worthwhile for a smaller subset. Even then, however, we would need to ensure that any such subset could be reliably identified and somehow separated from the full group if we were to feasibly implement practice change for only a subset of the whole group.

Equity and the missing side of economics

Economic evaluation is essentially a study of efficiency (getting the maximum utility, or happiness, from the resources that we have available). In practice, while the discipline of economics has dual concerns of efficiency and equity, the practice of economic evaluation is concerned overwhelmingly with efficiency. Equity concerns (analyzing who pays for health care, who accesses health care and/or who gains the benefits of health care) are either completely missing-in-action, or are parked at the door and only sometimes addressed as an afterthought. There are areas of health economics that look at equity in detail (e.g., in trying to improve formulae for the allocation of budgets to match clinical need), but these are separate to the practice of economic evaluation. However, a well-trained economist will always be aware of their missing half, and there are attempts underway to bring some consideration of equity back into the practice of economic evaluation (Cookson et al. 2009).

Economic evaluation is a large part of what health economists do and there are many textbooks available that set out what economic evaluation is and how to do it. For many, the bible is Methods for the Economic Evaluation of Health Care Programmes by Drummond and colleagues, currently in its third edition (Drummond et al. 2005). A more detailed coverage of issues in economic evaluation is provided in the summary of the United States Panel on Cost-Effectiveness in Health and Medicine (Gold et al. 1996); there are also more straightforward or simplified guides to economic evaluation in health care available (Jefferson et al. 2000). This section simply touches on the wealth of detail covered in these textbooks and any reader interested in how to actually do an economic evaluation should go straight out and buy a copy of the Drummond book.

Evaluation and economic evaluation

Economic evaluation is part of the evaluation process. By definition, an economic evaluation is a comparative assessment of one practice against another (i.e., an evaluation). The thing that makes it an economic evaluation is that it looks at both the consequences (outcomes) of the alternatives being evaluated and the costs of each alternative (the resources that need to be invested to deliver each alternative, plus the impact of each alternative on future resource use).

Forms of economic evaluation and more jargon

Economic evaluation can take many forms, with terms such as cost-effectiveness, cost-utility, and cost-benefit widely used (often incorrectly) to describe economic evaluation. Ultimately, there are two forms of economic evaluation: one that can tell you whether the consequences are worth the costs and one that cannot. The former is cost-benefit analysis (CBA) and requires some process to put a monetary valuation on the incremental outcomes of a change in practice (see below). The latter is generally termed cost-effectiveness analysis; this is by far the most common form of economic evaluation in health care and includes economic evaluations of the form of cost-consequences analysis, cost-effectiveness analysis, and cost-utility analysis. Cost-effectiveness analysis does not mean that decisions cannot be made on the worth of EBP change, it simply means that the results of the economic evaluation set out what are the additional costs and additional outcomes of a change in practice, and this information then needs to be presented to a third party (politician, manager, some form of “decision maker”) who will judge whether the additional outcomes are worth the additional costs.

Three steps of economic evaluation

Regardless of the form chosen, there are essentially three steps to economic evaluation: