Chapter 13 Quality in a public health service: the 3-5-7 Model

• Understanding service quality in public health services can be difficult because:

• different stakeholders have different perceptions and priorities

• different organisational contexts have differing resources

• outcome measurement, the dominant paradigm in health services, creates difficulties in respect of attribution, longitudinality and the need for change to take place

• This model emerged from a two-phase collective case study of the Irish public health nursing service to families with infants where data were collected from public health nurses, managers and clients

• The model draws on the work of Donabedian and comprises three main parts viz. organisational context, process and consequences, all of which are operationalised. Five key concepts, time, knowledge, communication, environment and orientation, are presented as the links between these three parts. Seven steps of process were identified

• Although developed in the Irish public health nursing service, the model can be applied in many other contexts. It can be used, for example, to provide a framework for reviewing existing services, for planning new services, for examining why things went wrong (or why they did not) and for setting standards

• The model presented here provides a mechanism for measuring service quality in a holistic way that takes account of key stakeholder perceptions; the process of the service and the organisational context within which people operate

Introduction

Longman’s (1990) definition of quality as ‘the degree to which something is excellent; a standard of goodness’ (p. 848) assumes that quality is tangible, identifiable and conceptually clear. This may be an appropriate assumption to make in the industrial sector where a clearly identifiable product emerges at the end of a strictly controlled process. In the health care setting, however, where there is infinite demand but finite resources, undemanding clients, complex consumers, a ‘high intangible’ content and a significant professional component (Øvretveit 1992), this is not the case. It is not surprising, therefore, that, within the heath care literature, the concept of quality has been variously described as slippery and elusive (Pfeffer & Coote 1991), nebulous and lacking consistent definition (Van Maanen 1979) and ‘enigmatic and multi-dimensional’ (Attree 1993, p. 369) and this has, in turn, led to a proliferation of different understandings of quality.

Quality has been identified, for example, as a political issue (Shaw 1997), a social construct (Redfern & Norman 1990), a philosophy (Øvretveit 1992), professional standards (Dozier 1998), customer satisfaction (Kleinsorge & Koeing 1991), an attribute of a product (Kerrison et al 1994), a system of management (National Standards Authority of Ireland (NSAI 1998) and effectiveness (Katz et al 2004). These differing conceptualisations result in different elements of the service being prioritised over others and this has implications for measurement. While budget holders, for example, will want to include in their assessment a measure of the cost of the service, this may not be the priority for professionals, whose main concern may be that they have adhered to a professional standard. A client who is unable to access the service because it is only available during normal working hours may assess the service as being of poor quality. Other stakeholders, such as the manager or the service provider, however, may evaluate the same service as being of good quality because their main focus may be on the advice given or the user friendliness of the hospital or health centre.

Øvretveit (1992), in the health care context, identifies three key stakeholder perspectives and these are:

• Client quality (what clients and carers want from the service).

• Professional quality (whether the service meets needs as defined by professional providers and referrers).

• Management quality (the most effective and productive use of resources).

In a holistic account of a service, an understanding of each of these perspectives is necessary and further, in the measurement of service quality, this must be operationalised.

Other issues also arise. Donabedian (1988) contends that before quality can be defined, broader principles must be considered, including whether one takes into account only the performance of practitioners or whether the contributions of patients and of the health care system should also be considered. While it is generally accepted that a service has an impact on a client’s needs, less attention has been paid to the impact of client’s needs on a service and this is particularly the case in community nursing services where even single differences in client group composition, such as social class, can have a significant impact on the work of a public health nurse or health visitor (Reading & Allen 1997, Horrocks et al 1998, Crofts et al 2000).

Other issues also arise, including, for example, how broadly health and responsibility for health are defined; whether maximally or optimally effective care is sought; and, crucially, on whose view that level of care is determined. It is clear, therefore, that quality is a multi-dimensional, multi-faceted concept that can be constructed in a variety of different ways depending on whose perspective or what perspective is dominant and on what elements of the service are taken into account.

While the difficulties outlined above could emerge in respect of almost all services, additional difficulties emerge in respect of public health work where prevention is often a key aim.

Categories of measurement

Two broad categories of measurement have been proposed and these are process measurement and outcome measurement. Judgements of the quality of a service can change depending on where the focus of the measurement is. The example shown in Box 13.1 demonstrates the difference in how the service is judged, depending on whether the focus of the assessment is on the process or outcome.

Box 13.1

Example of difference between process and outcome measurement

In the course of a home visit by the health visitor, a mother of a 4-month-old infant, with whom she has a very good relationship, indicates her intention to give up breastfeeding on her return to work the following month. The health visitor shares her knowledge about the area, communicates it in a non-judgemental way, provides written background material and gives the mother the phone number of a helpline where she can get support. The mother chooses to wean the infant and not continue breastfeeding

If the focus of the assessment is on the process then this service might be judged as very good. If the focus, however, is on the outcome (continuation of breastfeeding) then the process may well be judged as poor.

Outcome measurement has been the dominant paradigm in relation to service quality and an outcome has been defined as:

‘… not simply a measure of health, well-being or any other state but rather, a change in status confidently attributable to antecedent care.’

(Donabedian 1988, p. 1745)

Although some authors subscribe fully to this definition (Lohr 1988), others suggest that the predetermination of change as well as a change in status may not be necessary (Bond & Thomas 1991). In the context of public health a sole focus on outcome measurement is very problematic. Public health has been defined as:

‘… the science and art of preventing disease, prolonging life and promoting health through organised efforts of society.’

(Acheson 1998, p. 289)

A number of concerns arising from the use of outcomes as measures of service quality in public health have been identified in the literature and these are presented in Box 13.2 below.

A key focus of public health work can be understood to be preventive and this concept has proven very difficult to measure (Barriball & Mackenzie 1993, Campbell et al 1995, Hall 1996, Macleod Clark et al 1997). Factors influencing changes in behaviour have been identified as multi-factorial and, further, changes generally take place over a long period of time (Dines & Cribb 1993, Naidoo & Wills 1994, Tones & Tilford 1994). In these situations, questions arise in respect of longitudinality, attribution and valid outcome measures.

Attribution is considered by a number of authors to be a core element in the measurement of quality outcomes (Donabedian 1988, Redfern & Norman 1990, Schuster et al 1997). The multi-dimensional nature of many public health interventions, means however, that ‘outcomes are not predictable, and the lines of attribution are unclear’ (Campbell et al 1995, p. 30). Substantial challenges are posed in determining a cause–effect relationship and, indeed, the work of epidemiologists is almost exclusively focused on this area in respect of disease causation (Hennekens & Buring 1987, Mausner & Kramer 1985). In an epidemiological context, it is assumed that unless the principles of Koch’s postulates (strength of the association, dose–response, consistency of the association, temporally correct association, specificity of the association, and biological plausibility) can be satisfied, a relationship should not be considered cause-and-effect (Mausner & Kramer 1985). Even where only a single disease with a single intervention is under scrutiny, a considerable body of research is required to establish indicators for all of these postulates. How then can attribution be assigned in interventions that are multi-faceted, multi-disciplinary and multi-sectoral?

Predetermination of outcomes is also a key element of outcome identification. Public health is often focused on the identification of health needs in conjunction with the population itself. The search for, and stimulation of an awareness of health needs means that the specific predetermination of outcome is not possible. There is a paradigmatic gulf between a need for predetermined outcomes and an understanding of the service as one premised on the shared establishment of ‘need’ (i.e. shared by public health practitioners and the community).

Other difficulties have also been associated with outcome measurement. It has been suggested, for example, that the art of collecting data is not neutral and that, further, it exerts an influence on the activity it is intended to reflect (Cowley 1994). Whitehead (1993) raises concerns about the possibility that health authorities are interpreting health gain in a way that discriminates in favour of those for whom such gain can be easily achieved. Further, the cost of collecting data on outcome measures may be prohibitive in many situations.

The foregoing discussion clearly highlights the difficulties associated with outcome measurement in respect of the quality of a service. Despite these difficulties it is clear that some form of service quality measurement must take place since the maxims ‘What gets measured gets done’; and ‘If something cannot be measured, it cannot be improved’ have some merit in today’s health care environment. In an era where many Western countries are developing performance indicators for their health care services, it is simply untenable to suggest that public health work cannot be measured in some way. Issues relating to process are discussed later.

Themes, models, frameworks and dimensions of quality

The conceptual challenges outlined earlier are, in turn, mirrored in the multipli-city of approaches that have been identified, and themes, dimensions, models, frameworks and theories have all been presented as mechanisms through which an understanding of service quality can be facilitated.

The three main ‘gurus’ of what is known as total quality management (TQM) are Crosby, Demming and Juran. Joss and Kogan (1995) identify a number of common themes emerging from a comparison of aspects from their work:

• staff commitment (especially from management)

• the production of a medium- and long-term organisation-wide corporate plan

• putting structures in place

• process improvement through a commitment to continuous quality improvement

• costs of quality (and also cost of non-conformance)

• quality of information

• empowering and valuing all staff (through bottom-up approaches)

• focusing on the customer

• training and education

• monitoring and evaluation

• achieving a TQM culture.

There is much crossover between these themes and those of clinical governance, introduced in the UK and conceptually underpinned by:

‘Doing the right thing, for the right person at the right time and getting it right first time, every time.’

(Donaldson & Muir Gray 1998, p. 28).

Clinical governance focuses on the creation of systems, to ensure safe and effective clinical practice (Huntington 2000, Pringle 2000) as well as individual accountability (Allen 2000). Components include:

• evidence-based practice through the creation of opportunities for staff to undertake professional development (Scally & Donaldson 1998)

• continuous quality improvement (Baker et al 1999)

• cohesion, integration and co-operation at a system level (Malcolm & Mays 1999) and cultural change through the development of technical skills, structural skills and effective leadership (Cambell & Proctor 1999, Huntington 2000).

Others have presented ‘models’ of quality. In the corporate sector, Haywood-Farmer (1987), for example, proposed a ‘conceptual model of service quality’ that takes in to account three types of quality service attributes as follows: physical facilities, processes and procedures; people’s behaviour and conviviality; and professional judgement. Itagaki (1997, p. 26) presented a ‘hybrid evaluation mode’, which compared configurations of ‘application and adaptation’ in industrial settings. Jaros and Dostal (1999, p. 197) describe a model based on teleonics, where rather than being physically bounded, systems are informationally bonded. They report that their theory of teleonics ‘arose from a dialectic inter-action between theory and praxis’.

Models of quality have also been presented in the nursing and midwifery literature. Martin-Hirch and Wright (1998), for example, report on a ‘quality mode’ to measure effective midwifery services. Their model identifies client [constant independent (ethnicity, culture, language and religion) and inconstant specific variables (equity, information, continuity, choice and control, each of equal importance)] and service provider elements (tangible and intangible elements). Rantz et al (1999) report on the development of ‘a multi-dimensional theoretical model’ to integrate the views of consumers and providers in nursing-home care quality. Here, consumers’ and providers’ views were integrated around the dimensions of family involvement, communication, environment, staff, care and home. Williams (1998) reports on the use of grounded theory to develop a ‘substantive theory’ on the delivery of quality nursing care from the perspective of the nurse. This substantive theory included time available, conditions, quality of nursing care and selective focusing.

Yet others argue for different approaches to quality. Pffefer and Coote (1991) propose a democratic approach to quality that acknowledges differences between commerce and welfare as well as drawing on scientific (fitness for purpose), excellence (responsiveness), and consumerist (empowerment) approaches to quality. Maxwell (1984) presents the following dimensions and suggests each needs to be included when considering issues of quality: access to services; relevance to need (for the whole community); effectiveness (for individual patients); equity (fairness); social acceptability; efficiency and economy. Many of these were also presented by Donabedian (1990) in his seven ‘pillars’ of quality including efficacy, effectiveness, efficiency, optimality, acceptability, legitimacy, and equity.

Any one of the dimensions identified above, however, could form the whole focus for measurement leading to difficulties in prioritisation. Further, different dimensions may be in competition with each other. Equity, for example, may be forfeited in a situation where cost-effectiveness is to the fore, and cost-effectiveness forfeited in a situation where equity is to the fore (Vågerö 1994).

A foucs on process

Process measures have also been suggested, but are also problematic. Process has been defined as:

‘…what is actually done in giving and receiving care. It includes the patient’s activities in seeking care and carrying it out, as well as the prac-titioner’s activities in making a diagnosis and recommending or implementing treatment.’

(Donabedian 1988, p. 1745)

Many of the processes of the work of public health practitioners remain unexposed, unarticulated and unexplained. The literature on the process of health visiting in the UK demonstrates the complexity of the process when interacting with any individual client (Chalmers 1992, 1993, 1994, Cowley 1991, 1995a, 1995b, 1999, de la Cuesta 1993, 1994, Knott & Latter 1999, Luker & Chalmers 1989, Macleod Clark et al 1997). Cowley (1991, p. 655) writes:

‘It has often seemed difficult for health visitors to explain and articulate the deep processes buried within their work; considerable interpersonal skill and receptivity seem to be needed to negotiate the complexities embedded in the process of “stimulating an awareness of health needs”. Highly developed professional expertise and non-verbal knowledge may be internalized, thus become “invisible”, being recognized only as “intuition”.’

The complexity of the process when interacting with one individual is daunting. In a situation where the community or population is the ‘client’, it is likely that this complexity is magnified. Under the Ottawa Charter (WHO 1986) a number of key public health strategies have been proposed and these include:

• the creation of healthy public policy

• development of supportive environments

• strengthening of community action

• development of personal skills

• re-orientation of health service.

Public health practitioners may have an operational focus on individual elements of any one of these, or they may have a broader, more strategic focus. The processes through which these strategies can be enacted and the circumstances under which they are successfully implemented are likely to be multi-faceted, multi-layered and require many different and complex parallel processes. Most community development work is, by its nature, invisible and requires a strong local understanding of political barriers and needs (Griffiths et al 2003). A focus on social mobilisation, for example, might include community competency, community empowerment, social capital, social connectedness, peer and community norms, public opinion and public mandate for policy action, and community ownership of the programme (Nedham 2000). Any one of these elements, however, might include many different processes and, given the complex and hidden nature of these processes, many questions arise in respect of measurement.

The work of two authors has come to the fore in providing a framework for looking at the process of service delivery. One author focuses on the work of public health nurses (Byrd 1995, 1997, 1998), while the other focuses on the client’s journey through the health care system (Øvretveit 1992). The flow-process model presented by Øvretveit (1992) comprises eight components and these are: selection, entry, first contact, assessment, intervention, review, closure and follow-up. The author suggests that by tracing a typical client’s career through the service from the client’s point of view, it is possible to identify situations and encounters where the client may perceive the service to be poor or where his/her experiences are perceived to be worse than his/her expectations.

The elements of the model are similar to those of the process of home visiting observed by Byrd (1995) from a public health nursing perspective (Table 13.1). Byrd proposes that child-focused home visiting patterns are conducted in three distinct patterns: single, short and long term. The three models presented by Byrd are:

• preliminary process (Byrd 1997)

• child-focused, single home visiting (Byrd 1997)

• long-term maternal–child home visiting (Byrd 1998).

Table 13.1 Steps of three process models presented by Byrd (Byrd 1995, 1997, 1998)

Preliminary model	Child-focused single home visiting process	Long-term maternal–child home visiting
Identifying medium contacting, going to see, entering, seeing, terminating, telling	Surveying and designating, selling and scheduling, approaching the home, entering the home, gaining permission, making the care-giving judgement, ending the visit, haunting and telling	Responding and scheduling approaching the home, entering the the home, starting with mothers expressed concern, supporting and validating, care-giving, ending the visit, after the visit, maternal consequences, child consequences, environmental consequences

The steps of process outlined in each of Byrd’s model highlight differences according to the purpose of the visit. In addition, each of these processes is concerned with home visiting rather than with general interaction between public health practitioner and client. Nevertheless, the steps outlined above provide some explication of the process through which a service is delivered. Both models, however, are deficient in terms of quality because neither provide a mechanism through which the elusive and nebulous concept of quality can be made overt.

Donabedian

The most referenced framework used in service quality is that proposed by Donabedian: structure, process and outcome (SPO). It has been used in the setting of standards of nursing care (Barker 1991, Maycock 1989, Parsley & Corrigan 1994), ethical principles and health care quality (Huycke & All 2000), the identification of medical outcomes (Tarlov et al 1989), organisational effectiveness (Mark et al 1997), the assessment of palliative day care (Douglas et al 2000), evaluation of discharge planning (Closs & Tierney 1993), a primary health care setting (Coyle 2000), and auditing nursing care (Clarke et al 1998). Others have used Donabedian’s SPO as the basis for the development of their own models (Attree 1996, Holzemer 1994, Mitchell et al 1998).

Donabedian’s assertion that all three (structure, process and outcome) approaches are necessary suggests a holistic commitment to assessing service quality, see Box 13.3 below.

Box 13.3

Definition of elements of Donabedian’s framework

Structure denotes:

‘…the attributes of the settings in which care occurs. This includes the attributes of material resources (such as facilities, equipment and money), of human resources (such as the number and qualifications of personnel), and of organisational structure (such as medical staff organisation, methods of peer review and methods of reimbursement.’

(Donabedian 1988, p. 1745)

Process is:

‘…what is actually done in giving and receiving care. It includes the patient’s activities in seeking care and carrying it out, as well as the practitioner’s activities in making a diagnosis and recommending or implementing treatment.’

(Donabedian 1988, p. 1745)

Outcome is:

‘The effects of care on the health status of patients and populations. Improvements in the patient’s knowledge and salutary changes in the patient’s behaviour are included under a broad definition of health status, and so is the degree of the patient’s satisfaction with care.’

(Donabedian 1988, p. 1745)

‘For now, all that is needed is to accept provisionally that there are three major approaches to quality assessment: “structure,” “process,” and “outcome.” This three-fold approach is possible because there is a fundamental functional relationship among the three elements, which can be shown schematically.’

(Donabedian 1980, p. 83)

The relationship shown by Donabedian (1980) is as follows:

Structure → Process → Outcome

There have been a number of criticisms of Donabedian’s framework. Problems with delineation of the categories of structure, process and outcome have been identified by a number of authors (Closs & Tierney 1993, Fihn 2000, Parsley & Corrigan 1994). Fihn, for example, wrote that ‘knowledge is the key to good quality care and knowledge is in and of itself neither a structure nor a process’ (p. 1741). This is clearly problematic, because delineating concepts, a key element of theoretical development, can be difficult if one item crosses two categories (Davidson Reynolds 1971). Donabedian never dealt satisfactorily with this criticism. The second problem related to the way in which others constructed Donabedian’s work and many authors understood Donabedian’s work as giving primacy to outcome measurement over process (Badger 1999, Carr-Hill 1994, Coyle 2000, Mitchell et al 1998). Carr-Hill, in making a case for process measures of care, lamented the ‘over-indoctrination’ of people by Donabedian’s focus on outcome. Donabedian himself, however, has made the case many times for the primacy of process over outcome (Donabedian 1968, 1980, 1988, 1993). As recently as 1993, he wrote:

‘I place the interaction of patients and practitioners at the centre of the health care universe because I believe that it is there that the processes and decisions most critical to quality take place.’

(Donabedian 1993, p. 33).

Despite these criticisms, the SPO framework proposed by Donabedian has the capacity to take account of services in a holistic way that includes the organisational context within which people work, the process of the service and the consequences of the service. For these reasons, the study on developing a holistic model of service quality was broadly guided by his work.

A holistic model of service quality

The rest of the chapter draws heavily on a holistic model of service quality, which was developed through research about the public health nursing service to families with infants. The study methods are summarised in Box 13.4. The model is ‘holistic’ because it incorporates multiple stakeholders’ views, takes account of the organisational context within which the service is provided and enables both process and consequence to be taken into account.

Box 13.4

Methodology

A two-phase, collective case study approach was used to research the public health nursing service to families with infants in the Republic of Ireland [a more detailed explanation of the methodology is presented elsewhere (Hanafin 2003, Hanafin & Cowley 2006)]. Phase one involved collecting quantitative and qualitative data, through a national census of public health nurses (PHNs) (response rate 54%; n = 946), PHN managers (response rate 75%; n = 24) and small group interviews (n = 5). In phase two, group (n = 3) and individual (n = 14) interviews with clients, PHNs and PHN managers were carried out and data emerging from these were supplemented by non-participant observation of the public health nursing service at each of four case study sites.

Analysis of the questionnaire was undertaken using the software package SPSSx and descriptive and inferential techniques. Analysis of individual cases, using concepts, categories and codes was followed by cross-case analysis and triangulation of data, sources and methods. This was facilitated by qualitisation of quantitative data, which was undertaken on completion of the statistical analysis and involved writing the findings in profile documents according to each section of the questionnaire (Tashakkori & Teddlie 1998).

The analysis resulted in a thick description of the structure, process and consequences of the public health nursing service provided along with the identification of five key concepts. These elements form the basis for the model of service quality presented in this chapter.