Chapter 24. Delphi Studies
Hugh McKenna and Sinead Keeney
▪ Introduction: what is the Delphi technique?
▪ Background
▪ The Delphi process
▪ Application of the Delphi in nursing research
▪ Reliability and validity
▪ Conclusion
Introduction: what is the Delphi technique?
I have brought golden opinions from all sorts of people.(Macbeth vii:31)
The Delphi technique is a structured process, which uses a series of questionnaires (or rounds) to gather information. The process continues until ‘group’ consensus is reached (Beretta 1996 and Green 1999). It has grown in popularity recently within nursing research (McKenna 1994a). Like questionnaires, it allows the inclusion of a large number of individuals across diverse geographic locations and expertise; unlike questionnaires, the Delphi aims to gather consensus of opinion, judgement or choice. In face-to-face meetings a specific member or members might dominate the discussions. One of the advantages of the Delphi is that the members (experts) do not meet and their comments and preferences are gained through anonymised surveys leading towards consensus (Jaiarth & Weinstein 1994).
Background
The Delphi technique was originally developed by the RAND Corporation where its initial purpose was technological forecasting for the military. Since its inception the Delphi technique has evolved into a number of modifications. Each type of Delphi has the same aim – to gain consensus on the issue at hand – but differs in the process used to reach this consensus. The different types of Delphi include the classic Delphi (McIlfatrick & Keeney 2003), the modified Delphi (McKenna 1994a), the policy Delphi (Crisp at al 1997), the real-time Delphi or conference Delphi (Beretta 1996) and, more recently, the e-Delphi (Avery et al 2005). There are many studies reporting on these different manifestations, demonstrating the flexibility of the method.
The Delphi process
The classic Delphi involves the presentation of a questionnaire to a panel of ‘informed individuals’ in a specific field of application, to seek their opinion on a particular issue. Data are summarised and a new questionnaire is designed based solely on the results obtained from the first application. This second instrument is returned to each subject and they are asked (in the light of the first round results), to reconsider their initial opinions. Repeat rounds may be carried out until consensus of opinion, or a point of diminishing returns, has been reached.
In essence, the Delphi technique is a multistage approach with each stage building on the results of the previous one. Hitch and Murgatroyd (1983) see it as resembling a highly controlled meeting of experts, controlled by a chairperson who is adept at summing up the feelings of the meeting by reflecting the participants’ own views back to them in such a way that they can proceed further – the only difference is that the individual responses of the members are unknown to one another. In the first round of any Delphi investigation, a wide divergence of individual opinion is typical. Nevertheless, after several iterations there is a tendency for subjects to converge towards consensus.
The modified Delphi is similar to the classic Delphi but uses interviews or focus groups to gather the first round opinions (McKenna 1994a). The policy Delphi is used mostly within organisations to examine and explore policy issues (Turoff & Linstone 2002) with 10 to 50 people as a precursor to a committee meeting and used to explore the advantages and disadvantages of a certain issue. A face-to-face committee would then use the Delphi results to formulate the required policy.
The real-time or conference Delphi uses the Delphi process in a ‘real-time’ setting, eliminating the delay caused by the pen-and-paper type Delphi. It takes the form of panel members undertaking rounds face-to-face and analysis being undertaken in situ between each round (Linstone & Turoff 2002). The e-Delphi is a newer approach and has developed as a result of advancing technology. This is the administration of either the classic or modified Delphi electronically by email or completion of an online form (Avery et al 2005).
Defining the Delphi
According to Dalkey and Helmer (1963), the Delphi technique is ‘a method used to obtain the most reliable consensus of opinion of a group of experts by a series of intensive questionnaires interspersed with controlled feedback’ (p. 458). With increasing usage, broader definitions have been put forward. For instance, Reid (1998) believes the ‘Delphi’ is a method for the systematic collection and aggregation of informed judgement from a group of experts on specific issues.
Lynn et al (1998) defined the Delphi technique as an iterative process designed to combine expert opinion into group consensus. Most definitions attempt to encompass or highlight the ever-adapting Delphi process in one sentence, resulting in broad and varying interpretations. Regardless of definition, the purpose of the technique is to achieve consensus among a group of experts on a certain issue where no agreement previously existed.
The lack of uniformity in the use of the technique has spawned criticism from some researchers stating that the emergence of modifications of the technique poses a threat to its credibility and to the validity and reliability of the research findings (Sackman 1975). Box 24.1 outlines the characteristics of the original classic Delphi (McKenna 1994a), from which modifications have developed.
Box 24.1
Characteristics of a classic Delphi (McKenna 1994a)
1 The use of a panel of ‘experts’ for obtaining data
2 Participants do not meet in face-to-face discussions
3 The use of sequential questionnaires and/or interviews
4 The systematic emergence of a concurrence of judgement or opinion
5 The guarantee of anonymity for participants’ responses
6 The use of frequency distributions to identify patterns of agreement
7 The use of two or more rounds, between which a summary of the results of the previous round is communicated to and evaluated by panel members
Sampling and the use of experts
The Delphi does not use a random sample which is representative of the target population, but rather employs ‘experts’ as panel members. This means that each panel member is an expert in the area of interest. Experts have been defined as: a group of ‘informed individuals’ (McKenna 1994a); ‘specialists’ in their field (Goodman 1987); and someone who has knowledge about a specific subject (Green et al 1999). For example, a study that investigates the changing role of the midwife may include midwives who are knowledgeable about the subject under consideration (Lemmer 1998).
Sackman (1975) and Linstone and Turoff (1975) criticised the use of experts, as did Strauss and Zeigler (1975) who claimed that valid expert opinion is scientifically untenable and overstated. Similarly, in her critical examination of the Delphi, Goodman (1987) noted that:
It would seem more appropriate to recruit individuals who have knowledge of a particular topic and who are consequently willing to engage in discussion upon it without the potentially misleading title of ‘expert’ (p. 732).
One other criticism of the selection of experts is the potential for selection bias affecting the results. For example, if you were seeking consensus on whether the Labour Party should lead the UK government in the next 10 years, there is obvious bias in having the panel composed entirely of Labour Party members!
Panel selection
Deciding on the experts to include in the Delphi panel is regarded as the ‘linchpin of the method’ (Green et al 1999). There is little agreement in the literature regarding the relationship of the panel to the larger population of experts and the sampling method used to select such experts (Green 1999 and Williams 1994). Guidance from the literature suggests that, when using the Delphi, it is not necessary to calculate sample size using power calculations. Beretta (1996) stated ‘representative sample techniques may be inappropriate when expert opinions are required’ (p. 83). Previously, Goodman (1987) pointed out that, originally, the Delphi ‘tends not to advocate a random sample of panellists’ (p. 730). Moreover, Helmer (1977) argued that:
… a Delphi inquiry is not an opinion poll, relying on a random sample … rather once a set of experts has been selected (regardless of how) it provides a communication device for them, that uses the conductor of the exercise as a filter in order to preserve anonymity of responses. (p. 17).
In contrast, Reid (1998) maintained that the generalisability of results cannot be claimed unless it can be stated that these panels constitute a genuine population.
The importance of using ‘criteria’ to select a Delphi sample is an area that has grown in popularity (Keeney et al 2006). For example, these may include having published at least one paper in the area of investigation if it is an academic issue, or having 10 years’ clinical experience in a certain role if the area of investigation requires specific clinical knowledge. Criteria can be simple or multilayered depending on necessity. Simple criteria can denote gender, age or educational attainment.
Panel size
There is no universal agreement on what size the sample should be when using the Delphi technique. Linstone (1978)reported on studies using several hundred panellists and a Japanese Delphi that had several thousand panellists. Dalkey and Helmer (1963) stated that a suitable number is seven and that accuracy deteriorates with less and improves with more. Alexander and Kroposki (1999) asserted that the sample size should be over 60, while Burns (1998) argued for 15. Beech (1999) suggested that this disagreement over sample size is one of the disadvantages of conducting a Delphi study, whereas. Box 24.2 outlines the important factors to consider when deciding on the size of the sample for the Delphi technique.
Box 24.2
Factors to consider regarding sample size
▪ Consider the number of experts in the field and purposively select a sample, taking into account location, grade etc. (see below under sample selection).
▪ Select a manageable sample – there is considerable administration with a Delphi and the larger the sample the more work will be needed.
▪ Consider time constraints as this will have an impact on the sample size – the larger the sample, the more time will be spent following up people to return the questionnaire as you will want to maximise the response rate as much as possible.
▪ Generally the larger the sample, the poorer the response rate. If the sample can be kept small and incorporate interested experts, a good response rate is more likely, which in turn will increase the chances of a good level of consensus.
Anonymity and confidentiality
It should be noted that complete anonymity is not possible when using the Delphi technique. However, lack of complete anonymity is a fact that many researchers who use the Delphi do not address. The researcher needs to be able to link panel members with their responses, which threatens true anonymity. The reason for this is that the researcher may provide feedback in the form of individual response to the previous round as well as the overall group response.
This lack of complete anonymity may raise ethical issues in relation to confidentiality. It is vital that the researcher makes it clear that responses will be visible to the researcher but not to the other panel members. This will ensure that the participant is fully informed before deciding whether to participate.
Panel members often know other panel members, but cannot attribute responses. It is like being in an elite ‘expert’ club where the membership is known but they do not meet face-to-face. McKenna (1994a) uses the term ‘quasi-anonymity’ to describe this situation. Sackman (1975) argued that this can lead to a lack of accountability for the views expressed, while Goodman (1987) maintained that it encourages hasty ill-considered judgements. However, Rauch (1979) postulates that knowing who the other subjects are should have the effect of motivating the panellists to participate.
This promise of quasi-anonymity also facilitates panel members to be open and truthful about their views; this, in turn, provides insightful data for the researcher. The only difficulty may be if a panel member and the researcher know each other and the former’s responses are influenced because of this. Such panel members may be excluded from the study.
The basis of the Delphi is that panel members alter their views towards consensus as the study progresses. However, it is unclear whether they do this on the basis of new information or, despite the protection of quasi-anonymity, feel pressurised to conform to the ‘group think’.
Delphi rounds
Within the Delphi technique the number of rounds depends upon the time available and what type of Delphi the researcher has employed, raising the question of how many rounds it takes to reach consensus. The original classic Delphi used four rounds (Young & Hogben 1978). However, this has been modified by many to suit individual research aims and in some cases it has been shortened to two or three rounds (Beech 1997 and Green 1999) by using focus groups or interviews as a substitute for the first qualitative round (Keeney et al 2006). The difficulty with a three or four rounds Delphi is keeping the panel members interested and committed to returning each round. The topic needs to be of great interest to the panel members or they have to be rewarded in other ways, otherwise the response rate can suffer.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
