Sampling methods and external validity

5


Sampling methods and external validity




Introduction


Research in the health sciences usually involves the collection of information from a sample of participants, rather than on the entire population in which the investigator is interested. Studies that involve an entire population or group are called census studies but these are relatively rare and generally expensive to perform. A sample drawn from the target population is studied because it is usually impossible or too costly to study entire populations. For instance, when individuals who have conditions such as diabetes, cerebral palsy or emphysema are being studied, it is not possible to study everyone because of the large size of such populations. Also because many people do not seek treatment or may be wrongly diagnosed, we may not be able to identify all members of the entire population in order to study them. Therefore, in most research, the researcher studies a subset or sample of the target population, and then attempts to generalize the findings to the population from which the participants were drawn. This general principle applies to both qualitative and quantitative studies.


The aim of this chapter is to examine ways in which samples can be drawn to permit the investigator to make valid generalizations from the study sample to the target population.


We will also consider the question of generalizing the findings of an investigation to other samples and situations. This is referred to as ‘external validity’ or ‘generalizability’.


The specific aims of this chapter are to:




What is sampled in a study


While this chapter will focus on the selection of the research participants in a study, many other things are also selected or sampled. These include:



Many researchers focus on the selection of the research participants as the key or only issue in maximizing the generalizability of their research and they do not pay enough attention to the other factors they are sampling or the context in which the research is being conducted. It is not at all unusual to see studies that employ large and sophisticated participant samples yet with only one or two highly selected clinicians involved in the research in perhaps only one health setting. While the study sample may be highly representative the context in which the research is conducted may not be and it may be that the researchers and clinicians involved in the study have a particularly unusual or idiosyncratic approach to their work that is not reflective of others. It is our contention that, in many qualitative studies, there is strong consideration of the research context and its impact upon the research findings. However, there is often less emphasis upon sampling of research participants. This impacts upon the ability to generalize the findings more broadly from the actual research participants to other groups.



Basic issues in sampling


As we have discussed, often, because of the numbers involved, it is not within the resources of the researcher to study the whole target population. In any event, in most situations it would be wasteful to study all of the population. If a sample is representative, one can generalize validly from the sample’s results to the population without going to the expense of studying everyone.


The population is the target group of individuals or cases in which the researcher is interested. Examples of valid study populations include: all English women under 25; all children with diagnosed spina bifida in the state of Alberta; all the students at a particular Australian college. The researcher defines the population to which he/she wishes to generalize. Note that a population need not consist of human participants or animal subjects. Objects or events can also be sampled, as shown in Table 5.1.



As can be seen from Table 5.1, a population is an entire set of persons, objects or events which the researcher intends to study. A sample is a subset of the population. Sampling involves the selection of the sample from the population.



Representative samples


There is a variety of different ways by which one can select the sample from the population. These are called sampling methods.


The ultimate aim of all sampling methods is to draw a representative sample from the population. The advantage of a representative sample is clear: one can confidently generalize from a representative sample to the rest of the population without having to take the trouble of studying the rest of the population. If the sample is biased (not representative of the population) one can generalize less validly from the sample to the population. This might lead to quite incorrect conclusions or inferences about the population. This would mean that the results obtained in the study would not necessarily generalize to other studies using the same population. Figure 5.1 illustrates the concept of a representative sample.



Figure 5.1 illustrates a hypothetical population composed of three different types of study participants or categories of participants. A representative sample is a precise miniaturized representation of the population. An unrepresentative or biased sample does not adequately represent the key groups or characteristics in the population, and this may lead to mistaken conclusions about the state of the population.


The selection of the appropriate sampling method depends upon the aims and resources of the researchers. For instance, if someone is designing a very expensive health or social welfare program on the basis of a survey of clients’ needs, it is imperative that the researcher uses a good sampling method and obtains a representative sample of the clients, so that appropriate conclusions may be reached about the population. Good sampling methods are somewhat more expensive and more difficult to implement than poor methods but they are worth it. The main sampling methods used in health research are incidental and random sampling.



Incidental samples



Incidental sampling


Incidental sampling is the cheapest, easiest and most commonly used sampling method in clinical studies. It involves the selection of the most accessible and available members of the target population. For example, a researcher who stands in the middle of a city street and quizzes people about their health status is practising incidental sampling. However, it is quite likely that this sample would not be representative of the general voting population. There would probably be an over-representation of businessmen and white-collar workers, and an under-representation of factory workers and housewives. The sample is likely to be unrepresentative and biased.


A further example of incidental sampling might involve a researcher surveying the needs of a group of spina bifida children at a local community health centre. Their measured needs may be representative of those of other spina bifida children but then again they may not if these children are not typical of the wider population of children with spina bifida.


Thus, incidental sampling is cheap and easy to implement but may give a biased sample that is not representative of the population.



Quota sampling


Sometimes it is known in advance that there are important subgroups within the population that need to be included in the sample. Two such important groups within the human population are males and females. Further, it is known that they occur in the ratio of approximately 49:51 in the general population. Our researcher might decide that it is very important that the sexes are proportionally represented in the sample. Thus, the researcher would set two quotas of 49 male and 51 female respondents in a sample of 100. This is still a form of incidental sampling but has some significant advantages over simple incidental sampling because the study sample’s composition on this key demographic variable is guaranteed to match that of the target population.


More sophisticated examples involving more than two groups can be accommodated as shown in Table 5.2. We can see from Table 5.2 that, if our sample were to be representative regarding both sex and occupational status, in a sample of 100 people we would need 19 blue-collar males, 15 blue-collar females, and so on.



Quota sampling still has a number of shortcomings: before it can be used, one has to know which population groups are likely to be important to a particular question and the exact proportions of the various groups in the population. Sometimes we may not know these proportions. Also, the members of the sample within the quotas are still incidentally chosen. The blue-collar males, for example, selected in a city centre on a weekday may still be quite different from those working elsewhere. However, quota sampling is better than simple incidental sampling.

Related posts:

  1. Surveys and quasi-experimental designs
  2. Measurement
  3. Standard scores and normal distributions
  4. Measures of central tendency and variability

Stay updated, free articles. Join our Telegram channel
Tags:
Apr 12, 2017 | Posted by in MEDICAL ASSISSTANT | Comments Off on Sampling methods and external validity

Full access? Get Clinical Tree
Get Clinical Tree app for offline access