Epidemiological Methods and Measurements in Population-Based Nursing Practice: Part II

Patty A. Vitale and Ann L. Cupp Curley

In order to provide leadership in evidence-based practice, advanced practice registered nurses (APRNs) require skills in the analytic methods that are used to identify population trends and evaluate outcomes and systems of care (American Association of Colleges of Nursing [AACN], 2006). APRNs need to be able to carry out studies with strong designs and solid methodology, taking into account the factors that can affect study results. This chapter discusses the complexities of data collection and the strengths and weaknesses of study designs used in population research. Critical components of data analysis are discussed, including bias, causality, confounding, and interaction.

ERRORS IN MEASUREMENT

A dilemma that may occur with population research is the difficulty of controlling for variables that are not being studied but that may have an impact on the results. Finding a statistical association between an intervention and an outcome or an exposure and a particular disease is meaningful only if variables are correctly controlled, tested, and measured. The purpose of a well-designed study is to properly identify the impact of the variable (or variables) under study and to avoid bias and/or design flaws caused by another, unmeasured variable.

TABLE 4.1    •    Type I and Type II Errors

Systematic Error

There are several types of systematic error or bias and all of them can impact the validity of study results. Bias can occur in many ways and is commonly broken down into 2 categories: selection and information bias. Such things as how the study design is selected, how subjects are selected, how information is collected, how the study is carried out (the conduct), or how the study is interpreted by investigators are all forms of potential bias. These problems can result in a deviation from the truth, which can lead to false conclusions.

Selection Bias

Information Bias

Information bias deals with how information or data are collected for a study. This includes the source of data that are collected, such as hospital records, outpatient charts, or national databases. Many of these types of data are not collected for research purposes; so they may be incomplete, inaccurate, or contain information that is misleading. This can complicate data analysis as the information abstracted from these sources may be incorrect and can lead to invalid conclusions. Measurement bias is a form of information bias and occurs during data collection. It can be caused by an error in collecting information for an exposure or an outcome. Calibration errors can occur when using instruments to measure outcomes. This type of bias can also occur when an instrument is not sensitive enough to measure small differences between groups or when interventions are not applied equally (e.g., blood pressure measurements taken using the wrong cuff size). Information bias also includes how the data are recorded and classified. This can lead to misclassification bias, in which a control may be recorded as a case or a case is classified as having an exposure or exposures that he or she did not actually have. Misclassification bias can be subdivided into differential and nondifferential. For example, differential misclassification occurs when a case is misclassified into exposure groups more often than controls. In this case, this type of bias usually leads to the appearance of a greater association between exposure and the cases than one would find if this bias was not present (Gordis, 2014). In nondifferential misclassification, the misclassification occurs as a result of the data-collection methods such that a case is entered as a control or vice versa. In this situation, the association between exposure and outcome may be “diluted,” and one may conclude there is not an association when one really exists (Gordis, 2014). Another example of misclassification bias occurs when members of a control group are exposed to an intervention. This results in contamination bias. An example would be a nurse who floats from the floor where hourly rounding is being carried out to a control floor where no rounding is supposed to occur, but the nurse carries out hourly rounding on the control floor. In this case, contamination bias minimizes the true differences that would have been seen between groups. However, these cases should not be reassigned; in fact, any unexpected or unplanned crossover that occurs should be analyzed in the original group to which it was assigned by the investigator. This is known as the intent-to-treat principle. For example, patients who are assigned to one group or another and crossover intentionally or accidentally to the other group should be analyzed according to their original assignment. Intent to treat simply means that you assign patients to the original group you intended to treat them in from the start of the study regardless of the treatment they received.

Identification of confounding or other causes of spurious associations are important in population studies. A confounder is a variable that is linked to both a causative factor or an exposure and the outcome. There are many examples of confounders, such as age, gender, and socioeconomic status. Confounding occurs when a study is performed, and it appears from the study results that an association exists between an exposure and an outcome, when, in fact, the association is actually between the confounder and the outcome. Another example of confounding might occur if an APRN carried out a study to determine whether there is a relationship between age and medication compliance without controlling for income. Younger, working patients might be more compliant not because of the age factor, but because they have the resources to buy their medications. If confounding is ignored, there can be long-term implications as the APRN may implement interventions for medication compliance with education programs aimed at older patients without considering problems related to income. The intervention would ultimately not succeed because the relationship is false or not causal due to confounding. By definition, confounders must be known risk factors for the outcome and must not be affected by the exposure or the outcome (Gordis, 2014). Confounding, although difficult to avoid, must be recognized and accounted for in studies.

In individual matching, each individual case is matched to a control with similar characteristics of interest. This is referred to as matched pairs. One has to be careful not to match cases and controls for too many characteristics, as it can be difficult to find a control or the control may be too similar to the case and true differences may not be able to be demonstrated in the analysis phase. Using strict inclusion and exclusion criteria also can be helpful and should be applied similarly for comparison groups. There are limitations to the latter two methods.

Although it is possible to match for known confounding variables, there may be other unknown confounding variables that cannot be controlled for and, if not recognized, can impact study conclusions. If the study groups are matched for gender, then gender cannot be evaluated in the final analysis. Additionally, if the study is matched for too many variables, this can also limit the study as all of the matched variables cannot be studied, and this may limit the ability to make valid conclusions. There is a similar problem with inclusion and exclusion criteria as both criteria should be applied the same to each of the study groups. The method for analyzing data can also help reduce problems related to confounding.

Multivariable regression, for example, can measure the effects of multiple confounding variables. This method is useful only when the variables are recognized and acknowledged. Recognition of confounders requires a basic understanding of the relationship between an exposure and a disease or an outcome, and can also be identified by performing a stratified analysis first. Once confounders are determined, then these variables can be added in and removed from the model one at a time. Interaction needs to be assessed, and the exposure–disease relationship is determined. These inferential methods estimate the contribution of each variable to the outcome while holding all other variables constant in the model. The objective is to include a set of variables that are theoretically or actually correlated with both the intervention and the outcome to reduce the bias of treatment effect. Therefore, the goal of regression analysis is to identify causal relationships by recognizing the confounders to ensure found relationships are real and not spurious (Kellar & Kelvin, 2013; Starks, Diehr, & Curtis, 2009).

INTERACTION

There are clearly many sources of error that can occur while conducting a study. The informed APRN needs to identify and acknowledge these types of errors and work to minimize them. Therefore, it is essential that APRNs recognize when errors occur and how they can impact a study, and should be familiar with measures that can be taken to avoid or minimize errors.

RANDOMIZATION

Randomized controlled trials (RCTs) are considered inherently strong because of their rigorous design. Random selection of a sample and random assignment to groups are objective methods that can be used to prevent bias and produce comparable groups. Random assignment helps to minimize bias by ensuring that every subject has an equal chance of being selected and that results are more likely to be attributed to the intervention being tested and not to some other extraneous factor such as how subjects were assigned to the treatment or control group. It is impossible to know all of the characteristics that could influence results. The random assignment of subjects to different treatment groups helps to ensure that study groups are similar in the characteristics that might affect results (e.g., age, gender, ethnicity, and general health). As a general rule, the greater the number of subjects that are chosen and assigned into the treatment or nontreatment groups, the more likely that the groups will be similar for important characteristics (Shott, 1990).

Blinding

Another problem encountered in research occurs when investigators or subjects themselves have an effect on study results. This can happen when a researcher’s personal beliefs or expectations of subjects can influence his or her interpretation of the outcome. Sometimes observers can err in measuring data toward what is expected. If subjects know or believe that they are given a placebo or the nonexperimental treatment, it may cause them to exaggerate symptoms that they would dismiss if given the experimental treatment. These actions by both investigators and subjects are not necessarily intentional; they can occur subconsciously.

DATA COLLECTION

As mentioned earlier, how data are collected and analyzed can lead to bias when conducting a study. The training of investigators to ensure that data are collected uniformly from all subjects and the use of a strict methodology for data collection and analysis contribute to a strong study design. Objective criteria should be used for the collection of all data. Strict inclusion and exclusion criteria should be developed in writing so that there are no questions as to what criteria are to be applied to the study. Avoiding subjective criteria is important as it can lead to inconsistent application of criteria. For example, if you chose “ill appearance” as an exclusion criterion, it may be difficult to apply this criterion uniformly as each APRN may have different levels of experience making this assessment. Objective criteria, such as heart rate greater than 120 beats per minute, respiratory rate greater than 24 breaths per minute, or oxygen saturations less than 90%, are easy to apply uniformly. Of course, even those criteria can be incorrectly assessed by someone who is inexperienced; however, one can see that these types of data are more easily reproducible within and between studies. One way to assess reliability between raters in a study is by using the kappa statistic. This statistic tests how reliable different investigators or data collectors are in their assessment or interpretation of data beyond what one would expect by chance alone.

If you were to evaluate two observers without any training, you would expect them to agree a certain percentage of the time and that percentage represents the chance of agreement, usually around 50%. Using the kappa statistic, you can estimate how reliable this agreement is by subtracting out the percentage expected by chance alone. Kappa values that are greater than 0.75 represent excellent agreement, and those values less than 0.40 represent poor agreement. These values, although not perfect, can give an investigator an assessment of how well her or his observers are agreeing with each other in their data interpretation (Landis & Koch, 1977). The kappa statistic appears frequently in the literature, and the APRN should be familiar with its use and limitations (Maclure & Willett, 1987).

CAUSALITY

Ernst Mach, an Austrian professor of physics and mathematics and a philosopher, argued that all knowledge is based on sensation and that all scientific measurements are dependent on the observer’s perception. He proposed that “in nature there is no cause and effect” (Hutteman, 2013, p. 102). This is a relevant quote to begin a discussion of causality, because causality is a complex issue faced by all investigators. A single clinical disease can have many different “causes,” and one cause can have several clinical consequences. Causality becomes even more complex when we begin to look at chronic diseases. Chronic diseases can have multiple etiologies. Cardiac disease, for example, has multiple causes such as genetic predisposition, obesity, smoking, lack of exercise, poor diet, or any combination of these factors.

Causes can be both direct and indirect. An example of a direct cause would be an infectious agent that causes a disease. Pertussis (whooping cough, a bacterial infection) is caused by Bordetella pertussis. The disease is a direct cause of the organism. Toxic shock syndrome is an example of an indirect cause. Although the staphylococcal organism and its toxins are the direct cause of the syndrome, the indirect cause (and the first factor that was identified) is tampons.

Even the infectious disease process is not simple. Both the host and the environment can have an impact on the infectious disease process. Characteristics of the host (e.g., age, previous exposures, general health, and immune status) can influence the development of the disease. Environmental conditions also play a role. A good example is influenza, which is most prevalent during certain times of the year. Infectious disease departments document these seasonal trends during the year, and they are available for hospitals to review. Such information can assist in antibiotic selection, hospital staffing, and educational campaigns to ensure immunizations or prevention programs are put into place. Awareness of seasonal fluctuations in certain diseases, trends in drug resistance, or changes in the community that affect the overall management of a patient are important in an APRN’s practice. By following these trends, the APRN can better assess the needs of the community and ensure that appropriate resources are available to address the fluctuations that occur naturally in all communities.

SCIENTIFIC MISCONDUCT (FRAUD)

Perhaps one of the most infamous cases of fraud involved a well-respected peer-reviewed journal. In 1998, The Lancet published an article written by Andrew Wakefield and 12 others that implied a link between the measles–mumps–rubella (MMR) vaccine and autism and Crohn’s disease. Although epidemiologists pointed out several study weaknesses, including a small number of cases, no controls, and reliance on parental recall, it received wide notice in the popular press. It was 7 years before a journalist uncovered the fact that Wakefield altered facts to support his claim and exploited the MMR scare for financial gain. The Lancet retracted the paper in 2010 (Godlee, Smith, & Marcovitch, 2011). A series of articles in the British Medical Journal (Deer, 2011) revealed how Wakefield and his associates distorted data for financial gain. Before this article was retracted, it caused widespread fear among parents and accelerated an antivaccine movement that many blame for the resurgence of infectious diseases among children.

In 2006, a writer for The New York Times (Interlandi, 2006) wrote an article that described a case of fraud that involved a formerly tenured professor at the University of Vermont. Dr. Eric Poehlman was tried in a federal court and found guilty. He was sentenced to 1 year and 1 day in jail for fraudulent actions that spanned 10 years. His misconduct included using fraudulent data in lectures and in published papers, and using these data to obtain millions of dollars in federal grants from the National Institutes of Health (NIH). He pleaded guilty to fabricating data on obesity, menopause, and aging. Interlandi’s article, which includes a very detailed account of Dr. Poehlman’s actions and his downfall, documents how a “committed cheater can elude detection for years by playing on the trust—and self-interest—of his or her junior colleagues” (p. 3).