Directness of Measurement

To measure, the researcher must first identify the object, characteristic, element, event, or situation to be measured. In some cases, identifying the object to measure and determining how to measure it are quite simple, such as when the researcher measures a person’s weight and height. These are referred to as direct measures. Direct measures involve determining the value of concrete factors such as weight, waist circumference, temperature, heart rate, BP, and respiration. Technology is available to measure many bodily functions, biological indicators, and chemical characteristics. The focus of measurement in these instances is on the accuracy and precision of the measurement method and process. If a patient’s BP is to be accurate, it must be measured with a quality stethoscope and sphygmomanometer and must be precisely or consistently measured, as discussed earlier in the introduction. In research, three BP measurements are usually taken and averaged to determine the most accurate and precise BP reading. Nurse researchers are also experienced in gathering direct measures of demographic variables such as age, gender, ethnic origin, and diagnosis.

However, in many cases in nursing, the thing to be measured is not a concrete object but an abstract idea, characteristic, or concept such as pain, stress, caring, coping, depression, anxiety, and adherence. Researchers cannot directly measure an abstract idea, but they can capture some of its elements in their measurements, which are referred to as indirect measures or indicators of the concepts. Rarely, if ever, can a single measurement strategy measure all aspects of an abstract concept. Therefore multiple measurement methods or indicators are needed, and even then they cannot be expected to measure all elements of an abstract concept. For example, multiple measurement methods might be used to describe pain in a study, which decreases the measurement error and increases the understanding of pain. The measurement methods of pain might include the FACES Pain Scale, observation (rubbing and/or guarding the area that hurts, facial grimacing, and crying), and physiological measures, such as pulse and blood pressure. Figure 10-1 demonstrates multiple measures of the concept of pain and demonstrates how having more measurement methods increases the understanding of the concept. The bold, black-rimmed largest circle represents the concept of pain and the pale-colored smaller circles represent the measurement methods. A larger circle is represented by physiological measures indicating these measures (pulse, blood pressure, and respirations) add more to the objective measurement of pain. Even with three different types of measurement methods being used, the entire concept of pain is not completely measured, as indicated by the white areas within the black-rimmed large circle.

Levels of Measurement

Various measurement methods produce data that are at different levels of measurement. The traditional levels of measurement were developed by Stevens (1946), who organized the rules for assigning numbers to objects so that a hierarchy in measurement was established. The levels of measurement, from low to high, are nominal, ordinal, interval, and ratio.

Nominal-Level Measurement

Nominal-level measurement is the lowest of the four measurement categories. It is used when data can be organized into categories of a defined property but the categories cannot be rank-ordered. For example, you may decide to categorize potential study subjects by diagnosis. However, the category “kidney stone,” for example, cannot be rated higher than the category “gastric ulcer”; similarly, across categories, “ovarian cyst” is no closer to “kidney stone” than to “gastric ulcer.” The categories differ in quality but not quantity. Therefore, it is not possible to say that subject A possesses more of the property being categorized than subject B. (Rule: The categories must not be orderable.) Categories must be established in such a way that each datum will fit into only one of the categories. (Rule: The categories must be exclusive.) All the data must fit into the established categories. (Rule: The categories must be exhaustive.) Data such as gender, race and ethnicity, marital status, and diagnoses are examples of nominal data. The rules for the four levels of measurement are summarized in Figure 10-2.

Measurement Error

The ideal perfect measure is referred to as the true measure or score. However, some error is always present in any measurement strategy. Measurement error is the difference between the true measure and what is actually measured (Grove, Burns, & Gray, 2013). The amount of error in a measure varies from considerable error in one measurement to very little in another. Measurement error exists with direct and indirect measures. With direct measures, both the object and measurement method are visible. Direct measures, which generally are expected to be highly accurate, are subject to error. For example, a weight scale may be inaccurate for 0.5 pound, precisely calibrated BP equipment might decrease in precision with use, or a tape measure may not be held at exactly the same tension in measuring the waist of each patient. A subject in a study may be 65 years old but may write illegibly on the demographic form. As a result, the age may be entered inaccurately into the study database.

With indirect measures, the element being measured cannot be seen directly. For example, you cannot see pain. You may observe behaviors or hear words that you think represent pain, but pain is a sensation that is not always clearly recognized or expressed by the person experiencing it. The measurement of pain is usually conducted with a scale but can also include observation and physiological measures as shown in Figure 10-1. Efforts to measure concepts such as pain usually result in measuring only part of the concept. Sometimes measures may identify some aspects of the concept but may include other elements that are not part of the concept. In Figure 10-1, the measurement methods of scale, observation, and physiological measures include factors other than pain, as indicated by the parts of the circles that are outside the black-rimmed circle of the concept pain. For example, measurement methods for pain might be measuring aspects of anxiety and fear in addition to pain. However, using multiple methods to measure a concept or variable usually decreases the measurement error and increases the understanding of the concept being measured.

Two types of error are of concern in measurement, random error and systematic error. The difference between random and systematic error is in the direction of the error. In random measurement error, the difference between the measured value and the true value is without pattern or direction (random). In one measurement, the actual value obtained may be lower than the true value, whereas in the next measurement, the actual value obtained may be higher than the true value. A number of chance situations or factors can occur during the measurement process that can result in random error (Waltz, Strickland, & Lenz, 2010). For example, the person taking the measurements may not use the same procedure every time, a subject completing a paper and pencil scale may accidentally mark the wrong column, or the person entering the data into a computer may punch the wrong key. The purpose of measuring is to estimate the true value, usually by combining a number of values and calculating an average. An average value, such as the mean, is a closer estimate of the true measurement. As the number of random errors increases, the precision of the estimate decreases.

Measurement error that is not random is referred to as systematic error. In systematic measurement error, the variation in measurement values from the calculated average is primarily in the same direction. For example, most of the variation may be higher or lower than the average that was calculated. Systematic error occurs because something else is being measured in addition to the concept. For example, a paper and pencil rating scale designed to measure hope may actually also be measuring perceived support. When measuring subjects’ weights, a scale that shows weights that are 2 pounds over the true weights will give measures with systematic error. All the measured weights will be high, and as a result the mean will be higher than if an accurate weight scale were used. Some systematic error occurs in almost any measure. Because of the importance of this type of error in a study, researchers spend considerable time and effort refining their instruments to minimize systematic measurement error (Waltz et al., 2010).

In critically appraising a published study, you will not be able to judge the extent of measurement error directly. However, you may find clues about the amount of measurement error in the published report. For example, if the researchers have described the method of measurement in great detail and provided evidence of accuracy and precision of the measurement, then the probability of error typically is reduced. The measurement errors for BP readings can be minimized by checking the BP cuff and sphygmomanometer for accuracy and recalibrating them periodically during data collection, obtaining three BP readings and averaging them to determine one BP reading for each subject, and having a trained nurse using a protocol to take the BP readings. If a checklist of pain behaviors is developed for observation, less error occurs than if the observations for pain are unstructured. Measurement will also be more precise if researchers use a well-developed, reliable, and valid scale, such as the FACES Pain Scale, instead of developing a new pain scale for their study. In published studies, look for the steps that researchers have taken to decrease measurement error and increase the quality of their study findings.

Reliability

Reliability is concerned with the consistency of a measurement method. For example, if you are using a paper and pencil scale to measure depression, it should indicate similar depression scores each time a subject completes it within a short period of time. A scale that does not produce similar scores for a subject with repeat testing is considered unreliable and results in increased measurement error (Kerlinger & Lee, 2000; Waltz et al., 2010). For example, the Center for Epidemiologic Studies Depression Scale (CES-D) was developed to diagnose depression in mental health patients (Radloff, 1977). The CES-D has proven to be a quality measure of depression in research over the last 40 years. Figure 10-3 illustrates this 20-item Likert scale. If the items on this scale consistently measure what it was developed to measure, depression, then this scale is considered to be both reliable and valid. The different types of reliability and validity testing are discussed in the next sections (outlined in Table 10-1).

Validity

The validity of an instrument is a determination of how well the instrument reflects the abstract concept being examined. Validity, like reliability, is not an all or nothing phenomenon; it is measured on a continuum. No instrument is completely valid, so researchers determine the degree of validity of an instrument rather than whether validity exists (DeVon et al., 2007; Waltz et al., 2010). Validity will vary from one sample to another and one situation to another; therefore validity testing evaluates the use of an instrument for a specific group or purpose, rather than the instrument itself. An instrument may be valid in one situation but not another. For example, the CES-D was developed to measure the depression of patients in mental health settings. Will the same scale be valid as a measure of the depression of cancer patients? Researchers determine this by pilot-testing the scale to examine the validity of the instrument in a new population. In addition, the original CES-D (see Figure 10-3) was developed for adults, but the scale has been refined and tested with young children (4 to 6 years of age), school-age children, adolescents, and older adults. Thus different versions of this scale can be used with those of all ages, ranging from 4 years old to geriatric age (Sharp & Lipsky, 2002).

In this text, validity is considered a single broad method of measurement evaluation, referred to as construct validity, and includes content and predictive validity (Rew, Stuppy, & Becker, 1988). Content validity examines the extent to which the measurement method or scale includes all the major elements or items relevant to the construct being measured. The evidence for content validity of a scale includes the following: (1) how well the items of the scale reflect the description of the concept in the literature; (2) the content experts’ evaluation of the relevance of items on the scale that might be reported as an index (Grove et al., 2013); and (3) the potential subjects’ responses to the scale items.

Paper and pencil and electronic instruments or scales must be at a level that potential study subjects can read and understand. Readability level focuses on the study participants’ ability to read and comprehend the content of an instrument or scale. Readability is essential if an instrument is to be considered valid and reliable for a sample (see Table 10-1). Assessing the level of readability of an instrument is relatively simple and takes about 10 to 15 minutes. More than 30 readability formulas are available. These formulas use counts of language elements to provide an index of the probable degree of difficulty of comprehending the scale (Grove et al., 2013). Readability formulas are now a standard part of word-processing software.

Three common types of validity presented in published studies include evidence of validity from (1) contrasting groups, (2) convergence, and (3) divergence. An instrument’s evidence of validity from contrasting groups can be tested by identifying groups that are expected (or known) to have contrasting scores on an instrument. For example, researchers select samples from a group of individuals with a diagnosis of depression and a group that does not have this diagnosis. You would expect these two groups of individuals to have contrasting scores on the CES-D. The group with the diagnosis of depression would be expected to have higher scores than those without the depression diagnosis, which would add to the construct validity of this scale.

Evidence of validity from convergence is determined when a relatively new instrument is compared with an existing instrument(s) that measures the same construct. The instruments, the new and existing ones, are administered to a sample at the same time, and the results are evaluated with correlational analyses. If the measures are strongly positively correlated, the validity of each instrument is strengthened. For example, the CES-D has shown positive correlations ranging from 0.40 to 0.80 with the Hamilton Rating Scale for Depression, which supports the convergent validity of both scales (Locke & Putnam, 2002; Sharp & Lipsky, 2002).

Sometimes instruments can be located that measure a concept opposite to the concept measured by the newly developed instrument. For example, if the newly developed instrument is a measure of hope, you could make a search for an instrument that measures hopelessness or despair. Having study participants complete both these scales is a way to examine evidence of validity from divergence. Correlational procedures are performed with the measures of the two concepts. If the divergent measure (hopelessness scale) is negatively correlated (such as − 0.4 to − 0.8) with the other instrument (hope scale), validity for each of the instruments is strengthened (Waltz et al., 2010).

The evidence of an instrument’s validity from previous research and the current study needs to be included in the published report. In critically appraising a study, you need to judge the validity of the measurement methods that were used. However, you cannot consider validity apart from reliability (see Table 10-1). If a measurement method does not have acceptable reliability or is not consistently measuring a concept, then it is not valid.

Accuracy

Accuracy is comparable to validity in that it addresses the extent to which the instrument measures what it is supposed to measure in a study (Ryan-Wenger, 2010). For example, oxygen saturation measurements with pulse oximetry are considered comparable with measures of oxygen saturation with arterial blood gases. Because pulse oximetry is an accurate measure of oxygen saturation, it has been used in studies because it is easier, less expensive, less painful, and less invasive for research participants. Researchers need to document that previous research has been conducted to determine the accuracy of pulse oximetry for the measurement of individuals’ oxygen saturation levels in their study.

Precision

Precision is the degree of consistency or reproducibility of measurements made with physiological instruments. Precision is comparable to reliability. The precision of most physiological equipment depends on following the manufacturer’s instructions for care and routine testing of the equipment. Test-retest reliability is appropriate for physiological variables that have minimal fluctuations, such as cholesterol (lipid) levels, bone mineral density, or weight of adults (Ryan-Wenger, 2010). Test-retest reliability can be inappropriate if the variables’ values frequently fluctuate with various activities, such as with pulse, respirations, and BP. However, test-retest is a good measure of precision if the measurements are taken in rapid succession. For example, the national BP guidelines encourage taking three BP readings 1 to 2 minutes apart and then averaging them to obtain the most precise and accurate measure of BP (http://www.nhlbi.nih.gov/guidelines/hypertension).

Error

Sources of error in physiological measures can be grouped into the following five categories: environment, user, subject, equipment, and interpretation. The environment affects the equipment and subject. Environmental factors might include temperature, barometric pressure, and static electricity. User errors are caused by the person using the equipment and may be associated with variations by the same user, different users, or changes in supplies or procedures used to operate the equipment. Subject errors occur when the subject alters the equipment or the equipment alters the subject. In some cases, the equipment may not be used to its full capacity. Equipment error may be related to calibration or the stability of the equipment. Signals transmitted from the equipment are also a source of error and can result in misinterpretation (Gift & Soeken, 1988). Researchers need to report the protocols followed or steps taken to prevent errors in their physiological and biochemical measures in their published studies (Ryan-Wenger, 2010; Stone & Frazier, 2010).

Research Example

Directness, Level of Measurement, Reliability, and Validity of Scales, Accuracy, Precision, and Error of Physiological Measures

Research Excerpt

Whittenmore, Melkus, Wagner, Dziura, Northrup, and Grey (2009) studied the effects of a lifestyle change program on the outcomes for patients with type 2 diabetes. The lifestyle program was delivered by nurse practitioners (NPs) in primary care settings. The following excerpt describes some of the measurement methods used in this study.

Outcome Measures

“Data were collected at the individual (participant) and organizational (NP and site) levels at scheduled time points throughout the study. … All data were collected by trained research assistants blinded to group assignment, with the exception of … lipids, which were collected by experienced laboratory personnel at each site and sent to one laboratory for analysis.” (Whittenmore et al., 2009, p. 5)

“Efficacy data were collected on clinical outcomes on (weight loss, waist circumference, and lipid profiles); behavioral outcomes (nutrition and exercise); and psychological outcomes (depressive symptoms). … All data were collected at baseline, 3 months, and 6 months, with the exception of the laboratory data, which were collected at baseline and 6 months. … Efficacy data collection measures and times were based on the DPP [diabetes prevention program] study and modified for the short duration of this pilot study.

Weight loss was the primary outcome and was calculated as a percentage of weight loss from baseline to 6 months. … Waist circumference and lipid profiles were secondary clinical outcomes. Waist circumference was measured by positioning a tape measure snugly midway between the upper hip bone and the uppermost border of the iliac crest. In very overweight participants, the tape was placed at the level of the umbilicus (Klein et al., 2007). Lipid profiles (LDL [low-density lipoproteins], HDL [high-density lipoproteins], total cholesterol, and total triglycerides) were determined using fasting venous blood.

Diet and exercise health-promoting behaviors were measured with the exercise and nutrition subscales of the Health-Promoting Lifestyle Profile II (eight and nine items, respectively), which has items constructed on a 4-point Likert scale and measures patterns of diet and exercise behavior (Walker, Sechrist, & Pender, 1987). This instrument has been used in diverse samples, and demonstrates adequate internal consistency (r = .70 to .90 for subscales; Jefferson, Melkus, & Spollett, 2000). The alpha coefficients for the exercise and nutrition subscales in this study were .86 and .76, respectively.…

Psychosocial data were collected on depressive symptoms, as measured by the Center for Epidemiologic Studies Depression Scale (CES-D), a widely used scale (Radloff, 1977). The CES-D consists of 20 items that address depressed mood, guilt or worthlessness, helplessness or hopelessness, psychomotor retardation, loss of appetite, and sleep disturbance [see Figure 10-3]. Each item is rated on a scale of 0 to 3 in terms of frequency during the past week. The total score may range from 0 to 60, with a score of 16 or more indicating impairment. High internal consistency, acceptable test-retest reliability, and good construct validity have been demonstrated (Posner et al., 2001). The alpha coefficient was .93 for the CES-D in this sample.”

(Whittenmore et al., 2009, p. 6)

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related

Related posts:

1. Outcomes Research
2. Understanding Statistics in Research
3. Research Problems, Purposes, and Hypotheses
4. Clarifying Quantitative Research Designs

Stay updated, free articles. Join our Telegram channel

Join