4445CHAPTER 3

Selecting Advanced Practice Nursing Outcome Measures

Beth D. Quatrara and Katherine Dale Shaw

1.  Discuss reasons why measuring outcomes of advanced practice registered nurses (APRNs) is essential to the role of the APRN

2.  Identify at least three outcome measure categories that can be influenced by an APRN

3.  Discuss the benefits and pitfalls of using benchmark/aggregate data to measure APRN outcomes

4.  Describe methods to measure and monitor APRN outcome data

1.  Why is establishing a clear goal and defined outcome an important step early in the assessment process?

2.  Why are some data points considered less desirable APRN outcome measures than others?

3.  Why is a trending data over time a valuable strategy for recording APRN outcomes?

4.  What are the key elements of APRN outcome measures?

Changes in the health care landscape are emphasizing the need for providers to demonstrate outcomes. Government regulatory agencies, insurance companies, and institutional 46administrators expect high-quality patient care outcomes at the individual and aggregate levels in order to improve overall health and reduce fiscal waste. Standard process outcomes such as productivity and adherence are no longer as valued as the results of the care provided or the services rendered. Data pointing to the volume of patients seen or compliance with an established guideline is often considered inadequate if it fails to further demonstrate a resulting improvement in patient care.

As health care providers, APRNs are increasingly being asked to demonstrate the effectiveness of their roles. In some cases, the inability of an APRN to do so results in the dissolution of the role. APRNs must be thoughtful in demonstrating their effectiveness and “value-added” benefit to the institutions and communities in which they practice.

Well-controlled studies on APRN outcomes continue to be relatively scant (Bryant-Lukosius et al., 2015; Donald et al., 2015; Newhouse et al., 2011; Stanik-Hutt et al., 2013). However, some do exist and suggest that APRNs “provide safe, effective, quality care to a number of specific populations in a variety of settings” (Newhouse et al., 2011; Stanik-Hutt, 2013). The studies on APRN outcomes are extremely helpful as guides to APRN practice. While more studies are needed on APRN roles in specific settings, clinicians in practice must continue to demonstrate the effectiveness of their roles in less labor-intensive ways than performing research studies on a day-to-day basis. To that end, the purpose of this chapter is to describe a number of different methods and outcome measures that might be used to evaluate an APRN’s contributions to quality patient care. Examples of actual APRN outcome projects will illustrate the methods and demonstrate the importance of determining appropriate outcome data for measurement.

  SELECTING APRN ROLE-SENSITIVE OUTCOME MEASURES

A common approach used to determine APRN outcomes is to attempt to link aggregate data such as length of stay (LOS) and cost per case to APRN practice. While these types of measurements are important and helpful in some cases, they are generally not sensitive enough to clearly demonstrate the APRN’s unique contribution. Rarely does aggregate data show the causal effect of an individual on a patient population; there are simply too many intervening variables that may have contributed to the effect (this is discussed in depth later in the chapter). Thus, it is important to consider other, more sensitive indicators. There is also a practical reason for carefully selecting outcome variables; APRNs are busy and data collection takes time. The data that the APRN collects should be easy to obtain and should be specific to the APRN’s role. With the proliferation of electronic health records (EHRs) and other electronic systems and databases, data collection should be simplified through the generation of reports. However, the data collected should be limited to carefully selected role-sensitive indicators and the tendency to pull a plethora of semirelated data elements avoided. Reviewing unnecessary and tangentially related data distracts from the true outcome measure and wastes valuable APRN time. If hand-collected data are required they should be minimized to essential elements only. If the data-collection burden is too great, it is unlikely that the required metrics will be routinely collected. Unfortunately, many APRNs are averse to collecting data and having to demonstrate the value of their efforts. In addition, the APRNs may feel that data collection distracts them from their primary roles as clinical experts. But, if the outcome 47measures are carefully selected, the data will not only help to clarify the APRN’s value to the health care system, but may also be used to focus the role accordingly.

When an APRN is hired, the administrator and physician (if applicable) to whom the APRN will report will generally have a specific role or function in mind (clinical nurse specialist [CNS], nurse practitioner [NP], certified registered nurse anesthetist [CRNA], or certified nurse-midwife [CNM]). Following role negotiation, the APRN should begin establishing targeted outcome measures that are mutually agreed upon by the APRN and the individual who hired the APRN. Setting the outcome measures early in the role negotiation process (or prior to starting any new project) allows the APRN to identify currently available data sources and define the specific metrics within a defined time frame, which will be attributable to his or her efforts. Examples of role-specific outcome measures follow and an example of an APRN outcome planning, tracking, and reporting worksheet is found in Exhibit 3.1.

  ACUTE CARE NURSE PRACTITIONER FOR A MEDICAL ACUTE CARE UNIT

If the role is that of acute care nurse practitioner (ACNP) for a medical acute care floor, the collaborating physician and institutional leadership may be interested in the number of ACNP-managed patients requiring 30-day readmissions within a selected time interval. These data are relatively easy to collect and can be obtained from institutional EHRs or clinical data repositories. The ACNP will need to maintain a secure database of his or her discharged patients in order to query the system on a regular basis (e.g., every 2 months, quarterly, yearly). It may be desirable to compare these to other health care provider data as well, while being mindful that readmissions to outside facilities may not be easy to determine and are important for the accuracy of the report. The ACNP may consider further focusing his or her efforts on strategies to reduce 30-day readmissions within a specific high-risk population. For example, chronic obstructive pulmonary disease patients may be a complex population that warrants the attention of the ACNP. Further analyzing the 30-day readmission rates for this provider within the subcategory of a patient population could produce valuable insights into the care of these patients.

  ACUTE OR CRITICAL CARE CLINICAL NURSE SPECIALIST

The role of CNS generally encompasses all the domains of advanced practice (e.g., clinical management, education, research, consultation, and change agency). However, it may be that outcome measures may focus on only one or two measures of role effectiveness and that these may change over time. For example, the administrator and CNS may agree that one key objective for the year is to improve medication safety (i.e., medication errors). The CNS in this example might partner with the quality assurance (QA) department to track unit medication errors following the implementation of a CNS educational and evidence-based practice change initiative. A trended line chart might be used to graphically demonstrate the change in percentage of medication errors per time interval. In this case, the APRN does not actually need to collect data but instead partners with the QA department to ensure that the intervention date (i.e., training of staff and evidence-based change) is marked accurately on the chart. Very few examples like this one would be needed to demonstrate the APRN’s worth to the institution. More importantly, the APRN can use the data to readjust or change the intervention if necessary. See Figure 3.1 for an example of a trended line chart.

48EXHIBIT 3.1    APRN Outcome Planning, Tracking, and Reporting Worksheet

APRN, advanced practice registered nurse; IRB, institutional review board; LOS, length of stay; UTI, urinary tract infection.

Source: Adapted for use with permission of University of Virginia Professional Nursing Staff Organization, University of Virginia, Charlottesville, VA (copyright 2011).

The CNS may also monitor the effect of a change initiative on practice and patient outcomes. Perhaps the initiative is one designed to incorporate prone positioning into the care management protocol for patients with acute respiratory distress syndrome (ARDS). A prospective audit by the CNS following implementation of the educational and competency-based initiative would be relatively easy to accomplish while the CNS is on the unit. Elements to track would be protocol adherence (i.e., was it implemented), accuracy (i.e., was it done correctly), and the patient outcome (i.e., what was the patient’s response: PaO₂ following prone positioning, physiologic tolerance). Little data collection would be necessary since the patient population is discrete and maneuver is generally used infrequently. The reason such an initiative is a reasonable one to use as a CNS outcome is because the procedure is potentially risky and is almost entirely nurse managed. The CNS’s effectiveness in safely implementing such a protocol is essential as it speaks to integration of all the CNS role components and demonstrates effective leadership and follow-through. In addition, the CNS can use the data to quickly adapt and adjust the protocol as needed. This “real-time” monitoring with short cycles of intervention, evaluation, and correction ensures quality.

49

FIGURE 3.1 Trended line chart.

Note: Baseline data demonstrated a higher than desired percentage of medication error. The goal of the medication error initiative was to reduce the percentage of errors below 5. The initial intervention at month 5 demonstrated a reduction but did not achieve the goal. The subsequent revision to the plan at month 9 succeeded in achieving the goal. The trended line chart allowed the clinical nurse specialist to follow progress over time, intervene as appropriate, and demonstrate sustained improvement.

Some specific categories of outcome measures and the associated issues with each are discussed in the text that follows.

  CATEGORIES OF APRN OUTCOME DATA

A variety of indicators may be used to demonstrate the effectiveness of an APRN. The strengths and weaknesses of the indicators are described in examples that follow.

  Satisfaction (Patient, Family, Caregivers, and Physician)

Satisfaction is a variable that speaks directly to the institution’s “market share” of customers. If customer satisfaction is not good, the customer will not return. In addition, 50the customer’s negative advertisement of the hospital will have far-reaching implications for the institution. As noted in the business industry, 95% of customers will share a bad experience while 87% will share a good experience (Dimensional Research, 2013). Furthermore, 54% will share the bad experience with more than five people while 33% will share a good experience with more than five people (Dimensional Research, 2013). Few hospitals exist today that are arrogant enough to ignore satisfaction as an outcome measure. However, as we know, satisfaction is not always synonymous with quality. Regardless, it is an important variable to monitor, particularly for hospitals that participate in the Centers for Medicare and Medicaid’s Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) surveys. These publicly reported ratings of patient perceptions of care are measures with significant impact on financial outcomes.

Most institutions routinely measure patient satisfaction in a global manner. Like other aggregate data, it is hard to specifically attribute the outcomes to APRN practice. For example, in many cases, the satisfaction instrument will not distinguish the APRN from the bedside nurse or the physician. Further complicating the matter, many institutions collect data related only to the service line (i.e., medicine or surgery). Thus, for satisfaction to be linked to APRN practice, a separate survey may be necessary. However, even this may be a stumbling block. In some institutions, patient satisfaction surveys are closely controlled and may be distributed only via the institutional mechanism in place. There are good reasons for this; the institution does not want the patients and families “bothered” with numerous forms and questionnaires. Further, more specific and detailed questions are difficult to design, take the patient additional time to complete, and often require interpretation. If satisfaction is a desired APRN outcome measure, it is necessary to determine whether the existing institutional survey is sensitive enough. If not, the APRN may need to design his or her own survey, if permissible within the institutional structure.

If an institutional survey is used to measure APRN satisfaction outcomes, it is best to directly target a specific question(s). This is exemplified by an ACNP who wanted to improve patients’ pain management satisfaction scores. She identified the following survey questions as targeted outcome measures for unit: “how well was pain controlled” and “nurses kept me informed.” She designed a project to address these specific satisfaction measures by providing education about medication availability, timing, and by involving patients in their own pharmacologic pain management regimen. This nursing intervention provided patients with the information they needed to improve their pain management and empowered them to control the discomfort. The intervention was successful. Using the institutional survey data, the ACNP was able to demonstrate a sustained improvement in satisfaction scores, a valuable institutional outcome (Figure 3.2).

However, there are times when a broad institutional survey cannot address the unique attributes of a particular customer service initiative. Although difficult as described, it may be possible to use a separate survey to determine customer satisfaction with a specific intervention without overtaxing the survey process or overburdening patients and families. A key to success lies with introducing a well-timed, brief, and focused survey. For example, an operating room (OR) clinical research team implemented a new perioperative communication plan to evaluate the effect on family member anxiety and satisfaction. The communication plan included regularly scheduled OR nurse updates to a designated family member from the perioperative team on his or her loved one’s status throughout the surgery. With minimal intrusion, family members who were receiving the status updates responded to a short questionnaire about the experience. The survey participation rate was high. The satisfaction data directly demonstrated customer satisfaction with the new communication plan (Figure 3.3). Such interventions can directly contribute to an increased overall customer service rating but the outcome cannot be attributed to the APRN-led project without specific data.

51

FIGURE 3.2 APRN-directed pain management intervention: Results of patient satisfaction survey.

APRN, advanced practice registered nurse.

Surveys of staff and physician satisfaction may also be an effective and useful measure of APRN practice. The satisfaction of caregivers is important because their dissatisfaction can affect recruitment, retention, quality of care, and other financial outcomes. According to the 2016 National Health Care Retention Survey, institutional costs of nursing turnover are cited to be $37,700 to $58,400 (United States) per nurse (NSI Nursing Staffing Solutions, 2016). Additionally, frequent turnover makes the assurance of quality care difficult. It is costly (in time and money) to provide enough training to ensure the basic competency of “safe” care delivery following orientation. In this time of a “nursing shortage,” retention is essential, and nurses’ satisfaction with their work environment, professional development opportunities, and ability to “make a contribution in a collaborative manner” are important variables to consider. The APRN may well have an important part to play in satisfaction as it relates to one or more of these variables. Satisfaction surveys related to these and other specific aspects of APRN practice may be useful and relatively easy to accomplish via mechanisms such as the unit or service line intranet.

FIGURE 3.3 Survey results comparing preoperative and postoperative satisfaction of patients’ families related to communication intervention.

52Physician satisfaction is another variable that may be measured. The physician generates revenue and the APRNs with whom they work contribute to the efficiency and effectiveness of the physicians’ practice. In these cases, physician satisfaction with the collaborative relationship and the results of the same are important to follow.

Additional considerations for the APRN with regard to the development and use of satisfaction surveys include decisions related to whether or not the project must be reviewed and approved by the facility’s institutional review board (IRB) before implementation. If the information received from the survey is de-identified, it is likely that the project will receive an “exempt status” approval from the IRB. However, in some institutions, a satisfaction survey must seek approval via the QA department or sometimes a nursing research department. It is the responsibility of the APRN to proceed via appropriate channels. A second and important consideration when designing satisfaction surveys is that of the validity and reliability of the instrument. If it is not a tested instrument, the survey may provide inaccurate and erroneous answers. To avoid this potential stumbling block, use of an existing tested instrument is preferable.

  Clinical Outcome Measures

APRN-sensitive clinical outcomes may be difficult to identify because many factors may potentially affect them. It is important to remember that clinical outcome measures need not be inclusive of everything the APRN practice may affect, but rather those that are the most easily and clearly attributed to the APRN practice. For example, consider the NP charged with managing the care of a neurosurgical patient population. Because, in this case, the NP’s role is focused on managing the medical aspects of care of the patients, selected aspects of care may yield sensitive indicators of effectiveness. Examples include such indicators as urinary tract infection (because the NP is responsible for ordering catheter removal), decubitus ulcer formation (secondary to initiation of mobilization), and selected discharge outcomes.

A common error made by APRNs who manage large groups of patients is to collect a large data set in the hope that it will show something. This approach, “fishing in the data,” is unnecessary and a poor use of the APRN’s time. A rule of thumb, as with any research study, is that the question should be clear and the variables of interest, measurable. For example, perhaps the APRN’s role is focused on improving the outcomes of patients who require prolonged mechanical ventilation. It is important that the APRN 53have benchmark data available on ventilator duration so that a comparison may be made. LOS, though sometimes related to ventilator duration, may not be directly affected by the APRN since the clinician’s role may not extend to the unit to which the patient is transferred following successful weaning. In contrast, duration of ventilation may be attributable if the APRN is the one charged with ensuring proper application of, and adherence to, a weaning protocol. In this example, the data are relatively easy to collect because they are congruent with the role of the APRN and can be easily collected in the course of a practice day.

Another outcome directly attributable to an APRN may be defined through attention to the details of medication reconciliation. The 2016 National Patient Safety Goals for Hospitals includes this requirement:

Stay updated, free articles. Join our Telegram channel

Join