157CHAPTER 8

Assessing Outcomes in Clinical Nurse Specialist Practice

Judy Elisa Davidson, Melissa A. Morse, Cassia Yi, and Mary C. Hellyar

1.  List four different categories of outcomes measured as a product of clinical nurse specialist (CNS) work in hierarchical order of importance

2.  Describe an example of how to measure time-on activities, process measures, surrogate measures, and actual outcomes of CNS practice

3.  Describe how to integrate the CNS spheres of influence and categorical outcomes of a CNS project or activity

4.  Describe how outcomes from a project align with organizational goals, objectives, or pillars of performance

1.  List four different categories of outcomes discussed in this chapter in hierarchical order of importance.

2.  Using your own practice or experience during clinical rotations, how could you personally measure time-on activities, compliance with process measures, a surrogate measure of cost or quality, and an actual impact on patient or staff outcomes? Describe one example.

3.  Select one project in which you have been involved or the plan for your capstone project, and make a table to include at least one measure for each of the four categories of outcomes listed in Question 1. Map these outcomes for each sphere of influence, as done in Exhibit 8.9.

1584.  For the project discussed in question 3, how do the outcomes align to organizational goals or objectives?

5.  What are the pros and cons of recording activities and tracking outcomes?

The CNS role was created by the nursing profession to address the increasingly complex needs of patients (National Association of Clinical Nurse Specialists [NACNS], 2004). While the complexity of nursing care continues to increase, there is increasing pressure to contain costs while maintaining quality outcomes. CNSs have an impact on outcomes in three spheres of influence: (a) patients/clients, (b) nurses and nursing practice, and (c) systems and organizations (NACNS, 2004). While the positive impact of CNSs on outcomes is well-documented in the literature (Coen & Curry, 2016; Cunningham, 2004; Dickerson, Wu, & Kennedy, 2006; DiLibero, DeSanto-Madyea, & O’Dongohue, 2016; Duffy, 2002; Fabbruzzo-Cota et al., 2016; Forster et al., 2005; Fulton, 2006; Fulton & Baldwin, 2004; Hamilton & Hawley, 2006; Larsen, Neverett, & Larsen, 2001; Ley, 2001; McCabe, 2005; Prevost, 2002; Willoughby & Burroughs, 2001), the shift in health care to pay-for-performance models requires CNSs to articulate their impact on both clinical and financial outcomes. In this cost-containment environment, the ability to communicate outcomes of CNS practice is paramount to the survival of the CNS role in health care organizations (Davidson, 2010b, 2011; Wojner, 2001). This chapter provides a variety of strategies to select, measure, analyze, and promote outcomes using a hierarchical model familiar to operational leaders. Examples are provided, including a case example crosswalk, to visualize how outcome measures align to the CNS spheres of influence.

  FRAMEWORK FOR MEASURING CNS OUTCOMES

In 2010, NACNS revised and published CNS core competencies (National CNS Competency Task Force, 2010). The focus of this document is clearly differentiating CNS practice outcomes from those of other advanced practice registered nurses. While impact on the spheres of influence is important, it may not be meaningful to nonclinical leaders in health care organizations. It is critical to articulate the value of CNSs across health care organizations in an understandable manner. Table 8.1 provides examples of CNS practice outcomes across the three spheres of influence, which are reflective of the CNS competencies. Table 8.2 expands the outline of assessment of CNS practice to include the focus of practice, performance of subroles, and economic impact, in addition to the three spheres of influence. Translating the CNS impact on the three spheres of influence to measurable clinical and fiscal outcomes and communicating this impact across the organization is essential to making a business case for retaining the CNS role during austere economic times (Amber, Carreon, Agan, Johnson, & Cahill, 2012; Davidson, 2010a, 2011).

159TABLE 8.1    Categories of Outcomes of CNS Practice and Roles Across Three Spheres of Influence

CNS, clinical nurse specialist.

Source: Adapted from the National Association of Clinical Nurse Specialists (2004).

160TABLE 8.2    Summary of Assessments of CNS Practice

CNS, clinical nurse specialist; LOS, length of stay.

  THE HIERARCHY OF OUTCOME MEASUREMENT

The following hierarchical model (Figure 8.1) is proposed (by these authors) for sorting outcomes by level of value to the organization and is derived from the system widely used to sort levels of evidence (Craig & Smyth, 2007; Melnyk & Fineout-Overholt, 2011; Schmidt & Brown, 2014). The levels are divided from weakest to strongest into (a) time-on activities, (b) process measures, (c) surrogate outcome measures, and (d) patient or staff outcomes. The pyramid portrays the fact that the least-valued measure is more easily produced and often used, whereas the strongest measures of patient or staff outcomes are hardest to obtain and least frequently reported. Each level on the pyramid has its place in outcome measurement. For example, it is prudent for every CNS to account for his or her time by keeping track of time spent on projects and outcomes. During the beginning of a practice change, compliance with process measures helps to ensure success during the change process. Patient or nurse outcomes are long-term measures of success. If these end outcomes were the only outcomes measured to determine whether a practice change was successful, it would likely result in failure to course-correct as needed. However, the true success of a practice change depends on measuring the long-term patient or nurse outcomes even though they may take months to measure. In the sections to follow, we will describe examples of each of these forms of outcome measures. An example of measuring educational activities using Kirkpatrick’s four-level system will be presented (Kirkpatrick, 2009). In conclusion, a process for developing an end-of-year report will be described.

161

FIGURE 8.1 Clinical nurse specialist outcome hierarchy pyramid.

  Time-On Activities

Calendar

One easy strategy to track contributions without spending undue time is for every CNS to keep an electronic calendar populated with actual work done. At the end of the day, just before retiring, go back to the day’s calendar and populate the “white space” with projects, rounds, just-in-time education, and so forth. Populate meeting time with major decisions or outcomes. This will take 5 minutes or less if performed while the memory of 162the day is fresh. The data can then be used at the end of the month to populate an end-of-month productivity report.

Productivity Report/Organizational Alignment

Productivity reports describing CNS duties are a valuable tool to communicate CNS contributions to health care organizational leadership. To increase visibility of the contributions CNSs make to the organization, record summaries of monthly productivity on a template with subheadings for each of the organization’s pillars or key goals (Studer, 2008). Most organizations report a scorecard of metrics to the board of directors according to subheadings such as finance, quality, patient experience, workforce development, and so forth. Instead of hiding activities under the subcategories of the CNS role functions that are known and understood mainly by the CNS community, use the organizational pillars as subheadings, and have each CNS complete the tool at the end of each month. The nursing division likely has a professional practice model and nursing strategic plan. Alignment of project outcomes to these key overarching documents that guide practice should be overtly stated in the report. Proactive thought regarding alignment of project outcomes to organizational and nursing objectives helps CNSs to measure their direct contribution to the organization.

Wherever possible, use cost figures or cost surrogates to estimate impact on finance. A predicted obstacle will be to move the team from qualitative descriptions of their work to quantitative objective outcome reporting. The root of the problem is that most CNSs are not comfortable using estimates of cost reduction or savings, or literature-based cost estimates versus actual values. Because actual savings are difficult to capture, surrogates are acceptable and also save time in hunting for obscure organizational data. This normal business strategy will improve visibility and security. From our experience, moving from paragraph to concrete outcome descriptions took more than a year of practice (Davidson, 2011). In the case where a team is learning how to do this together, it is helpful to have the leader complete one of his or her own, leading by example. The leader’s productivity report is distributed to the staff with a timely reminder to complete the end-of-month report. This sets clear expectations for the level of detail. Then the entire team’s package can be attached to staff meeting minutes. In this manner, each member of the team can learn from peer examples. The leader of the team may call out especially positive examples and use the documents as a source of staff recognition. Exhibit 8.1 provides an example of how to structure a monthly productivity report. If each CNS submits one, the complete package can be forwarded up the chain of command each month for positive team exposure. The act of sending positive messaging about employees up the chain of command on a regular basis is called “managing up” and is promoted by Studer as a strong business tactic for organizational success (Studer, 2008).

Preformatted Productivity Spreadsheet

This second example is offered for collating monthly performance data. Preformatted electronic spreadsheets provide a quick way to organize CNS activities. The following process was used by a team of CNSs to organize a simple method of tabulating time-on activities. During a retreat, the group brainstormed the categories of activities for which each spent time. The total list was then numbered so that each activity received a unique identifier. Then, of the total list, individualized lists were made to reflect activities pertinent to each person’s scope of practice. The spreadsheets were then autopopulated with time spent on required meetings. At the end of the month, the CNS populated his or her own spreadsheet, which was then forwarded up the chain of command. The process was designed with the goal of spending less than 15 minutes per month tabulating time-on activities. The spreadsheet also provides the ability to capture time within the CNS spheres or within the organizational pillars, which allows the data to be tailored to the target audience (Exhibit 8.2).

163EXHIBIT 8.1    CNS Monthly Productivity Report

CNS, clinical nurse specialist.

Peer Review and CNS Visibility

Another strategy to increase CNS visibility and staff understanding of outcomes generated from activities within the CNS role is the peer evaluation process. One method to obtain peer review is to ensure first that the CNS job description is built according to the published role elements (National CNS Competency Task Force, 2010). The role-specific elements are then abstracted into a peer evaluation along with the standard organizational behaviors required by each employee. The supervisor may send out the peer evaluation electronically to physicians, operational leaders, and staff who work with the CNS on committees, projects, or in their area of practice (Exhibit 8.3). This accomplishes several goals. The recipient can see for what the CNS is accountable, as well as soliciting the input needed for evaluation. When sending out the peer evaluation, a leading message with at least two accomplishments can also be sent to “manage up” the CNS to others (Studer, 2008) and improve visibility of CNS outcomes (Exhibit 8.3). The peer evaluation can easily be sent as an electronic survey with the introductory e-mail to automatically summarize the data according to subheading to cut and paste into the evaluation.

164EXHIBIT 8.2    CNS-Specific Productivity Spreadsheet

Note: This is a CNS-specific productivity report culled from a larger list of possible CNS activities. Standard meeting time is formatted to automatically populate. If viewed electronically, a drop-down list of pillars would appear for selection, as well as the drop-down list of spheres of influence. A drop-down list of CNS role requirements also appears.

AWS, alcohol withdrawal syndrome; CNS, clinical nurse specialist.

165EXHIBIT 8.3    Example Script for Peer Evaluation Request

Note: Bold items are subheadings specific to clinical nurse specialist core competencies. Normal text subheadings are specific to required organizational behaviors and will change from facility to facility.

CNS, clinical nurse specialist; IHI, Institute for Healthcare Improvement; UTI, urinary tract infection.

  Process Measures

Rounding With a Purpose

It is known that educational offerings do little by themselves to change behavior (Bloom & Bloom, 2005) and, therefore, academic detailing in the form of bedside rounds is imperative to solidify practice change. Table 8.3 summarizes the effectiveness of different methods of education in changing practice (Bloom & Bloom, 2005). There are several forms of rounds that CNSs make during the course of a day. They may round on vulnerable staff, new employees, staff floating outside their normal work environment, or staff working with patients who have unusual diagnoses or complex care. Another form of round is targeted rounds to follow up on new projects, programs, or services. A third form might take the form of multidisciplinary rounds or discharge planning rounds. Last, the CNS might be responsible for certain quality metrics, such as restraints, falls, pressure ulcers, or hospital-acquired pneumonia and plans to round on patients with those issues to ensure that standards of care are being met and that the processes designed in meetings actually meet patients’ needs without undue burden on staff. In all of these situations, rounds should be performed with a purpose (Studer, 2008) and conducted using tools to gather data and input from staff. The data from these rounds may also be used for monthly productivity reports.

166TABLE 8.3    Expected Impact of Activity on Change in Practice

Rounding Tools

The CNS role was originally developed to foster evidence-based practice at the bedside, and this key function remains a critical aspect of the job. Although direct outcomes cannot always be measured, compliance with evidence-based process measures, which have been demonstrated to improve outcomes, can be captured prospectively enabling an opportunity for just-in-time education and practice improvement. CNS-led rounds provide an opportunity to showcase expert assessment and diagnostic skills, while encouraging application of evidence-based nursing interventions in real time.

Johnson et al. (2011) captured the types of evidence-based recommendations made during CNS-led rounds. The rounds were targeted at preventing ventilator-associated pneumonia (VAP), venous thromboembolism, and hospital-acquired infections, as well as use of progressive mobility and improved glycemic management. The number and types of recommendations and the frequency with which the recommendations were carried out could be captured. Johnson et al. found that CNS recommendations were carried out 63% of the time, and over the course of 1 year, CNS-led rounds led to 345 evidence-based interventions. Exhibits 8.4 and 8.5 are examples of tools tailored for data collection from different types of rounds.

167Another application of CNS process measurement targets key outcome indicators that change over time. For example, catheter-associated urinary tract infections (CAUTIs) occur infrequently and it may take many months to a year to see if an intervention has been effective. However, the Centers for Disease Control and Prevention has clear guidelines for the prevention of CAUTIs (Gould, Umscheid, Agarwal, Kuntz, & Pegues, 2010), and because CAUTIs are tied to pay for performance, this untoward outcome is ripe for CNS management. Figure 8.2 is an example of process data collected for the reduction of CAUTIs. Bar graphs are especially useful in displaying before and after data for a set of required process elements.

EXHIBIT 8.4    Outcome Measure: Data-Collection Tool for Outcomes From Rounds

ICU, intensive care unit; IV, intravenous; PO, by mouth; VAP, ventilator-associated pneumonia.

168EXHIBIT 8.5    Process Measure: Rounding Tool

GI proph., gastrointestinal prophylaxis; Glyc mgt, glycemic management; HOB, head of bed; ICU, intensive care unit; IV, intravenous; OT, occupational therapy; PO, by mouth; PT, physical therapy; Sed. hol., sedation holiday; ST, speech therapy; VAP, ventilator-associated pneumonia; Vent, ventilator; VTE proph, venous thromboembolism prophylaxis.

169

FIGURE 8.2 Process measure: Compliance with urinary catheter bundle.

ICU, intensive care unit.

Reflective Practice and Process Compliance

Another CNS result that can be used as an outcome measure is compliance with a new process standard. At one large academic center, a CNS was responsible for revising the pain assessment and management policy to include the pain assessment hierarchy as outlined by the Society of Critical Care Guidelines (Barr et al., 2013). In the previous version of the pain policy, pain reassessments after analgesia administration were required within 2 hours. The policy was revised to require reassessment dependent on the intervention, thus timing reassessment to the peak effect of the treatment (Pasero, 2010). This seemingly minor revision required a change in nursing culture. Previously, nurses focused on the 2-hour reassessment time frame from the former policy to satisfy regulatory standards. The new policy would require nurses to change practice from a time-based assessment to the critical thinking of a peak-effect-based assessment. Education to the change was rolled out through several different formats and avenues, such as educational symposiums, presentations at shared governance meetings, an online self-learning module, flyers, badge cards, and so on.

To measure the success of the policy change and subsequent education, compliance with pain reassessment was trended. The CNS worked with a pharmacy analyst to use the inpatient medication administration records and the nursing pain assessment documentation to create monthly hospital-wide, unit-specific, and even nurse-specific compliance rates. The compliance rates were distributed monthly to nursing leaders who then distributed them to their respective staff. The compliance rates were generated to the individual staff nurse level. The nursing leaders were also provided with medical record details for each staff nurse listing all as-needed analgesics given in the month along with the documented reassessment times.

170Posting the information in a nonpunitive manner, allowing nurses to see the pattern of their own work, is a form of reflective practice. This concept of reflective practice does two things. First, it allows for clinical nurses to use data to evaluate and reflect upon their individual practice. It also allows for nurse leaders to provide data to clinical nursing staff that allows for individual self-reflection, rather than time-intensive auditing and manager to nurse follow-up in a constructive (versus punitive) approach (Lau & Chan, 2005; Lawrence, 2011).

One unexpected result of this reflective practice data was the apparent impact of peer influence. When the data were given to the unit managers it was provided in two different formats. The first format listed the nurses by name; the second format listed them by their employee identification number. The managers were expected to post the unit compliance data in their break room or nursing station. It was up to the managers to decide how the data would be posted, knowing that if posted by employee identification number, the nurses on the unit would be able to identify only their own results.

The CNS kept track of which nursing units were posted by name, and saw an increase in overall compliance compared to those who were posted by employee identification number. Figures 8.3 and 8.4 show the results for two similar units that posted the compliance data starting in February 2016 (see arrows). Figure 8.3 shows that Unit A, which posted by employee identification number only, was unable to increase compliance; the goal was not met. Figure 8.4 shows the results for Unit B, a similar nursing unit, which posted compliance reports by nursing name. This unit saw a 19% increase in compliance.

  Surrogate Outcomes

Surrogate Outcome Measures

Cost avoidance and reduction in adverse outcomes can be difficult to capture. Surrogate values are a valuable tool in these circumstances and have been used in medical and allied health literature (Dasta et al., 2010; Fraser, Riker, Prato, & Wilkins, 2001; Stahl et al., 2009). Costs of a specific test or treatment can be obtained through administrative accounting data providing an actual cost per case figure, which is a better indicator than billing data. When a test or treatment is eliminated from care, the cost of the item plus the cost of staff time spent doing the test or treatment is calculated. Estimates of staff time are described in what follows.

FIGURE 8.3 Compliance posted using employee identification number.

171

FIGURE 8.4 Compliance posted using employee name.

When using a surrogate figure that was published in the literature for an episode of care (e.g., pressure ulcer or VAP), currency of the published figure is evaluated and then adjusted for inflation. If the figure was published in 2010, the inflation factor can be obtained from the department of finance and added to the published cost to bring the cost up to date. The inflation rate methodology is specific to the organization due to geographic variation in inflation. A CNS might find it helpful to proactively keep a list of cost or charge measures for common items such as average salary for each level of employee in the department, average revenue per day of stay, number of admissions per calendar year, average length of stay, and operating room or procedural area charge time in minutes.

Calculating Cost in Staff Time

One of the most desirable outcomes to measure is reduction in wasted time spent on unnecessary activities. For example, streamlined medication passes, decreasing the number of prompts in computerized documentation, or reduced time spent in hunting and gathering supplies could constitute cost savings. If a project has resulted in decreasing wasted process steps in the delivery of care, the hours of saved time can easily be converted into a surrogate figure for dollars saved. To do this, have your supervisor inform the department of finance that you are working on a project that requires this analysis. The data you will be requesting for the calculation is often considered sensitive and will not be granted unless there is verification that you are using it for a business need. If the change in practice will affect only one department, ask for the average nursing salary for that department. If the change will eventually affect all departments, ask for the organizational average salary. Average salary may differ widely based upon seniority. Also ask for the overhead rate that constitutes the percentage over wage that is spent by the organization on provision of benefits. Then, measure the amount of time the wasted 172process step takes in minutes by conducting a time and motion study. Perform several observations and average the result. Create a spreadsheet using the following variables:

Stay updated, free articles. Join our Telegram channel

Join