SIX
Cultural Competence Clinical Evaluation Tool (CCCET) and Clinical Setting Assessment Tool–Diversity and Disparity (CSAT–DD)
with contributions from Enis Dogan and Lynn Schmidt
The CCCET is a user-friendly tool designed to gather data about the provision of cultural-specific care, cultural assessments, and cultural sensitivity within clinical settings. The CSAT–DD is a user-friendly tool to collect data about the clinical practicum/agency site, specifically focusing on descriptions of diverse client populations (Part I) and clinical problems (Part II).
SNAPSHOT SCENARIO
At an advisory board meeting, nurse educators and administrators in academia and community health care organizations discuss accreditation and professional mandates concerning cultural competence education. Consider the following excerpts:
Nurse Administrator: Although our cultural competence programs during employee orientation, preceptor-mentoring, and subsequent unit-based education programs receive glowing participant feedback on surveys, I have one main question: To what extent is culturally specific care provided in the clinical setting?
Professor Ponder: This question is also pertinent to the education of our undergraduate and graduate students.
Professor Book: Well, cultural-specific care begins with consistently and systematically conducting cultural assessments. This means asking specific questions about the many dimensions of culture as advocated by Dr. Madeleine Leininger, transcultural nursing scholars, and noted cultural competence scholars in other health disciplines. If nurses don’t know what cultural values, beliefs, and health care practices are preferred by a patient, cultural-specific care cannot be preserved, maintained, accommodated, negotiated, and/or advocated. Sometimes cultural practices may be unsafe for a patient’s health and wellness; therefore, discovering a culturally congruent way to discuss repatterning or restructuring health care practices is essential.
Nurse Numbers: Gathering baseline information would be valuable in order to assist students, nurses, and other health professionals to systematically conduct cultural assessments. By knowing baseline information, educators can create focused educational programs on specific target areas, and continue building upon prior learning. We can’t do everything at once, but we can target specific focus areas and start by doing something now.
Professor Ponder: I guess some additional questions include:
Which cultural assessments are implemented most frequently with culturally diverse clients?
Which cultural assessments are implemented least frequently with culturally diverse clients?
Nurse Val: Cultural knowledge and skills are important, but without nurses and nursing students developing culturally sensitive and professionally appropriate attitudes, values, or beliefs concerning culturally competent patient care, they will not integrate cultural competence within their daily professional practice. Teachers, clinical instructors, preceptors, and agency evaluators play a powerful role in providing feedback to students and employees in the ongoing quest for optimal cultural competence development that then transfers into appropriate action in clinical settings with diverse patients.
Nurse Glass: That sparks some additional questions:
Are there any measurement tools that appraise cultural competence in clinical settings that measure provision of cultural-specific care, cultural assessment, culturally sensitive and professionally appropriate attitudes, values, and beliefs? Self-reflection is an expectation of a professional. Is there a comprehensive self-reflection tool and some mechanism for providing feedback to an employee or student? I wonder how closely evaluators’ assessments would mirror the employee’s or student’s self-evaluation.
Professor Ponder: Those are intriguing questions. The literature recognizes the importance of interacting with diverse clients as a valuable component for cultural competence development. For our students, I’m wondering to what extent culturally diverse clients are available in our clinical practice sites. What areas or topics of clinical practice are available and unavailable also need to be considered. Student and faculty perceptions about cultural and clinical diversity would also be important to appraise. This information could help guide simulation scenarios or exposure to new and different clinical experiences that expand cultural and clinical diversity, particularly matching course and program learning outcomes.
Nurse Ed: Yes, simulation and patient care conferences or case studies could be utilized to supplement and expand upon the diversity currently available in our employee settings too. The first step is to gather baseline data using comprehensive, valid, and reliable tools that capture all the areas we need to measure. We can then use the data to prioritize and guide our educational strategies, then implement, evaluate, and modify according to data trends.
Professor Val: We could also utilize validated cultural competence assessment tools during and after clinical experiences as part of a formative evaluation with individual students. The feedback would guide the ongoing process of optimal cultural competence development. Such formative feedback would also be valuable with new employees and their preceptors or mentors.
Such questions prompted the design of the CCCET and the development of the CSAT–DD. The CCCET is a user-friendly tool designed to gather data about the provision of cultural-specific care, cultural assessments, and cultural sensitivity (1) (see Jeffreys Toolkit 2016 Items 3–6). It can be used for multiple formative and summative evaluation purposes to guide individual, course, curricular, and employee educational program innovations and teaching–learning strategies. It can also be used in conjunction with other tools, such as the Transcultural Self-Efficacy Tool (TSET). The CSAT–DD is a user-friendly tool to collect data about the clinical practicum/agency site, specifically focusing on descriptions of diverse client populations (Part I) and clinical problems (Part II). (1) (see Jeffreys Toolkit 2016 Item 7). It can be used for multiple summative evaluation purposes to guide course and curricular decision making, innovations, teaching–learning strategies, and future clinical and/or simulation placements. This chapter highlights major components, features, psychometric properties (reliability and validity), purposes, scoring, and utilization. Exhibits, application strategies, and the “Educator-in-Action” vignette provide ideas for immediate application in a variety of settings.
What else do you seek to learn about the CCCET and CSAT–DD? How could learning more about the CCCET and CSAT–DD enhance your current and future professional role? In addition, how could it enhance cultural competence education, the provision of cultural-specific care, cultural assessments, and cultural sensitivity within your organization and/or health care setting?
CULTURAL COMPETENCE CLINICAL EVALUATION TOOL (CCCET)
Background
The CCCET is an assessment tool adapted from the TSET (1–11) (see Jeffreys Toolkit Items 1 and 2). Ten select points concerning the TSET with the most relevance to the CCCET are highlighted in the following sections (see Chapters 4 and 5 for more details about the TSET).
1. The TSET is an 83-item questionnaire designed to measure and evaluate learners’ confidence (transcultural self-efficacy) for performing general transcultural skills among diverse client populations.
2. Based upon the literature in transcultural nursing, self-efficacy, education, and psychometrics, the process of designing the TSET included item development, item sequence, subscale sequence, expert content review, expert psychometric review, revised draft, pretest, minor revisions, and a second pretest.
3. Content validity experts validated the 83 items and their categorization under three subscales presented in the following sequence: Cognitive (25 items), Practical (28 items), and Affective (30 items).
4. Repeated psychometric testing of the TSET among students, nurses, and health care providers in various languages demonstrate consistently high reliability and validity, consistent with the underlying conceptual framework of Jeffreys’s Cultural Competence and Confidence (CCC) model (see Chapter 3).
5. The CCC model (see Figure 3.1) aims to interrelate concepts that explain, describe, influence, and/or predict the phenomenon of learning (developing) cultural competence and incorporates the construct of transcultural self-efficacy (confidence) as a major influencing factor.
6. Cultural competence is a multidimensional learning process that integrates transcultural skills in all three dimensions (cognitive, practical, and affective), involves transcultural self-efficacy (confidence) as a major influencing factor, and aims to achieve culturally congruent care.
7. Transcultural self-efficacy (TSE) is the perceived confidence for performing or learning general transcultural nursing skills among culturally different clients.
8. Self-efficacy influences learning and performance of domain-specific behaviors (e.g., those within the specific domain of cultural competence), whereby individuals with resilient (strong, realistic) self-efficacy persist in learning and carrying out behaviors despite challenges; inefficacious (low confidence) individuals avoid and/or lack commitment and motivation in learning and carrying out behaviors; and supremely efficacious (overly confident) individuals overlook or ignore the task (learning and performance), prepare inadequately or not at all, and lack commitment and motivation (12).
9. The term learning process emphasizes that the cognitive, practical, and affective dimensions of TSE and transcultural skill development can change over time, as a result of formalized education and other learning experiences.
10. Research findings consistently support that the TSET detected differences in TSE perceptions within and between groups, such as prior to and after educational intervention. (13–30) (see TSET Research Exhibits 4.1–4.5, 7.1–7.3, 8.1, 9.1–9.3, 13.1–13.2, 16.1, and Appendix).
In addition, several noteworthy TSET findings reported by external reviewers provide essential background information relevant to understanding and appraising the adapted CCCET.
1. Harper’s Delphi study involving 35 expert international nurse researchers determined that of six other cultural questionnaires reported in the nursing literature, the TSET measures the most attributes of cultural competence identified by the expert panel (31).
2. In a systematic review of 45 instruments measuring cultural competence, Gozu et al. (32) reported that most instruments are poorly constructed, lacking acceptable psychometric properties; however, they noted that the TSET had more detailed psychometric testing and consistently demonstrated high validity and reliability.
3. Capell et al.’s (33) review of cultural competence instruments also noted the consistently high validity and reliability of the TSET, yet recommended further testing to determine if they correlate with culturally competent behaviors or enhanced client outcomes.
4. In addition to acknowledging the high validity and reliability of the TSET, Krentzman and Townsend’s review also noted its use with diverse respondents (34).
5. Shen’s (35) review of cultural competence models and instruments closely examined their components, theoretical backgrounds, empirical and psychometric validation, and concluded that the TSET consistently demonstrated high psychometric properties through various, rigorous tests and re-tests of validity and reliability.
6. Shen (35) noted that the CCC model underlying the TSET instrument was empirically supported via multiple psychometric tests and studies using an instrument with high psychometric properties.
7. Loftin and colleagues’ (36) integrative review of instruments that measure cultural competence in nursing and other health professionals concluded that: (a) the psychometric properties of the TSET has been extensively and thoroughly studied in initial development and subsequent testing, and (b) the results of the reliability and validity studies support that the TSET assesses the multidimensional aspects of TSE.
8. Loftin and colleagues (36) point out that the TSET validly measures all three constructs directly related to the AACN’s description of the three essential characteristics of culturally competent baccalaureate prepared nurses: awareness of personal culture, values, beliefs, attitudes, and behaviors; skill in assessing and communicating with individuals from other cultures; and assessment of cross-cultural variations.
Major Components and Features
The strong psychometric properties (validity and reliability) and comprehensiveness of the TSET (content validation) provided a strong conceptual and psychometric framework for the CCCET. Specifically, the TSET’s existing items were maintained; however, the rating scale and directions were adapted to permit the students’/learners’ self-evaluation of clinical cultural competence and teacher’s/agency evaluator’s evaluation of clinical cultural competence. During the CCCET design process, statistical consultation focused on issues concerning instructions, relationship of the item to the rating scale and directions, optical scanning capability, data processing and management, and proposed data analyses.
The CCCET contains three subscales measuring different dimensions of cultural competence clinical behaviors: (a) the extent of culturally specific care (Subscale 1); (b) cultural assessment (Subscale 2); and (c) culturally sensitive and professionally appropriate attitudes, values, or beliefs including awareness, acceptance, recognition, appreciation, and advocacy necessary for providing culturally sensitive professional nursing care (Subscale 3). The subscales contain 25 items, 28 items, and 30 items, respectively. Consistent with the TSET, a 10-point rating scale is used with anchors as indicated here:
• Subscale 1 (Provision of Culture-Specific Care): “not at all” (1) to “totally” (10)
• Subscale 2 (Cultural Assessment): “never” (1) to “always” (10)
• Subscale 3 (Cultural Sensitivity): “not at all” (1) to “to a great extent” (10)
In addition, respondents have the option of selecting “A” ([clinical] area not available) or “B” (diverse clients not available). For example, if nurses or students were not in the maternity/labor and delivery/postpartum unit, selection of choice “A” (area not available) would be appropriate for items related to birth and pregnancy.
The intended use of the CCCET was to determine baseline information; identify areas of strengths, weaknesses, and gaps; and to evaluate change following educational intervention and/or increased exposure to culturally diverse clients and/or increased exposure to the clinical topic/area not previously available. Another purpose is to provide formative evaluation so that feedback can guide future cognitive, practical, and affective learning needed for developing optimal cultural competence. “An important outcome of a nursing education program at all levels is the ability of students to evaluate their own learning and performance” (37, p. 210). Ongoing professional development in the workplace also necessitates reflection-in-action, reflection-on-action, and reflection-for-action (38–43). Consequently, the CCCET can also provide a structured, systematic, comprehensive mechanism/tool for periodic and ongoing self-reflection (self-appraisal).
Frequently, clinical evaluations have been reported as subjective appraisals (37, 44–45) and validity and objectivity between actual performance, assessment, and evaluation (judgment of the assessment) questioned. In addition, challenges in evaluating the many expected assessments, cultural-specific care interventions/behaviors, and professional attitudes, values, and beliefs within the large content domain of cultural competence and within clinical settings involving patients complicate the gathering and interpretation of such data. To address these concerns and challenges, a student version and a teacher version were created (see Jeffreys Toolkit Items 3 and 4). As part of a summative evaluation, students and teachers independently complete the CCCET at the end of the clinical experience; approximate completion time is 20 minutes (see Jeffreys Toolkit 2016 Items 3 and 4; see also Exhibits 6.1–6.3).
Following the design and preliminary psychometric evaluation of the CCCET–Student Version (CCCET–SV) and CCCET–Teacher Version (CCCET–TV) with undergraduate and graduate nursing students, an Employee Version (CCCET–EV) and an Agency Evaluator version (CCCET–AEV) were created for application in clinical agencies (see Jeffreys Toolkit 2016 Items 5 and 6). Questionnaire items remained the same; however, several words were changed in the directions to accommodate the change in population and setting. Formative evaluations conducted at designated points during a clinical course or employee orientation/preceptor period provide ongoing opportunities for guided self-reflection and feedback.
Psychometric Properties: Validity
What steps are needed for the design and evaluation of a valid and reliable assessment tool? What steps were followed in the design and evaluation of the CCCET?
Several studies were implemented to estimate the psychometric properties of the CCCET–SV and CCCET–TV.
Content Validity
Three content validity experts (doctoral-prepared nurses internationally recognized as experts in transcultural nursing) reviewed both versions of the CCCET. The Content Validity Index (CVI) was 0.91 for both the teacher and the student versions of the instrument, indicating that the experts found the items highly relevant and representative of the domain. Experts’ comments were appraised by the nurse researcher and statistician. No items required modification.
Additional Evidence of Validity
The CCCET–SV and CCCET–TV were piloted with both undergraduate and graduate nursing students. For the undergraduates, the CCCET was piloted with three semesters of second semester associate degree nursing students (n = 161) enrolled in a 15-week medical–surgical clinical course (1). Because the intended purpose was to evaluate cultural competence in the clinical setting, the particular undergraduate medical–surgical course was selected for the pilot because the course: (a) contained the most credit hours and clinical experiences of all the other courses in the associate degree curriculum; and (b) was the first course that required successful completion of the nursing fundamentals course in which students participate in multidimensional course activities incorporating transcultural nursing. Students independently completed the CCCET at the end of the clinical practicum (week 15). To allow matching of CCCET–TV and CCCET–SV, clinical instructors completed the CCCET–TV for each student at the end of the second clinical course, placing the completed CCCET–TV and student’s CCCET–SV in an envelope prior to sealing and returning to the researcher (1). Differences between student and teacher item mean ratings ranged from −1.39 to 0.12 (Subscale 1); −.05 to 1.35 (Subscale 2); and 0.16 to 2.17 (Subscale 3). “Student and teacher ratings were relatively close, suggesting that respondents took the task of CCCET completion seriously and honestly, that cultural competence was a visible theme throughout the course, and that students and instructors worked closely together in the clinical practicum setting to achieve learning objectives, including cultural competence” (1, p. 92). Rank-ordered outcomes via each subscale made sense conceptually when aligned with the course and curriculum. The findings demonstrated that the CCCET–SV and CCCET–TV were psychometrically valid (see Exhibit 6.1).
EXHIBIT 6.1
Disseminating Findings Via a Professional Conference: Sample Conference Abstract
Evaluating Cultural Competence in the Clinical Practicum: Undergraduate Students
Marianne R. Jeffreys, EdD, RN, Professor, Nursing
The City University of New York (CUNY) Graduate College, New York, NY, and CUNY College of Staten Island, Staten Island, NY
Enis Dogan, EdD, Associate Director for Research
Partnership for Assessment of Readiness for College and Careers (PARCC)
Washington, DC
Background
Evidence-based research models and educational practices that promote transcultural nursing require valid research methods for evaluating cultural competence. In addition, legal, ethical, and accreditation mandates demand theoretically based, valid, comprehensive tools to assess aspects of culturally specific care within the clinical practicum; yet, no relevant ones existed.
Purpose
Guided by Jeffreys’s Cultural Competence and Confidence (CCC) model for teaching cultural competence, the main purpose is to introduce a tool for evaluating the extent of culturally specific care provided for a diverse clientele, the frequency of cultural assessments, and the development of culturally sensitive and professionally appropriate attitudes, values, and beliefs. Specifically, the following research questions were explored:
1. To what extent is culturally specific care provided by students during the clinical practicum?
2. Which cultural assessments are implemented more frequently and which are implemented less frequently during the practicum?
3. To what extent do culturally sensitive and professionally appropriate attitudes, values, or beliefs change during the clinical practicum?
Instrument
Adapted from the psychometrically valid Transcultural Self-Efficacy Tool (TSET), the Cultural Competence Clinical Evaluation Tool (CCCET) contains three subscales measuring different dimensions of clinical cultural competence behaviors: (a) Subscale 1 (Provision of Cultural-Specific Care), 25 items; (b) Subscale 2 (Cultural Assessment), 28 items; (c) Subscale 3 (Cultural Sensitivity), 30 items. Review by three transcultural nursing experts yielded a Content Validity Index (CVI) of 0.91.
Method
Approval for the study was obtained from the institution’s institutional review board (IRB), chairperson, faculty, and grant-supporting agency.* The CCCET was administered at the end of a second semester medical–surgical nursing course (n = 161) following a series of cultural competence educational interventions with diverse students and faculty. Students independently completed the CCCET at the end of the clinical practicum. Clinical instructors completed the CCCET–TV (Teacher Version) for each student at the end of the second clinical course. Completed teacher and student forms were sealed in an envelope and returned to the researcher. To explore each of the research questions, item means, standard deviations, and frequency of each response category (in percent) were calculated for the CCCET student and teacher versions. In addition, the difference between teacher and student mean ratings was computed for each item. The Clinical Setting Assessment Tool–Diversity and Disparity (CSAT–DD) gathered important descriptive data about the clinical practicum/agency site; it specifically focuses on descriptions of diverse client populations (Part I) and clinical problems targeting Healthy People focus areas (Part II).
Results
After verifying that there were no significant demographic differences between the three semesters, data were collapsed for aggregate analyses. Reliability (Cronbach’s alpha) for the total CCCET–SV (Student Version) (0.99) and each subscale was quite high, ranging from 0.97 to 0.98. Cronbach’s alpha for the teacher version was also high, at 0.95 for the total CCCET–TV, and moderate to high for each subscale (0.85 to 0.98). Rank ordering of item means addressed each research question.
Conclusion
Student and teacher ratings were relatively close, suggesting that respondents took the task of CCCET completion seriously and honestly, that cultural competence was a visible theme throughout the course, and that students and instructors worked closely together in the clinical practicum setting to achieve learning objectives, including cultural competence. The findings demonstrated that the CCCET–SV and CCCET–TV were psychometrically valid and reliable. The rank-ordered outcomes via each subscale made sense when aligned with the course and curriculum. Notably, the greatest change on Subscale 3 occurred on the two items dealing with advocacy, suggesting that educational interventions via the clinical practicum experience were successful in moving students from more passive roles to more active patient advocacy. Implications for nurse educators in academia and practice conclude the presentation.
*Partially funded by the Mu Upsilon Chapter of Sigma Theta Tau International Honor Society in Nursing.
For the graduate student pilot, the CCCET was administered to graduate nursing students enrolled in either one of the two clinical courses in the clinical nurse specialist (CNS) adult health or geriatric curriculum. The CCCET was completed at the end of the semester by 24 students and clinical instructors/preceptors. To allow matching of CCCET–TV and CCCET–SV, clinical instructors/preceptors completed the CCCET–TV for each student at the end of the second clinical course, placed the completed CCCET–TV and student’s CCCET–SV in an envelope prior to sealing, and returned it to the researcher. There was high congruency between student and teacher item ratings with an average SV-TV difference of −1.75 (Subscale 1), −1.04 (Subscale 2), and −0.09 (Subscale 3). Student and preceptor ratings were relatively close, suggesting that respondents took the task of CCCET completion seriously and honestly, that cultural competence was a visible theme throughout the clinical practicum, and that students and preceptors worked closely together in the clinical practicum setting to achieve learning objectives (including cultural competence). Findings also made sense conceptually when aligned with the expectations of the advanced practice nursing curriculum, diversity of clients and clinical topics, and clinical settings (see Exhibit 6.2).
EXHIBIT 6.2
Evaluating Cultural Competence in the Clinical Practicum: Graduate Nursing Education
Marianne R. Jeffreys, EdD, RN, Professor, Nursing
The City University of New York (CUNY) Graduate College, New York, NY, and CUNY College of Staten Island, Staten Island, NY
Enis Dogan, EdD, Associate Director for Research
Partnership for Assessment of Readiness for College and Careers (PARCC),
Washington, DC
Data Collection
Approval for the study was obtained from the institution’s IRB, chairperson, faculty, and grant-supporting agency.* Preceptors agreed to administer the CCCET as part of the expected protocol for the graduate level (MS) clinical course during the three targeted semesters. Twenty-four students independently completed the CCCET at the end of the clinical practicum. To allow matching of CCCET–Teacher Version (TV) and CCCET–Student Version (SV), preceptors completed the CCCET–TV for each student at the end of the clinical practicum, placing the completed CCCET–TV in an envelope, sealing it, and then giving it to the student for return to the course coordinator. The student’s CCCET–SV was placed in an envelope prior to sealing it and returning it to the researcher.
Brief Tool Description
The CCCET is an 83-item questionnaire adapted from the Transcultural Self-Efficacy Tool (TSET), containing three subscales measuring different dimensions of clinical cultural competence behaviors as perceived by the student (Student Version) and the preceptor (Teacher Version):
• Provision of Cultural-Specific Care (Subscale 1) [25 items]
• Cultural Assessment (Subscale 2) [28 items]
• Cultural Sensitivity (Subscale 3) [30 items]
Data Analysis
Item means, standard deviations, and frequency of each response category (in percent) were calculated for CCCET student and teacher versions. In addition, the difference between teacher mean ratings and student mean ratings for each item was computed. Items with high responses of A ([clinical] area not available) or B (diverse clients not available) were excluded from further analyses. Four items on the Subscale 1 yielded A or B responses by at least 30% of student respondents; therefore, they were excluded from further analyses. For the most part, these items pertained to content areas not present in the clinical agency and not associated with the adult health and geriatric clinical nurse specialist and nurse practitioner tracks, such as pregnancy, birth, growth, and development; informed consent was also excluded. No item exclusion was indicated for Subscale 2 or Subscale 3. For the purposes of this report, rank ordering the five highest or five lowest means on each subscale is presented.
Select Results—(using item means for Student Version)
The five most frequent and five least frequent provisions of cultural-specific care:
Subscale 1 (Provision of Cultural-Specific Care)
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
