The development of an intervention manual, the careful selection of competent interventionists, and the intensive training of interventionists are strategies to promote fidelity of intervention delivery; however, they do not guarantee it. The actual delivery of an intervention may deviate from what is designed and described in the manual, and may vary for different clients. Interventionists, especially those experienced, view intervention manuals as being at odds with the principles of treatment, stating that treatment should be provided with flexibility and tailored to clients’ individual characteristics, concerns, and life circumstances. Interventionists also report that strictly adhering to the intervention manual interferes with building and maintaining a good rapport, therapeutic relationship, or working alliance, and with the quality of interactions between interventionists and clients; yet, interventionists value these interactions because they contribute to clients’ engagement in and enactment of treatment, and subsequently improvement in client outcomes (Brose et al., 2015). Accordingly, interventionists and health professionals have a tendency to not use or follow the intervention manual in delivering standardized and/or individualized interventions (Lorencatto et al., 2014; Wallace & von Ranson, 2011). This, in turn, leads to variability in intervention delivery, which has been reported widely in research (e.g. Webb et al., 2010) and in practice (e.g. Tschuschke et al., 2015; Verschuur et al., 2019). Variability in intervention delivery results in differences in the active ingredients to which clients are exposed, potentially yielding nonsignificant intervention’s effects on outcomes. Monitoring fidelity is critical for identifying deviations or variability in intervention’s delivery and rectifying them as necessary. Assessing fidelity is important for examining the impact of such variability on the outcomes expected of an intervention. Monitoring and assessing fidelity rest on a clear conceptualization of fidelity. The focus of this chapter is on the conceptualization and operationalization of fidelity. Definitions and levels of fidelity are reviewed. Strategies and methods for assessing fidelity are discussed. Traditionally, fidelity has received limited attention in intervention evaluation research. This was based on the assumption that well‐trained interventionists strictly follow the treatment manual, which ensures standardization and consistency in providing standardized and tailored interventions to clients. The assumption proved to be untenable in light of evidence showing differences in interventionists’ use of manual and performance in delivering interventions (as presented in Chapter 8). Fidelity emerged as a concern in intervention research in various fields of study, most notably program evaluation, psychotherapy, and, recently, behavioral medicine. Differences in theoretical and methodological orientations across fields of study may have contributed to differences in terminology and conceptualization of fidelity (McGee et al., 2018). Different terms are used to refer to fidelity. These include: (1) integrity, which appears primarily in the field of program evaluation; (2) adherence to treatment protocol, which is mentioned in the field of psychotherapy; and (3) fidelity, which has recently been reported in the fields of psychotherapy, behavioral medicine and implementation science. These terms are often used interchangeably to refer to the extent to which the delivery (in research) or implementation (in practice) of an intervention is consistent with the original intervention design. Recently, the term “flexible fidelity” has appeared in the literature. It reflects the increasing recognition of the need to adapt the delivery of interventions to fit the characteristics and circumstances of the client population and context, while maintaining fidelity in providing the intervention’s active ingredients that are operationalized in the specific or core components (e.g. Mignogna et al., 2018; Shelton et al., 2018) and responsible for its beneficial effects. The term fidelity is used throughout this book as it is the most widely employed in current writings. The widening interest in fidelity in different fields of study led to variability in its conceptualization. An increasing number of frameworks has been published; the recently mentioned ones are listed in Table 9.1. The frameworks present different definitions of fidelity that result in its operationalization in varying domains. The domains have been inconsistently defined and operationalized (McGee et al., 2018). These differences add to much confusion as what actually is fidelity and what domains are to be monitored and assessed (Song et al., 2010; Wainer & Ingersoll, 2013). Commonly used definitions of fidelity and its domains, synthesized from extant literature, are presented next. The definitions of fidelity, advanced by researchers in different fields, are mentioned in Table 9.2. The first definition is by far the most commonly embraced. The second definition does not actually characterize fidelity but confuses fidelity with the use of strategies to promote it (Song et al., 2010). The third definition emphasizes the consistency between the intervention’s theoretically identified active ingredients and their operationalization into specific components. This consistency is foundational for ensuring construct validity of interventions. Accordingly, the first and third definitions are relevant for a comprehensive conceptualization of fidelity. The conceptualization posits two levels of fidelity: TABLE 9.1 Recently mentioned frameworks of intervention fidelity. TABLE 9.2 Conceptual definitions of intervention fidelity. Most frameworks (Table 9.1) were primarily concerned with operational fidelity and extended it to cover domains representing the implementation of interventions by the interventionists and the clients. The extension is based on the realization that the implementation of health interventions is the responsibility of both. Interventionists are ascribed the functions of relaying content to and engaging clients in the activities as planned; whereas clients exposed to the intervention are expected to engage and enact treatment in daily life. Accordingly, in the frameworks, operational fidelity is represented in the following domains: adherence, competence and differentiation for interventionists, and responsiveness, exposure, receipt or engagement, and enactment for clients. Each of these domains is defined next. Adherence to the intervention is the core of fidelity. It refers to whether or not the intervention is delivered as designed or intended (Wainer & Ingersoll, 2013). Adherence implies that the interventionist performs the prescribed activities or behaviors for providing the treatment, as described in the intervention manual (Forsberg et al., 2015; Wojewodka et al., 2017) and avoids proscribed activities or behaviors. Prescribed activities are those reflecting the intervention’s active ingredients, represented in the specific components. Proscribed activities include those comprising other treatments such as the use of cognitive reframing in a purely behavioral intervention and general activities that detract from the treatment such as allowing the focus of a treatment session to shift to irrelevant topics (Campbell et al., 2013; DiRezze et al., 2013; Stein et al., 2007). Adherence represents the quantity of intervention delivery (Berkel et al., 2019). It is quantified as the number of the intervention’s components that are actually provided or the number of prescribed activities actually performed, out of those planned. Adherence is enhanced by: 1) having an intervention manual that specifies the activities to be performed and how, and the activities to be avoided; 2) training interventionists in the theory and skills for providing the intervention; and 3) requesting interventionists to follow the intervention manual when delivering the intervention. Competence (also called process fidelity by Dumas et al., 2001) focuses on the manner in which the interventionists deliver the intervention. Competence relates to the interventionists’ skillfulness at providing the intervention (Leeuw et al., 2009; Stein et al., 2007). A range of skills have been mentioned in the literature as reflecting interventionists’ competence. The most common skills pertain to: delivering the intervention while responding appropriately to client characteristics, concerns, and life circumstances (Hartley et al., 2014; Mars et al., 2013; Carpinteiro da Silva et al., 2014); engaging in nonspecific behaviors such as being flexible; adapting the content or activities to clients’ concerns and circumstances (Campbell et al., 2013); being client‐centered (Aggarwal et al., 2014); showing empathy (Aggarwal et al., 2014); communicating information clearly, at an appropriate pace and in an engaging or interactive way (Wojewodka et al., 2017); demonstrating understanding of clients’ life situation; clarifying information; providing constructive feedback; and collaborating with clients (Berkel et al., 2019; Carpinteiro da Silva et al., 2014; Hartley et al., 2014). The interventionists’ competence skills have been characterized slightly differently. Roth and Pilling (2008) identified them as generic skills and defined them as those exhibited in working collaboratively with clients. Dixon and Johnston (2010) characterized the skills as foundation competencies; these skills involve generic communication skills, ability to engage and collaborate with clients, and capacity to adapt treatment in response to clients’ concerns and feedback. Alternatively, these skills encompass communication, interactional style, and development and maintenance of a therapeutic relationship or working alliance, as described in Chapter 8. In general, competence entails generic (also called common factors) therapeutic skills of interventionists and accounts for the interventionist effects on client outcomes. Competence has been characterized as the quality of intervention delivery (Berkel et al., 2019; Southam‐Gerow & McLeod, 2013). It is enhanced with careful selection of interventionist and provision of constructive feedback on these generic skills. Differentiation is the domain of fidelity concerned with the distinctiveness of the health intervention and other treatments provided to clients in an evaluation study or in practice. It refers to the extent to which the treatments differ from one another in the intended ways (Hasson, 2010; Wainer & Ingersoll, 2013). This means that the interventionists perform the activities prescribed for each treatment and avoid the proscribed ones (Aggarwal et al., 2014; Forsberg et al., 2015), thereby avoiding contamination of the treatments. Client responsiveness has not been lucidly defined at the conceptual level. At the operational level, client responsiveness encompasses domains of fidelity that pertain to clients’ implementation of the intervention. It reflects clients’ exposure to, engagement in, and enactment of the intervention. Exposure represents the amount of the intervention to which clients are exposed. In other words, exposure is the dose of the intervention that clients actually receive (Hasson, 2010; Ibrahim & Sidani, 2016; Wainer & Ingersoll, 2013), reflecting the level of contact with the intervention content. It is usually quantified as attendance at the intervention sessions or self‐completion of the intervention modules. Both terms are often used interchangeably to denote clients’ active involvement in the planned intervention activities, comprehension of the content presented in the intervention sessions or modules, and capacity to employ the skills or perform the behaviors required for applying the treatment recommendations (Leeuw et al., 2009; Wainer & Ingersoll, 2013). Clients’ involvement in the intervention activities, such as participation in discussion of the treatment recommendations and reading accompanying materials, helps them to gain a good understanding the intervention and to retain the information. The acquired knowledge promotes their confidence to implement the skills, behaviors, or treatment recommendations in daily life (Prowse & Nagel, 2015; Walton et al., 2017). The results of two literature reviews on fidelity indicated that receipt or engagement is operationalized into: understanding or knowledge of the intervention content, self‐efficacy, and acceptability or satisfaction with the intervention (O’Shea et al., 2016; Rixon et al., 2016). Enactment is defined as the degree to which clients actually apply the treatment recommendations in their daily life during (i.e. in‐between intervention sessions) and following the treatment period (Ibrahim & Sidani, 2016; Prowse &, Nagel 2015; Wainer & Ingersoll, 2013). As defined, enactment has traditionally been discussed under the rubric of client adherence to treatment. The frameworks and domains of fidelity presented in Section 9.1.2 and 9.1.3 have been critiqued on conceptual grounds. First, the distinction between theoretical and operational fidelity has not been highlighted and the importance of evaluating theoretical fidelity has not been emphasized. Yet, maintaining theoretical fidelity is a prerequisite for enhancing construct validity of the intervention. Theoretical fidelity is maintained through the systematic process of designing health interventions described in Chapter 4 and 5. Briefly, the process begins by specifying the active ingredients of the intervention, followed by operationalizing the ingredients into components and ending with translating the components into specific activities. Evaluating the correspondence or alignment among activities, components, and active ingredients is important to determine the adequacy and accuracy of the operationalization and to delineate what distinguishes the intervention from others. Second, most domains of fidelity reflect concepts that have been posited and examined as underpinning the interventionist effects on client outcomes (i.e. competence), the mechanism mediating the intervention’s effects on outcomes (i.e. all domains under client responsiveness), or methodological issues (i.e. differentiation) (Gearing et al., 2011; Song et al., 2010). Furthermore, these domains are not in alignment with the conceptual definition of fidelity embraced by a large number of researchers (Table 9.2) and characterizing fidelity as the extent to which an intervention is delivered as designed. Empirical evidence, synthesized in two reviews, indicates that adherence is the most frequently examined and reported domain of fidelity for interventionist and technology‐based delivery of health interventions (Ibrahim & Sidani, 2015; O’Shea et al., 2016). This evidence, in combination with the theoretical points presented in the previous paragraph, has contributed to the increasing use of the Consolidated Framework for Implementation Fidelity (Carroll et al., 2007) to inform research on fidelity (McGee et al., 2018; Roy et al., 2018; Seys et al., 2019; Swindle et al., 2018). The Consolidated Framework for Implementation Fidelity presents a simplified conceptualization of fidelity that is coherent with the definition of fidelity and addresses the critique of available frameworks presented in the previous paragraphs. The simplified framework focuses on operational fidelity as it pertains to interventionists. The domains of operational fidelity pertaining to clients are viewed as embedded in the mechanism responsible for the intervention’s effects on the ultimate outcomes, and investigated as part of process evaluation (Chapter 13). In the simplified conceptualization, fidelity refers to the degree to which the interventionists deliver the health intervention as intended or designed. It is operationalized in terms of adherence to the intervention manual, which is reflected in the interventionists’ performance of prescribed activities and avoidance of proscribed activities. Four categories of factors affect (or moderate) interventionists’ delivery of the intervention with fidelity. The first category of factors consists of the characteristics of the intervention, with complexity (e.g. number of components) of the intervention being the most prominent. The second category of factors includes the characteristics of the interventionists that define the quality of the intervention delivery; these are represented in the interventionists’ perceptions of the intervention, confidence or self‐efficacy in providing the intervention, and competence (which encompasses communication skills, interactional style, and working alliance or therapeutic relationship). The third category of factors entails contextual factors such as the availability and use of supportive strategies (e.g. training) to facilitate intervention delivery. The last category involves characteristics of clients (e.g. severity of health problem, motivation) that impact what is provided and how. Interventionists’ delivery of the intervention with fidelity influences clients’ responsiveness, that is, clients’ exposure to, engagement in, and enactment of treatment (Barber et al., 2007; Hasson et al., 2012; Wang et al., 2015). Preliminary evidence supports some of the framework propositions: A positive moderate association was reported between the interventionists’ competence and adherence (Peters‐Scheffer et al., 2013). Assessment of theoretical fidelity is important for ensuring the valid operationalization of the intervention’s active ingredients into respective components, content, and activities. The results of such an assessment assist in the interpretation of the findings of an intervention evaluation study. The assessment results suggest whether findings indicating the intervention is ineffective are attributable to faulty intervention design and/or misalignment between the intervention’s active ingredients and respective components; this implies that the intervention components are inappropriate for initiating the hypothesized mechanism of action. Alternatively, results of theoretical fidelity assessment increase the confidence in attributing the observed improvement in the outcomes to the intervention. There are no formal published guidelines for assessing theoretical fidelity. Logically, the assessment should involve a thorough examination of the correspondence among the intervention’s active ingredients as identified in the intervention theory, and the components and activities as delineated in the intervention logic model, protocol, or manual. Assessment of theoretical fidelity is done prior to the implementation of the intervention. Two strategies can be applied to conduct this assessment: generation of a matrix and content validation. The matrix is generated to examine the alignment among the intervention’s active ingredients and their operationalization in respective components and activities. Pertinent information is gathered from the intervention theory, logic model, and manual; from relevant published or grey literature; and from discussion with intervention designers (Haynes et al., 2016). The intervention’s active ingredients are identified by reviewing the intervention theory and relevant literature, and clarified through discussion with the designers. The intervention’s components and activities are abstracted from the intervention logic model and manual.
CHAPTER 9
Assessment of Fidelity
9.1 CONCEPTUALIZATION OF FIDELITY
9.1.1 Terminology
9.1.2 Frameworks of Fidelity
9.1.3 Definition of Fidelity
Framework
Source
Definition and domains of fidelity
Consolidated Framework for Implementation Fidelity
Carroll et al. (2007)
Definition:
Fidelity is the degree to which interventions are implemented or delivered as intended
Domain:
Fidelity is operationalized in adherence to the intervention protocol or manual
Moderating factors:
Complexity of intervention, quality of delivery, facilitation strategies, client responsiveness
Treatment Fidelity Framework
Originally published by Bellg et al. (2004) on behalf of the National Institutes of Health—Behavioral Change Consortium
Definition:
Fidelity encompasses methodological strategies used to monitor and enhance the reliability and validity of interventions
Domains:
Design (or fidelity to theory), training, delivery or implementation, receipt, enactment
National Implementation Research Network
Domains:
Context, content, competence (covering interventionists’ decision making), client responsiveness
Comprehensive Intervention Fidelity Guide
Gearing et al. (2011)
Domains:
Design, training, monitoring of intervention fidelity, intervention receipt
Fidelity of Technology‐Based Interventions
DeVito Dabbs et al. (2011)
Domains:
Delivery, receipt, technology acceptance
Adapted Model of Fidelity
Wainer and Ingersoll (2013)
Domains:
Delivery (i.e. adherence, exposure, differentiation)
Moderating factors:
Complexity of intervention, facilitation strategies, competency, comprehension, social validity
Treatment Implementation Model
Lichstein et al. (1994)
Domains:
Delivery, receipt, enactment
Definition
Source
Aggarwal et al. (2014), Berkel et al. (2019), Breitenstein et al. (2010), Brandt et al. (2004), Campbell et al. (2013), Carroll et al. (2007), Carpinteiro da Silva et al. (2014), DeVito Dabbs et al. (2011), DiRezze et al. (2013), Dunst et al. (2013), Forsberg et al. (2015), French et al. (2015), Hasson (2010), Haynes et al. (2016), Ibrahim and Sidani (2016), Judge Santacrocce et al. (2004); Lorencatto et al. (2014), Mowbray et al. (2003), Prowse and Nagel (2015), Oxman et al. (2006), Roy et al. (2018), Schulte et al. (2009), Seys et al. (2019), Southam‐Gerow and McLeod (2013), Toomey et al. (2019), Wojewodka et al. (2017)
Bellg et al. (2004), Borrelli et al. (2005), Hart (2009), Mars et al. (2013), Resnick et al. (2005), Stein et al. (2007), Swindle et al. (2018)
Keller et al. (2009), Pearson et al. (2005)
Forsberg et al. (2015), Saunders et al. (2005)
9.1.3.1 Domains of Fidelity—Interventionist
Adherence
Competence
Differentiation
9.1.3.2 Domains of Fidelity—Client
Responsiveness
Exposure
Receipt or Engagement
Enactment
9.1.4 Simplified Conceptualization of Operational Fidelity
9.2 STRATEGIES AND METHODS FOR ASSESSING THEORETICAL FIDELITY
9.2.1 Generation of a Matrix
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
