Examination of feasibility is critical for the successful conduct of large‐scale studies aimed to evaluate the effects of health interventions. Generally, feasibility has to do with the practicality of an intervention delivery and the application of the evaluation research methods. Testing feasibility prior to a large‐scale intervention evaluation study helps in identifying potential challenges and in revising aspects of the intervention and research methods and procedures. The revisions ensure that the study’s implementation is logical, practical, convenient; the revisions also are done to lower the probability of biases or threats to validity (Tickle‐Degnen, 2013). Challenges in the delivery of the health intervention reduce the interventionists’ and clients’ enthusiasm for the intervention. The challenges also affect the quality and fidelity of the implementation of the intervention and, consequently, its effectiveness in improving the immediate, intermediate, and ultimate outcomes. Similarly, challenges in carrying out research methods and procedures (e.g. recruitment, randomization, data collection) can be detrimental to the validity of inferences about the intervention’s effects. For instance, challenges in recruiting participants result in a small sample size, which limits the power to detect meaningful intervention’s effects, leading to type II error. In this chapter, terms used to refer to feasibility, and the distinction between feasibility of the intervention and feasibility of research methods are introduced. Guidance for examining feasibility is presented. Issues with the interpretation of findings related to outcomes in small‐scale preliminary or pilot studies are discussed. Different terms have been used to refer to preliminary (i.e. prior to large‐scale evaluation) studies concerned with testing feasibility. For some, the terms are different and for others, the terms are synonymous (Whitehead et al., 2014). Three main terms are mentioned in the literature to refer to preliminary studies: feasibility, pilot, and proof‐of‐mechanism or proof‐of‐concept. The features that are claimed to distinguish them include: For many, the purpose and features of the three types of preliminary studies are not quite distinct. Instead, the features overlap and all types of studies are designed to examine the extent to which aspects of the main study, including the intervention and the research methods and procedures, are practical and feasible, and to make necessary changes in the aspects that would facilitate the conduct of the main study. As such, the terms feasibility studies, pilot studies, and proof‐of‐mechanism or proof‐of‐concept studies are used interchangeably (Abbott, 2014; Eldridge et al., 2016b; Lancaster, 2015; Leon et al., 2011; Thabane et al., 2010). The primary focus is on examining the practicality of delivering the intervention and of carrying out the research methods and procedures, as well as the extent to which the intervention, as planned and delivered, is effective in initiating the mechanism of action; thus, the outcomes included in these preliminary studies should reflect the immediate and intermediate outcomes, more so than the ultimate outcomes. Feasibility can be conceptualized in terms of the practicality of providing a health intervention and of applying the research design, methods, and procedures. Accordingly, strategies for examining feasibility of the intervention and feasibility of the research methods are distinct and discussed next. The importance of examining the feasibility of health interventions has been emphasized (e.g. Arain et al., 2010; Lancaster, 2015; Leon et al., 2011; Thabane et al., 2010; Tickle‐Degnen, 2013). However, the feasibility of interventions has not been explicitly defined. As a result, the indicators of feasibility are not clearly operationalized and guidance for testing the feasibility of interventions is not well delineated. Some investigators consider the acceptability of interventions as an indicator of feasibility (e.g. Arain et al., 2010; National Center for Complementary and Integrative Health, 2017), ignoring the distinction in the conceptualization of acceptability, which is defined as the perceived desirability of an intervention (see Chapter 11) and the conceptualization of feasibility, which reflects the practicality of implementation of an intervention as explained next. Feasibility of health interventions refers to the practicality and adequacy of the logistics required for delivering interventions (Becker, 2008). The logistics entail the resources and procedures for providing the intervention. In a feasibility test, the focus is on determining access to the resources and the capability of carrying out the components and activities of the intervention as planned. Challenges are identified in the provision of any component or the performance of any activity of the intervention. The challenges may be related to: (1) the availability in good quality and quantity of materials resources; (2) the availability, in adequate number, of well‐trained interventionists; (3) the comprehensiveness and clarity of the manual in guiding the interventionists’ delivery of the intervention with fidelity; and (4) the capacity or ability of clients to engage in the intervention and enact the treatment recommendations. The occurrence of challenges in the main large‐scale evaluation study may reduce the interventionists’ enthusiasm for the intervention and the participants’ perceptions of the intervention and motivation to complete the treatment and/or the study. Interventionists facing repeated challenges may experience frustration. They may modify, informally and spontaneously, aspects of the intervention to overcome the challenges encountered during delivery. These modifications may result in deviations that jeopardize the fidelity of intervention delivery and subsequently, the achievement of beneficial client outcomes (Chapter 6). Similarly, participants encountering challenges may react unfavorably, generating a sense of disappointment and dissatisfaction with the intervention; this in turn, interferes with their willingness to attend and engage in the intervention sessions, to adhere to treatment recommendations, or to continue their participation in the evaluation study, all of which negatively affect the validity of inferences (Chapters 10 and 11). Examination of the feasibility of intervention delivery is critical to minimize the potential occurrence of these challenges and their consequences in the main large‐scale evaluation study. Identifying difficulties in the delivery of the intervention and understanding their nature (i.e. what the challenges are and what contribute to them) are prerequisites for finding the most appropriate ways to address them. Resolution of the challenges should be done prior to the main evaluation study in order to enhance the delivery, engagement, and enactment of the intervention when testing its effects on outcomes. Several indicators of feasibility of interventions can be examined. The definition of feasibility highlights aspects of the intervention delivery that are assessed for practicality and adequacy in their execution. These aspects, and respective indicators, reflect material and human resources, contextual features, and intervention delivery. Feasibility is examined with the actual delivery of the intervention by trained and skilled interventionists to a select group of participants representative of the target client population, in the context specified as having the features required to facilitate intervention delivery. Examination of feasibility involves close monitoring of the activities performed in preparation of and throughout the delivery of the intervention. Relevant quantitative and qualitative methods are used to obtain data on the indicators. Feasibility is indicated by the availability of all material resources needed for the proper delivery of the intervention (Arain et al., 2010; Tickle‐Degnen, 2013). The types of resources differ with the nature of the intervention and its mode of delivery. As mentioned in Chapter 4, material resources are categorized into: (1) equipment such as laptop and projector for the presentation of educational material during a session, and personal computer for accessing online modules by participants; (2) infrastructure such as rooms with adequate seating; (3) supplies such as items for demonstration of a skill (e.g. inhaler) and for use by participants to apply the treatment recommendations (e.g. pedometer); and (4) written document such as booklet or other self‐help information guiding participants in the application of treatment recommendations. Material resources should be available in adequate number for all participants and be functioning properly. Strategies for accessing additional resources (e.g. contacting IT staff to resolve computer problems) as needed and promptly should be put in place. Information on the availability and accessibility of material resources is gathered formally by having the study personnel complete a checklist. The personnel include the research staff preparing for the delivery of the intervention, the interventionist delivering the intervention, or the observer monitoring the fidelity of intervention delivery. The checklist covers all material resources that should be prepared and made available for delivering the intervention as planned. The study personnel refers to the checklist to facilitate the preparation of the items, indicate the availability of the items; and document any challenges encountered with the accessibility and/or use of the items during the delivery of the intervention. In addition, formal interviews with the research staff, interventionists, and participants, scheduled upon completion of the intervention, are useful in identifying additional challenges in accessing and using material resources throughout the course of the intervention delivery. The information can also be obtained informally during regular meetings with research staff and interventionists. The quantitative data gathered by completion of the checklist are analyzed descriptively to determine the frequency of occurrence of challenges. The qualitative data obtained through formal interviews and discussion at meetings are content analyzed to determine the nature of the challenges and their impact on the delivery of the intervention. The quantitative and qualitative findings are integrated to inform the generation of relevant strategies to ensure preparation, availability, and accessibility of the material resources for providing the intervention. The strategies are embedded in the research protocol and intervention manual for use in the main evaluation study. Contextual features contribute to the feasibility of delivering the intervention. The intervention theory delineates features of the physical and psychosocial environment that facilitate provision of the intervention. The theory guides the selection of the setting (e.g. location, room) for giving the intervention and for promoting participants’ engagement. It also identifies aspects of the participants’ environment and life circumstances that may affect their enactment of treatment recommendations, and suggests ways to adapt the intervention to address them (see Chapter 5). Nonetheless, it is important to assess the logistics and practicality of delivering the intervention in the selected context. Specifically, the following contextual features are examined: The methods for obtaining information on the logistics and practicality of the context in which the intervention is provided are similar to those described for gathering information on material resources. A checklist is developed and used to assess the presence and adequacy level of the contextual features, as specified in the intervention theory. Two versions of the checklist are generated. One version focuses on the research staff, interventionists, and observers’ appraisal of the presence and adequacy of the features of the location and room in which the intervention is delivered. They complete the checklist prospectively throughout the intervention period. The other version focuses on participants’ judgment of the adequacy of the location, room, and timing of the intervention delivery. It also includes items for participants to report on features in their environment and life circumstances that facilitate or hinder the enactment of the treatment recommendations. Formal interviews with research staff, interventionists, and participants; informal discussion at research staff meetings; and complaints reported informally by participants to interventionists or research staff are all helpful in clarifying the nature of the contextual challenges and in delineating their influence, positive or negative, on the implementation of the intervention. Descriptive, quantitative and qualitative, findings point to contextual features that are inconsistent with those identified in the intervention theory and that interfere with the implementation of the intervention by interventionists and participants. Necessary modifications (e.g. selection of a more convenient location and timing, and revision of principles for adaptation or tailoring) are made prior to the delivery of the intervention in the main evaluation study. Feasibility of intervention delivery rests on the interventionists’ capacity to provide the intervention, as planned, to the predetermined number of participants (Tickle‐Degnen, 2013). Capacity is reflected in the availability of an adequate number of well‐prepared interventionists. Examination of this indicator of feasibility is founded on the logistics of delivering the intervention. This information is then used to estimate the number of interventionists needed for the main large‐scale evaluation study. In addition, assessment of capacity involves a determination of the extent to which training is adequate in preparing interventionists for intervention delivery. Estimation of the adequacy of the number of available interventionists is partly based on the nature, mode of delivery, and dose of the intervention as specified in the intervention theory. It is also informed by the logistics for providing the intervention, observed in the preliminary study. The logistics are related to: The estimation should also account for the plan to investigate the influence of interventionists (Chapter 8) and for possible interventionists’ resignation or withdrawal, and coverage for vacation and sick time. In addition to their availability in an adequate number, the interventionists should be well prepared to provide the intervention with fidelity. They are expected to attend the didactic part of the training to gain a comprehensive understanding of the intervention theory and protocol, including principles and strategies for adapting intervention content, activities, or treatment recommendations to fit the characteristics and life circumstance of individuals or subgroups of clients. Interventionists should also actively participate in the hands‐on training sessions or supervised delivery of the intervention in order to enhance their skills performance (Chapter 8). It is essential to monitor the training sessions for any issues that may arise during training and to assess the adequacy of the training in terms of content and duration, in preparing interventionists. Issues that may arise during training may be related to the following: These issues are identified informally through observation or reviewing audio recordings of the training sessions, or formally through discussion with the trainers or interviews with trainees. The interviews are scheduled upon completion of the didactic and the hands‐on practice parts of the training. The discussion and interviews focus on strengths and limitations of the intervention and of the training, and on ways to improve both as needed. Adequacy of the training reflects its effectiveness in enhancing interventionists’ theoretical knowledge, practical competence, and confidence in providing the intervention with fidelity. The effectiveness of training is evaluated by administering questionnaires before, during, and after the didactic and the practice sessions, as described in Chapter 8. This quantitative evaluation can be supplemented with interviews, held individually or in group, with the trainers and the trainees. The interviews are semi‐structured, concerned with identifying aspects of the training that are helpful and with exploring ways to improve the training. Results of the descriptive analysis of quantitative data and content analysis of qualitative data indicate aspects of the training that are useful or effective. The results may also identify challenges in carrying out the training and in enhancing interventionists’ capacity to deliver the intervention, and strategies to resolve the challenges. The feedback is integrated to refine the training sessions in adequately preparing interventionists or health professionals in future application of the intervention in research (main evaluation study) and practice. The implementation of health interventions by interventionists and clients is a key indicator of feasibility (Abbott, 2014; Arain et al., 2010; Leon et al., 2011; National Center for Complementary and Integrative Health, 2017; National Institute for Health Research, 2012; Tickle‐Degnen, 2013). Monitoring the fidelity with which interventionists deliver the intervention as well as the level of clients’ engagement and enactment is instrumental in identifying the adequacy or challenges in the application of the planned intervention activities. Quantitative and qualitative methods for assessing fidelity, engagement (e.g. attendance at intervention sessions), and enactment (e.g. adherence to treatment recommendations) are detailed in Chapters 9 and 13. When examining feasibility of the intervention, qualitative methods are appropriate. Feasibility is examined in a trial delivery of the intervention by an interventionist to a number of participants, such as four to six individuals or one group sessions. Close monitoring of the intervention delivery provides an excellent opportunity to address the adequacy of the intervention manual in guiding its delivery and the logistics of performing and engaging clients in the planning activities, and to recognize challenges. The challenges may be related to: For technology‐based interventions, examining the feasibility of delivery entails assessment of: Qualitative data on feasibility can be obtained through: (1) formal observation of the intervention sessions by the researchers or designates; (2) formal interviews with interventionists and participants; (3) review of formal documents (e.g. diary) completed by participants; (4) informal feedback given by interventionists; and (5) informal communication of challenges encountered by participants to either the interventionist or the research staff. Content analysis of the information gathered formally or informally points to aspects of the intervention that are feasible and those that are challenging and need to be modified and how. The modifications are integrated in the manual in order to enhance the ease and quality of providing the intervention in the main evaluation study.
CHAPTER 12
Examination of Feasibility: Intervention and Research Methods
12.1 TERMS REFLECTING PRELIMINARY STUDIES
12.2 FEASIBILITY OF INTERVENTIONS
12.2.1 Definition of Feasibility
12.2.2 Indicators of Feasibility
12.2.2.1 Material Resources
12.2.2.2 Contextual Features
12.2.2.3 Human Resources
12.2.2.4 Intervention Implementation
12.2.3 Research Design