R-11: Building Capacity for Full Community Participation
Methods
In the context of community and systems changes, socio-ecological theory suggests that changing the conditions in which people with disabilities live can influence their civic engagement. Group or community capacity to implement community processes is critical to bring about changes in community conditions (e.g., community and systems change) across all socio-ecological levels. These findings lead us to investigate the following research questions and hypotheses:
Research Question 1: Is the Community Participation Training and Technical Assistance (CPTTA) intervention effective?
Hypothesis 1: CILs that receive CPTTA will be more successful at initiating community and systems change than CILs that do not receive CPTTA.
Hypothesis 2: CILs that receive CPTTA will create more opportunities for civic engagement and community participation for people with disabilities in their community than those CILs that do not receive CPTTA.
Research Question 2: What are the contextual and influencing program factors distinguishing CILs’ successful implementation of community change, civic engagement and community participation for people with disabilities?
Participatory Action Research (PAR)
The investigators will involve a CET in all phases of the research. CILs and identified members of the community will be actively engaged in: assessing barriers to and assets for participation; helping develop strategic and action plans to address those factors that affect risk for isolation among local people with disabilities; implementing the multi-level plan across multiple sectors (e.g., schools, businesses, faith) using a variety of strategies; and evaluating the organization’s efforts. Regular review and analysis of community change data will enable local partners to evaluate and adjust their efforts to more effectively address the participation needs of consumers with disabilities and take advantage of emergent opportunities to address those needs.
Fidelity of Implementation
The fidelity or consistency with which the training and technical assistance intervention is implemented will be assessed by tracking the activities and products related to implementation. Training will be documented through records (e.g., records of curriculum modules completed by coalition participants; evidence that participants satisfactorily completed planning products, such as the elements of strategic plans, or plans for sustainability).
Technical assistance activities will also be tracked using a web-based online documentation support system located in the Community Tool Box (e.g., the frequency and type of instances of technical assistance provided to participant coalitions for different community processes). The quality of implementation will be assessed using surveys given to coalition participants after training and technical assistance activities (e.g., ratings of satisfaction with the quality and usefulness of training and technical assistance).
Sample
We will select 12 CILs serving people with disabilities in RSA Regions VI and VII to participate in the study. The National Council on Independent Living (NCIL) and the Association of Programs for Rural Independent Living (APRIL) have agreed to provide support for the identification and recruitment of CILs.
All CILs chosen to participate will have: a) the goal of increasing community participation among their consumers with disabilities, b) similar funding and staff assigned to the effort, and c) similar levels of maturity (≥ 5 years of existence). These criteria will help increase the likelihood that the CILs will continue their participation for the duration of the project.
Using current practice standards in single subject designs, we have planned our studyaccording to the published design standards and criteria of evidence rather than using a traditional power analysis to determine adequacy to examine the proposed research questions.
Design
This project will use a mixed-design study, incorporating both single-subject (quantitative) research and a grounded theory (qualitative) inquiry. The use of a mixed-methods approach will enable us to simultaneously address both confirmatory and exploratory questions. In this study, single subject design is the dominant approach and the qualitative inquiry is designed to supplement the approach. We will use a parallel data collection strategy, collecting and analyzing qualitative data throughout the duration of the project.
The single-subject design will allow researchers to intervene across 3 cohorts, using a wait-list replication. We will randomly assign 4 CILs to each of three cohorts. All cohorts will begin in baseline, with the first cohort receiving the training in about 6 months, while the remaining 2 cohorts continue in baseline. Six months following training of Cohort 1, the second cohort will receive training while the third cohort remains in baseline. This pattern repeats until all three cohorts have received the intervention. As depicted in the R-11 Gantt Chart (see Appendix G), the baseline time periods will range from about 5 months to 17 months.
For the qualitative component we will use a grounded theory, case study approach. We will select one site (3 sites total) from each cohort to serve as case study site. We will use maximum variation purposive sampling and advice from our CET to select sites that represent salient differences in communities (e.g., demographics, rural/urban, socioeconomic status of the overall community).
Implementation and Assessment of the Independent Variable. CPTTA: This intervention is composed of two components: a) training of representatives from CILs in core competencies related to promoting community change (e.g., assessing community needs and resources, advocacy) and b) providing technical assistance in implementing community processes (e.g., action planning) built on those skills.
Two CIL staff from each cohort will receive training in core competencies related to implementing community processes. This will involve a 2.5-day workshop at the University of Kansas (KU), including a selected subset of 13 field-tested modules in the Community Tool Box (CTB) Framework for Community Participation Training and TA project.
KU staff members will deliver the trainings consisting of presentations, small and large group exercises, and practice in completing plans or other products that demonstrate proficiency in each skill area (e.g., developing a vision and mission statement). Each curriculum module consists of: a) Participant’s Guide, b) Facilitator’s Outline, c) PowerPoint slides, and d) Assessments of knowledge and competence. The curriculum has been field tested in a variety of settings ( e.g., substance abuse prevention research projects, chronic disease prevention initiatives in Texas, and in the Community Anti-Drug Coalitions of America National Training Academy.
In addition, implementation of activities associated with each community process will be supported through technical assistance and consultation to each CIL when it moves to an experimental condition (group). A protocol will guide technical assistance, which will be delivered in appropriate forms including: a) consultation and coaching using telephone or web-conferences, b) e-mail and monthly phone contacts for six months as treatment group, then bi-monthly, c) ongoing web-based peer support through social media, and d) Internet-based resources for coalition work through the Community Tool Box. These data will provide a written record to assess fidelity of implementation of the technical assistance component of the independent variable.
Data Collection and Measurement
Data collection will occur monthly using the CTB online documentation system. For this project we will define the independent variable as a multi-component training package consisting of the CTB Curriculum for Promoting Community Health and Development, providing civic engagement skills training based on adapted CTB materials and additional best or emerging practice tools to increase consumer community participation solicited from CILs; and technical assistance in implementing best processes for CILs serving people with disabilities and best practices to people with disabilities whose goal is to become more engaged in community living.
We will define the dependent variable as the quantity and quality of community and systems changes (programs, practices and policies) that support full community participation of people with disabilities. Quantity will refer to the number of new or modified program, policies, and practices established and implemented by each group. The quality of the change will be measured using an impact rating of each, in which programs will be individually rated based on characteristics such as duration, efficacy of the strategy, level of implementation, and reach (e.g., number of people impacted). A count of high impact programs will be calculated for each group and this measure will serve as the dependent measure of quality. We will use a multiple baseline design across cohorts, where the dependent variable is measured repeatedly in each community group and the manipulation of the independent variable occurs at different points in time for different groups to help control for threats to internal validity.
We will start baseline measures in Fall 2012, and continue monthly to record the community and systems to observe changes in quantity and quality of community/systems changes brought about after CPTTA is implemented. To determine the efficacy of the CPTTA intervention on community/system, cohorts will all receive regular observation and receive treatment (intervention). Cohorts will receive treatment in a sequential fashion.
For the qualitative study, protocols for each of the three case study sites will be developed using a series of grand tour questions related to the primary research questions and modified for the different stakeholders to be interviewed (e.g., CIL staff receiving training, consumers, community policy makers targeted for change). We will conduct the first round of interviews at the conclusion of the 2.5-day training held at KU.
For subsequent interviews, researchers will identify CIL staff receiving technical assistance, and beginning with those individuals, use a snowball sampling process to access additional stakeholders (e.g., persons with disabilities, community agency staff involved in change activities) for interviews. The researcher will make at least one site visit to each of the three sites selected for each cohort, and in addition will conduct individual interviews via phone or Skype. All interviews will be audio recorded and transcribed verbatim.
Data Analysis
The data will be analyzed using visual analysis of the multiple baseline design measures. The first inspection will be conducted to ensure documentation of a predictable baseline pattern. Second, the within phase data will be examined for patterns. Based on prior research with similar communities, we expect some lag in change within phases as implementation of programs requires sufficient time before effects are evident.
We will examine the data across groups to determine consistencies of level, trend, variability, immediacy of effect and consistency of patterns across similar phases. Once trends are examined and consistency across phases is apparent, we will examine data at baseline and treatment to determine if implementation of the treatment can be associated with an effect in the outcome variable of interest. With our multiple baseline design of three cohorts we expect to find multiple demonstrations of the effect.
Single case design standards and criteria for evidence of a relation between independent variable and outcomes. According to current practice standards in single subject designs, we have planned our study according to the published design standards and criteria of evidence rather than using a traditional power analysis to determine adequacy to examine the proposed research questions. The proposed multiple baseline time series design meets the design standards.
These standards include the systematically manipulated independent variable, in this case the treatment training intervention implemented on 3 occasions in the multiple baseline design allowing for 6 phases (3 phase repetitions). Each phase has multiple data points above the minimum of 5 required to meet evidence standards for a multiple baseline design. There is systematic measurement of the outcome variables of quantity of programs as well as quality of programs.
The analysis plan also includes steps to meet the criteria of evidence of relations between the independent variable and outcomes. The planned visual analyses will document level, trend, and variability within phase and examine the immediacy of the effect as well as consistency of data across phases. External effects will be considered as well as examining data anomalies. The proposed multiple level analyses are appropriate for determining the effect-size estimation once an effect is evident from the visual inspection.
For the qualitative case studies, we will use a two-step process to analyze the data. First, we will utilize field note contact sheets and peer debriefing to begin the analytic process immediately after each interview, to produce initial, emergent themes, identify new probes and interview questions, and suggest additional potential interview respondents. The peer debriefing process is a form of triangulation, enhancing the dependability of findings. Second, we will analyze the transcripts using NVivo 9 qualitative software (QSR International, 2010), which enables coding of emergent themes and categories. The emergent themes from the initial debriefing will become the initial "tree nodes" (NVivo's term for categories) for coding, which will be modified as new ideas and concepts emerge throughout the analysis process.
We will use constant comparison analysis to compare coded themes from the second transcript with the first, and so on until all transcripts for a given site are included in the category framework. We will utilize the Notes feature of NVivo to track changes as well as to generate initial theories about relationships among categories and to begin model building. The coding will be completed by the primary qualitative investigator, with the assistance of two assistants, who will review and back-code a random sample of ten percent of the transcripts. This team will discuss any disagreements and will revise the codes and/or re-code as needed until they reach consensus (actual computations of reliability are inappropriate for qualitative studies).
We will organize a summary of these initial thematic findings into a member check document to share with all interview respondents to determine accuracy and dependability of the findings. Data from the three sites will be coded as separate sub-projects in order to produce case study profiles, and then combined to produce a grounded theory model to address our research question (i.e., the interrelationships among influencing factors and contexts affecting implementation of the Community Participation Training and Technical Assistance intervention).
We will present findings to the CET as well as to the quantitative team, to get broader insights and feedback. While no qualitative study can make claims to generalizability, this final step will enable some comparison with the other sites not included in the case studies and point to broader conclusions about primary factors leading to successful implementation.