A Program Evaluation Framework for P/CVE ^†

Abstract

P/CVE is a field of research and practice that focuses on complex social programs that, similarly to the field of public health, are most often multidisciplinary in nature. In the proposed program evaluation framework, adapted from the field of public health, we adopt a definition of “program” to include any type of organized activity (i.e., direct service interventions, community mobilization efforts, research initiatives, policy development activities, communication campaigns, etc.). The proposed framework includes three levels of evaluation, six implementation steps, and ten criteria for determining the quality of the evaluation. The three evaluation levels are based on the unit of analysis: system, population, or individual. System-level evaluations are studies conducted at the system level, where data are gathered to understand how an organizational model is developed or implemented (i.e., What type of inter-agency functions need to be developed to enhance the effectiveness of CVE programs?). In population-level evaluations, data are gathered from a segment of the population to understand the impact of the intervention on a group of individuals (i.e., Do school campaigns increase awareness about violent extremism? Do educational interventions directed to law enforcement enhance their ability to respond to cases of violent extremism?). Finally, there are programs executed at the individual level; evaluations of this type may be focused on what worked to reintegrate a specific individual—as such, the unit of analysis is at the individual level (i.e., What type of cognitive therapy interventions work in supporting reintegration efforts? What type of relationships with family and friends are necessary to support the individual in reintegrating into society?). The six implementation steps proposed in the framework are designed to guide the evaluator through the process of evidence gathering: (i) stakeholders’ engagement; (ii) description of the program and the context in which it is implemented; (iii) selection of the study design and framing of how activities are expected to be related to desired outcomes; (iv) collection of data; (v) interpretation of results; and (vi) delivering of results to stakeholders. Finally, the proposed 10 standards are based on questions we should ask to determine the quality of the evaluation. We determined that it is important to understand whether the evaluation is useful, context-specific, ethical, accurate, feasible, independent, culturally sensitive, systemic oriented, and transferable, and to include a counterfactual situation (what-if scenario).