Evaluation Warriorship: Raising Shields to Redress the Influence of Capitalism on Program Evaluation

Abstract

Evaluation warriorship, as defined by ¡Milwaukee Evaluation! Inc., links the practice of evaluation learning, reflection, and storytelling to the evaluator’s social responsibility as a warrior for justice. Unchecked global capitalism has led to extreme economic and racial injustice, undermined democracies, and accelerated environmental catastrophe. This paper argues that more evaluation warriorship is needed to resist this particular system of oppression. It presents examples of how evaluators reproduce neoliberal logic (e.g., in landscape analyses and collective impact assessments), which ultimately undermines transformative change. Evaluator reflexivity questions are proposed to incite change within the field and to help individual evaluators and evaluation teams unpack neoliberalism in their own practice. Evaluation education should include instruction on the effects of neoliberalism and how it shapes both programs and evaluation approaches. Future research should expand the body of knowledge of how neoliberalism has impacted the field of evaluation, support the development of an anti-capitalist praxis, and offer new opportunities for evaluation resistance.

“Right now, and in the very near future, program evaluators and evaluation researchers will be asked to explore many questions related to the current coronavirus pandemic. RFPs are already being prepared to study its effects on specific populations and issue areas. We will be responsible for setting the evaluation agenda for years to come. If we harness the full potential of our influence and power, we can help secure a fundamentally different society, one that is firmly rooted in social justice and celebrates the full scope of our humanity and the sacredness of this planet.”

—¡Milwaukee Evaluation! Inc. (2020)

1. Introduction

The field of program evaluation is undergoing a transformation, one that is inextricably linked to healing and resistance. For over a decade, program evaluators engaging in critical, transformative, decolonizing, feminist, and culturally responsive evaluation (Hall 2020; Sielbeck-Bowen et al. 2002; Smith 2012; Symonette et al. 2020) have gained prominence, elevating multiple, shared, and regenerative threads centered on our collective responsibility toward liberation and social justice for all. It is what Neubauer and Hall (2020) refer to as the “activist, and critical-action orientation to evaluation” (p. 130). Specialized institutions (e.g., the Culturally Responsive Evaluation Assessment/CREA), emerging evaluator pipelines (e.g., Graduate Education Diversity Internship/GEDI), theories, written accords (e.g., evaluator competency statements, textbooks), and convenings help to codify this transformation in the field; most notably among the culture, dominant ideation, and legacy of evaluators of color (Symonette et al. 2020). A main driver of these shifts is the recognition that evaluation has been a complicit to systems of oppression (Hall 2020); meaning, evaluation as it exists in the U.S., and as it is exported around the globe, helps to reproduce extreme poverty, colonialism, environmental oppression, and white supremacy (Hall 2020; Ofir 2018). The ways in which it does this are many and parallel social science research (see Zuberi and Bonilla-Silva 2008 for examples). Direct challenges to the accepted universalisms in evaluation theory and practice are happening all over the world. For example, the “Made in Africa” initiative by the African Evaluation Association is wrestling with the questions of how evaluation would look different if it had been built from African/Indigenous knowledge systems (IKS) (Ramasobana and Ngwabi 2018). Those spearheading this work seek to collectively restore Afrocentrism in evaluation practice throughout the continent, a process that will inevitably entail a direct challenge to and disruption of traditional forms of evaluation as mandated by international development aid programs and other outside organizations working on behalf of the countries in the Global North (Ofir 2018). In New Zealand, Indigenous critical scholars have developed decolonizing methodologies and research agendas that work toward self-determination using the processes of transformation, healing, mobilization, and decolonization. (Smith 2012). Linda Tuhiwai Smith (2012) has been instrumental in repositioning IKS as key to survival. She states:

“I believe that our survival as a people has come from our knowledge of our contexts, our environment… we had to know to survive. We had to work out ways of knowing, we had to predict, to learn and reflect, we had to preserve and protect, we had to defend and attack, we had to be mobile, we had to have social systems which enabled us to do these things. We still have to do these things.” (p. 13)

While the evaluation field’s metamorphosis is not yet complete (and perhaps will always be ongoing), common threads do more than just encompass healing; they center it in the immediate future of evaluation theory and practice.

Program evaluation has typically meant the assessment of policies, programs, interventions, and services, although recent discourse has reified its role to “incite social justice-oriented change” (p. 129) in communities of color (Neubauer and Hall 2020). As the liberation and social justice branches of the field of evaluation have been (re-)surging, more explication is needed on what inciting change means, language that denotes that evaluation has something to offer the resistance movements of people of color. After more than 15 years as program evaluator, I see a critical inciting change agenda that examines neoliberalism within evaluation practice, bringing it into the foreground for critique and eventual eradication. Understanding the influence of neoliberalism is essential for the field to claim that it can incite change and serve as an apparatus of various Black and Brown resistance movements, where anti-capitalist notions already thrive (Allen 1992; Davis 2020). Since the literature on neoliberalism is vast, I first present a high-level summary of the nexus between neoliberalism, healing, and evaluation warriorship. I then present two examples of how neoliberal logic (inside capitalist economic systems) has influenced U.S. evaluation practice, and several field-level resistance opportunities, including a series of reflexive questions for individual evaluators and evaluation teams to enhance the advocacy arm of their evaluation practice.

2. The Nexus between Neoliberalism, Healing, and Evaluation Warriorship

2.1. What Is Meant by “Neoliberalism”?

The definition of neoliberalism is hotly debated, and even its existence is contested (Peck 2013). It refers to everything from a theory with clear epistemological and ontological frames to a practice, an ideology, a doctrine, a framework or stance, a structure, or a set of policies and practices, political and economic in nature; it is a shorthand for a range of phenomena in the modern era (Hardin 2014). Hardin (2014) groups the common treatments of neoliberalism into three camps, all of which I find useful and draw on in my critique. The first camp, the Foucauldians, view neoliberalism as not just a form of government, but as a way of life. Hardin, citing many writers, notes that, in this camp, “the ‘neo-liberal political rationality’ casts all dimensions of life in terms of a market rationality” (p. 208). The second camp, stemming from a Marxist tradition, views neoliberalism as the dominant capitalist ideology of the modern era that preserves and serves class power. The Marxist economic geographer David Harvey falls into this camp according to Hardin. For Harvey (2007), neoliberalism denotes the process by which class power is restored to a small group of oligarchs; it is “a political project to re-establish the conditions for capital accumulation” among economic elites (Fisher 2009, p. 28 interpreting Harvey’s conception of neoliberalism). These processes include the corporatization, commodification, and privatization of public assets, services, infrastructure, and natural resources for private exploitation (Cohen 2018; Harvey 2007). The final camp, the epochalists, deal with the tangibles of neoliberalism; they use a modern lexicon to describe neoliberal systems and its effect on reorganizing society to serve market goals. The tangibles include “globalization, financialization, deregulation, economic inequality, individualization, entrepreneurialism, the extension of economics or market logic into all areas of life” (p. 207). Hardin offers her own definition focused on the “neo” part in neoliberalism, which she calls corporism (not to be confused with corporatism), to emphasize the power and domination of corporations as a result of neoliberal political-economic structures. Here, she argues that “the epistemological project of neoliberalism refigures society as an economic system of corporations. Individuals are refigured as corporations or entrepreneurs and corporations are treated as individuals. Rights are refigured as corporate rights, freedoms as corporate freedoms and even apparatuses of security are aimed at corporations (‘corporate welfare’)” (p. 215).

All of these definitions and applications of neoliberalism resonate and that there are variations in what constitutes neoliberalism is telling (Peck 2013). Peck (2013) and others provide important instruction on additional properties or attributes of neoliberalism that go beyond its processes. For example, it may be “omnipresent” (Peck 2013, p. 140), as some form of neoliberalism exists in most countries, despite different types of economies, some capitalist and some not (Harvey 2007). Even within the U.S., there are multiple, co-occurring variations in neoliberalism, though all seem to reconfigure the role of the state to support neoliberalism. For example, progressive neoliberalism adopts the core economic tenets (e.g., deregulation, tax breaks, free trade, financialization) and still finds congruence between the free market and cultural issues such as civil rights and the rights of lesbian, gay, bisexual, transgender, and non-binary individuals (Watkins and Seidelman 2019). This is the form of neoliberalism that exists within the Democratic Party. Neoliberalism within the Republican Party is influx under Trump, but its fascist leanings and willingness to overtly use state power for violence marks an important and recent shift in American politics (Watkins and Seidelman 2019). Neoliberalism reproduces itself, partly because powerholders at all levels of society subscribe to it, it is exceptional at alienating alternatives, and it is not hampered by its own contradictions, particularly those between theory and practice (e.g., aspirations for competition in theory, but monopolistic in practice) (Harvey 2007; Peck 2013). Its demise will not be its own doing (Fisher 2009; Peck 2013) or result from its failures (Harvey 2007).

2.2. Connecting Neoliberalism to Daily Struggles

The connections between neoliberalism and daily struggles are clear. At its crux, neoliberalism is a philosophy that human well-being and dignity, even for the most vulnerable among us, are best served by it (Harvey 2007). Neoliberalism transforms people and nature into commodities, so that meeting the needs of daily life occurs best through market terms. Challenging this notion, Polyani (as cited in Hardin 2014) contends, “liberties are not rights to redistribution of material resources for self-realization but rights to exist as cultural beings, rights to the protection of nature and rights to the protection of productive organization in society.” (p. 103) Moving toward the tangibles, neoliberalism (and the larger capitalist system) has a lot to answer for; the consequences of which are the subjects of major social justice movements (e.g., economic justice, racial justice, gender justice, environmental justice, education justice, and prison abolition). Pre-COVID-19, the evaluation field had sufficient evidence to problematize the current political-economic system, a system that is highly organized along the lines of neoliberalism, although again, the larger capitalist system is not exonerated. Through a complex web of nested systems, institutions, laws, and cultural traditions, the practices of capitalism produce a genocidal economy (Shiva and Shiva 2018) and a sociopathic economic system (Williamson 2020), causing excess deaths as a result of violence, drug addiction, lack of quality, affordable health insurance and care, and so on (Case and Deaton 2020). In the U.S., “people matter to the point to which they can produce, consume, or be owned” (Harriet’s Apothecary n.d.), with multiple sub-populations along race and gender lines excluded from the labor market or hidden and uncounted inside American prisons. Neoliberalism, through the privatization of American prisons and most criminal justice services, is implicated in the onset and maintenance of mass incarceration (Kotkin 2020; Pelaez 2020). This system requires racialized dehumanization processes that make the “criminal” subhuman and are part of a package of processes that erode civil liberties, human rights, and democratic and representative governments (Allen 1992; Cohen 2018; Kotkin 2020; Rodriguez 2009). Racial capitalism provides a framework for understanding this complex arrangement between race, labor exploitation and exclusion, and (white) wealth accumulation (Kelley 2017; Melamed 2006). As mentioned earlier these trends lean toward fascism, tyranny, and authoritarianism (Fanon 2018; Shiva and Shiva 2018). The deregulation and privatization of land and other public spaces is an explicit agenda for neoliberalism; it results in environmental exploitation and the destruction of natural resources (Harvey 2007; also see Baker 2020). Disaster capitalism demonstrates how neoliberal actors capitalize on natural disasters, including the current COVID-19 pandemic, to implement their agenda (Robinson and Habibi 2020). This is a continuation of colonialism in the U.S., but these pro-colonial projects are not limited to this country. Elsewhere, U.S. neoliberalism, operating through the country’s military forces, support U.S. imperialism (Narayan and Sealey-Huggins 2017). Wars and conflicts help the U.S. gain control over natural resources in other countries (Narayan and Sealey-Huggins 2017; Shiva and Shiva 2018). The current global COVID-19 pandemic exposes this economic system, or what Fanon (2018) refers to as the “colonial reconquest” (p. 566)—the economic, political, and military actions that eliminate self-determination and create economic dependency, from which he argues independence is only gained by “an end to feudalities and the destruction of all the economic structures of colonialization” (Fanon 2018, p. 573) in their modern forms. With COVID-19, these problems are exacerbated in full view, and we have even more evidence that the market cannot meet daily struggles; regenerative and restorative resistance forces are needed.

2.3. Connecting Neoliberalism to Evaluation Warriorship

That evaluation warriorship emphasizing healing and resistance is being added to the evaluators’ toolbox during the COVID-19 pandemic is both timely and needed. No global military block that might replace U.S. capitalism is on the horizon; as a result, the only recourse is to weaken it from within (Žižek et al. 2018). This is a difficult task because, as Giridharadas (2018) notes, we are “in an era where capitalism has no ideological opponent of a similar stature and influence, and in which it is hard to escape the market’s vocabulary, values, and assumptions, even when pondering a topic such as social change” (p. 17). Fisher (2009) concurs, stating that, without a credible and coherent alternative, capitalism will continue. Such systems of oppression are resilient (and aggressive) when losing ground (Fanon 2018). Against the backdrop of the COVID-19 pandemic, ¡Milwaukee Evaluation! Inc.’s (2020) clarion call, partially quoted below alongside the accompanying photo (Figure 1), asks program evaluators to actively discredit failed economic and political ideologies, specifically neoliberalism.

“Last July, local healer Tavita Martinez reminded evaluators that our ancestors have passed their wisdom on to us and that to be a warrior is to be a light for others, to be in service with others, and raising our shields is an inherent right as leaders. Evaluators amplify voices and set knowledge agendas. We cannot be silent or neutral on the adverse consequences of unregulated capitalism. We must confront it and name it in our evaluation work if we are accountable to social justice values. A warrior’s job is to protect.” (emphasis in original).

Evaluation warriorship provides an inciting change opportunity and a new agenda within the field to explore anti-capitalist evaluation praxis. This opportunity builds onto the artery within evaluation that already embraces advocacy (Greene 1997), and the use of findings to critique existing systems and mobilize resistance (Neubauer and Hall 2020). Evaluation Warriorship, as defined by ¡Milwaukee Evaluation! Inc., links the practice of evaluation learning, reflection, and storytelling to the evaluator’s social responsibility as a warrior for justice (Robinson and Habibi 2020). Sharing roots with culturally responsive evaluation (Hall 2020), which leverages knowledge for liberation, evaluation warriorship symbolizes the evaluator’s role in disrupting systems of oppression in order to replace them with systemic social justice and love. The ¡Milwaukee Evaluation! Inc. call asked: “How can we work toward the liberation of ourselves and others if we are capitalist evaluators (un)intentionally serving a profit-over-people agenda? What would it look like to investigate unregulated capitalism in your next evaluation or explore data metrics related to the impact of privatization on the people you serve?” (n.p.). Answering this call requires an act of evaluation warriorship to help delegitimize and destabilize neoliberal logic and move the evaluator’s imagination of what is and what can be away from colonialism and toward liberation. Culturally responsive and Indigenous evaluators, along with decolonizing evaluators, have acknowledged the relationship between ideology, white supremacy, colonialism, epistemology, and knowledge production (Kitossa 2012; Smith 2012). Despite their work, explicit direction on how to directly engage an anti-capitalist praxis in evaluation is still underdeveloped and much needed.

Evaluation warriorship in the context of neoliberalism has two aims. The first is to look inward at how evaluation maintains neoliberalism. The second is to orient evaluation findings to help dismantle it. The first aim would follow lines of inquiry similar to other fields that have started the journey to interrogate the effects of neoliberalism on their sector; see examples in education (Ford et al. 2015), health care (Himmelstein and Woolhandler 2008), social work (Ferguson 2008), and the human services (Abramovitz and Zelnick 2015). These studies have not only shown that the promises of neoliberalism have not been kept (e.g., Light 2001), but also the various ways in which neoliberalism reduces quality and increases inefficiencies and costs while transforming the person into a consumer and impeding democratic accountability. Our inciting change project would mirror Abramovitz and Zelnick’s (2015) analysis. They provide a historical review of the negative impacts of privatization in the human services sector in three stages demarcated by marketization (the delivery of public goods in the market), managerialism (the use of business principles to manage services), and financialization (the use of investment principals to finance and expand services). Over time, the rippling effects of these processes have led to de-skilled staff with precarious employment, fewer comprehensive services to meet complex social problems, and decreased quality of treatment and other services. Their analysis highlights the problematic role of cumbersome monitoring and evaluation activities tied to payment schedules and their adverse impact on service quality. The authors make a series of recommendations, including a call to the sector to “fight back” (p. 291).

In regard to the first aim, some work has already been done to tackle neoliberalism in evaluation; critical insights generated in adjacent fields can also benefit our work ahead. Program evaluation embodies the dominant socio-political ideology of the time, and at the present moment, the field normalizes and legitimizes neoliberal problem analysis and privatized solutions to public problems (Mathison 2009, 2018). Neoliberal logics exist throughout program development, financing, and implementation, which are then mirrored in the evaluation. It is easy to find neoliberal logics embedded within the market-based terms program evaluators use daily, such as “cost-benefit,” “return on investment,” “impact investment funds,” “optimization,” and “scale.” These terms are not benign; they carry with them a specific ideology and function. Even the terms “data-driven,” “evidence-based,” and “performance-based” are laden with values and metrics of colonial origin (e.g., assimilation) (Saltman 2009). Not too long ago, several of the plenary sessions of the American Evaluation Association’s annual meeting engaged the field in a discussion on the triple bottom line (people, planet, profit) as triadic outcomes on which to center sustainable social impacts, particularly for public-private and cross-sector interventions (American Evaluation Association 2014). Marketization in the field has already occurred; evaluations are bought and sold. Evaluation users are also “consumers,” and the field itself is referred to as a “marketplace” made up of complex supply and demand chains (see Nielsen et al.’s 2018 special issue of New Directions for Evaluation, which focuses exclusively on the evaluation marketplace and industry trends). Some evaluation consultants identify as “social entrepreneurs.” Evaluation activities (and the paperwork that comes with monitoring, quality control, reporting, and outcomes measurement) are common complaints in other sectors and seen as forms of neoliberalism (e.g., Abramovitz and Zelnick 2015). Evaluation activities are often used to fine tune aims and objectives, outcomes, mission statements, performance metrics, and assessments, all of which function to help the capitalist state levy “symbols of achievement over actual achievement” (Fisher 2009, p. 42).

3. Examples of Neoliberalism in Evaluation

In this section, I present two examples of how neoliberalism operates in the field of evaluation to showcase the force of such logic throughout evaluation practice. There are many more examples that, as a field, we should begin to name, interrogate, and oppose in evaluation projects. These examples tackle popular or common frameworks in the field, such as landscape analysis and collective impact, to make the inciting change work tangible to regular, everyday tasks within evaluation.

3.1. An Example of Neoliberalism in Landscape Analysis

Specific narratives are sold to the public to support neoliberalism as a rational act. In both its narrative and in practice, an important neoliberal logic rationalizes neoliberalism as a credible response to scarce resources, justifying the need for austerity, particularly as it concerns the state (Viajerx 2020); the state is also considered inefficient in delivering quality goods (Kosar 2006). In Figure 2, I present the conventional wisdom related to the provision of human services, which I call the problem of duplication, and trace how evaluation specifically upholds this logic (see Box 1). At the local level, one may have heard community, nonprofit, business, and government leaders issue complaints along the lines of “All these organizations are doing the same thing.” This complaint is backed by claims that such organizations are siloed and uncoordinated, creating unnecessary gaps in services, which in turn lowers the quality and quantity of the services delivered and hinders our overall ability to solve a particular social problem (see Box 2). In this logic, the duplication of services amounts to incompetence and chaos (e.g., referral networks are seen as convoluted, fractured, and difficult to navigate at the client and organizational levels, and the service network itself is viewed as unsustainable and wasteful). The problem of duplication requires careful planning and strategy to ensure a better use of resources, and thus a better chance of arriving at a permanent solution. Similar arguments are made at the national level; government, nonprofit, and private entities have to be strategic with their scarce resources and coordinate the deployment of those resources through various systems of collaboration and careful planning. Problems, after all, are complex and entail multiple actors and solutions, and without coming together, we have no chance of solving the whole problem in one strategic swoop. These planning and strategy development efforts ultimately prioritize a shortlist of possible solutions, set resource allocations (sometimes for several years), and coordinate the resources of the organizations responsible for implementing them.

The program evaluator plays an important role in legitimizing this logic. The role of the evaluator in this process is not to question the scarcity logic, but rather to uphold it and provide evidence that the powerholding class uses to defend its solutions and the organizations it selects to receive its scarce resources (see Box 3). The evaluator has many tools to gather such evidence and support this process. We conduct and prepare scans, situational maps, process-flow charts, and landscape analyses to identify the most common actors, what they do, and how they all work together (or not) to achieve progress. Directories, inventories, and searchable repositories are common deliverables. Evaluators also convene groups to encourage formal collaboration and strategic alliances, producing theories of change statements, diagrams, logic models, and other products to strengthen the argument that scarce resources are being used wisely, fairly, and objectively. This functions as a way of “getting smarter,” a common practice in neoliberal logic (Giridharadas 2018).

Much of the evaluator’s evidence and approaches are built on a system of science linking individual traits to desired program outcomes. For example, we conduct needs assessments of all kinds to help define the neediest and most vulnerable people in concrete terms (individuals with a specific behavior trait or membership in a particular group), which guides service eligibility criteria, cut scores, caps, diagnostics, quotas, and other programmatic operations (e.g., intake processes) premised on providing the right services to the right people in the right dose at just the right time. This knowledge is codified in a list of data-driven and practice-based blueprints, best practices, lessons learned, essential elements, tips, and guiding principles. In more formal situations, a parallel list of rules and regulations is strictly followed and is the subject of audits and other monitoring and accountability processes. As a result of our planning and strategy work, evaluators accept logics tied to who is disposable and who is not by, for example, affirming eligibility restrictions and building them into regression models to measure who benefits from the intervention and by how much. With these data in hand, evaluation reports contextualize a program intervention or an organization’s contribution by referencing the program’s uniqueness, added value, and ability to leverage the work and resources of other organizations. These reports conclude with recommendations that serve scarcity and austerity logics.

Neoliberal logics make up much so much of our everyday, commonsense, and conventional wisdom (Cohen 2018; Harvey 2007) that solving the problem of duplication is regarded as both a moral and efficient act. Yet, tensions arise when we unpack “duplication” as a moral problem (see Box 4). The public-facing problem is concerned with redundancies in the human services sector. If we set aside neoliberal logic, what is truly wrong with multiple organizations serving the same population or community? What is truly wrong with one person receiving the same service from more than one organization? What is truly wrong if it costs a lot to meet human needs (see Viajerx 2020)? Asking these questions in the context of a specific school or neighborhood helps us to see that we can reject the mythology of scarcity and the so-called “value” of planned austerity across places and settings. Posing these questions probably feels uneasy, as if it defies all logic. These feelings demonstrate the extent to which we have been indoctrinated by the scarcity logic and how the prevailing discourse narrowly confines us to neoliberal logic, erasing (or discrediting) other forms of logic. Evoking evaluation warriorship becomes necessary to be the lone voice in the room inciting change by asking these types of questions. At this point, unpacking the construct begs the question of who is responsible for filling service gaps, the market or the government?

To answer this, we turn to Box 5 to interrogate a sister logic formulated on who is and who is not disposable. In its cleverness, and without ever making these arguments directly, undergirding the market and public-facing problems, the problem of duplication is assumed to create fraud, laziness, and dependency; all of this is personified in “the welfare queen.” This form of rationality argues that scarce resources should go to those most deserving. This paternalistic view of the people who use social services is tied to colonialism and white supremacy. Fanon (2018) provides instruction here, stating that “the colonized people are presented ideologically as people arrested in their evolution, impervious to reason, incapable of directing their own affairs, requiring the permanent presence of an external ruling power” (p. 654). This logic strengthens paternalistic social control over the people who require “help” and the organizations providing such help (also see Rodriguez 2009). Program evaluators support both the limits placed on charity and, in the process, help to depict charity as a permanent solution. Program evaluators participate in the surveillance of human services, treating service providers as inherently incompetent, corrupt, and doomed to fail. Evaluation and evaluative activities such as strategy development are recognizable neoliberal social change processes (Jensen 2019). This cloud over the human services sector is part of a larger political project to reject the welfare state in favor of free market solutions to social problems and to conceal how (racial) capitalism creates a permanent underclass (see Kelley 2017). The entangling of multiple logics supports the larger neoliberal project, which requires constant narratives to shape the worldview that it is the underclass (not the market) who should be blamed for their condition and distrusted.

In neoliberalism, the problem of duplication requires solutions that consolidate and centralize services, resulting in some organizations being defunded and closed. It also requires privatization mechanisms, moving public goods and services to the private sector to increase efficiencies. Many contradictions between the public-facing concern and the market solution emerge that we must illuminate and reject in future landscape analyses and other evaluation endeavors (see Box 6). For example, in a free market, one can have a McDonald’s on every corner until market saturation is reached. All markets are growth markets, unsaturated markets, and expanding markets until the demand for that product or service has been met. Yet, powerholders do not have to prove that market saturation among human services is actually occurring or even harmful, they just have to demonstrate that more than one organization is providing a service or serving similar populations. In an almost circular logic, needs assessments are conducted to assess supply and demand, and rarely do these studies support the consolidation and tightening of services since many populations and service areas are unserved or underserved. More contradictions are illuminated when, simultaneously, we state that too many organizations are doing the same thing and also look for the “best program” to replicate, scale up, and disseminate as a model program and effective intervention (a task evaluators often participate in). Contradictions also arise when balancing the centralization of services and the desire for place-based, culturally, and contextually tailored programs. If duplication reflects a system that is chaotic and wasteful, it needs more regulation, not less. The scarcity argument, of course, has other functions. It limits best practices to a chosen set of programs likely to maintain existing power structures while starving programs that are constituent-led, of color, and radical (Scott et al. 2020). At the same time, the welfare state is rejected, and corporate-state power is legitimized as the best way to fill service gaps. The result facilitates elites’ control over human services, especially after such services are privatized, removed from democratically controlled government entities, and placed in the private or nonprofit sectors. If evaluators rejected scarcity logics in favor of people-centered solutions (see Box 7), organizations providing the same services to the same population would be allocated more resources, not fewer. The dignity of the people served would be respected. Infusing resources would reduce competition over limited resources, which often destroys inter-organizational collaborations and stifles organizational growth and maturity. This would return nonprofit partnerships to what they were before neoliberalism intensified competition and turf wars (Eikenberry and Kluver 2004).

This example demonstrates how the problem of service redundancies provides a pathway for discrediting the welfare state (and the people who need help) and promotes the privatization of public goods and services using evaluation activities. Other neoliberal logics are implicated, including those dictating who is deserving of help and who is not. Rather than the evaluator challenging and opposing this logic, we help legitimize it. When Peck (2013) notes that neoliberalism reboots itself, this means that this logic can play out in any number of ways. Recently, Milwaukee County issued funds for a research project to consolidate 11 municipal health departments in order to resolve “redundancies” and share resources across the southeastern county (Gallagher 2021). In a new twist, the project was pitched as a way to achieve racial equity, equitable health outcomes, and tackle racism. This was referred to as “equitable efficiencies” (n.p.). The study was presented as part of the county’s acknowledgement that “racism is a public health issue.” Public statements about the planned study did not explain the link between the consolidation and centralization of health departments and erasing racism and health inequalities. The absence of this narrative could presume the value is clear, that consolidation is less wasteful, and that everyone benefits from equitable efficiencies. Melamed’s (2006) concept of neoliberal multiculturalism is helpful here. After the civil rights movement, capitalists replaced overtly white supremacist views with ones that embraced multiculturalism. Contemporary neoliberal narratives may now be treating racial equity as a new way to justify old logic.

3.2. Examples of Neoliberalism in Philanthropic Evaluations

Grantmaking institutions, in particular philanthropic institutions or foundations granting to 501(c) organizations, are important direct and indirect evaluation users. They represent a major consumer and purchaser of evaluation (Kinarsky 2018). Most evaluation “gigs” assess philanthropic portfolios, strategic lines of work, and grantmaking tables; most projects also work with one or more grantees to conduct an external assessment that will be shared with the foundation in some form. Governmental grantmaking institutions similarly hire evaluators to track results and support grant reporting requirements. That we are beholden to a single type of institution for our livelihoods constitutes an important conundrum the field must face. First, Kotkin (2020) argues that academics and researchers (the titles and identities of a fair number of evaluators) share the worldview of the economic and political elite and, through social science evaluation research, help to spread that worldview. Mathison (2018), writing on program evaluators specifically, agrees. She states that evaluations reproduce and legitimize the dominant political-economic ideology, and that in the neoliberal era, the economic interests of corporations and foundations are clearly aligned and applied through philanthrocapitalism. As a result, the commodification of evaluation has hampered its independence, and thus its ability to protect human rights and the provision of public goods. Vandana Shiva with Kartikey Shiva (Shiva and Shiva 2018), Robert Allen (1992), and Anand Giridharadas (2018) all dedicate significant attention to the role of foundations in eroding public goods in favor of privatized solutions. The dominant approach for most U.S. foundations is to fund community-based solutions, rather than expand the welfare state. To preserve the economic system, foundations become a technology of control funding narrowly defined evidence-based or data-driven strategies that reflect their interests and construction of the problem, which is unlikely to disrupt the status quo (Beam 2014). Horvath and Powell (2016) have labeled this phenomenon “disruptive philanthropy,” and argue that it creates new funding models for public goods and positions philanthropic investments in competition with the state. These trends are particularly problematic given the billions pledged or committed to racial equity following the murder of George Floyd by police, since the solutions generally financed under philanthrocapitalism or disruptive philanthropy will likely reject the Black social imaginary of the role of the state, which is to provide health care, education, infrastructure, a clean environment, labor protections, and so forth (Lipsitz 2007). These philanthropic investments are also likely to reject radical changes to the criminal justice system (i.e., defund the police) and instead celebrate modest reforms (e.g., bans on chokeholds). For most foundations, achieving Black wealth will be something outside the purview of their racial equity portfolio.

Collective impact offers a clear example of these trends. Collective impact projects are long-term collaborative processes wherein multi-sector stakeholders work on a common agenda to solve a community problem by engaging in mutually reinforcing activities. This work entails shared coordination and measurement (Kania and Kramer 2011). For more than a decade, collective impact has influenced the philanthropic sector, leading to substantial financial investments and an industry of products and websites. Built on the simple premise of working together, power is “shared,” since the people most impacted have a seat at the table and are believed to have a meaningful hand in shaping the project. Yet, the structure of collective impact projects is inherently flawed. Power and resources are never truly shared, nor are they permanently redistributed. The participation of those most impacted is precarious and meant to satisfy the illusion of inclusion and voice. The involvement of those most impacted, including their surrogates (e.g., community leaders, nonprofit professionals, clergy, and advocates), is usually externally controlled by the funding or organizing body, who decides who can and cannot participate. Acceptable in-group and in-meeting behaviors are prescribed and known to all; they conform to notions of white space and white supremacy culture (see Jones and Okun 2001). This effectively neuters the oppositional voices of participants who must remain at the table. Collective impact is presented as a way to create social change without changing the underlying structures that degrade quality of life and challenge our collective dignities (i.e., racial capitalism, neoliberalism, and colonialism).

Below is an excerpt of a collective impact case by Kania and Kramer (2011) that was published in the Stanford Social Innovation Review:

Even companies are beginning to explore collective impact to tackle social problems. Mars, a manufacturer of chocolate brands such as M&M’s, Snickers, and Dove, is working with NGOs, local governments, and even direct competitors to improve the lives of more than 500,000 impoverished cocoa farms in Cote d’Ivoire, where Mars sources a large portion of its cocoa. Research suggests that better farming practices and improved plant stocks could triple the yield per hectare, dramatically increasing farmer incomes and improve the sustainability of Mars’s supply chain. To accomplish this, Mars must enlist the coordinated efforts of multiple organizations: the Cote d’Ivoire government needs to provide more agricultural extension workers, the World Bank needs to finance new roads, and bilateral donors need to support NGOs in improving health care, nutrition, and education in cocoa-growing communities. Moreover, Mars must find ways to work with its direct competitors on pre-competitive issues to reach farmers outside its supply chain (p. 38).

This case example demonstrates the flaws of this approach. Evaluators are often asked to conduct collective impact assessments by measuring efficiencies, as described in the previous section; the development and acceleration of formal collaborations; partnership satisfaction; and cross-sector work plan “alignment” (usually through a document review of each partner’s policies, practices, and procedures). In this example from 2011, the farmers’ perspective is excluded from the story; and the authors all but praise the for-profit motives behind the collective impact project. Their narrative also fails to analyze the inequitable financial benefits of the project if one compares the monetary value of a strengthened supply chain to the farmers’ (lack of) wealth. It also does not question the use of international development aid to support for-profit business motives. Nearly ten years after this paper was published, Whoriskey and Siegel (2019), writing in The Washington Post, reported that cocoa continues to be harvested by child laborers, despite pledges to address this issue more than two decades prior. Aboa and Bavier (2019) cited the findings from a recent Fairtrade International survey, which found that just 12% of farming households met the living income benchmark of USD 2.50 in earnings per person per day. Still, the work is celebrated in the Stanford Social Innovation Review because different factions, particularly large corporations, are involved in the social change process. The impact assessment, in this context, functions more as a marketing and communications product that heightens the brand of the individual, family, or business behind the various institutions. As the example above demonstrates, collective impact projects ultimately create parasitic relationships, narrowly define the boundaries of change, prioritize profit over people, and grossly distract from the U.S. political economic system by focusing resources and evaluation research on “modifiable variables” meant to pacify resistance. The evaluator gathers the data to support these processes and incrementalism. Not limited to collective impact grants, incremental outcomes are often celebrated as progress, perpetuate white savior and paternalistic narratives, and distract from radical outcomes (see Hobbes’ observations of Mark Zuckerberg’s impact on Newark public schools (Hobbes 2020)). In the case above, the government provides workers, but does it also provide labor protections and enforce environmental protections? Do the NGOs see their work as strengthening health care, nutrition, and education as forms of public goods?

There are alternatives to disruptive philanthropy. Decolonizing strategies can use wealth as a source of healing (Villanueva 2018). Social justice philanthropy, the more radical flank within philanthropy, is willing to fund controversial, progressive grassroots social change (Suarez 2012). Contributory philanthropy seeks the democratic and transparent expansion of public goods by the state; its investments support innovations that it anticipates the state will provide in the future (Horvath and Powell 2016). The challenge is that evaluation paradigms did not develop under these frameworks, and the field needs to rethink philanthropic evaluations to determine how assessments would be different if they had. For example, collective impact assessments could be redeemed in a collectivist framework that assessed whether or not the means of production were placed into the hands of publicly owned institutions. In this scenario, the evaluator would assert their independence to engage key learning questions that expose (rather than reproduce) the myths of neoliberal logic, and the evaluator would study liberatory outcomes that match the gravity of colonialism and imperialism (this would replace vague and amorphous outcomes like “racial equity”). The evaluator would make sense of and label philanthropic investments as “contributory” or “disruptive” in their findings, conclusions, and recommendations. The evaluator would also assess whether neoliberal logic drives the foundation’s theory of change and grantmaking processes.

4. Looking Ahead: Evaluation Resistance

4.1. Strengthen Evaluation Education

Neoliberal logic in evaluation has left blind spots in evaluation practice. Student and early career practitioner evaluation courses, how-to guides developed by foundations or governmental agencies, and degree programs emphasize data collection and analysis methods, along with techniques for working with program and community leaders to secure data and encourage its use. If economic structures are discussed, they are referenced in masked terms, such as “economy,” “poverty,” or “low income.” Opportunities for mid-career professionals are also limited, and continuing education and other professional development opportunities also often fail to illuminate neoliberalism in the context of evaluation practice; instead, they reduce class arguments to vague notions of “a marginalized and impoverished people.” These terms erase the reality of a coordinated economic system premised on the exploitation of people and land to allow the accumulation of wealth by a small group of people. To date, evaluators have rarely engaged neoliberalism directly in their assessments and, instead, devote themselves to quantifying disproportionality, cataloging the experiences of the underclass, and documenting how they survive in perpetual precarity. This occurs despite neoliberalism’s significant influence on the field and the popular view from other fields that evaluation itself is a neoliberal process.

To support the eradication of neoliberal logic from evaluation practice, there is an immediate need for evaluation education, whether through universities, professional affiliations, trainers, or capacity builders, to begin offering a substantive critical examination of neoliberalism and how it shapes problems, programs, and evaluative thinking. Since the field lags in its appreciation for critiquing various political-economic structures, it will have to tap allied organizations, such as the Action Center on Race and the Economy, which produces illustrative white papers, including one titled “Bankrolling Hate: How Wall Street Supports Racist Politicians and Enables White Supremacy” (Goodwin and Sloan 2019). Education training for mid- and late-career evaluators is also needed. Moving toward an anti-capitalist praxis is not an easy task; ¡Milwaukee Evaluation! Inc.’s call clarifies that “program evaluators must become attuned to and draw on the energy warrior within to challenge capitalist values and the erosion of universal public goods” (n.p.). If it feels challenging to raise the issue of race or utter the words “white supremacy” in an evaluation meeting, directly challenging neoliberal mythologies and structures will feel even more difficult and awkward without additional training, capacity building, and liminal spaces to grow. This training is needed, however. The boundaries constructed by neoliberal logic prevent program evaluators from seriously advocating for reparations policy, democratic redistributions of wealth, or abolitionist policy solutions such as defunding the police and closing prisons; see The Breathe Act by The Movement for Black Lives (2020) for an example of an anti-capitalist criminal justice policy.

4.2. Engage in Field-Level Critical Reflection

In the absence of formal training, culturally responsive evaluation offers some instruction. According to Symonette (2014), the role of the evaluator is to provide “helpful-help” (p. 110). In her framework, evaluators must understand self-in-context and have self-awareness of how their identities, roles, and actions in the larger social system either facilitate or preclude helpful-help. Symonette states, “investing in SELF-in-Context development work helps us become a dynamic social relations barometer and compass for navigating and negotiating complex, often turbulent, human systems dynamics. Because cultures and contexts are constantly morphing, this is a lifelong systematic-inquiry and reflective-practice pilgrimage” (p. 118). The evaluation field must engage in critical reflexivity on this issue, looking inward to discern what it can “Work-WITH versus Work-ON” (p. 120), as Symonette articulates. The following critical questions can be explored individually or collectively within an evaluation project. They can also be used to interrogate proposals, scopes of work, and other work products (e.g., requests for proposals) to interrupt the way neoliberal logics undergird all stages of an evaluation. The reflection questions are:

• Am I willing to confront capitalism in my professional work?
• Am I a capitalist evaluator, or an evaluator who subscribes to neoliberalism?
• How has capitalist conventional wisdom influenced the assumptions and worldview I bring to an evaluation? How do I normalize the economic elite’s ontology in my professional work?
• What do I believe about public goods?
• What public goods are being advanced or retracted in the program I am evaluating?
• What are the market’s failures and achievements, in terms of the problem program staff are trying to solve?
• Does the program favor market-based solutions over public solutions?
• Whose economic power am I advancing, and whose economic interests does my evaluation ultimately serve?
• Do I understand racial capitalism, the connection between capitalism, white supremacy, and colonization? How does this lens shape the final evaluation design? Have I done the work to understand how an anti-capitalist lens can still be pro-colonial? What are the synergies between anti-racist methodologies and anti-capitalist methodologies?
• What capitalist values and beliefs does my team hold?
• Am I measuring the effects of privatization and deregulation on program outcomes? Do I attend to how problems are constructed using neoliberal values; that is, as private, individual matters outside the purview of public solutions?
• Do my findings lay the groundwork for a profit-over-people agenda (i.e., the privatization of public goods)?
• What if I cannot use the word “capitalism” without scaring everyone away? Can I still maintain my integrity and use another word?

Evaluators seeking to incite change must critique “yourself, your practice, the profession, and your possible role in perpetuating inequities…the extent to which the field and its workers leave … social systems unchallenged” (Neubauer and Hall 2020, p. 132). In the context of weakening neoliberalism, we are not alone. Other fields have also been perplexed on how to do this (e.g., Mehrotra et al. 2016). The reflection questions above set up what will likely be an ongoing process that has remained underdeveloped in the evaluation field for too long. They may also support the development of a formalized anti-capitalist praxis in program evaluation. Unpacking these questions will help to identify the multiple, nuanced ways neoliberal logic is baked into popular evaluation frameworks. In generating such wisdom, this act becomes especially instructive to the field at large, as neoliberalism varies across social systems (Ford et al. 2015). Neoliberal logic, for example, will materialize one way in education and another way in criminal justice, for example. See the examination by Ford et al. (2015) on the role of media in infusing neoliberalism in education policy, and Kramer et al.’s (2013) inquiry into how neoliberal ideology operates in jail settings, particularly among corrections officers. Our task moving forward requires an interrogation of the program’s theory of change, assumptions, structure, and operations, in addition to an interrogation of the evaluator’s tools, many of which have been fashioned to serve neoliberalism and uphold its logic and myths with each evaluation project.

4.3. Seek Greater Independence

The evaluator’s oppositional stance to neoliberalism is complicated by the field’s reliance on grantmakers and the nonprofit sector for work. As Mathison (2018) explains, the evaluation field has wavered in its protection of, and even its views toward, public goods. As evaluators gain a stronger analysis of how deregulation, privatization, corporate influence, and financing mechanisms erode the accessibility, availability, and quality of public goods, they can design evaluation projects that support their continuation, expansion, and enhancement, as well as the notion of public goods as a value and orientation for meeting the daily struggles people face. In addition to the suggestions made in earlier sections, evaluators can label foundation program outcomes as public goods or as outcomes in service of public goods. Even a small evaluation of a single after-school program can measure and describe how the program is impacted by larger economic, political, and colonial forces that go beyond traditional deficit narratives (e.g., “lack of education causes poverty”). This type of evaluation would situate such programs as public goods serving whole students, not young consumers (see Baldridge 2014 for an example).

The nonprofit sector is important for democracy in the U.S., but it is changing as a result of neoliberalism, and it is considered a vehicle for privatization and marketization (Eikenberry and Kluver 2004; Kosar 2006). Through outsourcing, contracts, the creation of quasi-governmental entities, and contracting, nonprofits (along with for-profits) have absorbed goods and services once provided by the state. As allied fields, it will be difficult (but necessary) to oppose the nonprofit sector’s continued growth at the expense of public goods or the marketization of public goods. The central task in evaluation warriorship remains the same: to assess whether or not our current evaluation projects lay the groundwork for neoliberalism (e.g., deregulation, privatization, and centralization) and legitimize it as the best way to serve the wellbeing of the communities we love. It is the same question that Kosar (2006) leaves his audience with, “Which activities are essential to the state and should remain directly accountable to the elected representatives of the people and which may be carried out by the private sector?” (p. 33).

4.4. Incite Change

Turning back to the second aim of the inciting change project mentioned at the outset, evaluation findings must be usable in the work to weaken neoliberal logic. This will change our practice in several ways, starting with the questions we ask and the data we seek. For example, the popular outcomes for neighborhood development programs targeting people of color typically track the number of housing units built, the number of families placed, and the rate of on-time loan repayment, but rarely do such outcome assessments examine the impact on the target population’s wealth gains contrasted with the financial gains of banks and other industries involved in the project. Nor do they dig deep into the financialization of investments in low-income communities. By fixating on a smaller set of outcomes acceptable to the elite class, we fail to examine the larger forces of racial capitalism (see Kelley 2017 and Melamed 2006) that impact program outcomes. Using the same example, the evaluator assessing the impact of a neighborhood redevelopment plan would also assess the supply chain in the community at a technical level (e.g., L2 and L4 suppliers, distributers, assemblers) to document whether the supply chain was democratically controlled, sustainable, and reflected a reorganization of power. The fragility of the supply chain has been noted in previous catastrophic events, and more recently, during the first year of the COVID-19 pandemic. The expansion of public goods is connected to supply chains. For example, the Supplemental Nutrition Assistance Program (SNAP) program was created when the food chain began to break down during the Great Depression (Shapira 2018). Importantly, supply chains are implicated in the policy frameworks for Black liberation. For example, Allen (1992) states that “the Black community does not have control over all the essential goods and services which it requires for survival. Moreover, as long as corporate capitalism exists, the Black community is not likely to acquire such control” (p. 278). In the same evaluation example, the evaluator would engage in corporate research to discern whether parallel systems (via outsourcing, contracting out, public-private partnerships, or expansion of the nonprofit sector) are subverting public goods throughout each level in the supply chain, and how this affects power among residents of color. As public goods become new outcomes for evaluators to explore, so should supply chains.

4.5. Amplify Healing

Finally, given that an anti-capitalist praxis will disrupt major strands of accepted evaluation approaches and challenge the social imaginations of program staff, an evaluator seeking to implement such a new and conflict-generating paradigm must also support healing and regeneration among the project team to help them reimagine the problem, how they intend to solve it, and measure progress without neoliberalism. Such skills will be needed at the same level of sophistication as our ability to analyze data (Robinson and Thakrar 2020). Jennifer Lopez’s (2020) work Healing Is Rhizomatic can be useful in this regard, as it provides a framework for healing-oriented engagement across people, places, and systems. Healing also entails learning about alternative political-economic structures. Shiva and Shiva (2018) describe in Oneness and the 1%, a regenerative and restorative political economy. They propose a constructive dignity economy that is built from a collectivist ideology that respects people and the planet; prioritizes the universal provision of public goods like health care, housing, transportation, and education; and repairs intergenerational trauma.

5. Closing Remarks

Evaluation warriorship solidifies the evaluator’s connection to and lineage as a member of the warrior class. If the job of a warrior is to protect and respond to structural adversity, evaluators must openly incite change and treat capitalism as a tangible thing in evaluation practice, not something that is abstract and immaterial. Based on my experience as an evaluator for over 15 years, I present examples of how neoliberal logic is taken for granted in the field, resulting in consequential blind spots (e.g., the lack of attention paid to and substantive knowledge of supply chains in the field’s discourse, the use of evaluation to help erode public goods). All systems of oppression require accomplices to uphold them; capitalism needs accomplices (Fisher 2009). In this case, evaluators bend science to rationalize neoliberal solutions and support the narrative that capitalism can solve the daily struggles and injustices it has created. Naming, discrediting, and disrupting neoliberal logic in each evaluation project—and with real specificity to the program, problem area, discipline, system, or sector—is an important place to start and nurture evaluation warriorship. Davis (2020), Kelley (2017), the Black Panther Party, and Malcom X (as cited in Allen 1992) argue that, to be anti-racist, we must also be anti-capitalist. What would it mean for evaluators to expand anti-racist methodologies to include an anti-capitalist praxis as well?