Next Article in Journal
An Application of the A’WOT Analysis for the Management of Cultural Heritage Assets: The Case of the Historical Farmhouses in the Aglié Castle (Turin)
Next Article in Special Issue
Beyond Policies and Social Washing: How Social Procurement Unfolds in Practice
Previous Article in Journal
A Flood Risk Management Program of Wadi Baysh Dam on the Downstream Area: An Integration of Hydrologic and Hydraulic Models, Jizan Region, KSA
Previous Article in Special Issue
Social Accounting for Sustainability: A Study in the Social Economy
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

SROI Methodology for Public Administration Decisions about Financing with Social Criteria. A Case Study

1
Department of Finance and Accounting, Universidad Loyola Andalucía, 14004 Córdoba, Spain
2
Department of International Studies, Universidad Loyola Andalucía, 14004 Córdoba, Spain
3
Department of Management, Universidad Loyola Andalucía, 14004 Córdoba, Spain
4
Department of Business Administration, Universidad Autónoma de Chile, Santiago 425, Chile
5
Department of Economy, Universidad Loyola Andalucía, 14004 Córdoba, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(3), 1070; https://doi.org/10.3390/su12031070
Received: 20 December 2019 / Revised: 23 January 2020 / Accepted: 1 February 2020 / Published: 3 February 2020

Abstract

:
The measurement of impacts has been considered one of the best methodologies to evaluate the level of achievement of social entities’ objectives as well as of their contribution to resolving social problems. Those methodologies can guide public policies and subsidies granting, as they help to identify the organizations producing a higher social value, and the effects of their projects. Our research focused on the effectiveness and the efficiency of social entities, measured through their capacity to generate impacts on their stakeholders. The research was realized through the analysis of a case study: the special education center for disabled youths, CEE-SA, in Spain. The social return on investment (SROI) methodology has allowed us to monetarize the social value created for stakeholders through the activity carried out by CEE-SA, and it provides information about the whole value creation process that is generated, for which the analysis and follow-up through the indicators offers a contribution to its management system. This case study can serve as a reference in assessing the management processes of similar entities and can also highlight SROI usefulness for public administrations as an assessment tool for subsidies granted on social criteria. The originality of this research relies on the new SROI methodology provided for the assessment of public financing decisions, especially in a field that remains as under-researched as special education schools.

1. Introduction

Public policies and services’ efficiency and effectiveness have been analyzed for years throughout the results of projects and budget fulfilment. However, citizens have become more and more concerned by the usage of public funds in a context of expenses containment and a higher tax pressure. In this context, the public sector needs social value measurement and assessment tools [1] in order to increase citizens’ trust and gain legitimacy. On one hand, trust is promoted through transparency and showing the guarantee of a service’s sustainability. On the other hand, legitimacy increases as public entities satisfy their stakeholders’ expectations, sometimes delegating a service’s provision in social entities financed with public funds.
In the current environment, in which social challenges such as dependency, handicaps, or inequality are demanding a huge quantity of public resources, the analysis of public policies’ efficiency and effectiveness becomes increasingly meaningful. Frequently, the public sector delegates attention to these issues to social organizations working to give solutions to the most vulnerable groups [2]. In order to justify public subsidies to social entities, work is frequently done to relate the cost of the interventions with their results [2]. However, these investigations do not fully reflect the social value generated by these organizations, since the analysis is dominated by an eminently economic approach, despite the fact that many nonprofits prioritize social objectives of an intangible nature over monetary objectives [3,4,5]. In addition, all the stakeholders involved are not always included in the analysis, so the effects studied are limited to the direct beneficiaries [2].
Recently, the debate on the efficiency of nonprofit organizations (NPO) has focused on measuring the impacts on the groups that are affected by their activities [6,7]. The measurement of impacts has been considered one of the best methodologies when evaluating the level of achievement of social entities’ objectives and of demonstrating their contribution to resolving social problems [8,9,10,11]. Efficiency and effectiveness measurement in these entities shows, somehow, the suitable assignment of the public resources, as public administrations finance their activity. That explains the interest shown by the rulers in the usage of tools based on value analysis, as they combine economical assessments of the subsidies’ projects with the social impact generated, helping in this way to legitimize the granted subsidies in the eyes of the public [12].
In the education sector and, in particular, organizations dedicated to training young people with intellectual disabilities, impact studies are rare [13]. From a public service’s sustainability point of view, this approach is interesting as these entities receive a relevant percentage of public finance, and they produce a significant impact on crucial aspects related to vulnerable groups integration and equality of opportunities.
This requires a change in direction from an approach focused on evaluating the programs and services provided to the beneficiaries individually to the analysis of the changes generated by these entities in the social structures and in the community dynamics, where the beneficiaries are the participants in the entity’s activities [14].
The purpose of our study was to use the SROI (social return on investment) methodology to demonstrate, with basis in scientific evidence, the utility of measuring social impact, both for public funding evaluation and for the internal management of social entities. SROI monetizes the impacts that are not considered in the financial reports or in the evaluations of the programs of the social entities [13,15,16]. First, the information obtained throughout the application of this methodology will provide better scientific evidence to public administrations to evaluate the changes and impacts that their activities produce, in order to prioritize the resources they assign [1]. At the same time, citizens will be able to evaluate the social value obtained with the taxes they pay, as it is measured in money. Moreover, justification of the subsidies that are granted should be related to people’s wellbeing changes, which are crucial in their mission fulfilment as public entities. Secondly, the identification and quantification of the impacts provides new scientific evidence for the management of social entities, through a clearer visualization of the changes caused by their managerial decisions.
To achieve these purposes, our contribution relies on a case study that analyses the social value contributed to stakeholders by the activities carried out by a social entity that provides educational services to young people with intellectual disabilities (CEE-SA) in Spain. The selection of this case study is justified because CEE-SA works in a sector of high social relevance, as disability is. Moreover, despite the importance given by the public to this sector, research addressed at understanding the social value generated as SROI has not been applied so far. For this reason, our research can provide new scientific and practice implications. This case analysis can serve as an example of good practices in analyzing the social value created, and as a reference in assessing the management processes of similar entities. Moody et al. [15] also emphasized that despite the interest in SROI, case studies are still needed to analyze its implementation and to highlight its potential benefits.
This research contributes to deepening the knowledge of SROI methodology as an adequate scientific tool to understand and improve the effectiveness and efficiency of public administration funding and social entities management. Specifically, this study provides news that may be relevant in the internal management of special education schools, while also contributing to the generation of knowledge to justify the funding granted by the public sector based on the analysis of the social value of a NPO.
The originality of our case study relies on the application of the SROI methodology to evaluate public financing decisions. The application of SROI offers elements that can help public administration to decide about financing with social criteria, according to the social impacts generated by each kind of activity. In addition, even though scientific literature on SROI is a growing body of knowledge, there is still no application in the field of special education to young people with disabilities, which is a social challenge of increasing interest to the public.
The remainder of this article is organized as follows. Section 2 presents a review of the relevant literature on social value and highlights the importance of this concept for social entities. Section 3 presents our application of SROI methodology. Section 4 delves into the empirical research as well as the study results. Lastly, Section 5 includes a discussion of the study results, conclusions, limitations and further research.

2. The Measurement of Social Value in the Nonprofit Sector

Nonprofit organizations combine their social goals with the search for greater effectiveness and efficiency, which guarantee their ability to continue providing services [17]; therefore, evaluating their performance in developing their activities is gaining greater importance [10,18]. From the resources and capacities theory point of view [19], the performance analysis should allow researchers to demonstrate which entities are applying the funds received in the best possible way, and consequently, are more competitive, as in the current environment, characterized by expenses containment, public finance access has become more restricted.
Given the growing presence of social entities in the economy and their essential role in maintaining a state of well-being, the pressure to demonstrate their contribution to solving social problems has increased on the part of policymakers, resource providers, donors, users, volunteers, and other third parties [10,17,20,21]. In particular, the funders want to know if their funds are being well spent and are contributing to achieving the purposes for which they were contributed [16]. This concern has led to the development of initiatives for measuring social value since the 1990s [10].
The measurement and management of performance in social entities has two main aims: to demonstrate its contribution to value creation and to improve organizational performance [8,10,18].
However, these tasks are not without difficulties. With regard to the first aim, verifying the social value created, the first difficulty is associated with the fact that these entities work towards achieving social objectives that are intangible and, therefore, difficult to measure [22]. On the other hand, from the point of view of stakeholder theory [23], methodologies for measuring social value that respond to the expectations of a broad spectrum of groups and stakeholders are needed [20]. By only attending to their interests, these entities will get the legitimacy needed to operate. From the legitimacy theory [24] point of view, social impact evidence can support the granting of public funds, as the generated changes are clarified, improving transparency, confidence, and credibility.
Nevertheless, the lack of consistency in providing evidence on the social value created is another one of the difficulties added to the evaluation problem [7,21], since the indicators used can be highly subjective because they cover intangible aspects.
In this sense, the so-called participatory action methodologies [25] could contribute consistency to the results, as they are based on the evidence provided by those affected. This means that it is necessary to involve the stakeholders and rely on their collaboration [26], highlighting the importance of the dialogue and the relevance of the story they provide regarding the credibility of the impacts to be measured [4,9].
Regarding the second aim, improving performance, according to Costa and Pesci [17], each of the stakeholders evaluates the effectiveness of the organization in a different way according to their own perceptions, judgements, and priorities regarding what is relevant, which makes measuring performance difficult. Lee and Fisher [27] found that measures of performance remain a challenge for these organizations, particularly when the expected impacts on beneficiaries are influenced by external factors that are beyond the control of the entity and that question the attribution of the results to a particular intervention.
All these difficulties have generated both reluctance and enthusiasm with respect to some of the methodologies for measuring social value that have emerged recently [21] and that replace the traditional cost-effectiveness and cost–benefit analysis tools used so far [28]. Owen et al. [13] focused their criticism on the fact that these evaluations have been carried out in terms of economic viability and not of social impact.
These limitations are easier to overcome when performance measurement systems use methodologies based on measuring impacts and not on intervention outcomes [7,29,30]. These are mixed methodologies that combine the quantification of impacts with a narrative and dialogue with stakeholders [17,26].
Ebrahim and Rangan [31] defined the impacts as the long-term results derived from an organization’s activities, which produce economic, social, or environmental changes. The concept of impact is derived from the theory of change [32] and from the methodologies based on the logic models backed by some international governmental development aid agencies. These models relate inputs, activities, outputs, and outcomes. However, while the inputs, activities, and outputs depend on the efficacy and internal efficiency of the organization, the outcomes are highly influenced by environmental conditions and stakeholders’ perceptions [1,10].
One of the tools for measuring social value, the use of which has grown the most in recent years, is the SROI methodology [5,11].
Several reasons justify the usage of SROI to focus the public administration’s decisions. In the end, this methodology can help the public sector (and/or the social investors) to evaluate and compare the investment options by the analysis of the changes obtained; it also can facilitate an input–outcomes analysis. Moreover, the impact description made by the stakeholders shows their perception about risks and externalities associated with social interventions. That information helps the administrators to better understand how these organizations contribute to the fulfilment of their goals. The monetarization of impacts improves the public value accounting and the public’s understanding of policies becomes more visible [2]. Finally, this methodology can help managers of social organizations to evaluate and maximize social benefits; moreover, it can be a tool to compare the performance of different managers, allowing them to make more justifiable decisions.
According to Cooney [4], expanding on this methodology will help reduce the criticism of the proposed SROI analysis being an exercise in "rhetoric" and ensure that the lack of consistent results is remedied. This study also responds to the need expressed by Jones et al. [33] to advance the methodology and to share experiences that allow knowledge generation in this field.

3. Methodology

The scope of this study is a case analysis of the special education college CEE-SA, which is a center that receives public funding from the Education Department and is promoted and sponsored by a private foundation that also provides financial resources. CEE-SA serves students with special educational needs due to intellectual, physical, and sensory disabilities, aged between 3 and 21 years.
The pedagogical structure of the CEE-SA is organized around the following two types of teaching:
Basic Compulsory Training (Formación Básica Obligatoria, FBO), for boys and girls from 3 to 18 years old, with seven approved and subsidized units that are distributed as follows: four autistic units, one psychiatric unit, one multipurpose unit, and one hearing and language unit.
Program for the Transition to Adult and Working Life (Programa para la Transición a la Vida Adulta y Laboral, PTVAL), aimed at young people from 18 to 21 years of age, which has three approved and subsidized units, which include two autistic units and one psychiatric unit.
The center offers speech therapy, counselling, medical and rehabilitation services, programs for developing personal autonomy (pilot), swimming, and asinotherapy, among others. In addition, it provides school transportation, dining services, and morning classes.
As already mentioned, one of the most innovative methodologies in the field of measuring social value is the SROI methodology. In this case, it was applied to an evaluative analysis [34] about a real situation involving an activity developed during an academic course (2016–2017), which allowed for the identification of the priority stakeholders, analysis of the main changes generated, definition of a set of indicators that allow the monitoring of the changes, and, finally, the monetary quantification of the impact or social value contributed to the stakeholders.
Several reasons make the CEE-SA a significant case in order to analyze how public administrations can use impact assessment to direct their funding decisions and policies. Firstly, it is an institution for which feasibility is highly conditioned by public funding, as it provides a deficit service due to the huge quantity of activities and attention demanded by the beneficiaries, and is impossible to finance with the family’s contributions. On the other hand, it is a not financially sustainable institution through the beneficiary’s contributions, as the action is developed in an especially vulnerable environment, as the school is located in one of the most depressed neighborhoods of the city—that is why its sustainability can be measured only in social terms, using the measure of intangible impacts.
Secondly, the school deals with a problem that constitutes one of the big challenges in our environment: disability attention, which implies a wide group of people demanding a huge quantity of public resources. The significance of this group and the funds granted, give better opportunities to improve the efficient assignment of public resources—that is the reason why the case offers a higher learning effect.
Finally, the theory of change of CEE-SA is based on an intervention model developed in a special education school, opposite to other integrated education models. The case contributes to the debate on educational models for disabled youth, analyzing the impacts generated in this differentiated educational environment.
The information was obtained through interviews and questionnaires to the priority stakeholders (the university’s ethics committee has evaluated the project in accordance to the criteria established by the rules for the protection of people and the respect of human rights in scientific activities): the FBO students (30 students), the PTVAL students (12 students), the families (42 families), the center staff (15 subsidized employees and 10 contracted employees by the center), volunteers (11 people), the Education Department administration, public administrations, and other institutions (university, collaborating companies, city council, etc.).
The impacts were identified through a system of indicators. To obtain the data related to these indicators, 98 surveys were distributed among the different interest groups. At the time that the information about the impacts on students was gathered, the difficulty arose that the students are children under 18 years of age, and all of them have some type of intellectual disability. Therefore, the information related to these beneficiaries was collected from the students’ teachers and families. Two surveys were administered for each student, as follows: one completed by the staff of the center and another by the closest relative.

4. Results

The results obtained in analyzing the social impact of the CEE-SA correspond to the application of the different phases of the SROI methodology [34].

4.1. Stakeholder Identification

The quantification and monetization of the impacts that the CEE-SA generates in society requires the identification of its main stakeholders, as the process must be highly participatory, so that the changes that the school’s activity generates in them can be evaluated. According to MacIndoe and Barnman [6], social entities should identify two types of stakeholders: direct and indirect beneficiaries of the service provided, among which we highlight students and their families, and institutional stakeholders, who can influence the decisions from which the generated impacts depend. The latter stakeholder group can consist of the following three types: resource providers, internal staff, and collaboration networks.
Following the principles of the SROI methodology, the stakeholder identification phase is an ideal time to favor the involvement and participation of stakeholders throughout the process [15,35]. For this reason, different focus groups were organized with the representatives of the foundation, the members of the school’s administration team, and the president of the Association of Parents of Students (Asociación de Padres y Madres de Alumnos, AMPA). In the focus groups, the methodology was clearly explained and a brainstorming session was carried out to identify the priority interest groups.
After the analysis and consensus among the focus group participants, it was decided to focus the study on the following seven priority groups: the FBO students, the PTVAL students, the families, the center staff, volunteers, the Education Department administration, public administrations, and other institutions (university, collaborating companies, city council, etc.).

4.2. Identification of Outcomes by Stakeholders

Then, different meetings were held with different representatives of the interest groups to determine the core changes that the activity of the school generates. While these interviews encouraged the stakeholders to speak spontaneously about how they are involved in the center’s activity and how they perceive the impacts they receive [36], their participation in the process also provided greater consistency and robustness to the results and counteracted subjectivity in some measurements that are necessary for SROI application [5,35].
Following the type of analysis previously carried out by Mook et al. [16], the following three types of impacts were considered: economic, socioeconomic, and social, since it is sometimes difficult to separate some impacts from others. Likewise, the categories suggested in the study by Farr and Cressey [2] on the impacts of disability care programs were considered, which were as follows: individual impacts and impacts on interpersonal relationships, on institutional relationships, and on policies related to well-being.
Different authors suggested that given the restrictions on resources available to make the SROI system operational, it is necessary to limit the number of identified impacts to the most relevant ones [5,15,16]. Out of the set of possible outcomes, after excluding changes that lack materiality, either due to the lack of information, due to the difficulty in quantifying them, or because it is a minor change, a final relationship was established consisting of 31 outcomes (Table 1).

4.3. Quantification of Outcomes by Stakeholders

The process of quantifying outcomes requires identifying the indicators that can determine the occurrence of changes in the most objective possible way. In accordance with Slater and Aiken [12], the development of standardized indicators systems for nonprofit entities is highly important. Nevertheless, linking indicator to impacts may generate a performance measurement system based on stakeholder’s perceptions and in a deliberative process that confers the system a higher consistency.
For quantifying the impact, the indicator results were weighted by the number of people affected by the change, thus determining its outcome incidence [36].
Once the outcome incidence is quantified, it is necessary to assign a monetary value that estimates the value created. This monetary value estimation allows an approach to the economic discourse and the performance analysis. The choice of proxies is one of the most involved phases of the SROI methodology [11]. For this step, it was necessary to resort to secondary sources of information and to the subsequent comparison of the chosen proxies with the CEE-SA management team. The sources used by other studies were consulted [5], and some of them were adapted to the environment in question, for example, the salary costs of the XIV General Collective Agreement of Centers and Services for people with Disabilities have been used as proxies in the gaining of certain skills, abilities, and habits. To avoid overestimating the impacts, the most conservative assessment was always chosen.
Table 1 includes the indicators that measure the occurrence of the change. As seen, the indicators indicating the level of change perceived by stakeholders predominate, together with activity measurement indicators. In addition, the proxies chosen for the monetization of the impacts are summarized.
To ensure that the impacts generated by the organization are not overestimated, once the changes are assigned a value, the SROI methodology requires the following two adjustment coefficients to be applied: deadweight and attribution.
Deadweight considers the amount of impact that would have been generated in any case, even if the entity providing the service did not exist. In practice, this percentage is the result of a counterfactual analysis [3,28].
Attribution acknowledges that a portion of the changes could have been generated by the intervention of other actors, since it is very common to achieve social objectives through the intervention of organizations that work in a network in the same field and that complement each other.
According to Pathak and Dattani [3], in the absence of objective data on these coefficients, it is best to use estimates made internally by agents familiar with the project or intervention environment. According to Ryan and Lyne [37], to give robustness to this type of measurement, it is necessary to involve stakeholders in the SROI process. For these reasons, and in the absence of previous studies and statistics that allow us to determine which portion of the impacts cannot be interpreted as belonging to the school, the adjustment coefficients have been estimated through surveys to stakeholders, in which the stakeholder stated what percentage of the impact they consider should not be attributed to the CEE-SA intervention. Specifically, they were asked to valuate, from 0 (disagree) to 5 (totally agree), the possibility of obtaining the change if CEE-SA did not exist, or if the change could be generated by any other organization.
Table 2 includes the occurrence of changes, attributions, and corresponding deadweights, as well as the final impact created by the school for each type of stakeholder.
Although the methodology indicates the importance of analyzing the duration of the change [34], this study was carried out for an economic year, which coincides with the academic year and therefore with the valuation of the financial inputs of that year. In this case, being cautious in the assessment has been considered more opportune, since it is the first time that this methodology has been applied in these types of centers and because the study is about changes in the disabled and there is no certainty that these changes will last over time [38]. In turn, this limitation is evidence that the study has not overestimated the results.
Regarding the FBO student body, the changes with the greatest impact were those corresponding to the acquisition of basic skills (€12,972.68) and other skills whose valuation exceeded €9000. To a lesser extent was the contribution to the improvement of social skills (€4517.49), whose assessment was affected mainly by the lower availability of activities that offer this training. Taking into account that these changes affected a group of 30 children, we can deduce that the impact generated for each student amounted to €1842.16.
The changes in the acquisition of skills and abilities that were also present in the PTVAL students had similar results to those of the FBO students, although with a lower occurrence. To these changes were added other specific changes to facilitate the integration of PTVAL students into the world around them, and thus strengthen their autonomy. In fact, the acquisition of skills for insertion into the workforce showed the highest impact in this group of young people (€3316.55). In this case, each one of the 12 students of the program received an impact valued at €1423.35.
The total assessment of the changes that the school generated in the families of the students resulted in a value of €67,773.27. This is the group that most benefited from the change, with an impact per family of €1613.65. Although families did not take full advantage of the resources provided by the school, or despite the availability of other alternatives (as seen in the attribution and deadweight parameters), it was one of the most notable impacts among the different interest groups.
The impact of personnel was measured mainly in economic terms, by the contribution of the school to job creation. As seen in Table 2, the total amount of generated changes was €79,161.36. It should be noted that this impact included a change in social nature, which is related to the greater satisfaction felt by the employees of this type of entity due to their identification with the social purposes of the center (€6316.41). Likewise, it is worth highlighting not only the positive changes but also the negative changes, which is why we have identified how the decisions related to temporary contracts result in a negative impact, although of a smaller amount (€637.29), but whose identification provides very valuable information for the decision making of the entity’s management.
Regarding the volunteers, the CEE-SA and a neighboring school maintain a collaboration agreement whereby the students of the latter develop volunteer activities. Specifically, volunteers come to the CEE-SA on Fridays to share activities with the FBO and PTVAL students, initiating certain changes in, for example, developing values, such as equality, or the knowledge of new realities. The economic valuation of this relationship was quantified at €838.1.
The impact on the Department of Education was attributed to the recognition by the Andalucía School Council as an example of good educational practices in the field by the Transition to Adult and the Working Life Program (PTVAL). In this way, the school offers a practical and effective educational response for students to work on curricular aspects and, in turn, develop the necessary skills for later job placement. This good practice, which the Department of Education can transmit and implement in other special education centers of Andalusia, has been valued at €29,700.
Finally, regarding the stakeholder public administrations, job creation also causes a direct economic impact, either as a consequence of the contribution made by these workers to the social security contribution system, to the tax system for the personal income tax, or as a consequence of the saving that occurs in unemployment benefits. The sum of the three impacts considered amounted to €70,445.30.

4.4. SROI Rate Calculation

The next step of the SROI methodology involves calculating the ratio that relates the monetarized impacts with the available resources.
In this case, the financing entity provides resources that allow the sustainability of the school activity, which was valued at €106,261.56 in the academic year 2016–17.
Following the proposal of Millar and Hall [39], different SROI rates were calculated with the following two types of numerators: considering only the social impacts, which are qualitative as they have been previously monetarized through the allocation of a proxy; or also adding the economic impacts that are in themselves of a monetary nature.
First, SROI taking into account both social and economic changes presented a result of €3.01. This means that for each euro invested, €3.01 was returned to the different interest groups. On the other hand, if we only analyzed the strictly social impacts—that is, if we excluded the changes related to job creation, which affect both the staff and public administration stakeholders—the resulting SROI would be €1.68.
The SROI rate is much more than a number, since it is accompanied by the changes induced by the organization that support numbers [3,35]. What is truly important is the process followed until the SROI rate is obtained [4,15], which has allowed evidence about the sources of added value and the impacts generated by the CEE-SA school in its environment to be obtained.
Once the SROI has been calculated, it is necessary to recognize that, to quantify the variables and monetize them, subjective criteria have been used, resulting in some uncertainty about the reliability of the results obtained. The SROI methodology attempts to reduce this uncertainty by proposing different scenarios [5,15,16]. In this case, a sensitivity analysis was carried out for those changes that had a higher occurrence (excluding the job creation and its indirect effects), which consisted of determining the possible effect of an increase or decrease in the proxy by 10%. The results showed that the effect was practically insignificant, changing slightly from €3.01 to €3.102 or €2.926, respectively, which supports and strengthens the results of the valuation of the changes and values of the SROI previously presented.

5. Discussion and Conclusions

This research contributes to understand how the SROI methodology can help to elucidate the social impact generated by NPO activities, thus responding to one of the management challenges of social entities for which economic performance is not the main objective [4,5]. The application of the SROI methodology to the CEE-SA case study has not only allowed the impacts generated by the activity carried out for the different stakeholders to be quantified, but also demonstrated the whole value creation process that is generated. The monetarization of these impacts will allow to CEE-SA prioritize funding decisions according to the importance of the changes achieved, as well as promoting strategies more coherent with the social mission of these entities. Further, the methodology has allowed us to create a scorecard to support the process of decision-makings, for which analysis and follow-up through the impact indicators are contributions to the management system of these organizations.
Our analysis also contributes to provide orientation on the financing of public policies to NPOs, as these policies must guarantee the sustainability of those services facing social problems, such as the one experienced by CEE-SA. Quantifying social value in financial terms makes it possible to improve the understanding of impact by using a common language and by making it possible to visualize its importance [3], that is why this analysis allows us to evaluate the contribution to social value generation in those organizations receiving public funds; this analysis is consistent with the need to take into account social criteria to focus the activity of the public sector, in accordance with international assessments as that of the Directive 2014/24/EU, of February 26th, promoting social clauses in public contracts.
In accordance with Farr and Cressey [2], one of the main advantages of this instrument is that the parameters have been estimated in a participatory manner, which has shown the impact generated in students, families, and society in general; through the participants of the community with which it relates; and other entities with which best practices are shared, such as, in this case, the impact on the Department of Education. Stakeholders’ opinions about the changes they perceive thanks to the school’s actions and the SROI analysis performed led us to make some recommendations influencing the efficiency and effectiveness of the organization.
Firstly, regarding the impacts generated in the families, there are no similar previous studies with which the results can be compared, since the analysis requires that the study participants’ conditions be similar to each other. However, the data obtained showed that the family is a stakeholder that is affected greatly by the activities developed by the school [40] and benefits in accordance with the extent that it identifies and collaborates with the entity. This methodology application and the disclosure of the information obtained should be a way to encourage citizenship participation in public policy development. In this case, the low level of integration and collaboration between families and the school is worth noting, which is evident in evaluating the indicators and the correction coefficients and is perceived in the recognition by stakeholders that these changes are partially attributable to the school’s activity; therefore, the impact was lower than what could be expected. In any case, the results have shown the need for the center’s management team to strengthen relations with families and increase communication mechanisms and transparency with them.
In this sense, it should also be noted that the results of analyzing the changes in the PTVAL students revealed that, although this program has been chosen as an example of good practices by the Department of Education, the families and school staff recognize that the changes achieved and generated by the students are not as high as desired. This fact may result from the expectations placed on these groups not fully corresponding to reality, due primarily to the variability in the profiles of the students who access this program. All this leads us to prudently suggest a strategic review of the Transition to Adult and Working Life Program. One of the weaknesses that the previous analysis can present and that may be influencing the result of this change is that the assessment made regarding the PTVAL students has not taken into account the particular circumstances of each one of them. An improvement to be considered in future studies could be to define personalized objectives for each student according to their degree of disability and their personal situation, so that from that information, an individualized measurement of how they have been able to respond to such changes can be obtained. These results should be reviewed for resource planning and taking actions that allow students to make progress towards the changes.
Regarding the assessment of the impact on students, previous studies both support and contradict that people who are related to the disabled, whether they are employees of the service provider or relatives, can provide valid information to assess their well-being [41,42]. However, in the case of minors, or in the case of people who cannot make decisions for themselves, it is understood that the information obtained represents a multidimensional perspective that allows us to evaluate the change generated in them.
Another of the impacts that has been evidenced and that is key in these types of entities is the identity or sense of belonging of the employees. The degree of occurrence shows that the staff feels satisfied with the activities they perform. The staff do not perceive their work as just another job but rather identify with the mission of the entity and with the consequent impact that this has on the delivery of the activity, which is very necessary in NPOs. This result could be used by the institution to help strengthen the relationships with other stakeholders and facilitate the social objectives achievement.
In quantifying the impacts, it is noteworthy that the most important changes that the school generates in its stakeholders are those related to job creation, even though they were corrected by the employment rate in the service sector, both in the personnel and in public administrations.
In short, in line with the findings of Moody et al. [15], by applying this methodology, the organization learns about how it is responding to the achievement of the expected changes, involving the parties involved. Disclosure of the information obtained in this process can contribute to reviews of public policy suitability as a result of being offered key information to analyze the effectiveness of organizations, and in the end its efficacy.
In short, among the main contributions of this research is that it provides a coherent and clear approach of the activities of a special education school, which allows internal management of the entity based on the analysis of the changes generated by the organization in its main stakeholders, improving its contribution to the well-being of people with disabilities and their families. The mentioned processes could be a reference for this methodology to be adopted by those public or private administrations developing this kind of activity or offering these services.
In addition, this improvement in the understanding of the value sources contributes to inducing changes in the strategy, in the rendering of accounts, in the capacity to manage the risks, identify opportunities, and obtain the necessary financing to achieve its mission by providing evidence regarding the return on the investment, which can justify the use of the received funds [43,44], thus legitimizing the performance of the schools [4,11,22].
With this case study the potential of this methodology is demonstrated in analyzing the impacts produced and, consequently, encourage other institutions to adopt in higher or less depth this tool, as the analysis of the changes over time and their causes will be a management tool that promotes continuous improvement in nonprofit organizations.
However, between the limits of the study, we found that the SROI methodology had limitations, despite its contribution to representing the social value created. First, the social impact is difficult to identify [11,16]. Second, some problems regarding the quantification arise from subjectivity in the scoring and the assumptions required for identifying the different variables needed to apply the methodology [16,33,35]. According to Maier et al. [11], aspects such as the quality and availability of the data, the underlying measurement problems, the causality and the correlation, and the period of time analyzed are limiting factors that should lead SROI users to be cautious when making comparisons between organizations [3,11,16]. Third, if we use SROI to try to justify the financing obtained, sufficient evidence should be provided about the social impact so as not to question the methodology [4,16].
Further research should analyze other organizations with similar projects, using this methodology. That analysis should confirm the kind of impacts produced and would increase the reliability of the proxies found. This way, a response is given to the need for establishing clear and precise measures of the impacts in the social environment [15]. On the other hand, the compared analysis with other entities of the public sector should shed light on the efficiency and effectiveness of this kind of delegation to NPOs.

Author Contributions

Conceptualization, M.R.-L., P.T.-V., A.A.-M. and A.S.; Formal analysis, M.R.-L., A.A.-M. and A.S.; Investigation, M.R.-L., P.T.-V., A.A.-M. and A.S., V.F.-R. and M.C.L.-M.; Methodology, M.R.-L., P.T.-V., A.A.-M., A.S., V.F.-R. and M.C.L.-M.; Supervision, M.R.-L., P.T.-V., A.A.-M., A.S., V.F.-R. and M.C.L.-M.; Writing—Original draft, M.R.-L. and P.T.-V.; Writing—Review & editing, P.T.-V. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Purwohedi, U.; Gurd, B. Using Social Return on Investment (SROI) to measure project impact in local government. Public Money Manag. 2019, 39, 56–63. [Google Scholar] [CrossRef]
  2. Farr, M.; Cressey, P. The social impact of advice during disability welfare reform: From social return on investment to evidencing public value through realism and complexity. Public Manag. Rev. 2019, 21, 238–263. [Google Scholar] [CrossRef][Green Version]
  3. Pathak, P.; Dattani, P. Social return on investment: Three technical challenges. Soc. Enterp. J. 2014, 10, 91–104. [Google Scholar] [CrossRef][Green Version]
  4. Cooney, K. Legitimation dynamics: How SROI could mobilize resources for new constituencies. Eval. Program Plan. 2017, 64, 110–115. [Google Scholar] [CrossRef] [PubMed]
  5. Bellucci, M.; Nitti, C.; Franchi, S.; Testi, E.; Bagnoli, L. Accounting for social return on investment (SROI) The costs and benefits of family-centred care by the Ronald McDonald House Charities. Soc. Enterp. J. 2018. [Google Scholar] [CrossRef]
  6. MacIndoe, H.; Barman, E. How organizational stakeholders shape performance measurement in nonprofits: Exploring a multidimensional measure. Nonprof. Volunt. Sec. Q. 2013, 42, 716–738. [Google Scholar] [CrossRef]
  7. Haski-Leventhal, D.; Mehra, A. Impact measurement in social enterprises: Australia and India. Soc. Enterp. J. 2016, 12, 78–103. [Google Scholar] [CrossRef]
  8. Benjamin, L.M. Nonprofit organizations and outcome measurement: From tracking program activities to focusing on frontline work. Am. J. Eval. 2012, 33, 431–447. [Google Scholar] [CrossRef]
  9. Arvidson, M.; Lyon, F.; McKay, S.; Moro, D. Valuing the social? The nature and controversies of measuring social return on investment (SROI). Volunt. Sec. Rev. 2013, 4, 3–18. [Google Scholar] [CrossRef]
  10. Ebrahim, A.; Rangan, V.K. What impact? A framework for measuring the scale and scope of social performance. Calif. Manag. Rev. 2014, 56, 118–141. [Google Scholar] [CrossRef][Green Version]
  11. Maier, F.; Schober, C.; Simsa, R.; Millner, R. SROI as a method for evaluation research: Understanding merits and limitations. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2015, 26, 1805–1830. [Google Scholar] [CrossRef][Green Version]
  12. Slater, R.; Aiken, M. Can’t you count? Public service delivery and standardized measurement challenges–the case of community composting. Public Manag. Rev. 2015, 17, 1085–1102. [Google Scholar] [CrossRef][Green Version]
  13. Owen, F.; Li, J.; Whittingham, L.; Hope, J.; Bishop, C.; Readhead, A.; Mook, L. Social Return on Investment of an Innovative Employment Option for Persons with Developmental Disabilities: Common Ground Co-operative. Nonprofit Manag. Lead. 2015, 26, 209–228. [Google Scholar] [CrossRef]
  14. Brown, W. Classification of program activities: How nonprofits create social value. Admin. Sci. 2017, 7, 12. [Google Scholar] [CrossRef][Green Version]
  15. Moody, M.; Littlepage, L.; Paydar, N. Measuring social return on investment: Lessons from organizational implementation of SROI in the Netherlands and the United States. Nonprofit Manag. Lead. 2015, 26, 19–37. [Google Scholar] [CrossRef]
  16. Mook, L.; Chan, A.; Kershaw, D. Measuring Social Enterprise Value Creation: The Case of Furniture Bank. Nonprofit Manag. Lead. 2015, 26, 189–207. [Google Scholar] [CrossRef]
  17. Costa, E.; Pesci, C. Social impact measurement: Why do stakeholders matter? Sustain. Account. Manag. Policy J. 2016, 7, 99–124. [Google Scholar] [CrossRef]
  18. Noordin, N.H.; Haron, S.N.; Kassim, S. Developing a comprehensive performance measurement system for waqf institutions. Int. J. Soc. Econ. 2017, 44, 921–936. [Google Scholar] [CrossRef]
  19. Wernerfelt, B. A resource-based view of the firm. Strateg. Manag. J. 1984, 5, 171–180. [Google Scholar] [CrossRef]
  20. Luke, B.; Barraket, J.; Eversole, R. Measurement as legitimacy versus legitimacy of measures: Performance evaluation of social enterprise. Qual. Res. Account. Manag. 2013, 10, 234–258. [Google Scholar] [CrossRef][Green Version]
  21. Arvidson, M.; Lyon, F. Social impact measurement and non-profit organisations: Compliance, resistance, and promotion. VOLUNTAS Int. J. Volunt. Nonprofit Organ. 2014, 25, 869–886. [Google Scholar] [CrossRef]
  22. Klemelä, J. Licence to operate: Social Return on Investment as a multidimensional discursive means of legitimating organisational action. Soc. Enterp. J. 2016, 12, 387–408. [Google Scholar] [CrossRef]
  23. Freeman, R.E.; Reed, D.L. Stockholders and stakeholders: A new perspective on corporate governance. Calif. Manag. Rev. 1983, 25, 88–106. [Google Scholar] [CrossRef]
  24. Deegan, C.; Rankin, M.; Tobin, J. An examination of the corporate social and environmental disclosures of BHP from 1983-1997: A test of legitimacy theory. Account. Audit. Account. J. 2002, 15, 312–343. [Google Scholar] [CrossRef]
  25. Reason, P.; Bradbury, H. (Eds.) Handbook of Action Research. Participative Inquiry and Practice; Sage: New York, NY, USA, 2001. [Google Scholar]
  26. Rossetti, L.; Wall, T. The impact of story: Measuring the impact of story for organisational change. J. Work-Appl. Manag. 2017, 9, 170–184. [Google Scholar] [CrossRef][Green Version]
  27. Lee, J.; Fisher, G. The perceived usefulness and use of performance information in the Australian public sector. Account. Account. Performanc. 2007, 13, 42–73. [Google Scholar]
  28. Banke-Thomas, A.O.; Madaj, B.; Charles, A.; van den Broek, N. Social Return on Investment (SROI) methodology to account for value for money of public health interventions: A systematic review. BMC Public Health 2015, 15, 582. [Google Scholar] [CrossRef][Green Version]
  29. Fine, A.H.; Thayer, C.E.; Coghlan, A. Program evaluation practice in the nonprofit sector. Nonprofit Manag. Lead. 2000, 10, 331–339. [Google Scholar] [CrossRef]
  30. Grieco, C.; Michelini, L.; Lasevoli, G. Measuring value creation in social enterprises: A cluster analysis of social impact assessment models. Nonprof. Volunt. Sec. Q. 2015, 44, 1173–1193. [Google Scholar] [CrossRef]
  31. Ebrahim, A.; Rangan, V.K. The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance; No. 10-099; Harvard Business School: Cambridge, MA, USA, 2010. [Google Scholar]
  32. Keystone. Developing a Theory of Change. A Guide to Developing a Theory of Change as a Framework for Inclusive Dialogue, Learning and Accountability for Social Impact. IPAL GUIDE 2. 2009. Available online: www.keystoneaccountability.org (accessed on 5 October 2019).
  33. Jones, C.; Windle, G.; Edwards, R.T. Dementia and Imagination: A Social Return on Investment Analysis Framework for Art Activities for People Living With Dementia. Gerontologist 2020, 60, 112–123. [Google Scholar] [CrossRef]
  34. Nicholls, J.; Lawlor, E.; Neitzert, E.; Goodspeed, T. A Guide to Social Return on Investment; Office of the Third Sector, The Cabinet Office: London, UK, 2009. [Google Scholar]
  35. Wilson, D.; Bull, M. SROI in practice: The wooden canal boat society. Soc. Enterp. J. 2013, 9, 315–325. [Google Scholar] [CrossRef]
  36. Willis, E.; Semple, A.C.; de Waal, H. Quantifying the benefits of peer support for people with dementia: A Social Return on Investment (SROI) study. Dementia 2018, 17, 266–278. [Google Scholar] [CrossRef] [PubMed]
  37. Ryan, P.W.; Lyne, I. Social enterprise and the measurement of social value: Methodological issues with the calculation and application of the social return on investment. Educ. Knowl. Econ. 2008, 2, 223–237. [Google Scholar] [CrossRef]
  38. McFarlane, F.R. Personnel development in the field of disability with a focus on employment outcomes. Disabil. Soc. 1998, 13, 575–585. [Google Scholar] [CrossRef]
  39. Millar, R.; Hall, K. Social return on investment (SROI) and performance measurement: The opportunities and barriers for social enterprises in health and social care. Public Manag. Rev. 2013, 15, 923–941. [Google Scholar] [CrossRef][Green Version]
  40. Dowling, M.; Dolan, L. Families with children with disabilities-inequalities and the social model. Disabil. Soc. 2001, 16, 21–35. [Google Scholar] [CrossRef]
  41. Schalock, R.L.; Verdugo, M.A.; Braddock, D.L. Handbook on Quality of Life for Human Service Practitioners; American Association on Mental Retardation: Washington, DC, USA, 2002. [Google Scholar]
  42. Cummins, R.A. Proxy responding for subjective well-being: A review. Int. Rev. Res. Men. Ret. 2002, 25, 183–207. [Google Scholar]
  43. Gibbon, J.; Dey, C. Developments in social impact measurement in the third sector: Scaling up or dumbing down? Soc. Environ. Account. J. 2011, 31, 63–72. [Google Scholar] [CrossRef]
  44. Yates, B.T.; Marra, M. Social Return on Investment (SROI): Problems, solutions… and is SROI a good investment? Eval. Program Plan. 2017, 64, 136–144. [Google Scholar] [CrossRef]
Table 1. Parameters for impact quantification.
Table 1. Parameters for impact quantification.
ChangesIndicatorsProxy
StudentsAcquisition of basic skillsLevel of perceived changeHourly cost for a social educator
Acquisition of psychomotor skillsHourly cost for a physiotherapist
Acquisition of learning habitsHourly cost for a psychopedagogy practitioner
Development of cognitive and communication skills
Improvement in the self-regulation of feelings or emotionsHourly cost for a psychologist
Acquisition of social skillsCost for a 10-day special education camp each year
Acquisition of autonomy habitsHourly cost of a special education technical assistant
Incorporation in social life and working lifeHourly cost of an employment counsellor
Acquisition of pre-employment skillsCost of a 10-hour workshop
Contact with the working worldCost of a visit
FamiliesKnowledge of resources and network supportLevel of perceived changeHourly cost of a social worker
Free time (FBO)
(before school care)
Number of morning classroom hoursHourly cost for a free time monitor
Free time (FBO and PTVAL) (extracurricular activities)Number of hours of extracurricular activities
Free time (FBO-PTVAL) (compulsory training)Number of hours in the centerDaily cost for a daytime stay
Improvement of parenting skillsLevel of perceived changeHourly cost for a psychopedagogy practitioner
Family respite (camp activities organized by the AMPA)Number of days that families enjoyed free time because their children participated in the activityDaily cost for camp for disabled children
Guidance, more security and calm to face special educationLevel of perceived changeHourly cost for a psychologist
Decrease in family problemsHourly cost for a family mediator
StaffIdentify or sense of belongingLevel of perceived changeAverage of the annual salary difference to change jobs
Job creationNº of employeesAverage annual cost
Training and knowledge in special educationPercentage of employees who report having improved their knowledgeCost of a professional refresher course
Value loss due to temporary hiring (-)Number of hours of overworking until the person–position adjustment is achieved againHourly cost according to the category of the person affected
Volunteers Change in the conception of special educationLevel of perceived changeAverage course cost
Social commitment with other interest groupsCost of a similar activity
Personal growth, development of valuesNumber of volunteering hoursAdjusted hourly cost of an education technical assistant
Department of EducationImprovement of the performance in the Junta de Andalucía in matters of special educationNumber of good practices identifiedConsultancy contract for procedures in special education (270 hours)
Public AdministrationsIncrease in income from social contributionsNumber of employees who contribute to social securityAnnual average social security contributions
Increase in income from taxes (personal income tax—PIT)Number of employees who pay PITAnnual valuation according to the average PIT rate
Indirect savings in unemployment benefitsNumber of people who do not earn unemployment benefitsAverage annual value of unemployment benefit
Other institutionsChange in the conception of special educationNumber of institutions with which they are relatedNot applicable
Personal growth, development of valuesNumber of actions carried out with these institutions
Table 2. Summary of the impact assessment of CEE-SA.
Table 2. Summary of the impact assessment of CEE-SA.
ChangeOutcome IncidenceAttributionDeadweightSROI ImpactSubtotal
FBO studentsAcquisition of basic skills25.2546.25%50.42%€12,972.68
Acquisition of psychomotor skills24.1347.5%49.17%€9929.32
Acquisition of learning habits24.5039.17%42.08%€9037.29
Development of cognitive and communication skills24.0040%38.75%€9234.12
Improvement in the self-regulation of feelings or emotions24.3838.75%38.75%€9573.79
Acquisition of social skills24.0037.92%38.75%€4517.49
Total student impact (FBO)€55,264.7
PTVAL studentsAcquisition of basic skills8.1852.27%54.55%€2053.05
Acquisition of psychomotor skills7.5051.14%53.41%€2633.25
Acquisition of learning habits7.7746.59%47.73%€2271.91
Development of cognitive and communication skills7.5045.45%47.73%€2238.84
Improvement in the self-regulation of feelings or emotions6.9546.59%47.73%€938.2
Acquisition of autonomy habits8.4545.45%46.59%€2445.56
Incorporation in social life and work5.4540.91%40.91%€570.51
Acquisition of pre-employment skills (including the concept of company)6.4140.91%40.91%€3316.55
Contact with the working world5.5540.91%39.77%€612.31
Total student impact (PTVAL)€17,080.16
Families Knowledge of resources and the support network 24.0941.18%76.47%€603.32
Free time (FBO) (before school care)3206.250%61.9%€18,321.43
Free time (FBO and PTVAL) (extracurricular activities)6840%61.9%€3908.57
Free time (FBO) (compulsory training hours)31,3500%100%€0
Free time (PTVAL) (compulsory training hours)8621.250%43.75%€32,833.95
Improvement of parenting skills and raising children25.9439.71%76.47%€971.58
Family respite (activity camps organized by the AMPA)300%0%€1485
Guidance, more security and calm to face special education31.526.47%55%€5503.24
Decrease in family problems24.0943.75%64.58%€4146.19
Total Family impact€67,773.27
Staff Identity or sense of belonging23.440%38.75%€6316.41
Job creation (subsidized employees)150%88.8%€48,847.46
Job creation (contracted employees)100%88.8%€23,775.41
Training and knowledge in special education20.3150%61.54%€859.38
Value loss due to temporary hiring (-)300%0%€−637.29
Total Staff impact€79,161.36
VolunteersChange in the conception of special education82.64%0%0%€33.88
Social commitment with other interest groups74.59%0%21.21%€26.44
Personal growth, development of values449.090%9.3%€777.77
Total Volunteer impact€838.1
Department of EducationImprovement in the performance of the Junta de Andalucía in matters of Special Education10%0%€29.700
Total Department of Education impact€29,700.00
Public administrationsIncrease in income from social contributions2.80%88.8%€26,144.23
Increase in income from taxes (PIT)2.80%88.8%€12,157.07
Indirect savings in unemployment benefits2.80%88.8%€30,858.24
Total Public Administration impact€70,445.3
TOTAL€320,262.9

Share and Cite

MDPI and ACS Style

Ruiz-Lozano, M.; Tirado-Valencia, P.; Sianes, A.; Ariza-Montes, A.; Fernández-Rodríguez, V.; López-Martín, M.C. SROI Methodology for Public Administration Decisions about Financing with Social Criteria. A Case Study. Sustainability 2020, 12, 1070. https://doi.org/10.3390/su12031070

AMA Style

Ruiz-Lozano M, Tirado-Valencia P, Sianes A, Ariza-Montes A, Fernández-Rodríguez V, López-Martín MC. SROI Methodology for Public Administration Decisions about Financing with Social Criteria. A Case Study. Sustainability. 2020; 12(3):1070. https://doi.org/10.3390/su12031070

Chicago/Turabian Style

Ruiz-Lozano, Mercedes, Pilar Tirado-Valencia, Antonio Sianes, Antonio Ariza-Montes, Vicente Fernández-Rodríguez, and Mª Carmen López-Martín. 2020. "SROI Methodology for Public Administration Decisions about Financing with Social Criteria. A Case Study" Sustainability 12, no. 3: 1070. https://doi.org/10.3390/su12031070

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop