Developing and Validating Sustainability Indicators for Measuring Social Impact of University–Community Engagement Programs

: Universities are becoming more closely associated with communities, with many embracing a third mission as part of their recognized mission of sustainable development. Since holistic indicators are essential to measuring the post-intervention social impact of university–community engagement programs toward sustainable development, this study developed and validated a comprehensive set of sustainability indicators that would help universities conduct a meaningful measurement of social impact. Using a structured questionnaire, data were collected from 310 professionals and stakeholders in the Khanab Nak community in the Pak Phanang River Basin, Nakhon Si Tham-marat Province, Thailand. The collected data were analyzed using a five-stage process, including data reliability and validity, descriptive statistics, differences in group opinions, principal component analysis, model testing, and confirmatory factor analysis for fit statistics. As a result, 15 indicators were identified after synthesizing the common indicators from the sustainable development goals. The indicators were divided into three groups using exploratory factor analysis. Confirmatory factor analysis supported these findings with model fit indices, construct validity, and high reliability, as demonstrated below, i.e., social challenges, economic growth, and sustainable living. The practical implementation of the study’s findings could broaden the perspective of universities on promoting sustainable development goals and incorporating them into strategic plans to build capacity for scaling up engagement activities for community development


Introduction
Delegates from 122 nations gathered in Stockholm for the 1972 United Nations Conference on the Human Environment.Universities have become more active in promoting sustainable development (SD) [1].The number of universities participating in such programs has grown dramatically, especially in Europe, where most of the university declarations and charters were established in 1978 [2].Multiple declarations and charters were created to offer guidelines or frameworks for universities to improve sustainability within their systems.The initiatives involve developing a standard curriculum on sustainability [3,4], establishing partnerships [5], implementing sustainable campus activities [6][7][8], and engaging with the community [9,10].Therefore, universities play a crucial role in promoting SD and are vital for achieving the United Nations Sustainable Development Goals (SDGs) by integrating them into their strategies [11].
In recent years, universities have become more closely involved with communities, with many embracing a third mission of community engagement as part of their recognized mission of achieving SDGs [12].Community engagement is reflected as partnerships Sustainability 2024, 16, 5232 2 of 15 between universities and their external communities formed to address societal needs.Through their interactions with the communities in which they operate, universities can provide benefits for all parties and raise the standard of living in society [11,13].According to this definition, community engagement is classified into broad categories in which activities occur, such as policies and practices for building partnerships, community access to university facilities, community access to knowledge in teaching and learning, community engagement in research, community engagement in student engagement, and community engagement in intellectual staff engagement [14].These types of initiatives aim to form collaborations from which communities and universities can benefit and contribute to one another.
Many articles have explored university-community engagement for SD.Several studies have been published that offer comprehensive assessments of the state of knowledge, with the majority of case studies focusing on local outreach activities and their impacts [15].These studies mainly emphasize supporting rural small enterprises and developing local communities.For example, Strydom and Kempen [16] and Wang [17] described how universities enhanced the local economic sustainability of microenterprises and played a crucial role in promoting entrepreneurship in underserved local communities.Hill et al. [18] and Zeng et al. [19] analyzed the processes involved in knowledge and skills exchange from universities to rural small businesses and rural e-commerce industries as a part of academic outreach programs.These studies demonstrate the positive impacts of supporting rural small businesses.Bogedain and Hamm [20] confirmed that universities, through their roles in SD, can provide the impulse for sustainable economic transformation and engage in the co-creation of local enterprises for sustainability.
In terms of local community development, various studies have shown that universitycommunity engagement programs address social challenges.For example, Málovics et al. [21] reported on the effect of social justice-oriented university activities on local community engagement in promoting social justice.Anstadt [22] demonstrated how a local community engagement program reduced social isolation among older people and their caregivers by facilitating relationships between older people who felt valued by sharing their life experiences and international students seeking the security of extended family.Koh et al. [23] examined the effects of local community-based research and outreach programs on addressing the health impact and root causes of homelessness and finding solutions.Hayes et al. [24] investigated the social challenges of implementing programs and highlighted that authentic community engagement is crucial for promoting community health and well-being.Numerous other research studies emphasize outreach initiatives by universities that promote SD through local community engagement and outreach efforts.
Although these studies significantly increase our understanding of community engagement carried out by universities in the interest of SD, more research is needed about the measurement indicators to understand the post-intervention impacts of university initiatives on community engagement toward SD [25].Goyal et al. [1] emphasize the research gap in measuring the impact of universities on SD.Their findings indicate a lack of studies analyzing the impact from a holistic perspective, as well as the fact that the literature on SD at universities continues to focus primarily on case studies covering universities' operations, with little examination of the broader impacts on SD, particularly in terms of identifying and measuring the social impacts that occur with each university program [26].Social impact measurement (SIM) is defined as the process of defining, monitoring, and implementing measures to demonstrate benefits created for the target beneficiaries and communities through evidence of social impacts [27].SIM can effectively capture and address real-world policy changes, people's well-being, and social needs [28].
In addition, the discussion about how universities affect SD raises a couple of significant concerns.First, universities must effectively communicate their contributions to SD to meet the growing demand for accountability by various stakeholders, including supporters, politicians, accreditation agencies, students, and faculty [29,30].Second, Gooch et al. [31] and Koehn and Uitto [32] emphasized the difficulty in making informed strategic decisions and contributing to SD due to a lack of clarity and a diverse understanding of concepts.Thus, greater clarity and a deeper understanding of the social impacts are required to measure program achievement, make well-informed strategic decisions, and improve the contributions to university sustainability [33][34][35].
Researchers believe that the most effective way to address this situation would be to design measurement indicators regarding the social impacts of universities on community engagement programs toward SD.Such indicators could help universities understand and measure the social impact of shaping their community engagement strategies and building the capacity to scale up programs for community development.This study aims to develop and validate the SIM indicators and answer the following research question: What are the key indicators used for measuring the social impact of university-community engagement programs toward SD?This study utilized quantitative data from a questionnaire survey of professionals and stakeholders in a local community case study from the Khanab Nak community in Pak Phanang River Basin, Nakhon Si Thammarat Province, Thailand, a southern coastal community rich in biodiversity.Since 2014, it has been part of a plant genetic conservation project under the initiative of Her Royal Highness Princess Maha Chakri Sirindhorn, encouraging several universities to use this area for community engagement programs.
We organize the remaining sections of this paper as follows: In Section 2, we provide an overview of universities and sustainable development, discuss SIM indicators for sustainable development, and overview existing literature on these indicators.In Section 3, we describe the sample, the framework of data analysis, and the methodological structure.In Section 4, we present the quantitative results that confirm the SIM indicators for community engagement by universities towards sustainable development.In Section 5, we discuss the results.In Section 6, we conclude the study.Finally, in Section 7, we present the limitations and implications for future research.

Universities and Sustainable Development: The Need to Measure the Holistic Impact
Universities have evolved from being viewed as isolated "ivory towers" to being actively engaged with communities.They promote innovation, knowledge creation, and socioeconomic growth through knowledge transfer and experiential learning programs emphasizing SD [36].Modern universities face an increasing demand to produce research and academic services that have social impact and adopt strategies to meet stakeholders' expectations [37,38].To address these needs, universities must integrate sustainability into their strategic plans, enhance their stakeholder management strategies, and focus on community engagement.Collaborating with external stakeholders is crucial for innovation and knowledge transfer, and this requires universities to build capacity for scaling up activities for community development [39].
The growing body of research in academia and practice regarding the influence of community engagement programs by universities on SD is shown in the increasing number of papers published after 2014 [15].A review indicates that case studies concentrating on particular universities and their impacts on social issues [40,41] and local economic growth [20,42] have been the primary means of investigating the impact of universities on SD.These case studies shed light on how universities promote SD from various perspectives.However, there may be a knowledge gap; thus, more research is needed to examine the effects from a more holistic standpoint [43].Although the current study provides insights into the activities of certain universities, the field needs to be expanded to include a broader view of how all universities relate to and promote the Social Development Goals (SDGs).
The literature study made it clear that universities prioritize programs and their financial effects.Conducting a comprehensive examination of program impacts is less challenging than measuring the complicated relationships between research and educational activities and their effects on SD [35].Studies on the economic consequences of universities could use a variety of methodologies to quantify and aggregate their impact on local economics.

Social Impact Measurement Indicators for Sustainable Development
Universities seek ways to meet their needs and handle the pressure from internal and external stakeholders, as there is a growing interest in measuring social impact.This study defines social impact as a deliberate intervention by universities to improve the local community's social, environmental, and economic well-being [44].Our study focuses on post-intervention social impact, which occurs after implementation.We use SIM to differentiate and clarify how the practice of measuring social impact at different stages could be defined.Furthermore, because implementation typically occurs at the program level, investigating SIM is critical, especially when interventions considerably impact how well universities perform and share objectives such as the UN SDGs are achieved [45].
Throughout the literature review, we used the SDGs indicator to gauge the degree of change that university-community engagement programs had on social impact.The SDGs include a valuable set of common indicators that researchers can use to develop customized SIM indicators [46].The SDG indicators offer compatibility and a standardized approach while allowing researchers to measure in a way that is tailored to their specific requirements [25].Aiello et al. [47] noted that the SDGs are a vital reference point for measuring the social impact of research at the international level.Similarly, Alomoto et al. [48] and Lopez [49] highlighted the significance of the SDGs as a global standard for shaping measurement indicators when developing research on and addressing the challenges of promoting sustainability while considering public information from international rankings.

Overview of Existing Literature for Social Impact Measurement Indicators
In the literature on SIM indicators, the UN SDGs provide a good selection of common indicators that practitioners can use as a starting point when developing their own customized SIM indicators [46].This enables practitioners and stakeholders to measure in a way that best suits their unique needs while maintaining comparability and the appeal of a standardized approach.In other words, there is room for standardized and customized indicators when measuring social impact.University-based SDG indicators were most frequently discussed throughout the existing literature on SIM indicators.For example, Findler et al. [35] proposed impact indicators aimed primarily at the SDGs.They indicated that the economy, societal challenges, and natural environment were mainly the SD impact areas, while policymaking, culture, and demographics were seldom considered.Additionally, Yarime and Tanaka [50] categorized the sustainability assessment indicators from the SD impact framework of universities, including economic growth, societal and business practice changes, social cohesion, contribution to climate change, sustainable lifestyles, and community development.Furthermore, Jackson and McManus [51] described the SROI process and provided examples from organizations that used the indicators to measure the social impact of their projects and programs.

Materials and Methods
The aim of this study was to develop and validate sustainability indicators for measuring the social impact of university-community engagement programs.We used a deductive methodology based on a post-positivism philosophical perspective and utilized quantitative data obtained from a questionnaire survey of professionals and community stakeholders in the area.The structured questionnaire was used to collect data from many respondents in the area, making the research quantifiable and objective [52].There were two sections in the questionnaire.The information required for the respondents' demographic profiles is gathered in the first section.The community's perception of the SIM indicators for university-community engagement programs toward SD is collected in the second section of the questionnaire.A 5-point Likert-type scale was used in this section, where each point represents the following: 1 = strongly disagree, 2 = disagree, 3 = neither disagree nor agree, 4 = agree, and 5 = strongly agree.
Khanab Nak in Pak Phanang River Basin, Nakhon Si Thammarat Province, Thailand, is a southern coastal community rich in biodiversity.Since 2014, it has been part of a plant genetic conservation project under the initiative of Her Royal Highness Princess Maha Chakri Sirindhorn, encouraging numerous universities to use this area for community engagement programs [53].In this study, we used this rural community as a case study.Due to the difficulties in reaching professionals and stakeholders with expertise to assess the indicators for measuring the social impact of universities on community engagement programs toward SD, we used the method of snowball sampling.According to the sampling technique's propensity for increasing sample size [54], 310 respondents participated in the survey.The percentage approach to data analysis was used to examine the respondents' background data.Cronbach's alpha was utilized to assess the questionnaire's validity and reliability.Since the value was close to 1.00, an alpha value of 0.901 was calculated, indicating strong reliability [55].
As shown in Figure 1, a total of 15 indicators were found and initially evaluated using the mean item score (MIS) resulting from the literature review.The Kruskal-Wallis H-test was employed to determine whether the community responses differed significantly.Additionally, using IBM SPSS Statistics for Windows, Version 25.0, an exploratory factor analysis (EFA) was performed to assess the components' one-dimensionality and factor analysis potential.The measurement structure of a group of indicators was examined using EFA [56].
lized quantitative data obtained from a questionnaire survey of professionals and community stakeholders in the area.The structured questionnaire was used to collect data from many respondents in the area, making the research quantifiable and objective [52].There were two sections in the questionnaire.The information required for the respondents' demographic profiles is gathered in the first section.The community's perception of the SIM indicators for university-community engagement programs toward SD is collected in the second section of the questionnaire.A 5-point Likert-type scale was used in this section, where each point represents the following: 1 = strongly disagree, 2 = disagree, 3 = neither disagree nor agree, 4 = agree, and 5 = strongly agree.
Khanab Nak in Pak Phanang River Basin, Nakhon Si Thammarat Province, Thailand, is a southern coastal community rich in biodiversity.Since 2014, it has been part of a plant genetic conservation project under the initiative of Her Royal Highness Princess Maha Chakri Sirindhorn, encouraging numerous universities to use this area for community engagement programs [53].In this study, we used this rural community as a case study.Due to the difficulties in reaching professionals and stakeholders with expertise to assess the indicators for measuring the social impact of universities on community engagement programs toward SD, we used the method of snowball sampling.According to the sampling technique's propensity for increasing sample size [54], 310 respondents participated in the survey.The percentage approach to data analysis was used to examine the respondents' background data.Cronbach's alpha was utilized to assess the questionnaire's validity and reliability.Since the value was close to 1.00, an alpha value of 0.901 was calculated, indicating strong reliability [55].
As shown in Figure 1, a total of 15 indicators were found and initially evaluated using the mean item score (MIS) resulting from the literature review.The Kruskal-Wallis H-test was employed to determine whether the community responses differed significantly.Additionally, using IBM SPSS Statistics for Windows, Version 25.0, an exploratory factor analysis (EFA) was performed to assess the components' one-dimensionality and factor analysis potential.The measurement structure of a group of indicators was examined using EFA [56].Following the EFA, a confirmatory factor analysis (CFA) was conducted to evaluate the social impact measurement indicators for community interaction with SD among HEIs.LISREL version 11 was used to conduct the CFA.Chi-square test, comparative fit Following the EFA, a confirmatory factor analysis (CFA) was conducted to evaluate the social impact measurement indicators for community interaction with SD among HEIs.LISREL version 11 was used to conduct the CFA.Chi-square test, comparative fit index (CFI), goodness-of-fit index (GFI), adjust goodness-of-fit index (AGFI), root mean square error of approximation (RMSEA), root mean square residual (RMR), normed fit index (NFI), parsimonious normed fit index (PNFI), incremental fit index (IFI), and parsimonious goodness-of-fit index (PGFI) were employed in the study for model assessment [56].These indexes provide a robust evaluation of how well the identified factors fit into a sample of data.

Respondents' Background Data
Based on the demographic data given by the respondents, it was found that 70.7% were members of the community, 20.7% were leaders in the community, and the govern-ment officials employed 8.7% in a particular area.The respondents' demographic profiles indicate that the average local is married, male, and graduated from primary and secondary education.Most respondents (63.3%) are self-employed in the Khanap Nak community.Employees and those who work in agriculture make up the self-employed respondents.According to this scenario, most respondents engage directly with HEIs in this field.Their educational background indicates that they have limited career alternatives.

Descriptive Statistics and Kruskal-Wallis H-Test
The results of the Kruskal-Wallis H-test and mean ranking of the SIM indicators for community engagement toward SD are displayed in Table 1.The results show that the following SIM indicators are highly rated: development of production technology and innovation (MIS = 3.53), improvement of local products and services (MIS = 3.72), significance of production costs (MIS = 3.67), safety of life and property (MIS = 3.56), improvement of family relationships (MIS = 3.47), and preservation of the community's cultural heritage (MIS = 4.42).In contrast, the significance of household debt (MIS = 3.03), improvement of community well-being (MIS = 3.11), significance of expert and professional networks (MIS = 3.15), and improvement of health status (MIS = 3.16) are the minor indicators.The average response rate for all respondents was 3.43, while the individual response rates for local government officials, community leaders, and community members were 3.50, 3.88, and 3.29.The results of the Kruskal-Wallis H-test reveal that every variable has a p-value < 0.05, implying a significant difference in the responses to these indicators according to the group of people in the local area.Additionally, the internal consistency of the measurement indicators and the reliability of the study instrument were demonstrated by a Cronbach's alpha value of 0.901.

Exploratory Factor Analysis
The identified SIM indicators for community engagement toward SD at universities were evaluated by exploratory factor analysis (EFA).Table 2 shows a KMO value of 0.912, which is 0.6 above the study's criterion [57].Furthermore, a significant result of 1944.271 and a p-value of 0.000 were obtained from Bartlett's test of sphericity.These findings support the data's factorability and suitability for EFA.In addition, the output's correlation matrix was inspected to determine whether the data were suitable for analysis.According to the findings, most indicators had a value ≥ 0.3, indicating that the dataset was suitable.The commonalities of SIM indicators are shown in Table 3.An extraction value above 0.50 indicates the proportion of an indicator's variance that the retained factors can explain [58].Three aspects or components with an eigenvalue > 1 were extracted from the measurement indicators, and Figure 2 and Table 4 show the use of PCA in the varimax rotation.The cumulative variance was calculated as 60.016%, exceeding the study's 50% cutoff point [58].
SIM 12 Preservation of cultural heritage 1.000 0.758 SIM 13 Allocation of community resources 1.000 0.622 SIM 14 Significance of local resources 1.000 0.691 SIM 15 Improvement of community well-being 1.000 0.662 Three aspects or components with an eigenvalue > 1 were extracted from the measurement indicators, and Figure 2 and Table 4 show the use of PCA in the varimax rotation.The cumulative variance was calculated as 60.016%, exceeding the study's 50% cutoff point [58].The results of the SIM indicators for university-community engagement toward SD are shown in Table 5.The results show that the indicator's factor loading is higher than the study's threshold of 0.40.These results demonstrate that all indicators within a specific component have a good relationship when paired with the values of the extracted communalities.Three components were extracted based on the result of the rotated component matrix.The first component, which refers to social challenges (SOC), has factor loading ranging from 0.785 to 0.589.The second component is economic growth (ECG), with factor loading ranging from 0.787 to 0.611.The third factor, sustainable living (STL), falls between 0.860 and 0.540.

Confirmatory Factor Analysis
To verify the validity of group factors from the EFA, confirmatory factor analysis (CFA) was conducted.Figure 3 shows the results of calculating 15 path coefficients using structural equation modeling (SEM) to determine the causal relationships.The fit indices used for the analysis and the output estimates obtained for the data analysis are described in the LISREL output for the path analysis shown in Table 6.The values for CFI and GFI were 0.990 and 0.963, respectively.According to Iacobucci [59], CFI and GFI must have a value more than or equal to 0.90 and be adjustable to achieve an acceptable fit and meet the criteria.The values derived from the analysis satisfy the reasonable fit requirement.
RMSEA and RMR had values of 0.031 and 0.037, respectively.According to Bentler [60], a good fit can be defined by RMR and RMSEA values less than or equal to 0.05, and values less than 0.05 are considered acceptable.The RMR and RMSEA values met the good fit requirement based on the previously stated values.In addition, the model's sample data showed a p-value and a Chi-square with two degrees of freedom of 89.716.

Discussion
The increasing number of studies published after 2014 on sustainability indicates the growing interest in academia and practice in university-community engagement [15].The review indicates that the impact of universities on SD is primarily determined through case studies focusing on specific universities and their impact on social challenges [22,23] and economic growth [16][17][18].This study develops and validates SIM indicators and takes a more holistic approach and perspective.The SIM indicators were divided into three groups using exploratory factor analysis (EFA).Confirmatory factor analysis (CFA) supported the following findings with model fit indices, construct validity, and high reliability: social challenges, economic growth, and sustainable living.
For this study, the researchers employed SIM indicators, a standardized approach for synthesizing common indicators from the SDGs.There are universal indications and a method of measuring impact that are applicable and comparable in every situation, which reduces the cost and complexity [61].Even with these advantages, Lassarini [62] stated that standardized instruments fail to address the question of causation or whether the actions carried out by these corporations led to changes in the population they were targeting.Therefore, one option is to build customized indicators based on stakeholder demands or quantify the social effects utilizing a standardized approach that allows indicator comparability [63].Researchers can develop unique SIM indicators by drawing from a selection of common indicators provided by the SDGs [46].In addition to giving researchers and stakeholders the benefit of a standardized approach, the customized approach enables measurement of their specific demands to determine what best meets their needs.
The common themes and explanations for the variations in SIM indicators in the literature were divided into two groups [25].First, there were internal differences, such as the goal of continuing university performance.Achieving the SDGs, enhancing the knowledge of resource allocation, creating internal learning resources, spotting upscaling opportunities, enhancing the cost-effectiveness of inputs and activities, supporting universities' strategies, and boosting employee morale are all examples of the ways SIM promotes internal diversity.External influences, such as demands by stakeholders, constitute the second group.The following are some ways that SIM counteracts external pressures: it facilitates decision-making with pertinent data; challenges conventional political and economic thought processes; illustrates how actions affect society; shows how strategic efforts pay off; and raises the standards of accountability and transparency.

Conclusions
The study developed and validated sustainability indicators for measuring the social impact of university-community engagement programs.Using a structured questionnaire, data were collected from 310 professionals and stakeholders in the Khanab Nak community in the Pak Phanang River Basin, Nakhon Si Thammarat Province, Thailand.The collected data were analyzed using a five-stage process, including data reliability and validity, descriptive statistics, differences in group opinions, principal component analysis, model testing, and confirmatory factor analysis for fit statistics.Fifteen indicators were identified after synthesizing the common indicators from the UN SDGs.The SIM indicators were divided into three groups using exploratory factor analysis (EFA).Confirmatory factor analysis (CFA) supported these findings with model fit indices, construct validity (p-value = 0.56, RMSEA = 0.031, GFI = 0.963, CFI = 0.990, TLI = 0.984, NFI = 0.955, Chi-Square = 89.716),and high reliability (Cronbach's alpha = 0.901), as demonstrated below: social challenges, including household debt, community well-being, expert and professional networks, health status, educational attainment, and employment stability; economic growth, including local resources, products and services, production technology and innovation, income and assets, community resources, and production costs; and sustainable living, including cultural heritage preservation, safety of life and property, and improved family relationships.These findings provide a framework for SIM indicators and demonstrate more holistically the impact of university-community engagement programs toward SD.The practical implementation of these findings can broaden the perspective of universities in promoting the SDGs and incorporating them into their strategic plan to build capacity for scaling up engagement activities for community development.In addition, these measurement indicators can be used as a starting point for practitioners looking to implement a model for SIM.Social return on investment (SROI) has benefits, such as promoting engagement with stakeholders, includes qualitative and quantitative analysis, and is a valuable organizational learning tool.It also has some drawbacks.Despite the drawbacks, SROI remains a viable measurement model for those seeking a solution that offers uniformity for the SIM indicators, with the flexibility to customize indicators based on specific needs.

Limitation and Implications for Future Research
The findings of this study are helpful for practitioners seeking to measure the postintervention social impact of university-community engagement programs toward SD.This study employs the SDGs indicator as the standard or starting point for developing customized indicators.While comparability and data aggregation are essential, practitioners should have the flexibility to customize sustainability indicators that represent unique contexts when measuring social impact and engage stakeholders throughout every stage of the SIM process, particularly when identifying SIM indicators.
This study has limitations that must be acknowledged, introducing opportunities for future research.The indicators for measuring the social impact of university-community engagement programs toward SD were developed and validated only within a local community case study from the Khanab Nak community in Pak Phanang River Basin, Nakhon Si Thammarat Province, Thailand; therefore, care must be taken by not generalizing its findings.Future studies can be conducted in other local communities for a more robust and generalizable social impact.

Figure 1 .
Figure 1.Framework for data analysis.

Figure 1 .
Figure 1.Framework for data analysis.
The respondents, with members of the community, assessed the preservation of cultural heritage (MIS = 4.44), improvement of local products and services (MIS = 3.52), and significance of production costs (MIS = 3.52) as the essential variables for individual groups of people.Community leaders who responded gave the preservation of cultural legacy the highest rating (MIS = 4.53), followed by the improvement of local products and services (MIS = 4.32) and the significance of production costs (MIS = 4.16).Most individuals employed by the local government prioritized the preservation of cultural heritage (MIS = 4.01), improvement of local products and services (MIS = 3.96), and improvement of family relationships (MIS = 3.77).

Table 4 .
Total variance explained for SIM indicators for university-community engagement programs toward SD.

Table 4 .
Total variance explained for SIM indicators for university-community engagement programs toward SD.