You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Sustainability
Download PDF
  • Article
  • Open Access

4 October 2022

Using Modified Delphi Study to Develop Instrument for ESG Implementation: A Case Study at an Indonesian Higher Education Institution

,
,
and
1
Department of Industrial and Systems Engineering, Chung Yuan Christian University, Taoyuan City 320, Taiwan
2
Department of Industrial Engineering, Atma Jaya Catholic University of Indonesia, Jakarta 12930, Indonesia
*
Author to whom correspondence should be addressed.
Sustainability2022, 14(19), 12623;https://doi.org/10.3390/su141912623
This article belongs to the Special Issue Approach and Policy in Higher Education for Sustainability
Version Notes
Order Reprints

Abstract

Most research states that implementing environmental, social, and governance (ESG) has positive impacts. However, fewer studies have discussed ESG implementation in higher education. This study aimed to develop instruments to assess the ESG atmosphere in higher education institutions. A modified Delphi approach was employed. Experts were invited from a private higher education institution in Indonesia. A deductive study, discussion, and two stages of getting consensus from panelists were conducted. The instrument was distinguished into four types for four groups of higher education stakeholders: Students, Staff, Faculty Members, and Community Members. The I-CVIs ranged from 0.80–1.00, while the minimum values of S-CVI/Ave and S-CVI/UA were 0.98 and 0.91, respectively, meaning the content validity was excellent. The final version instrument has been tested and declared valid, reliable, and ready to be used for empirical research for universities to assess their contribution to the Sustainability Development Goals (SDGs). There are also opportunities to conduct further research on the existence of recursive and non-recursive models between factors.

1. Introduction

In 2015, all United Nations (UN) members adopted the 2030 Agenda for Sustainable Development Goals (SDGs). This agenda provides 17 Sustainable Development Goals (SDGs): (1) No poverty; (2) Zero hunger; (3) Good health and well-being; (4) Quality education; (5) Gender equality; (6) Clean water and sanitation; (7) Affordable and clean energy; (8) Decent work and economic growth; (9) Industry, innovation and infrastructure; (10) Reduce inequalities; (11) Sustainable cities and communities; (12) Responsible consumption and production; (13) Climate action; (14) Life below water; (15) Life on land; (16) Peace, justice and strong institutions; and (17) Partnerships for the goals []. Organizations or companies around the world are expected to participate in the achievement of some or all of these goals based on their respective abilities. Previously, companies, especially financial firms, have successfully implemented environmental, social, and governance (ESG) principles to support the company’s achievement and achieve SDGs simultaneously. ESG was first mentioned by the UN in 2006 when it launched the Principles for Responsible Investment (PRI) []. PRI suggested that investors should pay attention to other concerns rather than just focusing on profit.
Most research states that implementing ESG in financial firms has positive impacts: overall and individual ESG performance were significant positive indicators of creditworthiness as measured by credit rating [], ESG risks have the potential to reduce financial returns, and most superannuation fund members are interested in ESG investing [], conventional funds consider the firms in which Socially Responsible Investment (SRI) funds invest to integrate ESG criteria [].
Higher education has been identified as the crucial driver in achieving the SDGs, believed to influence people’s thoughts and actions, and plays an essential role in creating public awareness regarding the consequences and impact of unsustainability action on society [,]. Many higher education institutions have achieved the SDGs using their respective strategies. Universities conduct many activities related to sustainability, including environmental issues [,,,,,,], social issues [,,,,,], and governance issues [,]. Universities can imitate the successful application of ESG principles from financial companies to contribute towards achieving the SDGs. However, fewer studies have discussed ESG implementation in higher education. One reference states that the higher education sector is early in its ESG reporting journey []. Another reference argues that universities must carefully plan strategies to promote a broader ESG strategy, making higher education institutions attractive to stakeholders []. The implementation of ESG in higher education institutions will simultaneously improve their quality. This is supported by the fact that the global higher education institution ranking conducted by Quacquarelli Symonds (QS Rank) has included the ESG indicators in the requirements [].
Higher education in Indonesia has also begun to participate in sustainability. The University of Indonesia GreenMetric (UIGM) World University Ranking was launched in 2010. This ranking captures the latest conditions and policies related to Green Campus and Sustainability in universities worldwide []. The report stated that 101 higher education institutions in Indonesia participated in the UIGM, even though Indonesia has 4546 higher education institutions. This condition shows that there is still a wide gap between the number of higher education institutions concerned with sustainability and the total number of higher education institutions. Higher education in Indonesia still needs encouragement to participate in achieving the SDGs. One form of encouragement is providing references to them. A previous study has outlined the steps to becoming a sustainable university and determined the parties involved []. This study intended to expand knowledge of education by outlining the factors and indicators that support the implementation of ESG to achieve the SDGs. The available indicators were derived into items and formed into ready-to-use instruments. Thus, this study contributes to further research aiming to conduct empirical studies that prove the relationship between factors using this ready-to-use instrument.

2. Theory and Context of ESG

For decades, people have believed that modern businesses are responsible to their stakeholders, communities, and society. Stakeholders are increasingly looking for non-financial elements to help them make the best business decisions []. As a result, the organization’s information must be used to reveal valuable and trustworthy information and monitor corporate officials. ESG has become a competitive component for modern organizations; therefore, ESG’s responsibility to all stakeholders appears to be crucial []. This ESG implementation will be carried out by involving or impacting stakeholders both internally and externally.
Higher education institutions worldwide are being challenged to become more focused, efficient, and successful to meet the needs of a globalized and technologically advanced higher education market []. Universities adopt a business enterprise mindset to ensure survival in the face of resource constraints and rapid change in their working environment. Implementing ESG in universities is an opportunity to develop and improve the organization’s quality and simultaneously contribute to achieving the SDGs.
The global disruption caused by COVID-19 has had several environmental and climate consequences. Due to movement restrictions, air quality in many cities has improved, and water pollution has decreased. Furthermore, the increased use of personal protective equipment (PPE, e.g., face masks, hand gloves, etc.), their haphazard disposal, and the generation of a large amount of hospital waste have negative environmental consequences []. The global COVID-19 pandemic has also forced the temporary physical closure of schools and higher education institutions. Approximately 220 million students worldwide have been affected in higher education, presenting policymakers and educational institutions with unprecedented challenges []. Higher education must ensure that underrepresented, vulnerable, and disadvantaged learners are not left behind. The pandemic also emphasizes the value of university–community engagement. The pandemic has also directly impacted how universities operate and university governance, with management staff required to make various emergency decisions while allowing for additional flexibility. Those facts show that in the post-COVID-19 period, higher education needs to pay attention to environmental, social, and governance concerns.

3. Materials and Methods

The Delphi method has been proven as one of many techniques for organizing a group communication process so that a group of people can efficiently deal with a complex problem []. However, Gooman (1987) in Mao [] said that a traditional Delphi method begins with an open-ended questionnaire, is time-consuming, and usually leads to a low response rate. Therefore, a study that successfully developed a tool for measuring healthcare has designed a modified Delphi method that used a structured questionnaire to ask for experts’ opinions []. The structured questionnaire can be developed based on extensive literature reviews or a focus group interview study. This study adopted the modified Delphi method to compile the instrument. Furthermore, piloting was conducted to test the validity and reliability of the instrument to ensure that the instrument was ready to use. The research framework is shown in Figure 1, containing four steps in the modified Delphi method and one piloting step.
Figure 1. Research framework.

3.1. Stage 1: Determine Factors and Indicators

Three academics were involved in the discussion to determine what factors were related to the implementation of ESG. This discussion found that the three factors, namely Leadership, Strategy, and Performance, were closely related to the implementation of ESG. The discussion was combined with a deductive approach to corroborate the results. The following literature review has affirmed those findings.
The Sustainability Accounting Standards Board (SASB), a credible independent nonprofit organization, identifies Leadership as the factor affecting the organization’s performance on ESG issues []. Leadership involves managing issues inherent to the business model and common practice in the organization, an essential feature in the initiation step of effective governance and important to becoming a successful world-class university [,]. Any organization needs strategies to achieve performance. Likewise, in the field of education, strategies are required by higher education to deal with complicated problems and the competitive knowledge marketplace that has appeared in the last decade [,]. A new perspective requires that the performance of nonprofit organizations is also determined by how well and efficiently they meet the needs of their stakeholders []. Universities have unique, measurable performance related to stakeholders, for example, satisfaction in education services received (teaching–learning service, research service, community service, administrative service, and curricular service) [].
The proposed factors were latent variables that required measurable indicators. Therefore, the discussion continued with determining indicators. The ideas were collected from previous research. Similar arguments were grouped and given a name and identification code. Twenty-eight indicators have been found as selected indicators (Table 1). Furthermore, confirmation and consensus from the panelists were required before moving on to the following process.
Table 1. List of proposed indicators.

3.2. Stage 2: Verify Factors and Indicators

In the Delphi method, the consensus of expert panelists must be obtained []. The consensus was obtained by analyzing the responses of the panelists. Nine experts have been invited to participate as panelists. Crawford & Wright [] recommend obtaining the opinion of 5–20 experts with appropriate domain knowledge and heterogeneous background. A total of eight experts stated their willingness to be involved in this study. They were leaders from a private university in Indonesia with different backgrounds and position levels. Table 2 shows the demographics of the experts.
Table 2. Expert demographics.
Panelists were asked whether the proposed indicators were appropriate to support the relevant factors. Experts responded by giving a score of 1 to 5. Table 3 explains the definition of the suitable scale.
Table 3. Suitable scale.
In this study, the statistical analysis was conducted by following the steps of the previous research that has successfully developed an instrument []. The data from panelists’ opinions were inputted into SPSS (Statistical Package for the Social Scientists) software version 22.0 for Windows (IBM, Armonk, NY, USA). To obtain a consensus, measurement with a Likert scale of 1–5, the median, interquartile range (IQR), and the level of the agreement would be evaluated. The median should be between the top two measures (suitable/very suitable), the IQR should be one or less, and for the level of the agreement, the answers from the top two measures should be more than 70%.

3.3. Stage 3: Determine Items and Stakeholders

The process was continued by compiling items derived from each indicator. A total of 78 items were composed (Table 4). At the same time, the academics found the stakeholder groups mentioned most by previous studies. The stakeholder groups were Students (Stu), Staff (Sta), Faculty Members (FM), and Community Members (CM) [,,,,,]. Furthermore, a consensus from the panelists was needed to verify the suitability of the items to be asked to stakeholders.
Table 4. List of items.

3.4. Stage 4: Verify Items

In this stage, panelists were asked to rate the suitability of an item for a particular stakeholder group. Answers were on a scale of Yes/No. The answer YES indicated that the panelists agreed to address an item to the designated stakeholders, while the answer NO represented the converse.
The consensus was achieved when more than 67% of the panelists gave the same answer []. In this case, if more than 67% of the panelists answered YES, it is considered that the item was suitable for a particular stakeholder group. Items that agreed YES were retained, otherwise they were eliminated. The remaining items would become the prototype instrument aimed at specific stakeholder groups.
An analysis of the content validity index (CVI) was conducted. Content validity is an important procedure in scale development and is the degree to which an instrument has an appropriate sample of items for the measured construct []. Content validity was computed for each item (I-CVI) and overall scale (S-CVI), including the average (S-CVI/Ave) and the universal agreement (S-CVI/UA). I-CVI was computed as the number of experts giving YES for each item divided by the total number of experts. S-CVI/Ave was calculated by dividing the sum of the I-CVIs by the total number of items. Universal agreement (UA) was obtained by assigning a value of 1 to the item where all panelists answered YES; otherwise, they were given a value of 0. Then, S-CVI/UA was calculated by dividing the sum of UAs by the total number of items []. The statistical significance values for I-CVI, S-CVI/Ave, and S-CVI/UA were set at >0.78, 0.9, and 0.8, respectively [].

3.5. Stage 5: Piloting the Instrument Prototype

Based on the verified prototype instrument for each stakeholder group, the instruments were distributed to stakeholders. The stakeholders were from the same university as the panel experts. This test was intended to reveal the instrument’s reliability and validity. All tests were carried out using SPSS 22 software with α = 0.05. Table 5 shows the decision-making rules regarding validity and reliability.
Table 5. The decision-making rules regarding validity and reliability.

4. Results

Stage 1 found that leadership success can be measured by indicators of how a leader influences all members regarding ESG implementation through the vision and mission and utilizing available resources, socializing the ESG to become a culture and consensus among members. Strategy is crucial for implementing the mission, including the implementation of ESG. It includes financial planning and execution stages. Strategy development is needed to avoid gaps between planning and implementation. The presence of students and staff on campus greatly contributes to the environment around the campus. Therefore, the following issues are relevant to environmental concerns in the campus area: energy-saving (electricity and water), waste reduction and recycling, green campuses, and the impact of transportation systems. University activities are closely related to social relations, both on internal and external campuses. Therefore, the issue of gender and racial equality, as well as people’s respect, is very close to campus life. In addition, universities should also pay attention to students who are disabled or have economic weaknesses and maintain good relations with the community. Campus governance is needed to facilitate operational activities. Therefore, universities need to have the authority to regulate it. Academic development is entrusted to the study program and lecturers to manage it. The staff has the right to contribute to and control operational activities. Likewise, stakeholders have the right to obtain accountable and transparent information. Meanwhile, to measure performance, it is necessary to evaluate the plan and implementation, as well as measure the level of satisfaction of stakeholders regarding the implementation of ESG. The measurable indicator descriptions in this study have been systematically presented in Table 1, becoming references for the items’ compilation.
The data from Stage 2 was processed with SPSS and showed the results in median, IQR, and level of agreement, as shown in Table 6. The median values were in the range of 4 to 5, all IQRs were <1, and the levels of agreement were >0.70. These results indicated that all indicators were considered suitable to support the factors. The Funding planning indicator obtained the worst achievement with a median of 4, IQR of 1, and the lowest level of agreement (0.75). Even though the results were not good, they were still in the agreement range. These verified indicators were then derived to 78 items, as was done in Stage 3.
Table 6. The median, IQR, and level of agreement of items.
The Leadership Factor, which has five indicators, was derived into twelve items. The items compiled include questions about how a leader performs concerning directing, compiling a vision and mission about ESG, utilizing resources, encouraging members to participate, increasing awareness about ESG, and making ESG an inspiration and culture for the organization. The four indicators from the Strategies Factor were derived into eleven items, including the existence of a master plan and operational plan, especially those related to ESG, financial planning, leadership competence in implementing ESG, and evaluation and improvement of ESG implementation. Four indicators in the Environmental Factor were derived into seventeen items: how the university carries out electricity and water efficiency, encourages research and utilizes renewable energy, waste management, building and greenfield areas, and the provision of facilities and encouragement towards environmentally friendly transportation. Five indicators in social factors were derived into eleven items: respect for human rights, gender, and racial equality, solidarity, concern for disabled or economically weak students, care for the community, and other social organizations. In the Governance Factor, five factors were derived into fifteen items, including the university authority in preparing the vision and mission; planning; regulation; performance; dissemination of programs and achievements to stakeholders; study programs’ authority to manage curriculum and research; freedom for lecturers to apply lecture methods and materials, as well as carry out research and community service; and the involvement of supporting staff in preparing, implementing, and evaluating university programs related to ESG. Items in the performance factor involve assessing the implementation of ESG and stakeholders’ satisfaction in academic and non-academic activities and services, mainly those most commonly found with the application of ESG. There were twelve items arranged.
The items were arranged to be distributed to the respondents. Therefore, the questions must be easy to understand so that respondents will have a similar understanding of them. A total of 78 items were composed (Table 4).
Table 7 shows the level of agreement of the panelists regarding the suitability of each item with stakeholder groups. Items with a level of agreement < 67% were eliminated from the list. The result shows that the number of items eliminated from each stakeholder group was as follows: Students (Stu) 23 items, Staff (Sta) 10 items, Faculty Members (FM) 3 items, and Community Members (CM) 36 items. Consequently, the remaining items for each stakeholder group were 55, 68, 75, and 42, respectively.
Table 7. The level of agreement between the items.
A prototype instrument with a list of relevant questions for each stakeholder group was generated. The result showed a consensus on all items because all levels of agreement were > 67%. Therefore, the modified Delphi survey was completed.
The process continued by analyzing the content validity index, starting with calculating I-CVI and UA for each item, as shown in Table 8, Table 9, Table 10 and Table 11.
Table 8. Prototype instrument for Students (55 items).
Table 9. Prototype instrument for Staff (68 items).
Table 10. Prototype instrument for Faculty Members (75 items).
Table 11. Prototype instrument for Community Members (42 items).
Calculations for S-CVI/Ave and S-CVI/UA were performed. Figure 2 presents a summary of the content validity index. The I-CVI of the included items ranged from 0.80 to 1.00, exceeding the recommended level of 0.78, implying that the content of each item was excellent (a). While (b) and (c) show that the result of S-CVI/Ave had a minimum value of 0.98, and the S-CVI/UA had a minimum value of 0.91, meaning that the results were above the recommended level of 0.9 and 0.8, respectively. Therefore, it could be concluded that the content validity of the whole scale was excellent.
Figure 2. The content validity index. (a) Range of I-CVI; (b) Average of S-CVI; (c) Universal agreement.
In Stage 5, appropriate instruments were sent randomly to the group of stakeholders and received responses from 123 students, 97 staff, 80 faculty members, and 100 community members. The results of the Pearson correlation calculation were compared to the critical value in the R Product Moment Table using α = 0.05. The results are shown in Table 12. All values of the Pearson correlation were greater than the critical value. Likewise, all p-values were less than 0.05. Thus, all items were valid.
Table 12. The result of the validity test.
As for the reliability test, Cronbach’s Alpha results were above the critical value of 0.6 (Table 13). These results were statistically convincing that the existing instruments were consistent.
Table 13. The result of the reliability test.

5. Discussion

To the researchers’ best knowledge, this study was the first to validate an instrument regarding ESG implementation in higher education institutions through the modified Delphi study. Instruments have been explicitly compiled aimed at higher education stakeholders, namely Students, Staff, Faculty Members, and Community Members, each of which had 55, 68, 75, and 42 items, respectively.
Although there are no standards for defining expertise, having an expert panel is essential to the Delphi technique process []. In the current study, the expert panelists are key persons in their position level in private higher education in Indonesia. All panelists with experience managing higher education, teaching, research, and community service activities come from different backgrounds. Their competence and diversity provided enrichment to the consensus outcome.
A consensus among the expert panelists regarding the factors and indicators considered suitable to support the factors was reached. The median range of all the included indicators was 4 to 5, and the interquartile range was 0 to 1, while the level of agreement demonstrated > 0.875, indicating good consensus.
This study has contributed to higher education by identifying factors related to ESG implementation. It was concluded that Leadership, Strategies, and Performance were the associated factors. Furthermore, twenty-eight indicators have been determined that support the factors related to ESG implementation in higher education. Five indicators support the Leadership Factor: the Rector/Leader’s Leadership, Vision and Mission, Resource Input, ESG Education, and Culture and Consensus. Four indicators support the Strategies Factor: Strategy for ESG, Plan of Strategies, Funding Planning, and Execution. Four indicators support Environmental Concerns: Energy-saving (electricity and water), Waste Reduction and Recycling, Green Building and Campus, and Transportation. Five indicators support Social Concerns: People Respect, Gender and Racial Equality, Social Values Education, Care for Weak Students, and Social Relationships. Five indicators support Governance Concerns: University Autonomy, Academic Freedom, Professor Authority, Staff Empowerment, and Information Transparency. As for the Performance Factor, there are five supporting indicators: ESG Evaluation, Student Satisfaction, Staff Satisfaction, Professor Satisfaction, and Social Satisfaction.
The next stage was to compile items based on agreed indicators. Panelists were asked their opinion regarding indicators’ suitability with stakeholder groups. Items that did not receive panelist approval were eliminated. Thus, the remaining items were items that have received agreement from the panelists. The items were in the form of simple questions that the respondents easily understood; therefore, the respondents were expected to provide the proper answer accurately.
The Content Validity of this study was good. The I-CVI of the included items varied from 0.80 to 1.00, which was higher than the recommended level of 0.78, suggesting that each instrument’s content was excellent. The S-CVI/UA of each instrument was above 0.91, reaching the acceptable value of 0.8. Similarly, all of the S-CVI/Ave were above the proper value of 0.9, indicating the content validity of the whole scale was excellent. Higher education can use this instrument as a tool to monitor ESG implementation.
Although this study has been conducted very carefully, it is acknowledged that it has weaknesses and limitations. The deductive technique that supports the focus group discussion was used to collect as much information as possible from reliable sources; nonetheless, it was noticeable that this study could not cover all of them. On the other hand, the invited experts were all from the same university, which did not represent the diversity of higher education options.
Despite the limitations, this study provides essential information in the form of ready-to-use instruments. Figure 3 provides a summary of what resulted from each step in this study.
Figure 3. Results of each step.

6. Conclusions

This study has established an instrument for assessing ESG implementation in higher education. Six factors were identified: Leadership, Strategies, Environmental, Social, Governance, and Performance. The six factors were latent variables requiring measurable indicators to support them. Furthermore, twenty-eight supporting indicators have been identified. These indicators were derived into seventy-eight items in the form of simple statements.
In connection with the four stakeholder groups identified as potential respondents, an in-depth study was conducted to determine the appropriate items for each stakeholder group. Finally, it was identified that the number of items varies for each group as follows Students (55 items), Staff (68 items), Faculty Members (75 items), and Community Members (42 items).
Determination of factors, indicators, and items was performed through a deductive study, focus group discussion, and consensus-seeking from a panel of experts. The panelists were key persons in higher education management. The Modified Delphi Method was applied to obtain consensus. The questions to the panelists were not open-ended but structured questions that have been compiled based on reliable previous literature. The panel of experts reached a consensus on all items, and both the individual items and overall scale were found to have excellent content validity.
The final versions of these instruments were valid and reliable. These instruments were ready to be used to conduct empirical research in higher education that will provide information on the implementation of ESG in higher education institutions. The managerial impact is to increase the interest of higher education managers in implementing ESG to support the contribution of higher education institutions to the achievement of the SDGs. This study proposes that higher education leaders consider formulating policies, especially developing strategies related to education, implementation, and evaluation related to ESG, and assessing ESG performance as a measure of stakeholder satisfaction. In practice, higher education needs to give authority to study programs and lecturers to determine the curriculum, teaching materials, and activities related to ESG and involve students to participate.
The initiation of determining factors, indicators, and items from various reliable sources might not cover all available information. Therefore, it could become the potential weakness of this study. Likewise, panelist selection was limited to one institution. It might not be able to represent the diversity of higher education institutions. Therefore, further research can be conducted by finding information from the latest research and varied panelists. Thus, it is hoped that other factors or indicators can be found to enrich the instruments. The results of this study also provide insight for further research to prove the existence of a recursive or non-recursive model among the factors.

Author Contributions

Conceptualization, P.B.H. and C.-C.Y.; Data curation, M.M.W.I.; Formal analysis, M.M.W.I.; Investigation, M.M.W.I.; Methodology, P.B.H., C.-C.Y. and M.M.W.I.; Resources, P.B.H. and C.-C.Y.; Supervision, P.B.H. and C.-C.Y.; Validation, P.B.H., C.-C.Y. and R.S.; Visualization, M.M.W.I.; Writing—original draft, P.B.H., C.-C.Y., M.M.W.I. and R.S.; Writing—review & editing, P.B.H., C.-C.Y. and R.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. 17 Goals History. Available online: https://sdgs.un.org/goals (accessed on 25 April 2022).
  2. Secretary-General Launches ‘Principles For Responsible Investment’ Backed by World’s Largest Investors. Available online: https://www.un.org/press/en/2006/sg2111.doc.htm (accessed on 17 May 2021).
  3. Bhattacharya, S.; Sharma, D. Do environment, social and governance performance impact credit ratings: A study from India. Int. J. Ethic-Syst. 2019, 35, 466–484. [Google Scholar] [CrossRef]
  4. De Zwaan, L.; Brimble, M.; Stewart, J. Member perceptions of ESG investing through superannuation. Sustain. Account. Manag. Policy J. 2015, 6, 79–102. [Google Scholar] [CrossRef]
  5. Alda, M. The environmental, social, and governance (ESG) dimension of firms in which social responsible investment (SRI) and conventional pension funds invest: The mainstream SRI and the ESG inclusion. J. Clean. Prod. 2021, 298, 126812. [Google Scholar] [CrossRef]
  6. Lambrechts, W. The contribution of sustainability assessment to policy development in higher education. Assess. Eval. High. Educ. 2015, 40, 801–816. [Google Scholar] [CrossRef]
  7. Menon, S.; Suresh, M. Synergizing education, research, campus operations, and community engagements towards sustainability in higher education: A literature review. Int. J. Sustain. High. Educ. 2020, 21, 1015–1051. [Google Scholar] [CrossRef]
  8. Faghihi, V.; Hessami, A.R.; Ford, D.N. Sustainable campus improvement program design using energy efficiency and conservation. J. Clean. Prod. 2015, 107, 400–409. [Google Scholar] [CrossRef]
  9. Ozdemir, Y.; Kaya, S.K.; Turhan, E. A scale to measure sustainable campus services in higher education: “Sustainable Service Quality”. J. Clean. Prod. 2020, 245, 118839. [Google Scholar] [CrossRef]
  10. Berzosa, A.; Bernaldo, M.O.; Fernández-Sanchez, G. Sustainability assessment tools for higher education: An empirical comparative analysis. J. Clean. Prod. 2017, 161, 812–820. [Google Scholar] [CrossRef]
  11. Pimentel, D.; Pleasant, A.; Barron, J.; Gaudioso, J.; Pollock, N.; Chae, E.; Kim, Y.; Lassiter, A.; Schiavoni, C.; Jackson, A.; et al. US Energy Conservation and Efficiency: Benefits and Costs. Environ. Dev. Sustain. 2004, 6, 279–305. [Google Scholar] [CrossRef]
  12. Wan, K.K.; Li, D.H.; Liu, D.; Lam, J.C. Future trends of building heating and cooling loads and energy consumption in different climates. Build. Environ. 2011, 46, 223–234. [Google Scholar] [CrossRef]
  13. Sailor, D.J.; Elley, T.B.; Gibson, M. Exploring the building energy impacts of green roof design decisions—A modeling study of buildings in four distinct climates. J. Build. Phys. 2012, 35, 372–391. [Google Scholar] [CrossRef]
  14. Ralph, M.; Stubbs, W. Integrating environmental sustainability into universities. High. Educ. 2014, 67, 71–90. [Google Scholar] [CrossRef]
  15. Albareda-Tiana, S.; Vidal-Raméntol, S.; Fernández-Morilla, M. Implementing the sustainable development goals at University level. Int. J. Sustain. High. Educ. 2018, 19, 473–497. [Google Scholar] [CrossRef]
  16. GRI. Consolidated Set of GRI Sustainability Reporting Standards 2020; GSSB: Amsterdam, The Netherlands, 2020. [Google Scholar]
  17. Santos, G.; Marques, C.S.; Justino, E.; Mendes, L. Understanding social responsibility’s influence on service quality and student satisfaction in higher education. J. Clean. Prod. 2020, 256, 120597. [Google Scholar] [CrossRef]
  18. Vázquez, J.L.; Aza, C.L.; Lanero, A. University social responsibility as antecedent of students’ satisfaction. Int. Rev. Public Nonprofit Mark. 2016, 13, 137–149. [Google Scholar] [CrossRef]
  19. Jain, P.K.; Hazenberg, R.; Seddon, F.; Denny, S. Social Value as a Mechanism for Linking Public Administrators with Society: Identifying the Meaning, Forms and Process of Social Value Creation. Int. J. Public Adm. 2020, 43, 876–889. [Google Scholar] [CrossRef]
  20. Jaramillo, A.; Zaafrane, H. Benchmarking university governance in the MENA region. J. High. Educ. Manag. Policy 2014, 24, 7–36. [Google Scholar] [CrossRef]
  21. Higher Education is Investing in ESG. Available online: https://info.kpmg.us/news-perspectives/advancing-the-profession/higher-ed-investing-in-esg.html (accessed on 19 September 2022).
  22. ESG in Higher Education: From Strategy to Execution. Available online: https://www.ey.com/en_us/education/strategy-consulting/esg-in-higher-education-a-focus-on-the-environment (accessed on 19 September 2022).
  23. Data Appendix. Available online: https://support.qs.com/hc/en-gb/articles/4405027974290-Data-Appendix (accessed on 28 April 2022).
  24. UI GreenMetric World University Rankings: Background of The Ranking. Available online: https://greenmetric.ui.ac.id/about/welcome (accessed on 28 April 2022).
  25. Jusuf, E.; Herwany, A.; Kurniawan, P.S.; Gunardi, A. Sustainability Concept Implementation in Higher Education Institutions of Indonesia. J. Southwest Jiaotong Univ. 2020, 55, 1–18. [Google Scholar] [CrossRef]
  26. Hoang, T. The Role of the Integrated Reporting in Raising Awareness of Environmental, Social and Corporate Governance (ESG) Performance, in Stakeholders, Governance and Responsibility; Emerald Publishing Limited: Bradford, UK, 2018; pp. 47–69. [Google Scholar]
  27. Taliento, M.; Favino, C.; Netti, A. Impact of Environmental, Social, and Governance Information on Economic Performance: Evidence of a Corporate ‘Sustainability Advantage’ from Europe. Sustainability 2019, 11, 1738. [Google Scholar] [CrossRef]
  28. Harding, S.; Harrison, R. Changing Times, Changing Universities: Leadership, Governance and Management in a Dynamic Environment. Int. J. Chin. Educ. 2015, 4, 5–27. [Google Scholar] [CrossRef]
  29. Rume, T.; Islam, S.D.-U. Environmental effects of COVID-19 pandemic and potential strategies of sustainability. Heliyon 2020, 6, e04965. [Google Scholar] [CrossRef] [PubMed]
  30. Farnell, T.; Matijević, A.S.; Schmidt, N.Š. The Impact of COVID-19 on Higher Education: A Review of Emerging Evidence: Analytical Report; European Commission, Directorate-General for Education, Youth, Sport and Culture: Brussel, Belgium, 2021.
  31. Linstone, H.A.; Turoff, M. The Delphi Method: Techniques and Applications; Addison-Wesley Publishing Company, Advanced Book Program: Boston, MA, USA, 2002. [Google Scholar]
  32. Mao, X.; Loke, A.Y.; Hu, X. Developing a tool for measuring the disaster resilience of healthcare rescuers: A modified Delphi study. Scand. J. Trauma, Resusc. Emerg. Med. 2020, 28, 1–12. [Google Scholar] [CrossRef]
  33. SASB Materiality Map. Available online: https://materiality.sasb.org/ (accessed on 3 June 2021).
  34. Ehler, C.N. Indicators to measure governance performance in integrated coastal management. Ocean Coast. Manag. 2003, 46, 335–345. [Google Scholar] [CrossRef]
  35. Altbach, P.; Salmi, J. The Road to Academic Excellence: The Making of World-Class Research Universities; Bank, T.W., Ed.; The World Bank: Washington, DC, USA, 2011. [Google Scholar]
  36. Pucciarelli, F.; Kaplan, A. Competition and strategy in higher education: Managing complexity and uncertainty. Bus. Horiz. 2016, 59, 311–320. [Google Scholar] [CrossRef]
  37. Demir, R.; Wennberg, K.; McKelvie, A. The Strategic Management of High-Growth Firms: A Review and Theoretical Conceptualization. Long Range Plan. 2017, 50, 431–456. [Google Scholar] [CrossRef]
  38. Kaplan, R.S. Strategic Performance Measurement and Management in Nonprofit Organizations. Nonprofit Manag. Leadersh. 2001, 11, 353–370. [Google Scholar] [CrossRef]
  39. Sumardi, A.A.R.F. The mediating effect of service quality and organizational commitment on the effect of management process alignment on higher education performance in Makassar, Indonesia. J. Organ. Chang. Manag. 2018, 31, 410–425. [Google Scholar] [CrossRef]
  40. Northouse, P.G. Leadership Theory and Practices; SAGE Publication Inc.: Thousad Oaks, CA, USA, 2016. [Google Scholar]
  41. Khan, H.; Matlay, H. Implementing service excellence in higher education. Educ. Train. 2009, 51, 769–780. [Google Scholar] [CrossRef]
  42. Velazquez, L.; Munguia, N.; Platt, A.; Taddei, J. Sustainable university: What can be the matter? J. Clean. Prod. 2006, 14, 810–819. [Google Scholar] [CrossRef]
  43. Ling, K.; Piew, T.; Lau, T.-C. The Impact of Resource Input Model of Education Quality on the Overall Students’ Perceived Service Quality. Can. Soc. Sci. 2010, 6, 125–144. [Google Scholar] [CrossRef]
  44. Saqib, S. The Role of Leadership in Strategy Formulation and Implementation. Int. J. Manag. Organ. Stud. 2012, 1, 32. [Google Scholar]
  45. Engert, S.; Baumgartner, R.J. Corporate sustainability strategy—Bridging the gap between formulation and implementation. J. Clean. Prod. 2016, 113, 822–834. [Google Scholar] [CrossRef]
  46. OECD. Benchmarking Higher Education System Performance: Conceptual Framework and Data; OECD: Paris, France, 2017. [Google Scholar]
  47. Blasco, N.; Brusca, I.; Labrador, M. Drivers for Universities’ Contribution to the Sustainable Development Goals: An Analysis of Spanish Public Universities. Sustainability 2021, 13, 89. [Google Scholar] [CrossRef]
  48. Haid, M.; Albrecht, J.N.; Finkler, W. Sustainability implementation in destination management. J. Clean. Prod. 2021, 312, 127718. [Google Scholar] [CrossRef]
  49. Pearlmutter, D.; Rosenfeld, S. Performance analysis of a simple roof cooling system with irrigated soil and two shading alternatives. Energy Build. 2008, 40, 855–864. [Google Scholar] [CrossRef]
  50. Ansell, C.; Torfing, J. Introduction: Theories of Governance. In Handbook on Theories of Governance; Ansell, C., Torfing, J., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2016; pp. 1–17. [Google Scholar]
  51. Niedlich, S.; Bauer, M.; Doneliene, M.; Jaeger, L.; Rieckmann, M.; Bormann, I. Assessment of Sustainability Governance in Higher Education Institutions—A Systemic Tool Using a Governance Equalizer. Sustainability 2020, 12, 1816. [Google Scholar] [CrossRef]
  52. Arroyo, P. A new taxonomy for examining the multi-role of campus sustainability assessments in organizational change. J. Clean. Prod. 2017, 140, 1763–1774. [Google Scholar] [CrossRef]
  53. von der Gracht, H.A. Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technol. Forecast. Soc. Change 2012, 79, 1525–1536. [Google Scholar] [CrossRef]
  54. Crawford, M.; Wright, G. Delphi Method. In Wiley StatsRef: Statistics Reference Online; John and Wiley and Sons: Hoboken, NJ, USA, 2016; pp. 1–6. [Google Scholar]
  55. Jillson, I.A. The National Drug-Abuse Policy Delphi: Progress Report and Findings to Date. In The Delphi Method Techniques and Application; Linstone, H.A., Turoff, M., Helmer, O., Eds.; Addison Wesley Publishing Company: Boston, MA, USA, 2002; pp. 119–154. [Google Scholar]
  56. Naudé, P.; Ivy, J. The marketing strategies of universities in the United Kingdom. Int. J. Educ. Manag. 1999, 13, 126–136. [Google Scholar] [CrossRef]
  57. Langrafe, T.D.F.; Barakat, S.R.; Stocker, F.; Boaventura, J.M.G. A stakeholder theory approach to creating value in higher education institutions. Bottom Line 2020, 33, 297–313. [Google Scholar] [CrossRef]
  58. Perception on the Community Engagement of Students of Higher Education Institutions. Available online: https://ssrn.com/abstract=3664052 (accessed on 9 May 2021).
  59. Benneworth, P.; Jongbloed, B.W. Who matters to universities? A stakeholder perspective on humanities, arts and social sciences valorisation. High. Educ. 2010, 59, 567–588. [Google Scholar] [CrossRef]
  60. Sheila, N.A.; Zhu, C.; Kintu, M.J.; Kataike, J. Assessing higher education institutional stakeholders’ perceptions and needs for community engagement: An empirical evidence from Uganda. Heliyon 2021, 7, e06612. [Google Scholar] [CrossRef] [PubMed]
  61. Alexandrov, A.V.; Pullicino, P.M.; Meslin, E.M.; Norris, J.W. Agreement on Disease-Specific Criteria for Do-Not-Resuscitate Orders in Acute Stroke. Members of the Canadian and Western New York Stroke Consortiums. Stroke 1996, 27, 232–237. [Google Scholar] [CrossRef] [PubMed]
  62. Shi, J.; Mo, X.; Sun, Z. Content validity index in scale development. J. Cent. South Univ. 2012, 37, 152–155. [Google Scholar] [CrossRef]
  63. Rodrigues, I.B.; Adachi, J.D.; Beattie, K.A.; MacDermid, J.C. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet. Disord. 2017, 18, 540. [Google Scholar] [CrossRef]
  64. Polit, D.F.; Beck, C.T.; Owen, S.V. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health 2007, 30, 459–467. [Google Scholar] [CrossRef]
  65. Shaw, K.L.; Brook, L.; Cuddeford, L.; Fitzmaurice, N.; Thomas, C.; Thompson, A.; Wallis, M. Prognostic indicators for children and young people at the end of life: A Delphi study. Palliat. Med. 2014, 28, 501–512. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Identifying Similarities of National Yacht Qualifications Using Hierarchical Clustering Methods
Previous
Significant Fragmentation of Disposable Surgical Masks—Enormous Source for Problematic Micro/Nanoplastics Pollution in the Environment
Next