Next Article in Journal
Measuring and Integrating Risk Management into Green Innovation Practices for Green Manufacturing under the Global Value Chain
Next Article in Special Issue
Sustainability Assessment in Higher Education Institutions
Previous Article in Journal
A Novel Closed-Loop Supply Chain Network Design Considering Enterprise Profit and Service Level
Previous Article in Special Issue
A Matter of Responsible Management from Higher Education Institutions
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Sustainability Assessment and Benchmarking in Higher Education Institutions—A Critical Reflection

Department of Science and Technology, Universidade Aberta, 1269-001 Lisbon, Portugal
Center for Environmental and Sustainability Research, NOVA School of Science and Technology, NOVA University Lisbon, 2829-516 Caparica, Portugal
Department of Business Organization, Universidad Autónoma de Madrid, 28049 Madrid, Spain
Author to whom correspondence should be addressed.
Sustainability 2020, 12(2), 543;
Received: 13 November 2019 / Revised: 6 January 2020 / Accepted: 7 January 2020 / Published: 10 January 2020
(This article belongs to the Special Issue Sustainability Assessment in Higher Education Institutions)


Higher Education Institutions (HEIs) play a crucial role in implementing practices for Education for Sustainable Development (ESD). This implementation should be done in different dimensions according to a holistic and whole-school approach. Different tools have been adapted and developed to assess this integrated approach. The aim of this research is to critically reflect the existing tools to assess and benchmark ESD implementation and to discuss their applicability in two case studies. Two public Universities in Southern Europe, with headquarters in the capitals of Portugal and Spain were selected to assess and compare the integration of ESD according to a whole-school approach—Universidade Aberta in Portugal and Universidad Autónoma de Madrid in Spain. After a critical analysis of the existing tools based on literature review and a list of criteria classified by experts, two tools were selected to be applied in the case studies. The online Sustainability Tracking, Assessment & Rating System Reporting Tool was used in Universidade Aberta and Green Metrics tool was used in Universidad Autónoma de Madrid. The tools were complemented with focus group with key-actors in both universities. The results obtained allowed to identify the need to define a common objective of the assessment tools and limitations they still have. The tools need improvements on their development namely to integrate the external impact of Higher Education Institutions on sustainability, to integrate participatory processes and to assess non-traditional aspects of sustainability. This research hopes to contribute to the continuous research about the usefulness of these assessment and benchmarking tools as drivers to HEIs improve their sustainability performance and their role as agents of changes.

1. Introduction

Higher Education Institutions (HEIs) have a critical responsibility in education for sustainable development (ESD) due to “the main component for raising awareness of SD among the population” [1]. In other words, ESD provides the knowledge and skills for students to start to create SD initiatives. Furthermore, HEIs should “lead by example” [2].
Sustainable development as a development model integrates environmental, social and economic considerations [3] and so HEIs must assume a holistic focus in all activities. Indeed, HEI with or without governmental policies and recommendations are moving towards holistic and systemic approaches when addressing Education for Sustainable Development. The Sustainable Development Goals (SDG), recently adopted by United Nations in 2015 [4] and in particular, SDG 4, are an additional driver for the implementation of sustainability at HEI in an integrative way.
According to the United Nations guidelines followed by several researchers the integrative approach to implement sustainability in HEIs includes six major ESD dimensions to allow a whole-school approach—(i) Facilities or Operations; (ii) Teaching and Curriculum; (iii) Organizational Management; (iv) External Community; (v) Research; (vi) Assessment and Communication (based on [5,6]).
Based on early work [7,8], some authors stress that the integration of sustainability in HEIs can be done at various levels, from national to the institution level [9]. The institutions that integrate sustainability according to the whole-school approach (in terms of dimensions and involvement of the whole HEIs community) can achieve the sustainability maturity curve, thus enabling them to be agents of change and transformation. In this context, Kapitulcinová [9] introduced the model from a “business-as-usual university” to a “sustainable university” where sustainability has been fully integrated under three degrees, initiation/awakening, implementation/pioneering and institutionalization/transformation, including in this last stage a consolidation of changes. Thus, the HEIs are under mounting pressure to partner with societal stakeholders and organizations to collaboratively create and implement sustainability-advancing knowledge, tools and societal transformations [10]. So, in recent years, an increasing number of institutions have begun the adjustment and restructuring of education, research, campus operations and community outreach towards sustainability [8,11,12]. However, deep and full integration of sustainable development at all of dimensions and community of HEIs is still lacking [13,14].
Different tools have been developed to assess and benchmark ESD implementation at HEIs but how well are being performed in case studies and how well they evaluate sustainability and its impact is still an open question. This study attempts to provide a holistic sustainability maturation path reflecting essential dimensions that HEIs need to approach and assess as whole-school integration towards a potential sustainability management change. The aim of this research is, in the first part, to critically analyze the existing tools to assess and benchmark ESD implementation in HEIs. In the second part of the article, the aim is to assess the integration of ESD according to a whole-school approach through the use of those tools in the case studies and discuss their applicability. Two of the analyzed tools were selected and applied in two HEIs in Portugal and Spain. The assessment was complemented with stakeholder’s engagement. European HEIs have been ahead in the implementation of sustainability at the different dimensions [13,15]. In particular, in southern Europe neighbor countries like Portugal and Spain, despite the lack of national policies, HEIs are working towards ESD implementation through the development of plans and actions in sectorial areas. The two case studies are public Universities (one smaller and in distance learning regime and others in traditional learning and bigger) engaged in the last years in integrating sustainability in different dimensions. They have their headquarters in the capitals of Portugal and Spain and are classified in levels of excellence in their teaching and learning methods: Universidade Aberta, Portugal and Universidad Autónoma de Madrid in Spain.
This article starts after this Introduction with a literature review about the tools to assess sustainability in HEI (Section 2). Section 3 presents the methods for the tool’s assessment and case studies evaluation. Section 4 describes the case studies, Universidade Aberta and Universidad Autónoma de Madrid. Section 5 discuss the results and the final section is dedicated to final conclusions and contribution to new knowledge and research implications.

2. Sustainability Assessment in Higher Education

Given the rapid growth of ESD initiatives at HEIs, the measures, assessment and reports of HEIs progress toward SDG has become increasingly important [16]. The assessment of sustainability in HEIs is one of the most important dimensions of EDS implementation in HEIs and could be conducted base on specific tools, that allows to assess whether all possible dimensions to the implementation of sustainability are being implemented and whether they are doing so holistically [9]. Also, when well-developed these tools can be used as benchmarking practices, comparing HEIs processes and performance metrics. Nevertheless, these tools must be able to uniformly evaluate the implementation of sustainability in HEIs without necessarily placing them in a ranking or competition. So, the tools must identify the important themes, be measurable and comparable, go beyond eco-efficiency, measure progress and motivations and be understandable to a broad set of key actors [17]. According to other authors [18], these tools can be based on indicators and conceptual models that support sustainability decisions, as well as facilitating communication efficiently and for a wide audience, knowing how to respond to complex processes capable of assessing the transformation for sustainability.
Much work has been done on the development of tools to assess sustainability specifically in HEIs, showing the importance of the theme. Several articles have reviewed these tools from different perspectives (e.g., [7,9,14,16,17,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]). However, it is observed that in these various reviews, the systematization is not homogeneous, noting a lack of common designations and objectives and including tools that do not have as main objective the assessment of the implementation of ESD or are not per se an assessment tool. So, for example, the reviewed tools:
assess only one of the dimensions of implementing sustainability in HEIs such as Campus operations (e.g., Campus Sustainability Assessment Framework-CSAF, [20]; National Wildlife Federation’s State of the Campus, [34]) or Curricula (e.g., Sustainability Tool For Assessing Universities’ Curricula Holistically, STAUNCH, [11]);
evaluate only one pillar of sustainability, namely Environmental (e.g., Campus Ecology, [35], Environmental Performance Survey, [34]);
only serve as manuals or supportive conceptual models (e.g., Greening Campuses, [36]);
assess only the level of literacy and knowledge of population sustainability (e.g., SULITEST, [37]);
adapt by an Institution but based on other existing metrics (e.g., UNI-Metrics—Value Metrics and Policies for a Sustainable University Campus, [31]);
specific to a type of HEI (e.g., Business School Impact System BSIS, see Reference [14]);
serve only as guidelines for supporting communication of performance for sustainability but are not themselves an assessment tool. This is the case of the guidelines developed by the International Campus Sustainability Network and Global University Leader Forum [38], two international networks that suggest the organization of sustainability reports for HEIs, based on GRI indicators and the Sustainability Tracking tool, Assessment & Rating System—STARS [38]. This is also the case of the work developed by Nixon [19] who developed other guidelines for the implementation of sustainability in HEIs (under the CSARP project “Campus Sustainability Assessment Review Project” developed at Western Michigan University), in its different dimensions and practices, based on a literature review of existing tools, proposing no new tool.
Thus, for this study a systematic review of the tool for sustainability implementation assessment in HEIs was conducted, based on the following conditions:
developed specifically for assessing the performance of sustainability implementation in HEIs;
covering at least two of the various dimensions of sustainability implementation in HEIs ((i) facilities or campus operations; (ii) teaching and curriculum; (iii) organizational management; (iv) external community; (v) research; (vi) assessment and communication;
covering at least two of the sustainability pillars (environmental, social. and economic), to guarantee that the tools in some way are based on a holistic and whole-school approach and since most are based on only two of the pillars.
Based on these criteria 27 tools were searched on google scholar using the key-words: “Sustainability assessment” and “Higher Education,” from October 2018 and March 2019. Each tool was then characterized (see Table 1).
According to several authors, the overall implementation of these tools is still low and its development is still at an early stage [14,28,32].
The various tools for assessing the sustainability of HEIs are mostly based on indicators, using graphs or final rankings to communicate the results. Indicator-based tools have the advantage of being potentially more transparent, consistent and comparable, thus useful for monitoring and decision support [14,18], although support for decision making is not yet fully demonstrated [65].
Another common characteristic of these tools listed in Table 1 is the fact that they are filled out by self-assessment, requiring only a leader or researcher to complete them. In order to create a sustainable university, it is important not only to use assessment tools for a real application as well as integrate on the process different agents of Higher Education Institutions [32]. Stakeholder participatory approaches can be seen as a requirement, as well as a benefit towards the integration of SD into the university culture [66]. Furthermore, active stakeholder participation is essential to grow the model’s level of complexity, promote model ownership and use it in the organizational strategic planning process in a higher education organization [67]. Stakeholder engagement is also crucial to achieve the visions and goals for a Green university [68] and current SDG. The second draft of the People’s Sustainability Treaty on Higher Education [69] divided higher education stakeholders into three broad categories: (1) those engaged in the activities of higher education institutions: executive, academic managers, educators, researchers and students. (2) those engaged in the higher education system: administrative officers, ministries, assessment bodies, international organizations. (3) those forming part of the communities, which the HEI system serves: local communities, professional bodies, companies, among others. Hence, a socially responsible HEI considers stakeholder behavior and perception to better understand their expectations and priorities and use these to define the strategy and goals, to monitor the objectives in view of promoting activities and accountability and to enhance a community-university engagement. At the end it contributes to change management and to a more mature sustainable university [9]. Thus, sustainability reflects a condition based on the relationship between stakeholders and HEI [70].
Based on the characteristics of these tools (see Table 1), the 27 tools were critically analyzed to evaluate their real assessment of the sustainability implementation and integration in HEIs (see the sections of the methods and results).

3. Cases Studies

The Universidade Aberta (UAb) was founded in 1988 with the distinctive feature of being the only Portuguese public distance education university, a distinction that still remains today. Filling the Universidade Aberta vision and mission of being a global university, the Universidade Aberta offers undergraduate and graduate higher education courses and Lifelong Learning courses, which are especially dedicated to the whole Portuguese speaking country community. The Universidade Aberta campus corresponds to four facilities, namely, its headquarters in Lisbon and two other support buildings in Coimbra and Porto (regional offices). The Universidade Aberta also has other facilities in Lisbon and Local Learning Centers spread throughout the country but since they are not owned but rented, they were not considered in the sustainability evaluation. In 2018 Universidade Aberta community comprise the rounded numbers of 6000, which 5000 were full-time students and exclusively engaged in distance education and 340 employees, of whom 150 belongs to the academic and research staff and 190 to the administrative staff. It is structured in 5 academic departments: Science and Technology, Social and Management Sciences, Distance Learning and Humanities, Lifelong Learning, with an educational offer of 10 graduate Ph.D. programs; 22 graduated master programs, 11 under graduated programs and 9 post-graduated programs. UAb has 2 research institutes and 5 more research institutes with branches in this university.
The Universidade Aberta has focused on the quality of its service and has been distinguished by several national and international entities: (a) EFQUEL Award—European Foundation for Quality in E-learning in 2010; (b) the UNIQUe—The Quality Label for the use of ICT in Higher Education (Universities and Institutes) in 2010; (c) the 1st Level of Excellence Committed to Excellence (c2e2) of the European Foundation for Quality Management (EFQM) in 2011; (d) 2nd Level of Excellence Recognized for Excellence (R4E) of the European Foundation for Quality Management (EFQM) in 2016; (e) the certification of the International Standard Organization (ISO) 27001 by the Portuguese Association of Certification attesting the security of its platform of e-learning and ISO 9001 quality, in 2017. UAb has been applied sustainability in different ways, namely: (i) curricula (through a e-learning three cycle degrees system from undergraduate, to master and Ph.D. aiming to actively promote education for sustainable development, along with an increase in transdisciplinary across subjects and also through non formal courses about Climate Change, Education for Sustainability and Environment awareness), (ii) application of a quality management policy with a Recognition of Excellence, (iii) a specific inclusion program for students with disabilities. UAb formal compromise to Sustainability was achieved by being an institutional member of the Association for the Advancement of Sustainability in Higher Education (AASHE) and a signatory member of the Letter of Commitment for Sustainable Campus in Portugal, both in 2019. UAb has no formal office for Sustainability due to its small size but sustainability issues are informally addressed within the Quality Office.
The Universidad Autónoma de Madrid (UAM) is a Spanish public university established in 1968. In 2018, 26.733 students of all levels were enrolled at the UAM. This university has a teaching staff and/or researcher around 3.141 and an administrative and services staff around 1.053. The university is organized into eight schools: Sciences (biology, mathematics, physics and chemistry), Economics and Business, Law, Computer Science and Engineering, Arts (philosophy, history, philology, translation and interpretation), Education and Psychology, offering a wide range of programs in different scientific and technical fields and in the Humanities. UAM has 11 research institutes and these are located on campus, as well as the Madrid Science Park, with growing university-business collaboration (contracts, internships and sponsored chairs). In 2009, UAM was declared International Excellence Campus, at the same time with the Spanish Research Council (CSIC). Nowadays, UAM has been placed among the top universities for its levels of excellence in national and international rankings. In the QS World University Ranking 2019, the UAM has managed to locate itself in the first position in Spain and in the 159th place in the world.
Most of the faculties and specialized institutes are on the Cantoblanco campus, 15 km North of Madrid. The Faculty of Medicine is on another campus (near La Paz Hospital). These two campuses (Cantoblanco and La Paz) were considered in the sustainability evaluation. In the field of sustainability, this university, from the Rio de Janeiro Summit (1992) (where a global action plan for Sustainable Development was approved: Agenda 21), formalized its commitment to Agenda 21 through the ECOCAMPUS project. In 1997, the Ecocampus office was created, which has a special involvement in the maintenance of the campus as a sustainable territory and at the same time promotes activities related generally to sustainability. Social commitment and sustainability continue to be part of the frame of reference of the different lines of action for the entire institution in the 2025 strategy. UAM has also an SDG Lab that is a multi-stakeholder initiative that contributes to the implementation of the Sustainable Development Goals at the UAM.

4. Materials and Methods

4.1. Tools Assessment

The critical analysis of the tools listed in Table 1 was based on earlier reviews and according to the following criteria (based on the research of [9,18,24,28,65]: (i) comprehensibility; (ii) comparability; (iii) availability of baseline data; (iv) assessment of progress over time; (v) comprehensiveness and integration of sustainability dimensions in HEIs; (vi) usefulness for decision-making and communication; (vii) level of participation of the public or key actors and (viii) tool accessibility on the internet. The tools were then classified from 1 to 3 for each of the criteria (1. Low, 2. Medium and 3. High), based on documentary analysis of the tools and expert knowledge (according to the methodology referred in Reference [71]).
The classification of each tool was conducted independently by four judges (authors of this paper) using the defined criteria. The procedure proposed in Reference [72] has been followed in order to determine the level of agreement between the judges (values between 0 and 1 for each agreement between two judges):
Index of Agreement = (C1,2 + C1,3 + C1,4 + C2,3 + C2,4 + C3,4)/6
For data analysis, the mode and relative frequencies were calculated for each criterion. An average of the Index of the agreement was calculated for each criteria and tool. Two of the listed tools were not classified since their information was not available in English (DUK was only accessible in German and ASSC only accessible in Japanese).
Validity, reliability and generalizability are limitations associated with this type of qualitative approach [71] and were weighed up in the qualitative assessment and discussion of the results and when drawing the conclusions.

4.2. Sustainability Assessment in the Case Studies

The sustainability assessment process at the Universidade Aberta and the Universidad Autónoma de Madrid was carried out using different tools, STARS and GM, respectively, complemented with stakeholder’s engagement (focus groups/workshops) in each University. The focus groups and workshops were used for engagement and awareness of the assessment process and also to find paths of improvement (Table 2). The STARS and GM tools were selected since they were well classified according to the main criteria and also in terms of free access on the internet (see Section 5.1 heading the in results section).
For UAb, the implementation of the sustainability assessment was conducted across the year 2018, corresponding to a 3-year assessment analysis (2015 to 2017). STARS 2.1 version tool is based 5 dimensions with different weights each: Academic and Research (20%), Involvement of Key Actors (20%), Campus Operations (35%), Planning and Administration (15%), Innovation and Leadership (2%) and a total 74 indicators divided in each category. The indicators are quantified and filled in a web application with a written justification or document upload. The different indicators and methodological procedures are explained in AASHE [58]. The raw data collection was the first procedure, involving not only a web search as also the requirement of specific informants for the technical information, reach by a face to face interview or information request by email. Possessing the requested data, the STARS assessment was fulfilled, resulting in a diagnostic report on the implementation of sustainability at the Universidade Aberta, where the most and least punctuated indicators were identified, revealing the weaknesses and the strengths to the implementation of sustainability. Since UAb is an institutional member of AASHE, a final assessment of the STARS scores was conducted, through an internal (by the rectorate) and external validation (by the AASHE technical staff) to allow a final awarded rating between bronze, silver or gold label.
With the final STARS report information, two focus groups (one with university experts and others with different stakeholders from the all University—see Table 2) were held in June and July of 2018. The main objective of the participatory moments was to show and discuss the STARS results and development of proposals for improvement for the implementation of sustainability at the Universidade Aberta. In a first part of the focus groups the information provided in the STARS report was presented and improvements were then proposed according to STARS dimensions. The first focus group was conducted with university experts that have been working in UAb sustainability implementation, including the Vice-rector for quality (since the University does not have any green sustainability office and three researchers or teachers). The other focus group was organized with different stakeholders from the all University from the different departments (one professor per department chosen according to a convenience sampling), administrative staff (leader of each administrative service) and two students from environmental and sustainability graduation programs (chosen according to a convenience sampling). Detailed information about all the methods applied in UAb available in Reference [73]. These discussion groups were the first time the university stakeholders participated in discussions related to sustainability.
The process of evaluating the implementation of sustainability in the UAM has begun in 2013 and annually assessed (from 2014 to 2018), through collecting the data according to the six dimensions stipulated by GM with different weights each: Setting and Infrastructure (SI) (15%), Setting and Infrastructure (SI) (15%), Waste (WS) (18%), Water (WR) (10%), Transportation (TR) (18%), Education and Research (ED) (18%). All information about the indicators and methodologies are available at Green Metric [50]. The indicators are quantified and filled in a web application (survey type). Each year in a predefined calendar, Green Metrics validates the submitted universities and publishes the international rating scores.
Representatives of the various groups of the university have participated in the contribution of these data: administration and services staff, teacher and manager (see Table 2). They participated through face-to-face interviews and email, together with collecting the available information, for example in corporate reports and on the web page of the University. The GM assessment along the years allowed to observe the evolution of the university in each of the dimensions and to compare with the scores of participating universities worldwide in GM.
For a review of the sustainability implementation and improvement proposals of the university sustainability implementation, two participatory techniques were carried out, focus group and workshop) (see Table 2). The focus group and workshop were then conducted using semi-structured questions based on the four dimensions of STARS tools to allow better comparison with UAb case study (Academics; Engagement, Operations and Planning & Administration). The collection process was in May 2018. Participants were selected based on their crucial role in the university management system, specifically on the sustainability activities at the campus. In the 1st focus group, 4 members managers of the Eco-campus team participated and in the 2nd workshop the 3 Vice-Chancellors of UAM with responsibilities in the sustainability issues were called to participate. In the 3rd focus group, 6 students’ leaders from the Faculty of Business and Economic Sciences participated in the participative process (according to convenience sampling).
In a fourth participatory phase, SDGs LAB—Sustainable Development Goals Workshop was held. The UAM SDGs LAB was organized by the vice-chancellor of Sustainability and Campus and coordinated with professors and researchers in the field and 6 external experts from the town hall staff, already used to collaborate with the university, in a total number of 22. It took place in the UAM Campus for three days. It was the first participatory workshop to design a roadmap to improve the contribution of the university towards SDGs. This workshop was organized in three stages: (I) Inspiration: review of previous experiences and new inspiring ideas, (II) Observation: potential contribution to the campus in situ and proposals, (III) Implementation: discussion, analysis and outcome.
The results of the discussion groups in UAb and UAM were transcribed and then manually analyzed to identify recurring topics in the responses according to Reference [71]. The analyses were validated by the participants.

5. Results and Discussion

5.1. Tools Assessment

In an overall performance, Comprehensibility and Comparability were the criteria higher ranked by the judges (72% and 56% respectively classified as 3, see Table 3). Since the majority of the assessed tools are based on indicators (see Table 1) it is expected that they fulfil these criteria, as well as being a support tool for communication and decision-maker. Nevertheless, is still in doubt of real demonstration if indicators can indeed support decision-maker (only 48% were classified with the higher score in this criterion and also the index of agreement between judges was low—43%). This doubt is in accordance to Ramos [65] in his reflection article. Many of the assessed tools are adaptations to existing ones, often being used to contain geographic specificities (namely from North America, Latina America, Asia and Europe), what make the comparison at the national level easier but more difficult at the international level.
The more complex and complete the tools become, the more complex become to fill them and so the access to the basic data needed (only 32% of the analyzed tools had a maximum data availability rating, such as tools where is only need to answer “yes” or “no” or in the case of multiple-choice closed answers).
The heterogeneity of dimensions of sustainability implementation in HEIs covered by the tools is still remarkable, with the teaching and curriculum dimension and campus operations remaining the best addressed. Also, the environmental pillar is the more addressed, neglecting the social and economic pillars. These results are in accordance with [74], who also found in a study conducted in Spanish Universities that more attention is given to the environmental pillar and that is still necessary to achieve an integrated perspective of sustainability in universities. Those results justify the maximum ranking of only 36% of the tools in the criterion of coverage of sustainability implementation dimension and the same percentage in the criterion of measuring the progress of sustainability implementation in HEIs (see Table 3). Assessment of the progress over time was also the criterion with a higher level of disagreement between the judges (35%). According to Reference [75], variability between judges’ classification is usually expected. Nevertheless, this criterion can be subjective since the assessment along the years can also depend on the HEIs and not only the tool himself. These results are in line with recent studies (e.g., [14,16,32]). Also related to this criterion, the state of development of the tools is revealed by the lack of assessment of impact outside the institution, that means on society in general and long-term impact [76].
The Participation and Accessibility criteria were the ones that have the lowest classification (12% and 24% respectively classified as 3, see Table 3). Concerning the level of agreement between the judges, Accessibility was the criterion that reunited more consensus what is easily explained by the fact that the tool is or not available on the internet to be used (Index of agreement of 77%). Indeed, only a small number of tools have an easily accessible application on the internet to fill in the data and obtain the final result. Participatory approaches have gained increasing attention in the implementation of sustainability in higher education but often remain vague and less addressed in sustainability assessment procedures, as Disterhelft et al. [66] also defended. The policy agenda of ESD calls for innovative and more transformative approaches than reductionist practices, in order to respond better to the need for an institutional learning culture that envisions dialogue and change as stressed by the same authors.
The results of the classification of the tools by the judges (Table 3) highlighted that STARS and USAT have good performance in 6 criteria ranked with maximum classification (both had a sum of 22 points in a maximum of 24). More specifically, STARS ranked medium criteria only in Data access and Participation categories and USAT in Participation and Accessibility criteria. Additionally, from Berzosa et al. [32] experience of “USAT is simple to apply, however, complicates comparations and benchmarking, as there are not any mechanisms to standardize the interviewee and the evaluator.”
Following these tools were GASU, GM, SUM, SLS and GP (a sum of 20 points, see Table 3) with 5 criteria with the highest classification. For example, GASU has being pointed out by other authors (e.g., [7,16,29]) has having the advantages of giving institution visual illustration of sustainability, turning easy to compare and contracts the universities efforts towards sustainability within and among other universities. Also, AISHE was well classified with a sum of 20 points, with 4 criteria ranked with maximum classification (and a high index of agreement between the judges—71%). As discussed by other authors like Berzosa et al. [32] “AISHE score depends on stages of development of sustainability policies and actions within the organization and varies from an activity oriented (1) to a society oriented (5) approach.” In addition, according to Alghamdi et al. [16], AISHE was designed to incorporate only the most significant criteria and not necessarily the whole framework.
Regarding the tools that scored the lowest on the ranking, TUR, HE21 and GMID were the ones with 3 criteria with the lowest classification, where Comparability and Accessibility had poor scores in all three (see Table 3). HE21 [31] is difficult to measure and compare, also indicators may not cover most important issues. ESDGD, P&P and TUR are the tools that had a high divergence in the judgment agreement (44%, 50% and 42% respectively), evidencing a great variability in Understanding, Comparability, Progress Over Time and Support to Decision criteria. TUR and GMID are old tools only available in research papers. P&P, HE21 and ESDGD are tools adapted specifically to be applied in Universities in a geographical region (UK).
Considering the confounding results, THE and GM evidenced a disperse result for the Participation and the Support to Decision criteria, respectively, because the score was divided in the lowest and highest ranking (also with a low index of agreement between judges 50 and 56%, respectively). In GMID, there is a clear outlier in the Accessibility category (since the tool is not available online) and in THE, an outlier for the Comparability category (since it is an international ranking). These facts highlight the importance of the classification being made by several judges and reducing the bias of the results.
Overall, the results confirmed STARS as the tool that collected a high level of agreement among the judges (71%), with the four criteria expressing the same score, Understanding, Comparability, Sustainability Broadness, Support to Decision. Nonetheless, STARS major strength according to Sonetti et al. [31] is its preference for performance over strategy. Earlier studies that reviewed these kinds of tools highlighted that STARS, AISHE and SAQ have a higher incidence on the percentage of indicators for the Governance (in accordance with reference [25]) and Operations, as highlighted by Reference [28] dimensions. However, STARS has the widest coverage across all indicators, capturing a little of all areas compared, for example, with AISHE and SAQ [as also defended by other authors [24,28], SRC (in accordance with Shi and Lai [77]) or with GM [also as highlighted by Lauder et al. [49]. In addition, as authors like Berzosa et al. [32] claimed “the main weaknesses of SAQ are those related with open-ended questions, not establishing a final score so it is difficult to apply it as a tracking tool.”
In a more recent study, STARS appear to be comparable to AISHE and BIQ-AUA, considering the availability of academic and management staff as agents of change in the implementation of sustainability within universities [9]. Analyzing the filing process between STARS and AISHE, it can be seen that the former has a higher percentage of closed questions [24], so it is possible to deduce a greater ease of completeness and greater reliability when comparing the results, thus proving to be more efficient for a regular implementation [14,26,73]. This information is also underlined in a comparative study of STARS with other tools [78]. Authors such as Stough et al. [79] highlight various strengths in STARS namely innovation, understanding, popularity and be based on a holistic and integrative approach to sustainability, while also considering the United Nations Development Goals [79].
GM tool was also overall well classified, besides worst classification in terms of Participation and Progress Over Time criteria but has been also widely used since is free, easy to fill and good as a benchmark between universities worldwide, as stressed by many other authors (e.g., [16,49,74]). As Lauder et al. [49] stressed under scientific analysis, no ranking can be free from at least some limitations, resulting in unavoidable practical considerations, such as the need for the ranking to be at a level of complexity that can appeal to a wide audience. In addition, Sonetti et al. [31] mentioned one of the major weakness of GM is the use of generic quantitative indicators which does not underpin local dimensions as well as lack of the social dimension.

5.2. Application of the Tools in the Case Studies

Universidade Aberta scores for each STARS dimensions are shown in Table 4 and Figure 1 for the period of 2015–2017. The overall scores allowed to awarded the University with a Bronze label on April 5th of 2019 after a three-time external review process by STARS technical staff where some scores rectifications were made after documentation and numbers checked. UAb STARS report is now listed on the website of AASHE. Academic and Research dimensions are the ones where UAb is better classified leaving much room for implementing sustainability in its Planning and Management dimension, unlike in other universities where the focus is more on the Campus Operations [80]. Since UAb is a distance learning institution there is no formal campus for students’ classes and the resources demanding are low (in particular in terms of energy) what justifies that difference. Nevertheless, Campus Operations improvements can be put in place as discussed in the focus groups (see Table 4). However, it is worth mentioning the importance of this approach for recognizing the advantages of the UAb teaching model for sustainability, from the perspective of its social dimension, namely that it facilitates education for all and at all stages. This is evidenced as an innovation according to the STARS criteria [73]. Another strong point that the assessment highlighted is the university bet in sustainability teaching not only at formal programs from 1st, 2nd and 3rd level (bachelor to Ph.D.) but also at non-formal programs like open and massive on-line courses (see Table 4).
Through the assessment of the sustainability in UAb using STARS the key-actors were able to discuss ways of improvement namely new paths to implement sustainability practices (see Table 4). The improvements found are feasible and inexpensive but UAb should integrate sustainability into organizational practices and allocate financial and human resources in the next strategic plan. In particular, the application of STARS can be an important basis for the definition of a currently non-existent University sustainability strategy/policy, one of the fundamental pillars for the whole-school approach, which agrees with other authors [13,32]. In addition, the participatory process that took place during the completion and evaluation of the tool alerted and sensitized the focus group participants on sustainability issues at UAb, contributing to the fact that some of the proposals are already being implemented at the moment. Examples are the registration of the SDG in the resources uploaded in the Open Repository of the University, the development of more moments of joy and sociability (e.g., Christmas lunch) and ongoing process of the dematerialization of the administrative process.
This tool was first used in Portugal and at a distance learning university. Its application in the UAb has also identified some points of better adaptation to European reality (in particular related to units of measurement and benchmarks) and distance learning universities (since these institutions do not have a formal campus with students). These adaptations were communicated to the tool implementation support services of STARS. Earlier studies indicate that e-learning has a lower impact on greenhouse gas emissions and climate change, as observed in the UAb assessment and according to other studies (e.g., [81]) but its direct and indirect impacts on sustainability need to be better studied, as also advocated by Findler et al. [14]. Indeed, the long-term impact of the practices being implemented and the impact of UAb on a more sustainable society are issues that the tool has not been able to assess on its own.
The scores result of the GM assessment from 2014 to 2018 at Universidad Autónoma de Madrid are available in Figure 2 and Table 5. The results showed that UAM is in a good position compared with other worldwide universities, namely a 55th position in a total number of 719 in 2018. In 2018 UAM reached the highest scores due to considerable improvement in the Education and Infrastructures dimension but in 2014 its performance was better in terms of Water, Waste, Energy and Transportation. These results do not exactly mean the worst performance in the following years since the number of indicators related to these dimensions changed in the GM tool. An overall 34th position in the ranking of 2014 was obtained in a total of 361 universities ranked (corresponding to half of the universities compared to 2018).
According to the Vice-chancellors focus group (2nd focus group—Table 2), one critical area where the university should make an extra effort is the outreach, highlighting that university is a place to develop sciences, knowledge and new solutions able to contribute and affect to the society, local and global community. This factor is not considered in this tool, although considered in STARS. Participants from this 2nd focus group also highlighted the importance of undertaking the SDGs agenda in the university activities towards sustainability.
For the student focus group, a key factor to be included in the future is “the communication strategy” to foster the environmental message in the university community. This item is partly included in UI Green Metrics, in the Education and Research criteria, which requires the existence of a university-run sustainability website and the existence of a published sustainability report (Green Metrics [50]). This improvement was also highlighted in the UAb case study where it was proposed for better engagement of all university communities the development of an online collaborative platform for sustainable ideas (see Table 4).
Common barriers for sustainability implementation in both Southern European institutions are related to low levels of community participation and sustainability awareness, financial constraints and a lack of HEIs legislative framework. Those barriers are also found in earlier studies (e.g., [76]). Some weaknesses found in UAb are also found in UAM, namely in terms of SDG disclosure and low sustainability awareness from the university community and lack of financial support. Of course, in terms of campus operations, both universities’ reality is quite different due to their different sizes and regime of teaching. Nevertheless, they can learn from each other. For example, UAM, in terms of the Academic dimension can use teaching methods more based on new technologies like the ones used in distance learning at UAb, allowing also more transdisciplinary research and less theory (in accordance with authors like Lozano et al. [15]). UAM can also develop more lifelong learning programs or open courses to increase society sustainability awareness, like MOOCs, learning from the experience of UAb. In addition, UAb can learn from the experiences in terms of the SDG LAB and Eco campus initiatives that UAM uses and improve their worst performance in terms of Campus Operation. Even that UAb is a distance learning university without face to face students at the campus it has facilities where energy, water and waste measures can be implemented, learning from UAM experiences. More recently, at both Universities questionnaires are being developed to the students to understand their sustainability perceptions, engagement and motivations. With the questionnaires results new strategies can be put in place.
Looking into Figure 2 it can be seen that GM dimensions are mainly focused on environmental initiatives and actions on campus, hence, there are scarce indicators of policies, management, diversity, equity and community participation, what is in accordance with several authors (e.g., [17,29]). Also, this ranking’s main weakness could be considered that the information provided by the universities many times do not include evidence or in-situ verification. According to the experiences of the Eco campus focus group, the information sometimes can be considered subjective according to the understanding of the person in charge to provide the data. In the case of STARS, the data is validated by the rector team and also externally certified (like in UAb case study). However, the main strength is the accessibility and comparability of the outcome among the years (also in accordance with other authors like Lauder et al., [49]). The results from all university participants are available in the Green Metrics, easily accessible to compare by regions, countries and universities. From the benchmarking point of view, it could also facilitate the decision-making process for managers based on the potential analysis of the progress on these factors. The level of understanding is high and the authors of the GreenMetrics have provided the criteria, indicators and methodology used on their website (see for example the 2018 guide [50]).

5.3. Overall Discussion

The tools critical analysis on the tools and cases studies application, still raise the question of the effect of implementing these tools versus the actual integration of sustainability into an HEI. The integration of sustainability in HEIs should come along with the modification of existing structures and habits, which creates many challenges related to the involved actors, the available resources, values and strategic choices to be made [82,83]. Also, based on Kapitulcinova et al. [9], the “transformational change” should occur at the level of the entire HE. It is therefore essential that a critical mass of units comprising the institution adopt sustainable development principles in their respective tasks and duties. In addition, according to Alonso-Almeida et al. [84] and Beringer et al. [85], to achieve the sustainability maturation, the sustainability integration at HEIs should involve all dimensions into a whole-school approach.
For all those reasons it is considered that the implementation of integrative approaches and models still needs to be encouraged and further research is needed [9,14,31,76]. Regardless of this fact, the case studies demonstrated that the implementation of the tools make possible to assess the state of implementation of sustainability in HEIs, monitor it, communicate it, share it within and outside the organization and improve and stimulate change, often enabling low-cost measures to be implemented. These statements are also advocated by other authors (e.g., [16,24,28,32]).
A change process enforces an overall vision, an increasing need for change that is experienced by the stakeholders, resources to support the process and short-term gains that can be communicated [86]. In this research the application of the assessment tools was complemented with participatory processes. While participatory methods are not commonly considered in these tools, they are central support for more holistic implementation allowing for the best man-nature link and a reflection that can better respond to the transformation of the institution and individual towards sustainability [66]. These tools must have a stronger component of student community participation and involvement, as this community is a major agent of change. Given the characteristics of experimentation and research of HEIs, indicators that can be incorporated into these tools or that can be used independently, should be tested to allow a long-term evaluation of whether the transformation process has been successfully achieved. Disterheft et al. [66] suggested examples of these types of indicators, based on perspectives such as the whole-school approach, interconnection between man and nature, community cohesion, celebration and happiness and principles of democracy. As argued by Ramos [65] the challenges in the area of sustainability assessment indicators should be based on transdisciplinary, collaborative and innovative scientific development where communities and the individual play a central role. Greater emphasis should be given to the development of indicators to assess non-traditional aspects of sustainability, such as ethics, culture and art, aesthetics, governance efficiency, spirituality, solidarity, compassion and trust, which represent fewer tangible dimensions of society [65,87].
These tools are also too operational not evaluating the strategic processes, as also stressed by Arroyo [33]. being able to incorporate the unpredictable and not only knowing how to deal with linear problems but also being able to assess what external impact HEIs have in practice on sustainability and going beyond the limits of HEIs [14]. As an example, the impact assessment of the research that is developed in HEIs on ESD should be carried out. Since this cannot be done based on a simple citation counting and bibliometric analysis because they do not allow to accurately define the result of this same investigation in the SD, more qualitative and documentary analyses are necessary for a more robust evaluation [14].
Few studies have explored and evaluated the role of HEIs as agents of change [31]. Also, there is a weak link between HEIs and external networks and key actors and with local and regional policies, what does not contribute to change in organizational management [82], holistic integration of sustainability in HEIs and their impact abroad [14,88]. The external impact of implementing sustainable development in HEIs can be measured in the local economy and culture, in challenges in society, in the natural environment and in the policies. Impacts can be short-term and direct, such as by training skilled workers or reducing greenhouse gases in the facilities but also indirect and long-term, such as changing graduates’ sustainable lifestyles or in the implementation of Sustainable Development Goals (based on [14] suggestions).

6. Conclusions

A high increase has been seen in the implementation of sustainable development principles into Higher Education Institutions and in the research to assess its performance. In this article, a critical analysis of the existing tools to assess and benchmark ESD implementation was conducted by four judges according to a list of criteria. There are limitations associated with these kinds of critical review and qualitative evaluation but those limitations were weighed up in the mains findings. The tools applicability was then discussed in two case studies. The tool STARS—Sustainability Tracking, Assessment & Rating System was the one that was better classified in terms of understanding, comparability, sustainability broadness and support to decision, being one of the tools more worldwide used by different HEIs. The tool STARs was also used in one of the case studies. So, the use of STARS could be a good choice for HEIs that would like to assess and benchmark their sustainability performance according to a holistic and integrated approach.
The tools currently available for evaluating sustainability initiatives in HEIs do not all have the same objective or do not homogeneously evaluate the implementation of sustainability. As a consequence, some ambiguity is translated to its actual implementation and real contribution to the transformation for change. It is therefore recommended to set common general sustainability objectives in HEIs considering the integrative whole-school approach, regardless of some regions of institutions specificities (which may be indicated by specific objectives). Some improvements to the tools were discussed in this research and suggestions for future research. Sustainability assessment in HEIs should be viewed as a social construction, emerging from the different partners involved and according to mixed, bottom-up, top-down approaches, where the various actors, internal and external, contribute positively to the implementation of the sustainability assessment tool.
The implementation of Sustainability assessment tools in the HEIs case studies has shown that its use is an important driving force for the first diagnosis, a source for defining ways of improvement and also for future changes in organizational management. Also, the application of the tools in two HEIs of neighbor countries complemented with participation activities with key-actors, allowed to show the weakness of these tools namely the lack of the assessment of the impact of the sustainability performance of the university in the society and their real contribution for a sustainability transition. Bearing in mind that both countries have some similarities, namely cultural, this collaboration also brought up difficulties that need to be addressed in order to apply and implement EDS successfully. Barriers were identified and examples of improvement were given. Nevertheless, both universities learned from each other besides their differences and future developments for their sustainable performance improvement are already in place.
This article adds new insights about the main characteristic, common designations and objectives this type of tools should have and defined a list of criteria for their evaluation, trying to uniformize their meaning. Also, this research gives suggestions about improvements that are still needed on these tools so they can more fully answer their main purpose. Improvements can be in terms of incorporating in the tools an integrated process of stakeholder’s participation, adding of indicators to assess non-traditional aspects of sustainability and being able to assess what external impact HEIs have in practice on sustainability.

Author Contributions

S.C. led the process of the tools review and assessment and overall discussion of the results. R.M. and S.C. developed the case study in UAb and wrote the respective methods and results. L.A.S.H. and C.E.B.A. developed the literature review and case study in UAM and wrote the respective methods and results. All the authors classified the tools according to the listed criteria and contribute for the article writing, revision and approval of the manuscript. All authors have read and agreed to the published version of the manuscript.


CENSE is financed by Fundação para a Ciência e Tecnologia, I.P., Portugal (UID/AMB/04085/2019). The funding sources played no part in the design, analysis, interpretation or writing-up of the paper or in the decision to publish.


The authors wish to thank all participants in this research in the different focus groups and workshops held in UAb and UAM, in particular Ana Paula Martinho and Teresa Teixeira in the data collection at UAb. The authors want also to thanks the rectorate support on the assessment processes in each university, in particular to Vice-cancellor for Qualaty Carla Oliveira in UAb and Santiago Atrio Cerezo Vice-cancellor for Sustainability and Campus and Jesús Rodríguez Pomeda, Vice-cancellor of Strategy and Planning in UAM.

Conflicts of Interest

The authors declare no conflicts of interest.


  1. Mora, H.; Pujol-López, F.A.; Mendoza-Tello, J.C.; Morales-Morales, M.R. An education-based approach for enabling the sustainable development gear. Comput. Hum. Behav. 2018. [Google Scholar] [CrossRef]
  2. Wheeler, S.M. State and Municipal Climate Change Plans: The First Generation. J. Am. Plan. Assoc. 2008, 74, 481–496. [Google Scholar] [CrossRef]
  3. United Nations. World Commission on Environment and Development. In Our Common Future; A/42/427; United Nations: New York, NY, USA, 1987. [Google Scholar]
  4. U.N. Transforming Our World: The 2030 Agenda for Sustainable Development; A/RES/70/1; United Nations: New York, NY, USA, 2016. [Google Scholar]
  5. UNESCO. Education for Sustainable Development. Source Book; United Nations Educational, Scientific and Cultural Organization: Paris, France, 2012. [Google Scholar]
  6. Lozano, R.; Ceulemans, K.; Seatter, C.S. Teaching organisational change management for sustainability: Designing and delivering a course at the University of Leeds to better prepare future sustainability change agents. J. Clean. Prod. 2015, 106, 205–215. [Google Scholar] [CrossRef]
  7. Lozano, R. Incorporation and institutionalization of SD into universities: Breaking through barriers to change. J. Clean. Prod. 2006, 14, 787–796. [Google Scholar] [CrossRef]
  8. Velazquez, L.; Munguia, N.; Platt, A.; Taddei, J. Sustainable university: What can be the matter? J. Clean. Prod. 2006, 14, 810–819. [Google Scholar] [CrossRef]
  9. Kapitulcinova, D.; AtKisson, A.; Perdue, J.; Will, M. Towards integrated sustainability in higher education—Mapping the use of the Accelerator toolset in all dimensions of university practice. J. Clean. Prod. 2018, 172, 4367–4382. [Google Scholar] [CrossRef]
  10. Trencher, G.; Nagao, M.; Chen, C.; Ichiki, K.; Sadayoshi, T.; Kinai, M.; Kamitani, M.; Nakamura, S.; Yamauchi, A.; Yarime, M.; et al. Implementing Sustainability Co-Creation between Universities and Society: A Typology-Based Understanding. Sustainability 2017, 9, 594. [Google Scholar] [CrossRef][Green Version]
  11. Lozano, R. Diffusion of sustainable development in universities’ curricula: An empirical example from Cardiff University. J. Clean. Prod. 2010, 18, 637–644. [Google Scholar] [CrossRef]
  12. Lozano, R.; Lukman, R.; Lozano, F.J.; Huisingh, D.; Lambrechts, W. Declarations for sustainability in higher education: Becoming better leaders, through addressing the university system. J. Clean. Prod. 2013, 48, 10–19. [Google Scholar] [CrossRef]
  13. Lozano, R.; Ceulemans, K.; Alonso-Almeida, M.; Huisingh, D.; Lozano, F.J.; Waas, T.; Lambrechts, W.; Lukman, R.; Hugé, J. A review of commitment and implementation of sustainable development in higher education: Results from a worldwide survey. J. Clean. Prod. 2015, 108, 1–18. [Google Scholar] [CrossRef]
  14. Findler, F.; Schönherr, N.; Lozano, R.; Stacherl, B. Assessing the Impacts of Higher Education Institutions on Sustainable Development—An Analysis of Tools and Indicators. Sustainability 2018, 11, 59. [Google Scholar] [CrossRef][Green Version]
  15. Lozano, R.; Barreiro-Gen, M.; Lozano, F.J.; Sammalisto, K. Teaching Sustainability in European Higher Education Institutions: Assessing the Connections between Competences and Pedagogical Approaches. Sustainability 2019, 11, 1602. [Google Scholar] [CrossRef][Green Version]
  16. Alghamdi, N.; den Heijer, A.; de Jonge, H. Assessment tools’ indicators for sustainability in universities: An analytical overview. Int. J. Sustain. High. Educ. 2017, 18, 84–115. [Google Scholar] [CrossRef]
  17. Shriberg, M. Institutional assessment tools for sustainability in higher education. Int. J. Sustain. High. Educ. 2002, 3, 254–270. [Google Scholar] [CrossRef][Green Version]
  18. Ramos, T.; Pires, M.S. Sustainability Assessment: The Role of Indicators. In Sustainability Assessment Tools in Higher Education—Mapping Trends and Good Practices at Universities around the World; Caeiro, S., Leal Filho, W., Jabbour, C., Azeiteiro, U., Eds.; Springer: Cham, Switzerland, 2013; pp. 81–100. [Google Scholar]
  19. Nixon, A. Improving the Campus Sustainability Assessment Process. Bachelor’s Thesis, Western Michigan University, Kalamazoo, MI, USA, 2002. [Google Scholar]
  20. Cole, L. Assessing Sustainability on Canadian University Campuses: Development of a Campus Sustainability Assessment Framework. Master’s Thesis, Royal Roads University, Victoria, BC, Canada, 2003. [Google Scholar]
  21. Alshuwaikhat, H.M.; Abubakar, I. An integrated approach to achieving campus sustainability: Assessment of the current campus environmental management practices. J. Clean. Prod. 2008, 16, 1777–1785. [Google Scholar] [CrossRef][Green Version]
  22. Larouce, D.C. Tracking progress: Development and Use of Sustainability Indicators in Campus Planning and Management; Arizona State University: Phoenix, AZ, USA, 2009. [Google Scholar]
  23. Implementing Sustainability at the Campus—Towards a Better Understanding of Participation Processes within Sustainability Initiatives; Sustainable Development at Universities: New Horizons. Umweltbildung, Umweltkommunikation und Nachhaltigkeit—Environmental Education, Communication and Sustainability; Peter Lang: Frankfurt, Germany, 2012; Volume 34, pp. 345–361.
  24. Yarime, M.; Tanaka, Y. The Issues and Methodologies in Sustainability Assessment Tools for Higher Education Institutions: A Review of Recent Trends and Future Challenges. J. Educ. Sustain. Dev. 2012, 6, 63–77. [Google Scholar] [CrossRef]
  25. Kosta, K. Institutional Sustainability Assessment. In Encyclopedia of Sustainability in Higher Education; Leal Filho, W., Ed.; Springer Nature Switzerland AG: Cham, Switzerland, 2019; pp. 1–7. [Google Scholar] [CrossRef]
  26. Sayed, A.; Kamal, M.; Asmuss, M. Benchmarking tools for assessing and tracking sustainability in higher educational institutions. Int. J. Sustain. High. Educ. 2013, 14, 449–465. [Google Scholar] [CrossRef]
  27. Ceulemans, K.; Molderez, I.; Van Liedekerke, L. Sustainability reporting in higher education: A comprehensive review of the recent literature and paths for further research. J. Clean. Prod. 2015, 106, 127–143. [Google Scholar] [CrossRef]
  28. Fischer, D.; Jenssen, S.; Tappeser, V. Getting an empirical hold of the sustainable university: A comparative analysis of evaluation frameworks across 12 contemporary sustainability assessment tools. Assess. Eval. High. Educ. 2015, 40, 785–800. [Google Scholar] [CrossRef][Green Version]
  29. Gómez, F.; Sáez-Navarrete, C.; Lioi, S.; Marzuca, V. Adaptable Model for Assessing Sustainability in Higher Education. J. Clean. Prod. 2015, 107, 475–485. [Google Scholar] [CrossRef]
  30. Bullock, G. The comprehensiveness of competing higher education sustainability assessments. Int. J. Sustain. High. Educ. 2016, 17, 282–304. [Google Scholar] [CrossRef]
  31. Sonetti, G.; Lombardi, P.; Chelleri, L. True Green and Sustainable University Campuses? Toward a Clusters Approach. Sustainability 2016, 8, 83. [Google Scholar] [CrossRef][Green Version]
  32. Berzosa, A.; Bernaldo, M.O.; Fernández-Sanchez, G. Sustainability assessment tools for higher education: An empirical comparative analysis. J. Clean. Prod. 2017, 161, 812–820. [Google Scholar] [CrossRef][Green Version]
  33. Arroyo, P. A new taxonomy for examining the multi-role of campus sustainability assessments in organizational change. J. Clean. Prod. 2017, 140, 1763–1774. [Google Scholar] [CrossRef]
  34. National Wildlife Federation. Campus Environmental Report Card. Available online: (accessed on 5 October 2018).
  35. Smith, A.A. The Student Environmental Action Coalition. In Campus Ecology: A Guide to Assessing Environmental Quality and Creating Strategies for Change; Living Planet Press: Los Angeles, CA, USA, 1993. [Google Scholar]
  36. Chernushenko, D. Greening Campuses: Environmental Citizenship for Colleges and Universities; International Institute for Sustainable Development: Winnipeg, MB, Canada, 1996. [Google Scholar]
  37. Decamps, A.; Barbat, G.; Carteron, J.-C.; Hands, V.; Parkes, C. Sulitest: A collaborative initiative to support and assess sustainability literacy in higher education. Int. J. Manag. Educ. 2017, 15, 138–152. [Google Scholar] [CrossRef]
  38. GULF; ISCN. Implementation Guidelines to the ISCN-GULF Sustainable Campus Charter. Suggested Reporting Contents and Format; Global University Leader Forum (GULF); International Campus Sustainability Network (ICSN): Lausanne, Switzerland, 2016. [Google Scholar]
  39. Roorda, N. AISHE, Assessment Instrument for Sustainability in Higher Education; Atichting Duurzaam Hoger Onderwijs (DHO): Amsterdam, The Netherlands, 2001. [Google Scholar]
  40. Roorda, N.; Rammel, C.; Waara, S.; Fra Paleo, U. AISHE 2.0 Manual: Assessment Instrument for Sustainability in Higher Education. Available online: (accessed on 5 October 2018).
  41. CAS-NET Japan. Good Practices on Campus Sustainability in Japan. Available online: (accessed on 5 October 2018).
  42. CRUE; CSC. Evaluación de las Políticas Universitarias de Sostenibilidad como Facilitadoras Para el Desarrollo de los Campus de Excelencia Internacional; Conferencia de Rectores de las Universidades Españolas; Comisión Sectorial de Calidad: Madrid, Spain, 2011. [Google Scholar]
  43. CRUE; CSC. Diagnóstico De La Sostenibilidad Ambiental En Las Universidades Españolas; Conferencia de Rectores de las Universidades Españolas; Comisión Sectorial de Calidad: Madrid, Spain, 2018. [Google Scholar]
  44. ProSPER.Net. Alternative University Appraisal Model for ESD in Higher Education Institutions. Available online: (accessed on 5 October 2018).
  45. Razak, D.; Sanusi, Z.; Jegatesen, G.; Khelghat-Doost, H. Alternative University Appraisal (AUA). Reconstructing Universities’ Ranking and Rating Toward a Sustainable Future. In Sustainability Assessment Tolls in Higher Education in Higher Education Institutions: Mapping Trends and Good Practices Around the World; Caeiro, S., Leal Filho, W., Jabbour, C., Azeiteiro, U.M., Eds.; Springer: Cham, Switzerland, 2013; pp. 139–158. [Google Scholar]
  46. CITE-AMB M; ASCUN; RISU; RAUS; PNUMA; RFA-ALC; GUPES-LA; ARIUSA. Encuesta para el Diagnóstico de la Institucionalización Del Compromiso Ambiental En Las Universidades Colombianas; Red de Ciencia, Tecnología, Innovación y Educación Ambiental en Iberoamérica, Ministerio de Ambiente y Desarrollo Sostenible, Asociación Colombiana de Universidades, Red de Indicadores de Sostenibilidad en las Universidades, Red Ambiental de Universidades Colombianas, Programa de las Naciones Unidas para el Medio Ambiente, Red de Formación Ambiental para América Latina y el Caribe, Alianza Mundial de Universidades por el Ambiente y la Sostenibilidad and Alianza de Redes Iberoamericanas de Universidades por la Sustentabilidad y el Ambiente: New York, NY, USA, 2014. [Google Scholar]
  47. Glover, A.; Jones, Y.; Claricoates, J.; Morgan, J.; Peters, C. Developing and Piloting a Baselining Tool for Education for Sustainable Development and Global Citizenship (ESDGC) in Welsh Higher Education. Innov. High. Educ. 2012, 38. [Google Scholar] [CrossRef]
  48. Good Company. Sustainable Pathways Toolkit for Universities and Colleges: Social and Environmental Indicators for Campuses. Part II: Toolkit Technical Manual; Good Company: Eugene, OR, USA, 2002. [Google Scholar]
  49. Lauder, A.; Sari, R.F.; Suwartha, N.; Tjahjono, G. Critical review of a global campus sustainability ranking: GreenMetric. J. Clean. Prod. 2015, 108, 852–863. [Google Scholar] [CrossRef]
  50. UI Green Metric. Available online: (accessed on 5 October 2018).
  51. Mader, C. Sustainability process assessment on transformative potentials: The Graz Model for Integrative Development. J. Clean. Prod. 2013, 49, 54–63. [Google Scholar] [CrossRef]
  52. CdGE & CdPdU, Plan Vert—Version 2012. Available online: (accessed on 5 October 2018).
  53. Buckland, H.; Brookes, F.; Seddon, D.; Johnston, A.; Parkin, S. The UK Higher Education Partnership for Sustainability (HEPS). 2001. Available online: (accessed on 5 October 2018).
  54. Penn State Green Destiny Council. Penn State Indicators Report 2000: Steps toward A Sustainable University; Penn State Green Destiny Council: State College, PA, USA, 2000. [Google Scholar]
  55. People & Planet University League. How Sustainable Is Your University? Available online: (accessed on 5 October 2018).
  56. ULSF. Sustainability Assessment Questionnaire (SAQ). Available online: (accessed on 5 October 2018).
  57. Sustainable Endowments Institute. The College Sustainability Report Card. Available online: (accessed on 5 October 2018).
  58. AASHE. STARS Technical Manual—Version 2.1 Administrative Update Three; Association for the Advancement of Sustainability in Higher Education: Philadelphia, PA, USA, 2017. [Google Scholar]
  59. EUAC. Sustainability Leadership Scorecard—Launching a Transformational Tool for the Education Sector —The Sustainability Leadership Scorecard. Available online: (accessed on 5 October 2018).
  60. Washington, U. The Program Sustainability Assessment Tool v2. Available online: (accessed on 5 October 2018).
  61. Times, The World University Ranking. The University Impact Ranking. Available online: (accessed on 5 October 2018).
  62. Lukman, R.; Krajnc, D.; Glavič, P. University ranking using research, educational and environmental indicators. J. Clean. Prod. 2010, 18, 619–628. [Google Scholar] [CrossRef]
  63. Togo, M.; Lotz-Sisitka, H. Unit Based Sustainability Assessment Tool; A Resource Book to Complement the UNEP Mainstreaming Environment and Sustainability in African Universities Partnership; UNEP: Howick, South Africa, 2009. [Google Scholar]
  64. Waheed, B.; Khan, F.; Veitch, B.; Hawboldt, K. Uncertainty-based quantitative assessment of sustainability for higher education institutions. J. Clean. Prod. 2011, 19, 720–732. [Google Scholar] [CrossRef]
  65. Ramos, T. Sustainability Assessment: Exploring the Frontiers and Paradigms of Indicator Approaches. Sustainability 2019, 11, 824. [Google Scholar] [CrossRef][Green Version]
  66. Disterheft, A.; Caeiro, S.S.; Leal Filho, W.; Azeiteiro, U.M. The INDICARE-model—Measuring and caring about participation in higher education’s sustainability assessment. Ecol. Indic. 2016, 63, 172–186. [Google Scholar] [CrossRef]
  67. Zaini, R.M.; Pavlov, O.V.; Saeed, K.; Radzicki, M.J.; Hoffman, A.H.; Tichenor, K.R. Let’s Talk Change in a University: A Simple Model for Addressing a Complex Agenda. Syst. Res. Behav. Sci. 2017, 34, 250–266. [Google Scholar] [CrossRef]
  68. Yuan, X.; Zuo, J.; Huisingh, D. Green Universities in China—What matters? J. Clean. Prod. 2013, 61, 36–45. [Google Scholar] [CrossRef]
  69. UN. People’s Sustainability Treaty on Higher Education; United Nations: New York, NY, USA, 2012. [Google Scholar]
  70. Salvioni, D.M.; Franzoni, S.; Cassano, R. Sustainability in the Higher Education System: An Opportunity to Improve Quality and Image. Sustainability 2017, 9, 914. [Google Scholar] [CrossRef][Green Version]
  71. Saunders, M.; Lewis, P.; Thornhill, A. Research Methods for Business Students, 6th ed.; Pearson Education Limited: England, UK, 2012. [Google Scholar]
  72. Hernández Sampieri, R.; Férnandez Collado, C.; Baptista Lucio, P. Metodologia De Pesquisa [Research Methodology]; McGraw-Hill: São Paulo, Brazil, 2006. [Google Scholar]
  73. Martins, R. Ferramentas de Avaliação Da Sustentabilidade no Ensino Superior: A adequação ao Contexto de uma Universidade a Distância. Master’s Thesis, Universidade Aberta, Lisboa, Portugal, 2019. [Google Scholar]
  74. Blasco, N.; Brusca, I.; Labrador, M. Assessing Sustainability and Its Performance Implications: An Empirical Analysis in Spanish Public Universities. Sustainability 2019, 11, 5302. [Google Scholar] [CrossRef][Green Version]
  75. Krippendorff, K. Content Analysis. An Introduction to Its Methodology; Sage: Thousand Oaks, CA, USA, 2004. [Google Scholar]
  76. Leal Filho, W.; Wu, Y.-C.J.; Brandli, L.L.; Avila, L.V.; Azeiteiro, U.M.; Caeiro, S.; Madruga, L.R.D.R.G. Identifying and overcoming obstacles to the implementation of sustainable development at universities. J. Integr. Environ. Sci. 2017, 14, 93–108. [Google Scholar] [CrossRef]
  77. Shi, H.; Lai, E. An alternative university sustainability rating framework with a structured criteria tree. J. Clean. Prod. 2013, 61, 59–69. [Google Scholar] [CrossRef]
  78. Maragakis, A.; van den Dobbelsteen, A. Sustainability in Higher Education: Analysis and Selection of Assessment Systems. J. Sustain. Dev. 2015, 8, 1–9. [Google Scholar] [CrossRef]
  79. Stough, T.; Ceulemans, K.; Lambrechts, W.; Cappuyns, V. Assessing sustainability in higher education curricula: A critical reflection on validity issues. J. Clean. Prod. 2018, 172, 4456–4466. [Google Scholar] [CrossRef]
  80. Lidstone, L.; Wright, T.; Sherren, K. An analysis of Canadian STARS-rated higher education sustainability policies. Environ. Dev. Sustain. 2014, 17, 259–278. [Google Scholar] [CrossRef]
  81. Versteijlen, M.; Perez Salgado, F.; Janssen Groesbeek, M.; Counotte, A. Pros and cons of online education as a measure to reduce carbon emissions in higher education in the Netherlands. Curr. Opin. Environ. Sustain. 2017, 28, 80–89. [Google Scholar] [CrossRef]
  82. Verhulst, E.; Lambrechts, W. Fostering the incorporation of sustainable development in higher education. Lessons learned from a change management perspective. J. Clean. Prod. 2015, 106, 189–204. [Google Scholar] [CrossRef]
  83. Hugé, J.; Mac-Lean, C.; Vargas, L. Maturation of sustainability in engineering faculties—From emerging issue to strategy? J. Clean. Prod. 2018, 172, 4277–4285. [Google Scholar] [CrossRef]
  84. Alonso-Almeida, M.; Marimon, F.; Casani, F.; Rodriguez-Pomeda, J. Diffusion of sustainability reporting in universities: Current situation and future perspectives. J. Clean. Prod. 2015, 106, 144–154. [Google Scholar] [CrossRef]
  85. Beringer, A.; Adomßent, M. Sustainable university research and development: Inspecting sustainability in higher education research. Environ. Educ. Res. 2008, 14, 607–623. [Google Scholar] [CrossRef]
  86. Barth, M. Many roads lead to sustainability: A process-oriented analysis of change in higher education. Int. J. Sustain. High. Educ. 2013, 14, 160–175. [Google Scholar] [CrossRef]
  87. Viegas, O.; Caeiro, S.; Ramos, T. A Conceptual Model for the integration of Non-Material components in Sustainability Assessment. Ambiente Soc. 2018, 21, 1–20. [Google Scholar] [CrossRef]
  88. Vargas, V.R.; Lawthom, R.; Prowse, A.; Randles, S.; Tzoulas, K. Sustainable development stakeholder networks for organisational change in higher education institutions: A case study from the UK. J. Clean. Prod. 2019, 208, 470–478. [Google Scholar] [CrossRef]
Figure 1. Results of the Sustainability Tracking tool, Assessment & Rating System (STARS) in the six dimensions for Universidade Aberta (UAb) in the period of 2015–2017. Maximum scores calculated based on version of STARS 2.1 [58].
Figure 1. Results of the Sustainability Tracking tool, Assessment & Rating System (STARS) in the six dimensions for Universidade Aberta (UAb) in the period of 2015–2017. Maximum scores calculated based on version of STARS 2.1 [58].
Sustainability 12 00543 g001
Figure 2. Scores of GreenMentrics for Universidad Autónoma de Madrid from 2014 to 2018 in total and in each dimension and also international ranking compare to total of other universities. Maximum scores calculated based on version of Green Metrics 2018 [50].
Figure 2. Scores of GreenMentrics for Universidad Autónoma de Madrid from 2014 to 2018 in total and in each dimension and also international ranking compare to total of other universities. Maximum scores calculated based on version of Green Metrics 2018 [50].
Sustainability 12 00543 g002
Table 1. Sustainability Assessment Tools in Higher Education (HE) and brief description.
Table 1. Sustainability Assessment Tools in Higher Education (HE) and brief description.
Tool and SourceBrief Description and Region Where Applicable
AISHE 2.0 Assessment Instrument for Sustainability in Higher Education.
(Latest version) AISHE 2.0 Assessment Instrument for Sustainability in Higher Education [39,40]
Based on narrative and indicators: 30 indicators, 5 dimensions (Operations, Education, Research, Society, Identity), less emphasis on the environmental component (just 1 indicator); incorporating the Deming cycle approach; the intended target is the university system; with a wide world application across the universities; the application domain adapts according to the university structure as an entire university, campus, buildings or research institute; AISHE first version was developed in 2000 and 2001 only focused in the educational role of universities, however, AISHE 2.0 has a wider scope in terms of the research, operations and relation with the society;
Developed by a researcher in Europe;
Tool not available online, only the manual.
The Netherlands / international
AMAS—Adaptable Model for Assessing, Sustainability in Higher Education [29]Based on 3 domains (Institutional commitment, Leadership, Advanced sustainability); with 4 levels of application hierarchy that lead to the use of standardized indicators (based on other existing tools), with different weights and key actors’ participation, allowing to be adapted by each institution but comparable in the same country; With an expert consultation system;
Developed by a researcher in Chile;
Tool not available online.
ASSC—Assessment System for Sustainable Campus [41]Based on a questionnaire and reported on graphical form; 26 indicators; 4 dimensions (Management, Education and Research, Environment, Local Community); based on other tools (STARS, Uni-metrics, GM, BIQ - AUA). Rating system with 4 levels, allowing to obtain a certification: platinum, gold, silver and bronze; It provides information on the strengths and weaknesses of implementing sustainability in HEIs and helps them decide future strategies; includes specificities of the country where it was developed (e.g., natural disasters);
Developed by Hokkaido University in 2013, within CAS-NET JAPAN (Campus Sustainability Network in Japan) but used in other universities in Japan;
Tool available online, (see Reference [41]).
AUSP—Evaluación de las políticas universitarias de sostenibilidad como facilitadoras para el desarrollo de los campus de excelencia internacional [29,42,43]Based on 4 areas (Organization, Teaching, Research, Environmental management); less emphasis on the social component; 176 indicators; data collection by questionnaire and interviews (self-assessment) and reviewed by an external organization; the purpose is to improve performance and policies in terms of social responsibility, environment (including public procurement), economy and implementation of the Sustainable Development Goals; graphical representation of indicators; with several updates (last in 2018);
Developed by the De La Crue Sectoral Commission for Environmental Quality, Sustainable Development and Risk Prevention of CRUE - Conference of Rectors of Spanish Universities specifically for HEIs in Spain and tested in several Spanish Universities;
Questionnaire available online, (see Reference [42]).
BIQ-AUA—Alternative University Appraisal [29,44,45]Based on self-assessment with questions to calculate indicators for benchmark (BIQ) and Dialogue. BIQ has a special focus on governance, education, research and communication. It is divided into 4 categories, 15 sub categories (with equal weight) and includes 30 indicators and 50 questions; does not include environmental management and social responsibility indicators; the highest rating is 100, thus allowing comparison; dialogue is the component that enables institutions to share their concerns, best practices and learning about ESD. Applied to 28 universities in Asia and the Pacific;
Developed by ProSPER (Promotion of Sustainability in Postgraduate Education and Research Network), an academic alliance between Asia and the Pacific. It is composed of three components based on the evaluation according to the United Nations Decade for Sustainable Development;
Tool not available online
CITE AMB—Red de Ciencia, Tecnologia, Innovacion y Educación Ambienal em Iberoamerica [46]Based on a questionnaire with 27 questions (Yes/No answers); 4 areas (Management, Research, Education, Community), without focusing on the environmental component of campus infrastructure and social component;
Developed by the Network of Science, Technology, Innovation and Environmental Education in Iberoamerica;
Not available online (existence of a link in the report for “google docs” but is no longer available), developed in 2014 but with no updates available.
DUK—German Commission for UNESCO
AG HS (2011) fidé [14,28]
Based on indicators in 4 areas (Operations, Research, Education, Community); With a strong focus on the institutional part, the tool operates as moderator in the whole-school approach. It contains 10 action fields and each one offers 5 stages of implementation, allowing the HEIs option;
Developed by the German Commission for UNESCO in 2011 for the German context;
Tool not available online; only a report about the tool in German is available.
ESDGC—Education for Sustainable Development and Global Citizenship [9,47]Based on a ranking system with 5 areas (Commitment and leadership, Teaching and learning, Institutional Management, Partnerships, Research and Monitoring) and the categorization of 4 levels; adaptation of a maturity model and training usually applied to companies and the industrial sector; results with a semaphore system;
Developed specifically for HEIs in Wales, UK and outlined by the Government of Wales to enable an assessment of implementation of ESD in Universities;
Tool not available online.
Wales / United Kingdom
GASU—Graphical Assessment of Sustainability in Universities tool [7]Based on GRI report with adaptations to HEIs; applied in many universities, 8 dimensions (Direct economic impact, Environmental, Labor practices and decent work, Human rights, Society, Product liability, Curricula, Research), up to 126 indicators; graphical presentation of results;
Developed by a researcher in Europe and marketed through a company;
Tool not available online for free, only with a fee payment.
United Kingdom / international
GC—Good Company’s Sustainable Pathways Toolkit [24,48]Based on 20 key performance indicators plus an additional 10 indicators; more emphasis on campus operations; without focusing 2 categories of sustainability implementation in HEIs, namely, Research and Stakeholder involvement; the purpose is to aid decision support / management and benchmarking;
Developed by a US company (Good company) and without support to key experts / actors;
Tool not available online, neither report nor update.
USA / international
GM—Green Metrics University Ranking [49,50]Based on 6 domains (Scenario and infrastructure, Energy and climate change, Waste, Water, Transport, Education & Research); 33 indicators, two focus on the environment, no community involvement or other social components; ranking point system allowing benchmarking and comparison; with a wide world application across the universities;
Tool available online, (see Reference [50].
GMID—Graz Model for Integrative Development [51] Based on narrative and domains: 5 domains (Leadership, Social Networks, Participation, Education and Learning, Research); applicable but not specific to HEIs; applied to the RCE-an international network of formal, non-formal and informal education organizations -, mobilized to provide ESD to the local and regional community at 3 levels;
Developed by a researcher in Europe;
Tool not available online.
Austria / international
GP—Green Plan and the Label DD&RS ou Plan Vert [52]Based on 5 domains (Strategy governance, Education and training research, Environmental management, Social policy, Regional presence); 44 indicators; can be audited and certified by internal and external stakeholders concerning the ISO 26000; purpose of assisting in the elaboration of sustainability plans/policies;
Developed by Conférence des Grandes Ecoles, Conference of University Presidents, French Government and Non-Governmental Organizations within the Grenelle Environment Roundtable;
Tool not available online at the present
HE 21—Higher Education 21’s sustainability Indicators or
HEPS Higher Education Partnership for Sustainability [24,53]
Based on indicators (12 key indicators and 8 strategic management indicators); focusing mainly on parameters of organizational management change; less emphasis on social indicators and does not encompass in a balanced way all the dimensions of ESD in HEIs (more emphasis on governance); difficult to benchmarking; latest version and network activity in 2003;
Developed for 18 universities in the UK that have partnered to support English universities and their monitoring in the implementation of sustainability-HEPS Higher Education Partnership for Sustainability;
Tool not available online.
United Kingdom
PSIR—Penn State Indicator Report [24,54]Based on 33 indicators, covering the environmental dimensions of the campus, transport, decision support, research and community; results of each indicator reported in 4 levels of implementation and with proposals for improvement; less emphasis on social indicators and without teaching and curriculum components; last version available in 2000;
Developed by the Penn State Green Destiny Council to be applied at US universities, in the State of Pennsylvania and to be communicated to the general public how sustainability is being implemented;
Tool not available online, only on the report, (see Reference [54]).
USA, Pennsylvania State
P&P—People & Planet University League [55] Based on 13 indicators (not divided into dimensions), greater focus on environmental operations and less on community; graphical presentation of results; in operation for several years allowing the annual comparison and an annual ranking; data collection is carried out in the universities’ webpages and the UK Higher Education Statistics Agency;
Developed by a network of UK students - People & Planet for universities in the UK and tested at various UK universities;
Tool available online. (see Reference [55]).
United Kingdom
SAQ—Sustainability Assessment Questionnaire [56] Based on narrative and indicators: 35 indicators, 8 dimensions (Curriculum, Research and scholarship, Operations, Faculty and staff, Extension and services, Student opportunities, Administration, Mission and planning); with greater emphasis on campus operations; presented through a questionnaire addressed to various internal stakeholders;
Developed by the secretariat of the signatories of the Tailloires Declaration - Association of University Leaders for a Sustainable Future;
Tool available online, (see Reference [56]).
SRC—Sustainability Report Card [57]Based on narrative and indicators: 52 indicators, 5 dimensions (Campus operations, Meal service, Donation investment, Transportation, Involvement of key stakeholders); more focus on energy saving and less emphasis on education; presented through a questionnaire with a final grade from A to D; suspended in 2012;
Developed by a North American Non-Governmental Institution - Sustainable Endowments Institute;
Tool not available online.
USA / Canada
STARS—Sustainability Tracking, Assessment & Rating System [58]Based on narrative and indicators: 74 indicators, 5 dimensions (Academic, Involvement of key actors, Operations, Planning and Administration, Innovation and leadership); 5 levels of final classification, allowing the ranking (reporter, bronze, silver, gold, platinum); one of the most used tools internationally; updated every year;
developed by a North American Non-Governmental Institution-Association for the Advancement of Sustainability in Higher Education and initially developed for HEIs in the US and Canada but applicable to any region;
tool available online, (see Reference [58]).
USA / international
SUM—Sustainable University Model [8]Based on narrative and indicators: 23 indicators, 4 dimensions (Education, Research, Dissemination and partnership, Campus sustainability); divided in 4 phases (Vision development, Mission, Sustainable committee, Audit of sustainability strategies) incorporating the Deming cycle approach; tested at various world universities; without updates;
Developed by a researcher in Mexico;
Tool not available online.
Mexico / international
SLS—Sustainability Leadership Scorecard [59] Based on performance indicators; 4 domains (leadership and governance, learning, teaching and research, operations); self-assessment developed specifically for colleges and universities to improve social responsibility and environmental performance through a whole-school approach; final scores with a range from 0–4 no weights in the indicators and final result in a dashboard index; adapted from the Green Scorecard and linked to Sustainable Development Objectives standards;
Developed by a Non-Governmental Association of the United Kingdom and Ireland—The Alliance for Sustainability Leadership in Education;
Tool available online for free to United Kingdom and Ireland,!/login (see Reference [59]).
United Kingdom / Ireland
SustainTool—Program Sustainable Assessment Tool [60]Based on indicators, focused on areas / programs or at the institution level; 8 dimensions (Environmental support, Funding stability, Partnership, Organizational capacity, Program, Evaluation, Program adaptation, Communications, Strategic planning) with a low weight in the environmental component; presented in a 40 multiple-choice questions in self-assessment questionnaire, with answers being given individually or in a group; allows the communication, review and development of an action plan; available for several years with updates;
Developed by a north American university—Washington University for any university, particularly in the North American context but especially directed to the health area;
Tool available online, (see Reference [60]).
USA / international
THE—Times Higher Education Impact University Ranking [61] Based on the evaluation of the implementation of the Sustainable Development Objectives (ODS) in HEIs: 11 ODSs: 3,4,5,8,9,10,11,12,13,16, 17; each ODS has a small number of indicators associated with it; equal weight is given to each ODS; first version available for 7 ODS but still in development (1st version April 2019);
Developed by the Times;
Tool available online, requesting by email, (see Reference [61]).
TUR—Three Dimensional University Ranking [62] Based on indicators: 15 indicators, weighted based on a participatory process and Analytical Hierarchical Process (AHP), 3 dimensions (Research, Education, Environment); less holistic approach; graphical presentation of results; allows ranking based on rankings of world universities, simplified sustainability only in 5 indicators; tested in the best universities but without updates;
Developed by researchers in Europe;
Tool not available online.
UEMS—University Environmental Management System [21]Based on EMAS / ISO14001 with a social responsibility component and indicators: 27 indicators, 3 dimensions (University EMS, Public participation and social responsibility, Teaching and research in sustainability); greater focus on the environment and campus areas;
Developed by researchers in Saudi Arabia;
Tool not available online.
USAT—Unit-Based Sustainability Assessment tool [63] Based on indicators: 75 indicators, 4 domains (Teaching, Research and community services, Operation and management, Student involvement, Written policy and statement); score of 1 to 4 indicators; adapted from SAQ, AISHE and GASU; can be used in the department, college or HE unit; without updates;
Developed by the United Nations Environment Program (UNEP) for the African context;
Tool not available online but questionnaire available online on report, (see Reference [63]).
uD-SiM model—Uncertainty-based quantitative assessment of sustainability for HEIs [9,64]Based on indicators and the Models of Pressure, Exposure, Effects, Action (DPSEEA) and a multicriteria decision process (applying Fuzzy logic). Aggregate score in a final index that integrates the non-linear effects of the indicators, with different weights and normalized indicators. Indicators based on the GASU model; 4 areas (Environmental, Economic, Social and Education); applied to Canadian Universities but its implementation is international; calculation method is complex;
Developed by researchers;
Tool not available online.
Table 2. Operationalization design of the sustainability assessment in the case studies.
Table 2. Operationalization design of the sustainability assessment in the case studies.
UniversityData CollectionMethodParticipants
Universidade AbertaRaw dataDocuments Analysis: Activity plans and reports, Strategic Plan and programs and courses study guides, databases.Academics, Administrative services,
Campus Operations
and Rectorate
Data standardizationSearch in databases /websites-
Improvement Proposals1st Focus groupVice-rector for quality and 3 researchers (experts in sustainability assessment)
2nd Focus group16 (researchers, students, teachers, administrative staff)
Universidad Autónoma de MadridRaw dataDocuments Analysis:
Reports and University web page information
Administration and services staff, teacher
1st Focus group4 members of Eco-campus team (Eco-campus Manager, Infrastructure Manager, Environmental Participation Officer, Electric Cars on Campus Officer)
2nd Focus group3 Vice-Chancellors (Undergraduate Studies,
Sustainability and Campus and Strategy and Planning)
Improvement Proposals3rd Focus group6 students’ leaders from Faculty of Business
and Economic Sciences
22 Professors and researchers
6 External experts
Table 3. Mode and relative frequencies for each criterion within 25 HE sustainability assessment tools. Sum and Level of agreement for each tool and criteria.
Table 3. Mode and relative frequencies for each criterion within 25 HE sustainability assessment tools. Sum and Level of agreement for each tool and criteria.
ToolComprehensibilityComparabilityData AccessProgress over TimeSustainability BroadnessSupport to DecisionParticipationAccessibilitySumIndex of Agreement
1. AISHEMode332322322070.8%
R.F. (1)
R.F. (2) 10.50.750.750.51
R.F. (3)11
2. AMASMode222222211552.1%
R.F. (1) 0.250.25 0.25 1
R.F. (2)0.510.750.50.750.50.75
R.F. (3)0.5
3. AUSPMode322123121654.1%
R.F. (1) 0.5 0.750.25
R.F. (2)0.5110.250.750.50.250.75
R.F. (3)0.5
4. BIQ AUAMode222222211560.5%
R.F. (1) 0.25 0.25 1
R.F. (2)0.750.7510.510.50.75
R.F. (3)0.250.25 0.25 0.250.25
5. CITE AMBMode323122211648.0%
R.F. (1)0.25 0.250.750.25 0.251
R.F. (2)0.250.750.250.250.750.750.75
R.F. (3) 0.25
6. ESDGCMode322333111843.8%
R.F. (1)0.25 0.25 0.250.51
R.F. (2)0.250.510.
R.F. (3)0.50.5
7. GASUMode332333212056.1%
R.F. (1) 0.5 0.250.75
R.F. (2) 0.750.25
R.F. (3)0.50.75 0.511
8. GCMode232222111545.9%
R.F. (1)0.25 0.250.25 0.251
R.F. (2)
R.F. (3) 0.25
9. GMMode333123131956.1%
R.F. (1) 0.5 0.50.5
R.F. (2) 0.5
R.F. (3)10.750.750.250.250.5 1
10. GMIDMode211222211335.4%
R.F. (1) 0.750.50.250.25 0.75
R.F. (2)
R.F. (3)0.5
11. GPMode332333212041.6%
R.F. (1) 0.25 0.250.5
R.F. (2)0.250.2510.
R.F. (3)0.750.5 0.25
12. HE21Mode112222211352%
R.F. (1)0.50.75
R.F. (2)0.250.2510.750.50.50.5
R.F. (3)0.25
13. P&P People & PlanetMode333122131850.1%
R.F. (1)0.250.25 0.5 0.251
R.F. (2)
R.F. (3) 0.25 1
14. PSIRMode333223111856.3%
R.F. (1) 0.250.25 11
R.F. (2)
R.F. (3)0.750.50.50.25 0.5
15. SAQMode313221231754.1%
R.F. (1) 0.5 0.5 0.750.25
R.F. (2) 0.25 0.50.750.250.750.5
R.F. (3)10.251 0.25 0.5
16. SRCMode322133211739.5%
R.F. (1) 0.251
R.F. (2)
R.F. (3)0.50.25 0.750.5
17. STARSMode332333232270.8%
R.F. (1) 0.25
R.F. (2) 0.750.25 0.50.5
Freq. Relative (3)11 0.75110.50.5
18. SUMMode332333212068.8%
R.F. (1) 0.251
R.F. (2) 0.250.75
R.F. (3)0.750.750.25110.75
19. SLSMode332333122054.1%
R.F. (1) 0.5
R.F. (2) 0.250.750.25
R.F. (3)10.750.250.7510.50.250.5
20. SustainToolMode333122231964.5%
R.F. (1) 0.75
R.F. (2)0.50.5 0.2510.750.75
R.F. (3)0.50.51 0.250.251
21. THEMode332223131949.9%
R.F. (1) 0.25 0.5
R.F. (3)0.750.750.
22. TURMode313122311641.8%
R.F. (1) 0.5 0.5
R.F. (2)
R.F. (3)0.750.250.750.25 0.250.5
23. UEMSMode222222311654.1%
R.F. (1) 0.25 1
R.F. (2)0.750.750.750.750.50.750.25
R.F. (3)0.25
24. USATMode333333222254.1%
R.F. (1) 0.25
R.F. (2)
R.F. (3) 0.25
25. uD-SiM modelMode232332211858.3%
R.F. (1)0.25 0.25 0.251
R.F. (2)0.750.50.750.5 0.750.75
R.F. (3) 0.5 0.510.25
Index of Agreement 51.3%47.3%59.3%35.3%70.0%4.6%47.3%77.3%--
Legend: R.F. (1)—Relative Frequency in (1) low; R.F. (2)—Relative Frequency in (2) medium; R.F. (3) —Relative Frequency in (3) high.
Table 4. Summary of the results of Assessment and Improvements for Universidade Aberta regarding the STARS dimensions.
Table 4. Summary of the results of Assessment and Improvements for Universidade Aberta regarding the STARS dimensions.
  • Comprehensive formal cycle: BSc Environmental science, MSc Environmental Citizenship and Participation, PhD Social Sustainability and Development
  • Most of undergraduate programs with a least a module about Sustainability
  • Non formal courses (Open classes, Massive Open Online Courses, MOOCS, related with Environment)
  • Strategic research line about “Sustainability and Environment” (but with no financial support from University)
  • There is no available place to register the SDG activities of the university.
  • Institution specific sustainability learning outcomes for all students
  • All undergraduate students should enroll at least a sustainability course (3 ETCS)
  • Enlarge research on sustainability linking students with the labor market, according to transdisciplinary research
Engagement, Planning and Administration
  • Staff training about better sustainable practices at work
  • Open courses on Sustainability and Environment
  • PhD thesis aiming at solving local problems related with SDG (action research, transdisciplinary)
  • No formal Institutional sustainability policy/strategy
  • Formal support to Students with disabilities
  • Assessed employee satisfaction
  • Participation of community members in the Institution governance (General Council)
  • Engagement of UAb community in DREAMLAB for Sustainability (Dragon Dreaming technic)
  • Develop a University policy for sustainability
  • Integrated in the Quality office the Sustainability Practices
  • Inclusion within the program of Welcome of new employees (already existing) a sustainability performance kit
  • Promote awareness within academic community for students to work with the local community problems in the realm of Local Learning Centers;
  • Give more emphasis to the sustainability academic offer of the University
Operations and Innovation
  • No formal campus; e-learning regime (low ecological footprint)
  • LED lighting has reduced consumption
  • Use of local food/resources/services for events
  • Videoconference as prime communication service to all events
  • Decentralized Local Learning Centers in areas of low population density/close contact with society
  • Disseminate/Monitor the Ecological footprint – CO2 equivalent, water, waste (improvement of the GEE inventory)
  • Sustainable procurement practices (also within national policies): e.g., recycle paper, hybrid cars, cleaning material)
  • Separate bins in all facilities
  • Engage all university community in an online collaborative platform for sustainable ideas
Table 5. Summary of the results of Assessment and Improvements for Universidad Autónoma de Madrid regarding the STARS dimensions.
Table 5. Summary of the results of Assessment and Improvements for Universidad Autónoma de Madrid regarding the STARS dimensions.
  • Sustainability plays a role in many subjects on the majority of the faculties, 632 courses are connected to environment and sustainability.
  • The university is attempting to promote a participatory approach to sustainability activities together with the campus community, integrating an overall vision to a local impact
  • There is no available tracking system to register and follow the SDG activities of the university. The SDG approach is implicit in the academic programs. Also, the research activities are scattered.
  • When it comes to research and teaching, there are leading teams in different aspects of environmental, economic and social sustainability.
  • UAM was one of the first universities in Spain to offer a degree in the environmental science and incorporate environment into other degrees.
  • However, the learning method for these topics should consider a more practical method than only theory
  • Potential outreach in the society through scientific contribution including SDG areas in research and academic content
  • Use the campus as an environmental classroom for practicum, thesis, etc.
  • Collaborative online platform to show publications, projects, thesis, events connected with the SDGs
  • Observatory for dissemination - Inventory research team SDG (dynamic, active)
  • Involvement of teachers to motivate student participation
Engagement, Planning and Administration
  • There is a lack of awareness from the university community and society, as well as a low level of involvement and participation.
  • UAM must manage energy and water consumption according to the restrictions defined by the City Council and National Government.
  • UAM is working on a list of rigorous requirements for the process for suppliers in 2019.
  • Authorities are aware of the university’s great expenses related to unsustainable actions on the campus.
  • There is no evolution on the metrics or rankings due to a lack of financial support.
  • Very high engagement of the UAM to sustainability: “SDG Lab Campus”; the presence of the UAM in Sustainability Conference of Rectors of Spanish Universities about Sustainability
  • Social center for sustainability reference: store, organic coffee shop, distribution point Association of Parents and Friends of the Disabled of the UAM (APADUAM) Social and community involvement
  • Guide the UAM towards the circular economy
  • Implementation of a SDGs Road Map 2018
Operations and Innovation
  • A good connection between Madrid and UAM campus by public transportation, a daily average of 8k12k cars on the campus
  • Recycling initiatives by the implementation of technological resources.
  • Green areas have a great impact on students’ attitudes toward the university.
  • UAM nature protection areas are a great improvement in air quality.
  • LED lighting has reduced consumption from 100% to 10%.
  • There is not an evolution of the environmental image of the UAM.
  • ECOCAMPUS is a leader of environmental programs at HEIs since 1997
  • Agro-ecological use: fruit trees, orchards, livestock, beekeeping.
  • Analyze waste production and improve separation
  • Pedestrianize the historic core area.
  • Improve the treatment plant, analysis of water quality (contaminants); geofilter design
  • Metro minute (Distance in meters and time on feet) located at access points (train station, bus stop) and central (Rectorate, Plaza Mayor)

Share and Cite

MDPI and ACS Style

Caeiro, S.; Sandoval Hamón, L.A.; Martins, R.; Bayas Aldaz, C.E. Sustainability Assessment and Benchmarking in Higher Education Institutions—A Critical Reflection. Sustainability 2020, 12, 543.

AMA Style

Caeiro S, Sandoval Hamón LA, Martins R, Bayas Aldaz CE. Sustainability Assessment and Benchmarking in Higher Education Institutions—A Critical Reflection. Sustainability. 2020; 12(2):543.

Chicago/Turabian Style

Caeiro, Sandra, Leyla Angélica Sandoval Hamón, Rute Martins, and Cecilia Elizabeth Bayas Aldaz. 2020. "Sustainability Assessment and Benchmarking in Higher Education Institutions—A Critical Reflection" Sustainability 12, no. 2: 543.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop