A Systemic Tool and Process for Sustainability Assessment

Abstract

Sustainability assessment is a growing concern worldwide with United Nations’ Agenda 2030 being implemented. As sustainability refers to the consideration of environmental, social and economic issues in light of cultural, historic—retrospective and prospective—and institutional perspectives, appropriate tools are needed to ensure the complete coverage of these aspects and allow the participation of multiple stakeholders. This article presents a scientifically robust and flexible tool, developed over the last 25 years and tested in different cultural and development contexts to build a framework for sustainability assessment of policies, strategies, programs and projects in light of Agenda 2030. A selected case study conducted on a major mining project in Québec (Canada) illustrates the Sustainable Development Analytical Grid performance for sustainability assessment. This tool and process is part of the United Nations’ Sustainable Development Goals Acceleration Toolkit; it is one of the most adaptable, addresses all 17 SDGs and is fully accessible for free. Other advantages and limitations of the tool and process are discussed.

1. Introduction

For the last 45 years, the concept of sustainable development (SD) has been seen as an alternative to conventional economic development based on gross domestic product (GDP) growth. Rallying around this paradigm is a response to the failures of the dominant model, including its failure to fairly distribute wealth, and its impacts on the local and global environment, cultural diversity and social cohesion. Although its roots can be traced millenaries ago in human history [1], it has flourished in the late 20th century [2].

At the United Nations (UN) Conference on the Human Environment held in Stockholm in 1972 [3], the international community agreed that both development and the environment, hitherto addressed as separate issues, could be managed in a mutually beneficial way. That conference, and many others afterwards, was the occasion for the international community to adopt virtuous principles to be integrated in development policies, strategies, programs or projects (PSPP). This approach led to the adoption of Agenda 21 at the Earth summit held in Rio de Janeiro in 1992.

Since then, sustainability is discussed in the political world, in business sectors as well as at community level, in non-governmental organizations and in the academic world but attempts for its implementation are rewarded by varying degrees of success. As long as the concept is based on virtuous principles, “things” keep on going mostly as usual and no real in-depth changes are implemented.

Despite abundant literature on SD and another World UN Summit held in Johannesburg (2002), sustainability did not enter the realm of decision makers sufficiently to prevent multiple crisis (food, climate, water, poverty, etc.) that culminated with the 2008 financial crisis. As these would have been improbable in a SD ruled world, the financial crisis finally called for a real change towards a conversion in the measure of progress from GDP growth to wellbeing and sustainability [2,4,5,6,7]. The United Nations Conference on Sustainable Development (Rio+20), held in Rio de Janeiro in 2012, led to the call of a new Agenda for 2030 to face those challenges. The 2030 Agenda for Sustainable Development and its 17 Sustainable Development Goals (SDG) [8] were adopted by the UN General Assembly in September 2015 and came into force on 1 January 2016. This approach, including targets and indicators, addressing complexity and interactions, goes beyond the simple wish list of previous conferences.

Sustainability relates to multiple objectives pertaining to various dimensions. It is also linked to various timeframes. As decisions are generally made on short term perceived issues in order to satisfy political agendas or quarterly results, long term issues and complexity have a limited influence on everyday management. Thus, we can consider SD as a hypothesis on the evolution of different indicators for the welfare of human beings enhancing the quality of their local and global environment over a long time perspective [9] (p. 53). However, SD still presents difficulties in terms of its planning, implementation, monitoring and evaluation [10,11].

Sustainability can be assessed but, as a whole, it allows different views to coexist. According to Waas et al. [1], it must abide by four principles:

(1)

normativity;

(2)

equity;

(3)

integration; and

(4)

dynamic principle.

The normativity principle supposes that sustainability is a matter of social definition and can change with time and values. It is a temporary representation of the kind of world we want to live in and want to leave as a legacy for future generations. The equity principle is central to sustainability and can be divided in inter-generational, intra-generational, geographical, procedural and inter-species equity. The integration principle implies that all sustainability principles should be considered together in a holistic perspective and that mutually supportive benefits must always be sought. The dynamic principle refers to SD as a process of directed change that is not defined by an end-state: SD is therefore cybernetic by essence [12].

Sustainability assessment (SA), according to Sala et al. [10], “is one of the most complex types of appraisal methodologies” that entails multidisciplinary aspects and is conducted for supporting decision-making and policy development. Many tools, indicators system and frameworks have been proposed to characterize and assess sustainability at different level [13,14,15,16,17,18,19]. For this paper, we consider as SD assessment tools: checklists, computer models, processes and approaches, institutionalized or not, used to evaluate and analyze operations for informed decisions on sustainable development. They can be used for planning, simulation, monitoring and management follow-ups or as repositories. While some tools give partial answers for an overall assessment, they can be combined for a more complete sustainability assessment. Furthermore, they should refer to the eight principles of “The New BellagioSTAMP” [2]:

(1)

Guiding vision;

(2)

Essential considerations;

(3)

Adequate scope;

(4)

Framework and indicators;

(5)

Transparency;

(6)

Effective communications;

(7)

Broad participation; and

(8)

Continuity and capacity.

However, are the tools aforementioned able to address the complexity of a systemic representation of sustainability? Most SA tools were developed from a given perspective to obtain the labeling of sustainability and most cannot account for the complexity and full scope of interactions pertaining to a dynamic socio-ecological system submitted to multiple driving forces and diverse limits. As stated in Sala et al. (2015), “Performing a sustainability assessment requires integrating sustainability principles, thresholds and targets in the evaluation, as well as moving from a mere multidisciplinary to inter- and trans-disciplinary approaches” [10]. Moreover, the projection of this system in a foreseeable future demands the common understanding of challenges, pathways and means by multiple stakeholders in order to get their involvement towards common goals.

On 17 July 2017, UN Development Group and UNICEF launched the “SDG Acceleration Toolkit” [20] where a collection of sixty-one tools developed worldwide are presented in three broad categories:

(1)

Integration tools;

(2)

Last-mile analysis tools; and

(3)

Risk-informed planning tools.

The first category encompasses “Integration tools for analyzing interconnections, synergies, trade-offs and bottlenecks among SDGs” including Dynamic Simulation, Indicators and Assessment and Static Inter-linkage Analysis tools. The second category is defined as: “Last-mile tools to support meeting the SDGs and targets and unlocking bottlenecks for all segments of society and reaching the furthest behind first”; it includes tools targeting Community-based planning and Multi-stakeholder Engagement, Data Revolution, Fragility Assessment, Human Rights Mainstreaming, Financing for SDGs and Vulnerability Assessment. Finally, the third category comprises “Risk-informed planning tools for resilience building, adaptation and identifying and unlocking bottlenecks”; referring to Disaster Risk Reduction, Economic Forecasting, Environmental Degradation Risk Reduction, Epidemic and Pandemic Analysis, Peace and Conflict Analysis, Risk, Foresight and Scenario tools and tools for Financing Resilience [20].

In the SDG Acceleration Toolkit, tools are also classified according to their type: Diagnostics, Guidelines, Computer models and programs, Set of monitoring indicators and indices, Financing instruments and funds, Technology access protocols, Training programs, Communication plans, Capacity building, Knowledge management platforms, Econometric models, Scenario builders, forecasting and back-casting methodologies, and Narratives (storytelling) guidelines.

This paper is about a sustainability assessment tool (the Sustainable Development Analytical Grid (SDAG)) and its related process, developed and tested on Policies, Strategies, Programs and Projects (PSPP) at local and national levels from 1988 onwards. It was created and refined over the past 25+ years because SD early practitioners needed to have a complete, thorough assessment tool and methodology, able to evolve in accordance with SD conceptualization and deal with its inherent complexity. The SDAG is recognized by the UN and is part of the SDG Acceleration toolkit where it is classified under Integration tools. The SDAG tool and process meet the eight principles of “The New BellagioSTAMP” [2]. Its assessment methodology is simple, efficient, scientifically robust and the tool is freely accessible to anyone via a website. The SDAG stands out by its flexibility as its methodology allows questioning on a very complete range of themes pertaining to the six sustainability dimensions identified in the literature by Waas et al., and can be applied to policies, strategies, programs or projects of different scales (local to national). The SDAG imposes questioning and reflection in an ethical dialogue which permits and favors inclusion. The scientific robustness of the content and the flexibility of the methodology make the SDAG a tool distinct from the others by making complexity accessible to SA.

The SDAG has been tested in developed, as well as developing and emergent countries, around the world (Algeria, Benin, Burkina Faso, Cameroon, Canada, China, Comoros, Côte d’Ivoire, Democratic Republic of the Congo, France, Gabon, Guinea, Haïti, Madagascar, Mali, Morocco, Sao Tomé and Principe, Senegal, Togo and USA). According to SDG Acceleration Toolkit classification, the SDAG is a diagnostic type of tool. It is an integration tool as well as an Indicator and assessment tool and a Static inter-linkage analysis tool. It also appears in the Last-mile analysis tool category because it is a Community-based planning, partnership and multi-stakeholder engagement tool. Because of its methodology and complete content (it addresses all 17 SDGs), the SDAG is one of the three tools in the SDG Acceleration Toolkit that fits that profile and the only one of the three fully accessible for free.

In this paper, we present the SDAG and its application to a case study, chosen as the best real context example to demonstrate the SDAG’s complete coverage of SD issues in a holistic perspective, stakeholders’ involvement for co-constructing solutions, priority setting in a transdisciplinary approach and continuous improvement perspectives, thus meeting Sala et al. requirements [21], “The New BellagioSTAMP” principles for SA [2] and Waas et al. principles of sustainability [1]. The Apatite Arnaud mine in Sept-Iles (QC, Canada) was chosen as the case study because of its limited and adequate scope (a mining project), the compliance of process (the study was conducted with research team support), its completeness (the project was fully documented and submitted to a legal examination process by an independent commission) and because of its recentness (2012–2014). The case study is here presented to illustrate the process and show examples of real results, graphics and interpretations. However, other assessments have been conducted with more advanced versions of the SDAG, on National planning documents for instance, but they are not shown in this paper as they did not fill all the requirements mentioned above. The decision-making process following assessment, another order of complexity by itself, will not be discussed herein.

2. Materials and Methods

In this section, the tool described is the 2011 version of the SDAG as it was the version used for the case study. Changes made in ulterior versions will be noted in the text to provide readers with the latest information on the SDAG.

2.1. The Eco-Advising Chair Sustainable Development Analytical Grid

Mastering tools to explicitly take into account the principles and SD issues at all levels are of outmost importance. The Sustainable Development Analytical Grid (SDAG) [22], developed by the Eco-Advising Chair of the University of Quebec in Chicoutimi (UQAC) stemming from the initial work of Villeneuve, provides stakeholders with a tool that allows them to fully play their roles in assessing sustainability. The SDAG was first created to evaluate simple projects, but then was enhanced and adapted to the evaluation of any national, regional, local or sectorial PSPPs. The SDAG presents as a Microsoft Excel spreadsheet with a user guide in English or French version that can be uploaded for free on ecoconseil.uqac.ca .

The latest (2016) version of the SDAG is an analytical tool based on six dimensions (ecological, social, economic, ethical, cultural and governance), which can be used to assess to what extend PSPP can improve human conditions through the leverage it provides without adverse effect to other components of sustainability. It enables its users to situate themselves in a sustainability process and presents ways to improve their PSPP, while aiming for continuous improvement. The analysis can also be used to set objectives, find indicators, make more knowledgeable decisions, and/or find manageable tradeoffs and new research questions. The SDAG evaluates PSPP performance on objectives in order to assess dimensions and overall sustainability performance.

The Brundtland report definition of SD focusing on the satisfaction of present and future human needs was the first inspiration for the SDAG. It was built from principles, themes, and objectives drawn from a literary review of the proceedings of international conferences and conventions, standards such as ISO 26000 best practices, as well as a diversity of authors. Since 1988, components and ideas have been collected, crossed, reformulated and the wording has been tested with students, laypersons involved in projects and sustainable development experts after 1991. For example, the UN SDGs and their targets were introduced in the latter version of the tool launched in May 2016 [22] from the reformulation of the earlier 2014 version following a review by twelve experts. The objectives set out for each dimension are used to build a qualitative index consisting of a global score by dimension that measures the capacity of a PSPP to comply with the conditions of SD. The approach has the advantage of considering at once global concerns and the more local types of impacts, limited in both space and time.

Over time, the tool has evolved. After the introduction of “ethical” dimension in 1992, governance was added in the 2011 version and then cultural dimension in 2014. These new dimensions were introduced from the addition of new objectives and reorganization of previous ones, each dimension referring to the satisfaction of specific needs corresponding to the Guiding vision principle of the New BellagioSTAMP [2]. With these six dimensions, SDAG covers the six pillars identified in the literature by Waas et al. [1] as well as the five Ps—People, Planet, Prosperity, Peace and Partnership—of UN Agenda 2030.

• The ethical dimension addresses the need of equity, justice, solidarity and coherence;
• The ecological dimension addresses the need for quality environments and long-term availability of resources;
• The social dimension addresses social needs and individual aspirations, peace and security needs, to improve the quality of life and cohesion of societies;
• The economic dimension addresses the material needs to maintain and improve the mechanisms that allow human societies to meet their needs through exchanges of their comparative advantages;
• The dimension of governance addresses the needs of participation, democracy, integration and transparency; and
• The cultural dimension (not present in 2011 SDAG) addresses the needs of cultural traits affirmation, protection and enhancement.

Although content has been updated in recent years, the analysis process has been unchanged since 1999. However, the Excel spreadsheets were progressively automated to cross results and present figures and priorities that make the analyst’s work and interpretation of results easier.

The case study presented in this article arises from the use of the SDAG 2011 edition, which comprises five dimensions (ethical, social, ecological, economic, and governance), 32 themes and 101 objectives (Figure 1). This project was chosen as an example as the authors were involved (via the Eco-advising Chair research team) from beginning to end to perform research and acquire additional knowledge during the sustainability assessment process. The exercise was therefore recent, complete and could be used to demonstrate the SDAG’s full potential.

The 2011 SDAG version was published in English in 2012. The SDAG is a Microsoft Excel file where each dimension is on a separate spreadsheet (Figure 2). In addition, each dimension/sheet highlights a principle pertaining to the rationale of the dimension. Dimensions are subdivided in themes that are subdivided in objectives, which can be weighted and evaluated.

For each principle, theme and objective, explanations, justifications and examples are available directly in the file for ease of use (accessed by a commentary window in each cell) (Figure 3).

2.2. Assessment Methodology

Sustainability being normative and dynamic, the SDAG should above all be considered as an asset in a continuous improvement process. It can be used at different steps of a PSPP where such an approach involves making regular evaluations to ensure the PSPP evolves in the desired direction. The PSPP can be revisited periodically as needed, e.g., an organization could decide to make its assessment every three to five years. The assessment should be made ex-ante (in the conception phase, prior to implementation) warranting that knowledge acquisition and stakeholders’ need analysis are sufficient and to verify which elements may be under-represented or missing. It is also useful to set priorities, to select performance indicators, and to determine the needs for future research.

During implementation, the evaluation is used to check the relevance and performance of the PSPP, to look for compromises, to redirect actions and to articulate priority improvements. Ex-post evaluations allow validating the overall direction of the development resulting from the implementation of the PSPP. It reviews the actions in place to ensure they do not produce counter-intuitive effects. It can also be used to gather information for the improvement of future PSPPs to report or communicate.

The assessment procedure with the SDAG consists of five major steps that will be explained in detail in the following sections. They are:

(1)

Preparing for the assessment;

(2)

Weighting;

(3)

Evaluation;

(4)

Current and future actions and Improvements; and

(5)

Results interpretation and sustainability assessment.

An assessment is an iterative process and analysts, with the Committee, are free to make adjustments throughout. Because an assessment should be based on the latest data available, they can add documentation, re-discuss weights, or reevaluate objectives enlightened by new data.

2.3. Preparing for the Assessment

Before using the SDAG, the assessors must pay particular attention to the scope of analysis, the needs of the assessment, the data collection and the analysis committee.

2.3.1. Scope of the Analysis

To measure SD progress, it is important to first determine the scope of the analysis. In accordance to the Adequate scope principle (New BellagioSTAMP), the scope should adopt an appropriate time horizon and geographical scope [2]. The limits of the PSPP must be identified to manage the amount of information needed and to relate the results in a significant way. Use of the SDAG first requires asking how it will be applied, and what are the objectives of the analysis (e.g., project design, progress evaluation, specific reporting, etc.). The scope defines the limits in terms of organization, authority, infrastructures, planning steps, production steps, territory, temporality, sectors, etc. This information can be reported in a table of the spreadsheet for further analysis.

2.3.2. Needs Assessment

The second prerequisite concerns the needs that the evaluated PSPP should meet. Numerous issues, objectives, and indicators can be targeted by a PSPP. Hence, PSPPs can identify the tradeoffs necessary to satisfy the most basic needs of the greatest number of people before meeting the accessories needs or preferences of a few. The needs assessment is essential to fully grasp the nature of the actions that the stakeholders will need to focus on as they commit to a sustainable PSPP. This information can be obtained from interviews, media analysis, statistical data, contextual documents and stakeholder’s declarations to public authorities, for example. The needs analysis should be included in the final report as it justifies the PSPP. Needs analysis helps the stakeholders in the weighting process. It can also be performed as a sub-step of weighting depending upon the type of PSPP considered. Needs assessment is related to the equity principle of sustainability as it uncovers more than the promoter’s needs. A larger view of the various stakeholders’ needs is useful for further steps of analysis. The needs assessment can be linked with the Guiding vision, the Essential considerations and the Framework and indicators principles of the New BellagioSTAMP [2].

2.3.3. Data Collection

Finally, the last prerequisite to the evaluation relates to the best possible understanding of the PSPPs and their context of implementation. This is done through research in literature, expert testimony, field trips, readings, etc., within the boundaries of the defined scope. Data collection can be in various domains such as technical, legal, moral, social, economic, environmental, and others. Knowledge gaps must be identified. Every piece of literature quoted in the grid should be available at all time during and after the analysis as suggested by the Transparency principle New BellagioSTAMP, [2]. Assessing a knowledge gap may require reconsideration of the PSPP if it is linked to a prioritized objective and necessitates an evaluative search to supplement the missing information.

The data collection step helps to determine the assessment committee composition, to identify analysts or to select experts for the evaluation. That step must be initiated before assessment but can be revisited or completed during the following assessment steps.

2.3.4. Analysis Committee

For the weighting step, a committee must be constituted. Depending on circumstances, the committee may be composed of a combination of stakeholders (5–12) with diverse backgrounds: promoters, experts in the related field, people or communities impacted by a project, and, ideally, a SDAG expert. Ideally, members of the committee are open minded people with great listening skills, dialogue and systemic view capacities. If and when more people want to be involved, multiple committees can compare and discuss their results after weighting.

2.4. Weighting

Weighting each objective calibrates the grid according to their relevance and significance for a given PSPP in a given country or region at a particular moment. This step is crucial to calibrate the SDAG to the realities and context in which it applies. To comply with the equity principle, weighting must be conducted by a facilitator in a free discussion between a team of persons including stakeholders, operators, and planners. The analysts must decide through dialogue the weight of each component by consensus. For each SDAG’s objectives, the group must answer the following question: Is achieving this objective desirable, important or essential for our PSPP?

Values of 1, 2 and 3 are used to qualify the significance of a given PSPP objective:

(1)

Desirable objective: Achieving this objective is deemed unimportant, or it has low value to fulfill identified needs;

(2)

Important objective: Achieving this objective is deemed important but is not directly associated with identified needs regarding the PSPP; and

(3)

Essential objective: Achieving this objective is deemed essential and will contribute directly to the satisfaction of identified needs. It is considered essential to the success of the PSPP.

It should be noted that the value 0 cannot be granted in the weighting process because every objective in the grid is relevant in a SD perspective. Consequently, all objectives are subject to evaluation and improvement over time. This insures respect of the integration principle as the complete set of objectives will be questioned and their interactions considered during the assessment process.

Weighting reflects the values and experience of the committee members. Every objective must be discussed and a common decision made. If an agreement is not possible, minority opinions must be noted and justified for further sensitivity analysis. Those are conducted to determine to what extent their weighting would have changed overall priority results. The 2016 SDAG version was modified to gather more information by adding a column for weighting justification.

The weighting can remain or be changed with further assessments, by the same group or different people according to each PSPP’s circumstances, but a good practice would be to go over it and exert consensual changes if need be.

2.5. Evaluation

If the weighting reflects stakeholder point of view, evaluation records the performance of a given PSPP for every objective in the SDAG. Therefore, once all weighted, each objective is evaluated by answering the following question: “How does the PSPP address this objective?”

In the 2011 SDAG, percentile values are used to determine a PSPP’s performance regarding a given objective. The following scale serves as a guide for the evaluation:

• From 0% to 9%: An objective that is ignored in the PSPP;
• From 10% to 24%: An objective on which the PSPP has indirect effects; that is not tied to PSPP’s outcomes;
• From 25% to 39%: An objective that is marginally addressed by the PSPP;
• From 40% to 59%: An objective that is moderately addressed by the PSPP, the objective benefits from a certain amount of attention or applies regulatory compliance;
• From 60% to 79%: An objective that is taken into account, but still improvable;
• From 80% to 89%: The PSPP stands out by its innovations and the amount of consideration given to this objective by the choices made; and
• From 90% to 100%: The PSPP is a model of innovation regarding this given objective.

In the 2016 SDAG, the evaluation was modified with the addition of 10% graduation of scores for more accuracy.

To perform an evaluation, analysts can average their respective scores or agree on an overall score based on their discussions. Evaluations should be based on current and/or future actions documented in the material relating to the PSPP under analysis and noted in the grid.

Evaluation may be performed by the same team than weighting but it is not compulsory. The evaluation team may need more technical skills and in-depth knowledge coming from experts, consultants or academics. However, any document supporting the evaluation must be accessible to every participant and be kept for reference in the final report. Justification of the evaluation is noted in the “Actual and projected actions”.

2.6. Actual and Projected Actions and Improvements

To comply with the dynamic principle of sustainability, during both weighting and the evaluation processes, ideas for PSPP improvement may be suggested and added. At this step, it should be done in a brainstorm format without censorship or discussion. These ideas are noted in the appropriate space in the grid. As these are integrated, the same improvement ideas can apply to more than one objective in more than one dimension. Actual and/or projected actions, as well as improvements, must be recorded in the appropriate boxes in the analysis grid. To be admitted as actual or projected action, it needs to be defined in a written description or results from a commitment from a qualified authority. These projects and commitments must be kept available at all time during the analysis process. It will further serve as a justification for action in the assessment report.

Regarding the improvements, the software generates red cells for the prioritized objectives “React” and “Act”—cf. Section 2.7. A red cell implies that the analysts must identify improvements for the corresponding objective (Figure 4).

2.7. Results Interpretation and Analysis

To support sustainability assessment, an analysis report should be produced every time the SDAG is used. The software generates four types of results:

(1)

Weighted average ratings for every dimensions and themes;

(2)

Prioritization for every objective;

(3)

Radar graphics for every dimensions and themes; and

(4)

Priority stakes for dimensions and themes.

The spreadsheet automatically calculates all ratings; examples are provided in the case study Results Section. Equations are simple operations and can be consulted by the analysts for transparency. Ratings are first order indicators of the performance of PSPPs regarding dimensions or themes. It can be used to compare the performance between dimensions, between themes and between similar PSPPs or the same PSPP at different times or with different options. In the 2011 SDAG, a qualitative assessment rating can be obtained for a dimension or a theme:

• Below 20%: Dimension or theme is not considered in the PSPP;
• Between 20% and 39%: Dimension or theme is insufficiently considered in the PSPP;
• Between 40% and 59%: Dimension or theme is moderately considered in the PSPP;
• Between 60% and 79%: Dimension or theme is considered in the PSPP; and
• Between 80% and 100%: Dimension or theme is strongly considered in the PSPP.

That rating was changed in the 2016 SDAG for more sensitivity. Prioritizing objectives is essential to maximize the efforts in the most relevant improvement avenues. The more significant an objective is deemed to be (high weight) and the poorer its performance (low evaluation), the more urgent is the need to act and implement corrective actions. Figure 5 shows a prioritization index grid.

The report should finally include a list of suggested prioritized improvements. These improvements include the ones that address priority objectives (“React” and “Act”) and those that positively impact several objectives. This latter analysis must rely on a compilation and a grouping of improvements collected in the process. The software does not automatically generate a list of improvements ranked by priority. Such a list must be compiled by the analysts. Risk analysis or multi-criteria analysis can be performed to refine improvements ranking.

Weighting is qualitative and relative but it drives the assessment process, as actions linked to objectives noted “essential” will require more attention if they received only a low rating through the evaluation. As such, weighting reflects the importance of each objective as assessed by a group, whereas the evaluation speaks to the performance of a given PSPP for every objective in the SDAG, but it is the combination of both that establishes the priorities to attain SD through corrective actions.

Sustainability assessment is the result of the application of the SDAG to a PSPP. In the following sections, we will present results from a recent case study using the 2011 version of the SDAG. Selected results from the study illustrate the tool potential and will be presented as material for discussion.

3. Results

This section presents selected results to support further discussion on sustainability assessment with the SDAG based on the case study of the Apatite Arnaud mine in Sept-Iles (QC, Canada). The complete results and the sustainability assessment can be found in the full report [23]. Scope of the assessment covers the effects of the project in every dimension of the SDAG referring to positive and negative impacts for the mine project’s life cycle, including mitigation measures to be implemented by the proponent and other stakeholders during mining operations, as well as landscape remediation after mining operations.

3.1. Description of the Case Study

Mine Arnaud is a mining project for an apatite deposit located within the Sept-Iles city limits in northeastern QC, Canada. In 2012, the City of Sept-Iles mandated one of its para-municipal bodies in charge of the environment (La Corporation de la protection de l’environnement de Sept-Iles (CPESI)) to search for tools to guide their development within a sustainability perspective. To do so, the CPESI chose the SDAG. A partnership with the Eco-advising Chair at UQAC was devised to guide them in their first use of the tool. The CPESI’s objective was to evaluate, in a sustainability perspective, an open-pit mining project located inside municipal boundaries. By law, the promoter produced an Environmental impact study. The city’s objective was to better equip policymakers and to inform the public. The Eco-advising Chair’s mandate was to support CPESI in their analysis as a capacity building exercise for municipal authorities to perform other analysis autonomously.

To perform the assessment, a Sustainable Development Assessment Committee (SDC) was formed by CPESI. The SDC was an independent committee, which included experienced professionals with various trainings, from the social, economic, environmental and municipal sectors. Once constituted, the committee members received one-day training on the SDAG. Then, the SDC members carefully read the reference documents, which included the Environmental Impact Assessment (EIA) study required by the Quebec “Loi sur la qualité de l’environnement” (LQE) (Environmental Quality Act), documents from the promoter and others pertaining to the mining industry in general or to the project specifically. That literature covered technical, legal, social, economic and environmental topics. A research team accompanied the SDC, providing help with the weighting, keeping track of consensus weights, evaluation results and filling the grid with improvements suggested by the group members. The analysis was then produced and discussed with the committee prior to the writing of the final report presented to the City council. The final report was also filed with the Bureau d’audiences publiques sur l’environnement (BAPE) commission mandated by Quebec’s government to conduct public hearings on the project, as requested by LQE.

3.2. Weighting

For the weighting step, the SDC members individually weighted the 101 objectives prior to the meeting. Then, members of the Chair supervised the dialogues on individual weighting to define a consensus weighting for all 101 objectives. Through dialogue, members of the SDC put their own subjectivity aside to accept a consensual intersubjectivity; a step closer to objectivity in the sense of being unbiased and free from prejudices [24].

The weighting distribution (Figure 6) shows that 75% of the objectives were deemed “Essential”, 21% “Important” and 4% “Desirable”. Although the weighting value “3” (Essential) is the one that is mostly attributed in all dimension, the weighting distribution is uneven from one dimension to another. For example, for the Governance dimension, 95% of the objectives were weighted to “3” (Essential) as compared to 59% in the Ethical dimension. It should also be noted that for the social and governance dimensions, no objective weighted as “Desirable”.

The average weights for the five dimensions are all ≥ 2.5 (

Table 1. Dimensions and themes’ average weight for the Mine Arnaud project.

DIMENSION/Theme	Avg. Weight	DIMENSION/Theme	Avg. Weight
ETHICAL	2.5	ECONOMICAL	2.8
Poverty	1.7	Ownership and use of goods and capital	3
Solidarity	2.5	Quality of goods and services	2.5
Restoration and compensation	3	Responsible production and consumption	3
Originality and innovation	3	Financial viability	3
Common values	2.5	Wealth creation	2.5
ECOLOGICAL	2.7	Wealth sharing opportunities	2.3
Use of renewable resources	3	Work conditions	3
Use of non-renewable resources	3	GOVERNANCE	3
Use of energy	2.5	Management and decision-making processes	3
Outputs from human activity	3	Participation and democracy	2.8
Biodiversity	2.7	Monitoring and evaluation	3
Use of lands	2.7	Project integration	3
Pollutants globally affecting the biosphere	2	Subsidiarity	3
SOCIAL	2.6	Risk management	3
Health conditions for the overall population	2.5
Safety/security	3
Educational standards	3
Integration of individuals in society	2.5
Individual freedoms and collective responsibilities	2.4
Recognition for individuals and engagement	2.7
Culture	2.3

). There are no major differences between dimensions. The average weights for the 32 themes are also presented in Table 1. We note that one theme (Poverty) averages below 2 (Important). Half of the 32 themes averaged the maximum value, 3.

3.3. Evaluation

The same process as for the weighting was used for the evaluation of the objectives. This time, instead of consensus, the value for each objective was determined by averaging individual committee member’s evaluations. They based their evaluations on actual and projected actions identified in the literature. The analysts also suggested improvements for each objective to complete the evaluation (Figure 7).

Table 2. Evaluation of the five Dimensions for the Mine Arnaud project by the Sustainable Development Assessment Committee (SDC).

DIMENSIONS	Evaluation
ETHICAL	58%
ECOLOGICAL	55%
SOCIAL	48%
ECONOMICAL	58%
GOVERNANCE	60%

and Figure 8 illustrate the project evaluation according to the SDC. The governance dimension obtained the highest score with 60%, while the social dimension was the lowest with 48%. The results show that the project is balanced between dimensions, but that progress is achievable in each dimension but particularly in the social dimension. The prioritized improvement list further supported this assessment of the project’s sustainability.

The graphical tools visually present the overall scores for each dimension (5) and each theme (32), as illustrated in Figure 9. Scores are performance indicators for a given PSPP. Issued from a qualitative method, they have no “scientific” or “absolute” value, but significance is based on inter-subjectivity. They can be used to compare performance between dimensions, between themes and between similar PSPP or between different options for a given PSPP.

It is also possible to comment on each dimension. Theme’s evaluation (Figure 9) also allows underlining particular strengths and weaknesses in each dimension. For example, based on the SDAG content in the ethical dimension (Figure 9A), four themes—Poverty, Solidarity, Restoration and compensation—as well as Common values have very similar evaluations, between 59% and 61%. One theme—Originality and innovation—stands apart slightly with an appreciation of 50%. Therefore, it can be said that it is moderately considered in the project according to the SDC.

The distribution of ratings is more heterogeneous in the ecological dimension (Figure 9B). The theme “Pollutants” is the weakest of the entire evaluation with 38%. On the other hand, “Energy” and “Outputs from human activity” are among those that stand out the most. In the ecological dimension, one can observe that environment characterization in the EIA and the ecosystem knowledge acquisition represents the strengths of the project. However, shortcomings too were identified, such as the non-integration of particular fields and biological areas as requested by some stakeholders. It is also possible, in the ecological dimension, to underline priority objectives “Act” and “React”. For Mine Arnaud, the promoter should “React” on two objectives: “Assessing the potential for alternatives resources” and “Planning global change adaptation measures”.

The social dimension theme’s evaluations also are unevenly distributed (Figure 9C). The two weakest themes are “Individuals integration” with 39% and “Freedom and responsibilities” with 40%. These values support the need to implement improvements for the objectives in these themes. Evaluations of economic themes are more homogeneous (Figure 9D). Except for the “Responsible production and consumption” theme, which scored 47%, there is no big differences between the other themes with “Wealth creation” and “Work conditions” that are the highest with 62%. Finally, the governance theme’s evaluations are all close to the average of the dimension (60%), as shown in Figure 9E. One theme, “Risk management”, underperformed with 53% while four themes—Participation and democracy, Monitoring and evaluation, PSPP integration, Subsidiarity—scored above 60%.

3.4. Priorities

As mentioned in Section 2.7, assigning priority among the objectives is essential. The juxtaposition of weight and performance of an objective indicates the level of priority for implementation of corrective actions.

Figure 10 shows distribution of priorities for the entire evaluation (Figure 10A) and for each dimension—Figure 10B, Ethical; Figure 10C, Ecological; Figure 10D, Social; Figure 10E, Economical; and Figure 10F, Governance. The overall distribution reveals that, for half the objectives, the promoter should implement improvements—priorities Act and React—and that, for 6% of these, implementation is urgent. Only 4% are long term issues and the remaining 46% are to be perpetuated, meaning that actual and projected measures are achieving the objectives. Dimension’s distribution is a visual guide allowing to see where the high priorities are. For example, 71% of the objectives of the social dimension should be improved upon for the project to be more sustainable. On the other hand, only 23% of the ethical dimension’s objectives should be improved. However, as will be discussed in the next section, improvements can affect multiple dimensions.

Results illustrated in Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 and Table 1 and Table 2 are first order outputs. To deepen the sustainability assessment, more steps must be performed after the analysis process. The Chair’s research team furthered the analysis and discussed their results with the SDC before the final report was issued.

After the analysis of the Mine Arnaud project, priority improvements were suggested and directed to stakeholders for consideration. The analysts listed thirteen multidimensional and priority improvements that can be summarize into the followings:

• Partnership with stakeholders;
• Employment support/Training/Recognition of skills and competencies;
• Granting of donations/sponsorship and patronage;
• Establishment of a local development fund;
• Restoration and closure plan specifications;
• Electrification of equipment;
• Special considerations for cumulative impacts on the Bay of Sept-Iles;
• Offsetting greenhouse gases/Climate Change adaptation;
• Innovation, research and development;
• Use of SD tools: awareness, social marketing, standards and certification, action plan, responsible procurement policy, etc.;
• Adoption of best SD practices;
• Establishment of a monitoring and control committee; and
• Communication.

These improvements are explained in the analysis report, which also highlights which dimensions, themes and objectives are impacted. For example, as shown in

Table 3. Improvement “Partnership with stakeholders”.

Dimensions	Themes	Objectives	Priority
			React	Act
Ethical	Poverty	1.2, 4.2, 4.3		4.2, 4.3
	Originality and innovation
Ecological	Non-renewable resources	2.2, 5.2		5.2
	Biodiversity
Social	Health	1.1, 1.4,
	Safety/security	2.1, 2.2,		1.1, 1.4
	Education	3.1, 3.2,		2.1, 2.2
	Freedom and responsibilities	5.3,		5.3, 6.1
	Individuals Integration	6.1,		7.2
	Culture	7.2, 7.3
Economical	Wealth sharing	6.1
Governance	Participation and democracy	2.1

, the “Partnership with stakeholders” improvement impacts objectives in all five dimensions, for twelve themes and seventeen objectives including ten priorities.

These improvements were used by the CPESI during the BAPE public hearings. Other stakeholders also used the report to support their position on the project. Conclusions of the BAPE commission and improvements suggested in its report were mostly in line with the SDAG analysis conclusions and prioritized improvements.

Different Users, Different Perspectives

The Mine Arnaud project allowed considering a complex view of sustainability using the SDAG. It is interesting to compare the results obtained when the SDAG was completed both by a committee of stakeholders (SDC) and by the project proponent—Mine Arnaud Project Team (MAPT).

All interested stakeholders were invited to produce their own evaluation using the SDAG or to provide the SDC with their opinions on the project. For this purpose, a tutorial on the use of the SDAG was produced and made available online for the promoter as well as other interested stakeholders. Usually, promoters must participate, but, with the Mine Arnaud project case study, the promoter was not involved into the process, as the City of Sept-Iles wanted to have an independent assessment. However, the promoter was invited to run its own assessment for comparison purposes and to share supplementary information on the project with the SDC at any time. All documents collected for the analysis were publicly accessible on the same dedicated website.

The SDC committee, based on the SDAG analysis, assessed that the Mine Arnaud project should not be implemented as such, without considering some important improvements. The MAPT’s in-house evaluation scored visibly better than the SDC evaluation. They concluded that the project was sustainable with some improvements. Figure 11 compares the SDC’s and MAPT’s overall assessment.

The comparison between the two groups was performed to highlight the promoter’s actions on issues that were not raised by the SDC, or those that meet all or part of the suggested improvements. It also allows comparison between weights and evaluation and helps to identify differences and similarities.

Comparing the two weightings and themes’ averages show notable differences between the committee and the promoter’s views on the project, the latter being less critical of its own project (

Table 4. Comparison between the weighting of the Dimensions and Themes obtained by the SDC and the MAPT for the Mine Arnaud project.

Dimensions and Themes	Average Weight		Dimensions and Themes	Average Weight
	SDC	MAPT		SDC	MAPT
ETHICAL	2.5	1.7	ECONOMICAL	2.8	1.8
Poverty	1.7	1	Ownership and use of goods and capital	3	1.3
Solidarity	2.5	1.3	Quality of goods and services	2.5	3
Restoration and compensation	3	3	Responsible production and consumption	3	1.5
Originality and innovation	3	1.7	Financial viability	3	2.7
Common values	2.5	2	Wealth creation	2.5	2.5
ECOLOGICAL	2.7	2	Wealth sharing opportunities	2.3	2
Use of renewable resources	3	2	Work conditions	3	2
Use of non-renewable resources	3	1.5	GOVERNANCE	3	2.2
Use of energy	2.5	2	Management and decision-making processes	3	2
Outputs from human activity	3	2.6	Participation and democracy	2.8	2
Biodiversity	2.7	2	Monitoring and evaluation	3	2.5
Use of lands	2.7	1.7	Project integration	3	2.6
Pollutants globally affecting the biosphere	2	2	Subsidiarity	3	1.5
SOCIAL	2.6	1.7	Risk management	3	2.3
Health conditions for the overall population	2.5	1.5
Safety/security	3	2.7
Educational standards	3	1.5
Integration of individuals in society	2.5	1.5
Individual freedoms and collective responsibilities	2.4	1.4
Recognition for individuals and engagement	2.7	1.7
Culture	2.3	2

). Twenty-eight of the 32 themes received an average higher weight by the SDC than those indicated by the MAPT, three had equal weighting and only one was higher on the promoter side. On the other hand, the MAPT gave better evaluation to their lesser weighted themes. This reflects the difference in perception between the two teams.

Given that both weighting and evaluation are considered for the prioritization, the promoter’s lower weighting and higher evaluation also influenced the distribution of priorities (

Table 5. Total objectives prioritization comparison.

PRIORITIZATION	SDC	MAPT
REACT	6	0
ACT	44	2
PERPETUATE	47	72
LONG TERM ISSUE	4	7
NO PRIORITY	0	20
NOT EVALUATED	0	2

). According to the SDC, fifty objectives require improvements before the project can take place against only two according to the promoter whose evaluation also shows twenty non-priority objectives versus none for the committee.

4. Discussion

The SDAG is a tool to support decision-making that allows the assessment of the sustainability performance of any PSPP based on six dimensions—Ecological, Social, Economic, Ethical, Cultural and Governance—separated into principles, themes and objectives. With the grid, strengths and weaknesses of the PSPP can be identified, improvements can be found and prioritized to increase performance through time and indicators can be selected to follow evolution. As an SA tool, the SDAG and process integrate the eight principles of the New BellagioSTAMP [2].

The case presented to illustrate how the grid can be used to aim at SD was the Mine Arnaud study in Sept-Îles, Québec where both the promoter as well as an independent committee completed the SDAG assessment process. Observed differences between the promoter’s and other stakeholders’ assessment of the project probably rely on the importance and consideration they give to social and environmental issues of the project. Promoters often are more concerned by technical and economic issues, with environment and society issues seen as compliance issues or corporate social responsibilities, or simple externalities. For the community, however, the perspective is quite different and that probably explains the discrepancies between the two sets of evaluation and action prioritization. The way that the SDAG assessment of Mine Arnaud project was received by the City council and defended by CPESI and other stakeholders in the following BAPE public hearings demonstrate that the results corresponded to stakeholders concerns also allowing them to advocate improvement of the project [25]. However, the comparison shows the vast difference in views and demonstrates the need to include various stakeholders as soon as possible in the process, to reflect a broader opinion, avoid bias and encourage dialogue to foster social acceptability as mentioned in the Broad participation principle (New BellagioSTAMP), [2].

SD is multidimensional and the SDAG handles and manages this reality, essential for PSPP analysis. SD also promotes stakeholders’ involvement and as a process, requires priority setting and continuous improvement. A SD assessment performed ex-ante and planned at the beginning, opens the door to wider considerations and alternative solutions. It also allows for improvements of PSPP right at the gates, i.e., at the planning stage. SD objectives are crosscutting, that is, they may be associated with more than one dimension. This must be accounted for by the final assessment.

Sustainability can allow different views to coexist but must, according to Waas et al. [1], answer to four principles: the normativity, equity, integration and the dynamic principle. The SDAG meets these four principles: normativity: as it looks for consensus in weighting and evaluation may change in time with new facts or when deeper understanding arises; equity: as stakeholders are part of the whole process, can bring new literature or observations, ask for new studies, identify knowledge gaps and express their dissidences without censorship and these are considered in sensitivity analysis at the end of the process; integration: as every objective must be considered and cannot be weighted zero. Links between objectives are also built-in the SDAG; dynamic: as it can be performed before and repeated as need be for continuous improvement until the end, when original hypothesis can be revisited in the light of performance.

4.1. Framework

The classical three-pillar representation of SD has proven insufficient to categorize and represent the diversity of issues. As the recent adoption of the 2030 Agenda by the UN General Assembly illustrates, SD embraces very large areas of concerns such as the 5 Ps: people, planet, peace, prosperity and partnership [8]. Waas et al. [1] identifies six dimensions defined by scholars: environmental, economic, social, institutional (governance), cultural and time (ethical). The SDAG comprised five dimensions in the 2011 version (ecological, social, economic, ethical and governance) and a sixth (cultural) was added in 2014 to harmonize with the Francophonie’s definition of SD, adopted in 2002. The SDAG therefore now covers these six “pillars”.

In 2016, a new version of the SDAG was produced in order to address the whole asset of the UN’s Sustainable Development Goals and most of their 169 targets [26], thus ensuring that stakeholders in the conception or evaluation of PSPP have a holistic view of what needs to be addressed. Weighting each objective and classifying actions and improvements allows in-depth discussions and appropriation of individual objectives, giving appreciation of the complex nature of SD assessment. Discussions, during the analysis process, allow uncovering the interactions between objectives as synergic or antagonistic. This relates to the latest research [27,28,29,30] to qualify the complex relationship between the SDGs and their targets.

In the Mine Arnaud example, it can be noted from the analysis of Table 1 that such themes as education, safety/security, monitoring and evaluation are weighted as “essential” by the SDC. This is surprising for a mining project in an industrialized country context. However, it is possible to make the connection with SDGs 4 (Quality education), 8 (Decent working conditions and economic growth), 9 (Industry, innovation and infrastructure) and 12 (Responsible production and consumption), as the mining project is assessed for sustainability by the SDC. The connections between prioritized themes supports synergic effects between areas of concern of sustainability identified by the SDGs and the SDAG as well.

Such coverage encompasses all the stakeholders’ concerns about PSPP and it warrants a systemic understanding of sustainability. It also sets for the selection of indicators in various fields to report on progress towards every prioritized themes or objectives. However, there is a possibility that the common understanding of sustainability, more simplistic than the SDAG, may not correspond to the dominant thinking, or that prioritized objectives from the SDAG may differ from more urgent or perceived priorities in a given society. For example, sustainable development priorities must be reconsidered in areas affected by natural disasters or in areas where poverty is of outmost importance. However, the wider scope of concerns raised by the SDAG is a good start for every PSPP conception or evaluation, as proven by the multiple experiments with the SDAG performed in developed as well as developing and emerging countries.

Waas et al. [1] identifies two criticisms for modeling sustainability: first the static and separate nature of dimensions which minors the capacity to understand interactions between objectives; and, second, the fact that models do not address the planetary limits. The SDAG is built on a dynamic model where dimensions are closely interrelated and where performance can be seen by the changes in the diagram shape for dimensions as well as for sub-dimension themes. Finally, the SDAG does not address the planetary boundaries as formal limits but presents each with a precautionary approach, thus leaving room for the discussion of cumulative impacts of PSPPs on the carrying capacity of the planet and/or local environment. That follows the normative nature of sustainability. It also presents sustainability as a dynamic issue, or in short: a hypothesis.

4.2. Stakeholders Participation

The complex nature of sustainability calls for a major shift in governance [1] to include the participation of as many stakeholders as possible illustrating the equity principle. In that sense, it is particularly interesting to note for the Mine Arnaud mining project that the Governance dimension was the most prioritized by SDC with 95% of the objectives weighted as “3”—Essential. This indicates that the importance of compliance to existing regulations, as well as public involvement in the control processes and planning of the mine over its lifespan, are regarded as a minimum requirement by stakeholders. This situation is observed in most assessments conducted elsewhere with the SDAG.

Sustainability assessment is holistic. A SD assessment with the SDAG aims to broaden stakeholders’ thinking and stimulate dialogue on a given PSPP by integrating various SD issues at multiple levels. It can also serve as support for project planning and decision-making. The involvement of diverse participants, at least for weighting and at best for the complete process, makes people aware of the in-depth considerations underlying PSPPs. For the promoter, dialogue with and between various stakeholders also is a positive asset. The results obtained during an analysis illustrate what needs to be considered and addressed to ensure SD according to the members of the analysis, underlining the need for a broad array of stakeholders being involved in the assessment. Difficulties encountered during the assessment often stems from analytical team not properly and/or unanimously define the scope of the analysis.

A SD assessment is not de facto validation of PSPPs, as some have criticized [31] (p. 370). It can lead to the relevance of a specific project being questioned [32], often as its opponents compare the extent of environmental disturbances to limited social and economic benefits. In this perspective, ex-ante sustainability assessment with the SDAG involving the project promoter and various stakeholders is preferable and favored. When the promoter and other stakeholders are part of the weighting and evaluation processes, consensuses are attained on most objectives and the entailed discussions allow more in-depth sharing between parties, possibly more understanding and even social acceptability. Using the SDAG prior to decision making can reveal unsuspected benefits or impacts and allows well informed parties to search for solutions. It follows best practices principles, where stakeholders, e.g., indigenous people concerned by a PSPP, can give free, prior and informed consent [33].

4.3. Priority Settings

SD is a process and cannot be achieved with one action. Priorities must be set, as every weighted and evaluated objective in the assessment process will justify actions to be implemented in a world with limited resources.

When using the SDAG, an analysis report is produced at each use to identify objectives that should be improved first, but also to highlight the strengths and weaknesses of a PSPP. The analysis report is also used to communicate and share the results to the public. In accordance to the Effective communication principle (New BellagioSTAMP), [2], the report use clear and plain language, present information in an objective way, use visuals and graphics and make data available. To do so, analysts can comment on the diagrams and organize the information contained in the “Actual and projected measures” and “Improvements” columns. “Actual and projected measures” will give information on highlights while “Improvements” should guide planning the priority actions to be implemented.

In the Mine Arnaud case study, strengths and weaknesses analysis permitted to highlight actions that should be considered as priorities to enhance the sustainability of the whole project, following the combination of weighting and evaluation. For example, analysis of the theme “Global pollutants”, which was prioritized “2—Important”, had two objectives weighted “3—Essential”: “reducing greenhouse gases (GHG) emissions” and “planning global change adaptation measures”. Those two objectives were “moderately addressed” and “poorly considered” respectively with 54% and 17%. The two other theme’s objectives, “reducing ozone layer depleting substances” and “reducing persistent organic pollutants” were weighted “1-Desirable”. Hence, even if the theme as a whole cannot be considered a priority, the analysis underlines the necessity to react on the GHG abatement and climate change adaptation issues. Thus, improvements were suggested to reduce GHG in the extraction process such as the use of trolley trucks, renewable hydroelectricity being available near site. This one improvement could have many positive effects, enhancing performance of seven objectives in six Themes pertaining to three Dimensions, such as “Originality and innovation” in the Ethical dimension, “Risk management” in Governance and “Renewable resources” in the Ecological dimension. It was therefore prioritized in the final list of improvements. That result illustrates the SDAG capacity to comply with the integration principle.

4.4. Continuous Improvement

SD bears normative content and SA must be dynamic allowing stakeholders to follow and adjust to the achievement (or not) of expected results, i.e., if targets or objectives, are attained. To meet the dynamic principle, periodic evaluations of PSPPs must be performed. With the SDAG, each PSPP is evaluated according to its own objectives weighting. In accordance with the Continuity and capacity principle (New BellagioSTAMP) [2], it is the objectives’ progression and not necessarily their initial position that matters. This approach avoids the pitfalls of unnatural comparison or rating in the absolute. Although PSPPs of different scales can be analyzed with the SDAG, such as a community garden project or a National Sustainable Development Strategy, comparing performance of the two is senseless: they do not relate to the same needs, and cannot be implemented in the same context.

Improvements identified during the analysis with the SDAG should guide the promoter in a continuous improvement process. The improvements combined with the prioritization index, give an overview of the most relevant actions to be taken to enhance the project. Some improvements appear more than once and can affect several SD dimensions, reinforcing the integration principle. Other improvements will be more relevant because they relate to the “React” priority and should be targeted in the first place.

The prioritized improvements list shows the improvements identified during the evaluation process and how their consideration could improve the project performance on one or more objectives. It should be noted that the assessment of improvements’ feasibility was not performed in the Mine Arnaud analysis. It was to be carried out by the promoter. In the 2016 SDAG, additions on prioritized actions were made to bridge this gap. It includes a preliminary identification of synergies and antagonisms as well as a basic feasibility tool.

4.5. The SDAG’s Advantages and Limitations

There are other tools and processes to achieve SD assessment, varying in scope, flexibility and overall quality [34,35]. The aim of this paper is not to offer a comparison but rather to present the SDAG as one of the possible SD assessment tool and process. Amongst its advantages one can note the very broad spectrum of SD stakes it can cover by its complete and rigorous methodology and its refined interpretation modalities to help decision-making. Using the SDAG allows for a complete interpretation of the results, including looking at the equilibrium between the dimensions and the themes, prioritizing the objectives to be improved as well as a mechanism to prioritize the steps to be implemented and a brief feasibility report. The SDAG’s content stems from international declarations and is updated regularly, starting with the Brundtland’s report 30 years ago to the SDGs of the 2030 Agenda for SD. The SDAG tool and process can, and often is, used as an SD quality control assessment, being used to analyze National Sustainable Development Strategies developed with other SD tools [36].

Although well thought out and perfected over time, there are limitations to what the SDAG process can do. One of the drawbacks of using the SDAG, a complete and thorough tool, is the time required to go through the process. Working with groups of diverse stakeholders may require up to three full workdays only for weighting with the help of a skilled moderator. The subsequent evaluation depends on the available information and may require more time. The latter analysis for priority setting is a matter of one week. A worthy time-investment and short compared to a financial analysis but many promoters, companies or others, still need convincing. The versatility of the SDAG also means that it must be adapted to each context through weighting and this step generates intersubjectivity. People may want to value an objective at “0”, as they do not see the connection with the particular PSPP.

The analysis process using the SDAG requires a change in paradigm, where stakeholders must change from a competing frame of mind to a collaborative one. For the dialog to be productive, the analysts must be trusting, open-minded, practice active listening and should benefit from an adequate accompaniment. Obviously, these perfect predispositions are not always attainable without a trained moderator.

Using the SDAG can stir up fears in the stakeholders: fear of realizing and confronting the balance of power, fear of being judged, fear of conflicts, fear of causing additional costs and even the fear of seeing the project abort [34]. Navigating through these human issues requires abilities and communication skills by the analysts.

Another limit of the SDAG relates to the arbitration that must take place between the ease of use of the tool and the intrinsic complexity of SD. PSPP analysis can underline redundancies, crosschecks, synergies and antagonisms between SD objectives. The analysis of the synergies among objectives, themes and dimensions, cannot be automated and should be performed by the analysts: requiring specific knowledge. The lack of expertise at the interface of a specific discipline can hinder the application of a real transdisciplinary approach yet at the core of the implementation of an integrated SD approach.

5. Conclusions

Sustainability assessment using the SDAG has proven efficient in many contexts over the years. It is a rigorous, effective and powerful tool that can be used for different levels of policy or project planning and allows for public participation and education regarding sustainability issues. However, pertinent at the local level, sustainability initiatives will only make a difference at the global level if their results are propagated and replicated around the globe. In that perspective, the SDAG gives a coherent framework to assess sustainability of a local project, to compare with similar ones and to connect to higher levels of governance. For example, a project such as Mine Arnaud assessed with the SDAG can be followed through future evaluations and implementations over time. It can also be linked to a program such as the “Plan Nord” in Quebec—aiming to develop the Northern areas—a Climate change mitigation strategy and to a policy such as Quebec’s “Loi sur le développement durable”—Québec SD law. SD assessments performed with the SDAG allow for priority setting with strong background documentation, stakeholders support and objectivized tracking for decision makers. Initiatives and use of the SDAG by numerous countries engaged in the process of Agenda 2030 will provide abundant further material and examples in a near future.

The SDAG fits into a complete and integrated range of tools, called Systemic Sustainability Assessment, which ensures structural consistency in assessing sustainability by taking into account the various dimensions of SD, prioritizing multilevel actions, integration of inputs from different stakeholders, evaluation and continued improvement in an iterative approach and ensuring accountability at all levels [37]. These tools, currently in development jointly with the Organisation Internationale de la Francophonie (OIF), are built consistently and complementarily. In addition to the SDAG, they combine a Grid for Prioritizing SDG targets (GPT-SDGs), which is a participatory prioritization of the targets at the different levels of developmental planning; a SDGs Atlas encompassing the prioritized targets with the monitoring indicators; a Competency Analysis assessing the quality of the human resources required for the implementation of the PSPP; and a Sustainable Development Governance Sheet (SDGS), also selected in the UNDG Sustainability Acceleration Toolkit, for accountability in relation to the previous strategic planning for sustainable development. Systemic Sustainability Assessment is part of ongoing research by the Eco-Advising Chair at UQAC in partnership with the OIF and numerous users in industrial, state and municipal agencies.