Next Article in Journal
An Assessment Method of Urban Traffic Crash Severity Considering Traveling Delay and Non-Essential Fuel Consumption of Third Parties
Next Article in Special Issue
Agricultural Workforce Crisis in Light of the COVID-19 Pandemic
Previous Article in Journal
Research on Network Patterns and Influencing Factors of Population Flow and Migration in the Yangtze River Delta Urban Agglomeration, China
Previous Article in Special Issue
Application of Artificial Neural Networks for Natural Gas Consumption Forecasting
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Sustainability Assessment of Investments Based on a Multiple Criteria Methodological Framework

Paraskevi Ovezikoglou
Dimitrios Aidonis
Charisios Achillas
Christos Vlachokostas
3 and
Dionysis Bochtis
Department of Quality Management and Technology, School of Science and Technology, Hellenic Open University, GR 26335 Patra, Greece
Department of Supply Chain Management, School of Economics and Business Administration, International Hellenic University, GR 60100 Katerini, Greece
Laboratory of Heat Transfer and Environmental Engineering, Aristotle University Thessaloniki, GR 54124 Thessaloniki, Greece
Institute for Bio-Economy and Agri-Technology (iBO), Center for Research and Technology–Hellas (CERTH), 6th km Charilaou-Thermi Rd, Thermi, GR 57001 Thessaloniki, Greece
Author to whom correspondence should be addressed.
Sustainability 2020, 12(17), 6805;
Submission received: 26 July 2020 / Revised: 17 August 2020 / Accepted: 20 August 2020 / Published: 21 August 2020
(This article belongs to the Special Issue Green, Closed Loop, Circular Bio-Economy)


The assessment of an investment is currently carried out by using mainly financial tools. This work presents a new model for the assessment of the sustainability of an industrial investment and focuses on the development of a holistic framework with the use of indicators. With the use of multi-criteria decision analysis, the framework evaluates a total of eighteen (18) alternative indicators in order to select the optimal bundle to be used for the assessment of future industrial investments. The proposed indicators are selected based on relevant data from the literature, taking into account the principles of prevention, planning and designing. The alternatives are assessed over four (4) criteria, namely environment, society, economy and technology, which are grounded on the principles of sustainable development. Depending on the special characteristics of the programme that is foreseen to fund the potential investments, the decision-maker is provided with a hierarchized set of indicators over which the alternative investments could be optimally assessed in parallel with widely used indicators that strictly assess economic performance. In the present work, twelve (12) different scenarios are examined, incorporating different values in the coefficients of the criteria. For the majority of the scenarios examined (a sensitivity analysis is also provided), the alternative indicator that is assessed with the highest score is “Resource Savings”, followed by “Recycling” and “Research, Innovation, Development”.

1. Introduction

Environmental performance indicators are considered to assist decision-making processes in managing important environmental, social and financial aspects and perspectives and improving the assessment of the impact caused by business activities [1,2,3,4,5]. Improving environmental performance requires effective control of the activities, products and processes of the business that may trigger a significant burden [6,7,8]. Improving the business performance can be achieved through a wide spectrum of modifications in corporate activities, products or processes and can range from small fragmentary changes to integrated environmental management.
Changes in this direction include, among others, inter-alias environmental education of workers and stakeholders, informing and/or sensitizing customers and suppliers, investing in environmental protection technologies, adopting optimally available techniques to minimize gaseous pollutants, etc. A significant process is the so called “Design for Environment” (DfE), i.e., the ecological design of products and services and the creation of environmental reports (corporate environmental reporting), as well as the installation of environmental management systems [9,10,11,12].
Environmental performance appraisal is an internal process of business and is essentially a tool designed to provide management with reliable and verifiable information on an ongoing basis to determine whether an enterprise’s environmental performance meets the criteria set by the organization’s management [13,14,15]. The literature presents several available environmental indicators that are used at different scales of business activities, namely international, national or local. As evaluation methods vary, environmental performance indicators, as well as the concept of sustainability, also differ and include diverse groups of indicators [16,17]. In particular, indicators related to specific environmental consequences (e.g., climate change) have been developed, using resources (e.g., water footprint) with ecological efficiency measures [18].
There are different approaches to measuring environmental performance, namely production, control, ecological, accounting, economics and quality [19]. These approaches have different guidelines, focus and measurements. It is obvious that performance measurement activities vary in different countries and also among different industries, due to the variety of environmental issues, organizational variables (e.g., the size of the organization or the way it is structured), national conditions and individual corporate strategies [20].
According to the definition provided by the Organization for Economic Co-operation and Development (OECD), an environmental indicator is characterized as “a parameter that describes the state of a phenomenon/environment/area, with significance that extends beyond that directly related to a parameter value” [21]. Therefore, an indicator needs to provide important information about the parameter to be described. If a parameter is complex, such as sustainability, more than one indicator may be required. On the other hand, a group of various indicators may be combined to produce a single indicator, i.e., a Composite Sustainable Development Index (CSDI), if desired. The Environmental Sustainability Index (ESI), the Dow Jones Sustainability Index (DJSI) and the Global Reporting Initiative (GRI) are examples of CSDI. Furthermore, there are various environmental performance indicators, but also specific indicators focused mainly on environmental impacts, such as climate change [22], air pollutants [23] and ad-hoc application of systems of indicators as decision-support tools towards sustainable urban development [24].
The indicators that best describe environmental performance can be divided into the following groups [25]: (a) lagging indicators, which are measures at the end of a process, such as the amount of emitted pollutants; (b) leading indicators, which are performance measurement measures, i.e., they measure the implementation of practices or measures that are expected to lead to improved environmental performance, such as the percentage of facilities that carry out self-monitoring; (c) Environmental Condition Indicators (ECIs) measure the direct impact of an activity on the environment, such as air, water, groundwater and soil concentrations, changes in the size of a population of a particular species in a given area. Each group of indicators has discrete strengths and weaknesses, aiming a different target audience, and this is the reason why many companies use a mix of indicators to measure their environmental performance [25].
In addition to the use of indicators, progress on environmental issues can be assessed by comparative evaluation between companies or by the average performance of the industry to which a company belongs. The International Organization for Standardization (ISO) has compiled a list of conditions that a marker must meet to be useful and relevant to the measurement of environmental performance. According to this list, an indicator must be: (i) in accordance with the environmental policy and the important environmental aspects; (ii) suitable for management, business or environmental activities; (iii) useful and representative of the environmental performance criteria; (iv) understandable to internal and external stakeholders; (v) easily accessible, measurable and informative; (vi) adequate in relation to the quality and quantity of data; (vii) able to respond to changes in environmental performance [26].
This paper focuses on the development of a holistic framework for the assessment of industrial investments’ sustainability with the use of indicators. The key research question that is examined in the present work is the identification of the optimal bundle of indicators which could be used for the assessment of alternative industrial investments. It is evident that in most cases, available funds are not adequate to cover all needs, thus decision-makers (either industrial or public) require a concrete methodology for the assessment of alternative investments and the identification of the optimal one, based on sustainability criteria. To date, the evaluation and selection of an investment over competition in most cases is solely realized based on their economic performance and indicators (Return on Investment, Net Present Value, etc.). However, the selection of the optimal criteria for the assessment of investments is, without any doubt, a highly multi-dimensional problem. The proposed framework is the outcome of a research effort that incorporates collection of data with the use of a structured questionnaire. For the determination of the optimal set of indicators that best capture the environmental performance of a given investment, the opinions of a number of different stakeholders, with mutually contradicting priorities (e.g., environmental organizations, companies, universities, public bodies, environmentalists and economists), are considered in the present work. The proposed environmental (non-monetary) indicators are proposed to be used in parallel to the currently widely used economic ones, in order to more efficiently and holistically assess alternative industrial investments.

2. Methodology

A critical point in multidimensional management problems is the evaluation and combination of different types of available information that are ultimately able to lead to an optimal solution [27]. Multi-criteria methods provide the framework for collecting, registering, and ultimately promoting all relevant information, thus making the decision-making process detectable and transparent [28,29]. In this light, the adoption of a decision is based on the result of the analysis of the conflicting parameters and goals of socio-political, economic and environmental nature, thus creating a multidimensional problem that needs special treatment [30]. The nature of decision-making processes makes it difficult to represent them in descriptive models. Furthermore, uncertainty and inaccuracy are inalienable elements of their structure.
Multidisciplinary analysis can be defined as a systematic and mathematically standardized effort to solve problems arising from conflicting goals in an effort to reconcile them [31]. Making a decision is the study of identifying and selecting alternative solutions based on the preferences of the recipient’s decision. Decision making also implies that there are alternatives being considered. In this case, the goal is not only to identify as many of the alternatives as possible, but to choose the one that best fits the goals and desires of the decision maker [32]. Decision making with the use of multi-criteria analysis is realized in four discrete steps, as follows: The first step comprises the determination of the alternative scenarios for the selection of environmental indicators. The second step includes the selection of the criteria, the scoring scale of the alternative scenarios and the assessment of the weighting factors by the decision makers. The next step includes the application of the multi-criteria analysis and the extraction of the results, followed by a last step, where the decision is realized for the selection of the appropriate set of indicators, taking into consideration the use results of a sensitivity analysis.
The selection of environmental indicators for investment evaluation is a very complex process. A considerable number of alternatives, often presenting equivalent weightings, need to be evaluated [33]. To efficiently achieve such an assessment, it is necessary to analyze and grade a series of critical parameters, or other criteria. In particular, the formulation of an integrated policy regarding the creation of environmental indicators for investment evaluation is considered a rather complex process, given that the number of environmental indicators (alternatives) can be quite large, and at the same time, each indicator shows advantages and disadvantages on different levels, namely economic, environmental, social or technological (criteria).
The combination of all the parameters that appear makes the selection of an environmental indicator a rather complex problem. The final selection of the most appropriate buddle of indicators between alternative scenarios requires the consideration and evaluation of several parameters, therefore, it is necessary to apply the multi-criteria analysis. In the literature, there are over 50 multi-criteria analysis techniques [34,35], and a different classification can be attempted according to their content and scope.
In the present work, the ELECTRE III technique is selected to process the collected data via the distributed questionnaires. ELECTRE III is a well-known method of multi-criteria analysis, with a long history of successful practical applications internationally [29,36,37]. The method was used often in the past to compare different scenarios in different thematic areas, such as energy, construction, waste management, services, public policy, and transportation. An important advantage of the ELECTRE III method over other methods is its usefulness in examining environmental problems [38]. In addition, ELECTRE III has the ability to include a fairly large number of evaluation criteria for the selection of environmental indicators, combined with the ability of a large number of decision makers [39]. The method requires the determination of three threshold values of the criteria used, i.e., the indifference, the preference and the veto threshold. These thresholds allow the uncertainties of the evaluation criteria to be incorporated into the decision-making process [40].
Determining the recipient’s preference data for a decision expressed as a criterion is one of the most important factors of ELECTRE III. It is already noted that the method uses the thresholds of preference and indifference, and includes an additional parameter, the concept of veto [41]. By using these parameters, the method examines not only the two extremes of the problem, strong and weak, but also a whole family of intermediate levels, from the overall strongest to the overall weakest alternative. The process is achieved by assessing, comparing and finalizing the various environmental selection indicators (alternatives) over the criteria considered.
As a ranking technique, ELECTRE III calculates a ranking hierarchy among alternatives. The ranking is based on concordance (cj) and non-discordance (dj) binary outranking. In brief, concordance is valid for the cases where alternative X outranks alternative Y when most of the criteria X’s performance is better than the alternative’s Y. Respectively, non-discordance is valid for the cases where none of the criteria in the minority are strongly opposed to alternative’s Y outranking by alternative X. The ELECTRE III methodology calculates a credibility index, which characterizes that X outranks Y. The credibility index shows the real degree of the aforementioned assertion [42].
Following this, alternatives are pairwise compared for every criterion by inserting two more pseudo-criteria, namely the preference (pj) threshold and the indifference (qj) threshold. In the case where the difference between the performances of X and Y is lower than the indifference threshold for a specific criterion, then the two alternatives are regarded as similar for that criterion j, and the credibility index cj(X,Y) equals zero. On the other end, in the case where the difference between the performances of X and Y is larger than the preference threshold for a specific criterion, then X is strongly preferred to Y for that specific criterion j, and the credibility index cj(X,Y) equals one. In this context, the concordance index cj(X,Y) for any criterion j is mathematically described with Equation (1).
V j X V j Y q j c j X , Y = 0 q j < V j X V j Y < p j c j X , Y = V j X V j Y q j p j q j V X V Y p j c j X , Y = 1
Taking into account all the concordance indices calculated for each criterion j, and also the weighting factor (relative importance) of each criterion j(wj), a global concordance index is calculated for every pair of alternatives (X,Y). The global concordance index (CXY) is mathematically described with Equation (2).
C X Y = j = 1 n w j × c j X , Y j = 1 n w j
As a next step in the methodology, a discordance index (dj) is calculated, with the assistance of preference (pj) and indifference (qj) thresholds, as those were described above, and with the use of a third threshold, namely veto (vj), that gives the acceptable (maximum) difference between the performances of two given alternatives X and Y in criterion j for not rejecting the assertion that X outranks Y, regardless of the alternative’s performance in all other criteria. More specifically, in the case that the difference between the performances of X and Y is lower than the preference threshold for a specific criterion, then no discordance exists and dj(X,Y) equals zero. On the other end, in the case that the difference between the performances of X and Y is larger than the veto threshold for a specific criterion, then Y is globally preferred to X, no matter the performances in all other criteria, and the discordance index dj(X,Y) equals one. In this context, the discordance index dj(X,Y) for any criterion j is mathematically described with Equation (3).
V j Y V j X p j d j X , Y = 0 p j < V j Y   V j X < v j d j X , Y = V j Y V j X p j v j p j V j Y V j X v j d j X , Y = 1
The credibility index δΧΥ of the assertion “X outranks Y” is mathematically formulated with the use of Equation (4).
δ X Y = Π j F ¯ 1 d j X , Y 1 C X Y ,   where   F ¯ = j F , d j X , Y > C X Y
In order to determine the optimal set of environmental indicators (alternatives) for the assessment of potential investments within the present work, a questionnaire was used, aiming at the collection of the required data that would feed the ELECTRE III methodology. The questionnaire considered a number of indicators that are commonly used to evaluate the environmental performance of a company in operation, the indicators for the evaluation of environmental performance in highly industrialized countries (e.g., USA, UK, EU, Japan) [43], as well as the ISO 14031/2013 standard [44]. The aim was to explore the dominant aspect of tackling the problem, namely the optimal choice of environmental indicators, through the views of experts. A total of 80 experts’ responses were collected, representing all different stakeholders involved in the decision-making process, namely NGOs, business, academia, authorities, certified assessors, and scientist/practitioners activated in the field, in order to capture different needs and requirements. More specifically, half of the experts’ sample (40 out of 80) represented the private sector (31 practitioners and business consultants activated in industrial investments and nine senior staff in private companies), while the other half represented the public sector (17 representatives from public authorities at local-to-regional level, 9 representatives from the academia sector, 3 certified assessors and 11 representatives from NGOs activated in the field of environmental protection and sustainability). The aim was to evaluate alternative indicators from all different aspects, with the involvement of a balanced sample of experts, since the former (practitioners, business consultants, enterprises) focus mostly on the business success of the proposed investment, while the latter (public authorities, academia, NGOs) place emphasis on the common public interest.
The questionnaire consists of 18 indicators (alternatives) that were coded to facilitate the processing of the results (Table 1). Potential indicators of sustainable development were selected taking into account the pillars of sustainable development and research work in international literature [43,44,45] and assessed over four discrete criteria by the experts involved in the framework of the present work. The criteria used for the assessment of indicators’ suitability are grounded on a set of sustainable development’s pillars and principles. The pillars are the environment (Criterion C1), society (Criterion C2), and economy (Criterion C3). To efficiently evaluate an investment, the above criteria are the main priorities for its successful operation and a more sustainable future. In addition, technology (Criterion C4) was selected as the criterion for evaluation, given that environmental technology and technological infrastructure provide the basis for faster and cost-effective development.

3. Results and Discussion

Within the framework of the present study, an assessments’ matrix was formed, consisting of the alternative scenarios of environmental indicators, and criteria over which the selected alternative scenarios were assessed by the experts. In Table 2, the evaluations of 80 experts are depicted in a scale from 1 to 9, where 1 represents the worst-case assessment and 9 the best-case one. The assessment of each alternative for each criterion is calculated as follows:
V i j = 1 N v i j N ,   i A 1 , A 2 , , A 18 ,   j C 1 , C 2 , C 3 , C 4
  • V i j : Assessment of alternative scenario i based on criterion j for all experts
  • v i j : Performance of an alternative scenario i based on criterion j for each expert
  • N: Total number of experts
In the present work, the weighting factors of each criterion were determined by the experts participating in the research. In particular, experts were required to assign a percentage of importance to each criterion according to their personal opinion. The values of the criterion weighting factors emerged as averages of the views of the various experts, who took part in the qualitative evaluation of the environmental selection scenarios. For the assessment of the importance of the selected criteria (environment, society, economy, technology) a weight scale from 0 to 100% was used.
In respect to the preference and indifference thresholds that are required for the ELECTRE III methodology, the following equations were considered [38,46,47,48,49]
p i = ( v i m a x   v i m i n )   N
q i = 0.3 × p i
  • v i m a x : The maximum value displayed by alternative i for criterion j
  • v i m i n : The minimum value displayed by alternative i for criterion j
  • N: The number of alternative scenarios (here N = 18)
According to the ELECTRE III technique, two distillations are calculated (namely, ascending and descending), before the determination of the final order of the available alternatives. In the case under study, the LAMSADE software was used. In Figure 1, the distribution of the ascending and descending distillations is illustrated for the Baseline Scenario (BS), i.e., taking into account: (a) the opinion of the 80 experts with respect to their assessment for the performance of the 18 alternatives over the four selected criteria, (b) the opinion of the 80 experts with respect to the weighting of the four criteria’s importance, (c) Equations (6) and (7) for the calculation of the preference and indifference thresholds, in particular, in the x-axis, the ascending distillation (i.e., from the worst to the optimal alternative), while in the y-axis, the descending distillation (i.e., from the optimal to the worst alternative), are provided. For instance, alternative A03 (resources’ savings) is the optimal one in both distillations. Correspondingly, alternative A01 (recycling) is ranked third (following A03 and A14) when considering the optimal hierarchy from the worst to the best one (ascending distillation), while ranked second (only after A01) when considering the optimal hierarchy from the best to the worst one (descending distillation).
The final ranking of the alternatives (indicators) is calculated taking into account the two aforementioned distillations. For the BS, the ranking of the alternative indicators is the following; (i) Resources’ savings [A03], (ii) Recycling [A01], (iii) Research, Innovation, Development [A14], (iv) Impact restoration [A05], (v) Health and safety [A08], (vi) Gas emissions [A02], (vii) Social actions [A11], (viii) Pollution prevention [A12], (ix) Alternative energy forms [A06], (x) Information across supply chain [A10], (xi) Legal framework [A16], (xii) Communication and public awareness [A09], (xiii) Environmental policy [A15], (xiv) Environmental accounting [A13], (xv) Biodiversity [A04], (xvi) Education [A07], Environmental standards [A17], Corporate governance [A18].
In addition to the Baseline Scenario (BS), nine (9) additional scenarios were considered for sensitivity analysis purposes. In other words, additional scenarios are examined to study whether changes in the parameters of the problem affect the final solution, with the aim of providing further confidence in decision-making. More specifically, in order to globally assess the alternative indicators (taking into account their performance in the four described criteria, namely environment, society, economy and technology), the following parameters need to be determined: (a) the weighting factor (relative importance) of each criterion, and (b) two pseudo-criteria, namely the preference and the indifference threshold. In the present study, sensitivity analysis is selected to be applied in comparison to the Baseline Scenario. In Table 3, the key parameters (thresholds and weighting factors) of the scenarios (Sx) examined are depicted. The Baseline Scenario (BS) reflects the solution of the mathematical algorithm (Equations (1)–(5)), taking into account the weighting factors (relative importance) of the criteria as averages of the experts’ views. In this light, the weighting factor for the environmental criterion [C1], the social criterion [C2], the economic criterion [C3] and the technological criterion [C4], are 31.9%, 23.6%, 25.1% and 19.4%, respectively. Moreover, for the determination of the preference and indifference thresholds, we used the referenced Equations (6) and (7). However, since both the views of the experts are subjective, while the equations for the determination of the pseudo-criteria (preference and indifference thresholds) are based on assumptions, the algorithmic model presented in the methodology is solved with alternative values with respect to weighting factors and thresholds, so as to provide a “what-if” analysis. In this context, the hierarchy of the alternative indicators is re-assessed with the use of different weighting factors (putting more emphasis on different criteria) in each scenario examined. More specifically, in Scenario 1 (S1), all criteria are equally weighted (25%), compared to the Baseline Scenario (BS), where more emphasis is placed on the environmental pillar of sustainability. In Scenario S2, more emphasis is put on the economic criterion [C3] which weighs 50%, while each of the environmental [C1] and social [C2] criteria weigh 25% and the technological criterion [C4] is neglected. Similarly, different weight factors are applied in the case of the rest of the different scenarios (S3–S9). For the scenarios S10 and S11, the weighting factors are based on the experts’ views (similarly to the Baseline Scenario), while the preference and indifference thresholds are altered compared to the values calculated with Equations (6) and (7) in order to assess the sensitivity of those thresholds in the optimal hierarchy of the indicators. The analysis, apart from providing robustness to the ranking of the Baseline Scenario, can be used for altering the weights of the criteria based on the individual needs of the funding programmes.
In Table 4, the final ranking (hierarchy) of the alternative indicators are illustrated for the scenarios examined. Apparently, the ranking shows significant changes in cases of change in the thresholds of preference and indifference. However, the results provide adequate information in the selection of a bundle of indicators to be used for the assessment of alternative investments. More specifically, for the Baseline Scenario, Alternative A03 (Resources’ savings) is the highest ranked indicator to be considered in the environmental assessment of potential investments. The same applies for all examined scenarios, except S9, where the focus is solely on economic and technological criteria, which provides robustness to the optimal solution and confidence to the decision-maker so as to always consider the indicator when assessing the environmental performance of any given investment.
Since, in most programmes, more than one criterion is simultaneously considered in order to select the optimal investment, the bundle of five top indicators in the Baseline Scenario was comprised, apart from Alternative A03, by Alternatives A01 (Recycling), A14 (Research, Innovation, Development), A05 (Impact restoration) and A08 (Health and safety). Research, Innovation, Development (A14) is included in the top five criteria for all examined scenarios (being the highest ranked alternative indicator for 3 out of 11 scenarios), while recycling (A01) and impact restoration (A05) are included in the top five criteria for 10 out of 11 examined scenarios (A01 is ranked 6th for S8, while A05 is ranked 6th for S10). In this light, the aforementioned criteria should be considered in the top bundle of indicators to be selected when designing a funding programme, accompanied by indicators like A14 (Research, Innovation, Development), A11 (Gas emissions) and A06 (Alternative energy forms), which are placed comparatively highly for most of the examined scenarios. On the contrary, Alternative A18 (Corporate governance) is the last indicator to be considered for the assessment of investments, as it is ranked 18th for 8 out of 11 scenarios.
It should be highlighted from the analysis of the variation of the coefficients that the ranking of the alternative indicators in the Baseline Scenario (BS) and the Scenario S1 where all criteria (environmental, social, economic, technological) are equally considered (with a weighting factor of 25%), are identical in all ranking positions. Consequently, the experts’ opinion on the significance of the criteria does not significantly affect the ranking of the indicators. In this light, all criteria could be equally considered (as realized in S1) in a real-world case.
Overall, the ranking of environmental indicators for investments’ assessment was observed to be significantly influenced by the weighting factors and preference and indifference thresholds. The latter demonstrates the choice of A03 as the optimal alternative indicator, but, on the other hand, also reveals that the final choice of the optimal bundle of environmental indicators for the assessment and evaluation of investments is left purely to the decision-maker and to the thresholds used. This provides the funding authority with appropriate “freedom” to apply the most suitable weighting factors and thresholds that best suit the particular needs of the funding programme and the pillar of sustainable development that should be promoted.

4. Conclusions

In the past, the evaluation and selection of an investment over competition was mostly realized based on their economic performance, using mainly financial tools. Nevertheless, the determination of the optimal investment is an interdisciplinary problem and apart from the economic performance, other sustainability criteria need to be considered. The need for an effective bundle of environmental indicators that would lead to best possible investments and cost-efficient use of available funds triggered the need for a multi-criteria analysis methodology, such as the one herein described.
The survey conducted in the framework of the present work, with the involvement of 80 experts closely related to the field, reveals that design for energy and other resources (materials, water, etc.) savings, and reduction in non-renewable resources consumption, are the most appropriate criteria, apart from cost, to introduce as additional indicators in the investments’ evaluation. Additionally, the provision for research and development of high-tech and/or innovative products and the development of green products (design for disassembly/recycling/reuse) could be supplemented as a second-best assessment indicator.
The importance of multi-criteria analysis is critical in environmental problems. The ELECTRE III method was preferred over other multi-criteria techniques for selecting environmental indicators to be applied to investment evaluation. In the proposed methodology, 18 scenarios were selected as alternatives, based on relevant data from the literature and taking into account the principles of prevention, planning and design. Moreover, four criteria are considered, namely environmental, social, economic and technological, in order to capture the different pillars of sustainable development. With the use of the ELECTRE III, the optimal bundle of criteria is extracted for twelve (12) scenarios with differentiated weighting factors for the four criteria, and also preference and indifference thresholds.
From the analysis of the scenarios, it is evident that the of environmental indicators is influenced by the selected parameters of the methodology, however, there is a dominant trend demonstrating that specific indicators (resources’ savings, recycling, Research–Innovation–Development, impact restoration) should undeniably be considered for the overall assessment of investments. At the same time, the modified rankings, such as those resulting from the realized sensitivity analysis, demonstrate that the final word for the selected bundle of indicators is left to the decision-maker. The latter, on the basis of the particular needs of the funding programme, is responsible for drafting the weights for the criteria and also determines the thresholds for investments’ evaluation.
The proposed methodology can be seen as a tool through which decision-makers may select additional indicators that can create a framework of sustainable assessment of potential investments. Based on the results presented herein, the legislative framework could be improved so that sustainable growth indicators can also be incorporated in the decision-making process for the evaluation of an investment. Undeniably, this research can be extended not only to public authorities, but also to businesses in their effort to promote sustainable products and solutions. The present work represents an initial attempt to reach this goal, however, further research is required in terms of sample size (including international bodies and funding agencies) and the criteria considered, so as to demonstrate the optimal indicators that should be incorporated as key performance indicators, alongside financial ones, for the assessment of investments.

Author Contributions

Conceptualization, P.O. and D.A.; methodology, P.O. and D.A.; software, C.A.; validation, C.V. and D.B.; resources, P.O. and C.A.; data curation, P.O., D.A. and C.A.; writing—original draft preparation, P.O.; writing—review and editing, C.A.; visualization, C.A.; supervision, D.A. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.


The authors would like to wholeheartedly thank the experts that participated in the survey.

Conflicts of Interest

The authors declare no conflict of interest


  1. Medne, A.; Lapina, I. Sustainability and continuous improvement of organization: Review of process-oriented performance indicators. J. Open Innov. Technol. Mark. Complex. 2019, 5, 49. [Google Scholar] [CrossRef] [Green Version]
  2. Di Vaio, A.; Varriale, L.; Alvino, F. Key performance indicators for developing environmentally sustainable and energy efficient ports: Evidence from Italy. Energy Policy 2018, 122, 229–240. [Google Scholar] [CrossRef]
  3. Maslesa, E.; Jensen, P.A.; Birkved, M. Indicators for quantifying environmental building performance: A systematic literature review. J. Build. Eng. 2018, 19, 552–560. [Google Scholar] [CrossRef]
  4. Pilouk, S.; Koottatep, T. Environmental performance indicators as the key for eco-industrial parks in Thailand. J. Clean. Prod. 2017, 156, 614–623. [Google Scholar] [CrossRef]
  5. Achillas, C.; Vlachokostas, C.; Moussiopoulos, N.; Perkoulidis, G.; Banias, G.; Mastropavlos, M. Electronic waste management cost: A scenario-based analysis for Greece. Waste Manag. Res. 2011, 29, 963–972. [Google Scholar] [CrossRef]
  6. Feleki, E.; Vlachokostas, C.; Moussiopoulos, N. Characterisation of sustainability in urban areas: An analysis of assessment tools with emphasis on European cities. Sustain. Cities Soc. 2018, 43, 563–577. [Google Scholar] [CrossRef]
  7. Manrique, S.; Ballester, M.; Pilar, C. Analyzing the effect of corporate environmental performance on corporate financial performance in developed and developing countries. Sustainability 2017, 9, 1957. [Google Scholar] [CrossRef] [Green Version]
  8. Banias, G.; Achillas, C.; Vlachokostas, C.; Moussiopoulos, N.; Stefanou, M. Environmental impacts in the life cycle of olive oil: A literature review. J. Sci. Food Agric. 2017, 97, 1686–1697. [Google Scholar] [CrossRef]
  9. Zheng, Y.; Chen, Z.; Pearson, T.; Zhao, J.; Hu, H.; Prosperi, M. Design and methodology challenges of environment-wide association studies: A systematic review. Environ. Res. 2020, 183. [Google Scholar] [CrossRef]
  10. Saad, E.; Hegab, H.; Kishawy, H.A. Design for sustainable manufacturing: Approach, implementation, and assessment. Sustainability 2018, 10, 3604. [Google Scholar]
  11. Ariffin, R.; Ghazilla, R.; Sakundarini, N.; Taha, Z.; Abdul-Rashid, S.; Yusoff, S. Design for environment and design for disassembly practices in Malaysia: A practitioner’s perspectives. J. Clean. Prod. 2015, 108, 331–342. [Google Scholar]
  12. Arnette, A.; Brewer, B.; Choal, T. Design for sustainability (DFS): The intersection of supply chain and environment. J. Clean. Prod. 2014, 83, 374–390. [Google Scholar] [CrossRef]
  13. Tian, J.F.; Pan, C.; Xue, R.; Yang, X.T.; Wang, C.; Ji, X.Z.; Shan, Y.L. Corporate innovation and environmental investment: The moderating role of institutional environment. Adv. Clim. Chang. Res. 2020, 11, 85–91. [Google Scholar] [CrossRef]
  14. Lee, K.H.; Min, B.; Yook, K.H. The impacts of carbon (CO2) emissions and environmental research and development (R&D) investment on firm performance. Int. J. Prod. Econ. 2015, 167, 1–11. [Google Scholar]
  15. Theofanidou. Environmental Performance of Businesses after the Implementation of Environmental Management System; National Technical University of Athens: Athens, Greece; University of Piraeus: Athens, Greece, 2008; pp. 6–24. [Google Scholar]
  16. Chirico, A.; Hristov, I. The role of sustainability Key Performance Indicators (KPIs) in implementing sustainable strategies. Sustainability 2019, 11, 5742. [Google Scholar]
  17. Gaidajis, G.; Angelakoglou, K. A conceptual framework to evaluate the environmental sustainability performance of mining industrial facilities. Sustainability 2020, 12, 2135. [Google Scholar]
  18. Michailidou, A.V.; Vlachokostas, C.; Moussiopoulos, N. A methodology to assess the overall environmental pressure attributed to tourism areas: A combined approach for typical all-sized hotels in Chalkidiki, Greece. Ecol. Indic. 2015, 50, 108–119. [Google Scholar] [CrossRef]
  19. Berghout, F.; Hertin, J.; Azzone, G.; Carlens, J.; Drunen, M.; Jasch, C.; Noci, G.; Olsthoorn, X.; Tyteca, D.; van der Woerd, F.; et al. Measuring the Environmental Performance of Industry (MEPI). Final Report. 2001. Available online: (accessed on 7 July 2020).
  20. Dizdaroglu, D. The role of indicator-based sustainability assessment in policy and the decision-making process: A review and outlook. Sustainability 2017, 9, 1018. [Google Scholar] [CrossRef] [Green Version]
  21. European Commission. Science for Environment Policy—Indicators for Sustainable Cities. In-Depth Report Produced for the European Commission DG Environment by the Science Communication Unit; UWE: Bristol, UK, 2018; Available online: (accessed on 7 July 2020).
  22. Laurent, A.; Olsen, S.I.; Hauschild, M.Z. Carbon footprint as environmental performance indicator for the manufacturing industry. CIRP Annals 2010, 59, 37–40. [Google Scholar] [CrossRef]
  23. Vlachokostas, C.; Achillas, C.; Chourdakis, E.; Moussiopoulos, N. Combining regression analysis and air quality modelling to predict benzene concentration levels. Atmos. Environ. 2011, 45, 2585–2592. [Google Scholar] [CrossRef]
  24. Feleki, E.; Vlachokostas, C.; Moussiopoulos, N. Holistic methodological framework for the characterization of urban sustainability and strategic planning. J. Clean. Prod. 2020, 243, 118432. [Google Scholar] [CrossRef]
  25. Ethridge, M.A. Measuring Environmental Performance: A Primer and Survey of Metrics in Use; Global Environmental Management Initiative (GEMI): Washington, DC, USA, 1998. [Google Scholar]
  26. International Organization for Standardization. ISO 14031:1999: Environmental Management—Environmental Performance Evaluation—Guidelines; International Organization for Standardization: Geneva, Switzerland, 1999. [Google Scholar]
  27. Watróbski, J.; Jankowski, J.; Ziemba, P.; Karczmarczyk, A.; Zioło, M. Generalised framework for multi-criteria method selection. Omega 2019, 86, 107–124. [Google Scholar] [CrossRef]
  28. Cinelli, M.; Coles, S.; Kirwan, K. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecol. Indic. 2014, 46, 138–148. [Google Scholar] [CrossRef] [Green Version]
  29. Hokkanen, J.; Salminen, P. Choosing a solid waste management system using multicriteria decision analysis. Eur. J. Oper. Res. 1997, 98, 19–36. [Google Scholar] [CrossRef]
  30. Achillas, C.; Aidonis, D.; Vlachokostas, C.; Folinas, D.; Moussiopoulos, N. Re-designing industrial products on a multi-objective basis: A case study. J. Oper. Res. Soc. 2013, 64, 1336–1346. [Google Scholar] [CrossRef]
  31. Hashemi, S.S.; Hajiagha, S.H.R.; Zavadskas, E.K.; Mahdiraji, H.A. Multicriteria group decision making with ELECTRE III method based on interval-valued intuitionistic fuzzy information. Appl. Math. Model. 2016, 40, 1554–1564. [Google Scholar] [CrossRef]
  32. Zopounidis, C.; Pardalos, P. (Eds.) Handbook of Multicriteria Analysis; Applied Optimization Series; Springer: Berlin, Germany, 2010. [Google Scholar]
  33. Hass, J.L.; Palm, V. Using the Right Environmental Indicators: Tracking Progress, Raising Awareness and Supporting Analysis; Nordic Council of Ministers: Copenhagen, Denmark, 2012. [Google Scholar]
  34. Arcibugi, F.; Nijkamp, P. Economy and Ecology: Towards Sustainable Development; Klumer Academic Publishers: Dordrecht, The Netherlands, 1989. [Google Scholar]
  35. Wu, Z.; Abdul-Nour, G. Comparison of multi-criteria group decision-making methods for urban sewer network plan selection. CivilEng 2020, 1, 26-48. [Google Scholar] [CrossRef]
  36. Azziz, M.B.-H. Multiple Criteria Outranking Algorithm: Implementation and Computational Tests. ELECTRE III Method; Department of Engineering and Management, Instituto Superior Técnico: Lisbon, Portugal, 2015. [Google Scholar]
  37. Vlachokostas, C.; Michailidou, A.V.; Matziris, E.; Achillas, C.; Moussiopoulos, N. A multiple criteria decision-making approach to put forward tree species in urban environment. Urban Clim. 2014, 10, 105–118. [Google Scholar] [CrossRef]
  38. Rogers, M.; Bruen, M. Choosing realistic values of indifference, preference and veto thresholds for use with environmental criteria within ELECTRE. Eur. J. Oper. Res. 1998, 107, 542–551. [Google Scholar] [CrossRef]
  39. Xiaoting, W. Study of Ranking Irregularities When Evaluating Alternatives by Using Some ELECTRE Methods and A Proposed New MCDM Method Based on Regret and Rejoicing. Master’s Thesis, Louisiana State University, Baton Rouge, LA, USA, 2007. [Google Scholar]
  40. Roy, B.; Electre, A. III lgorithme de classement base sur une representation floue des preferences en presence de criteres multiples. Cahiers de CERO 1978, 20, 3–24. [Google Scholar]
  41. Roy, B.; Bouyssou, D. Aide Multicritere a la Decision: Methods et Cas; Economica: Paris, France, 1993. [Google Scholar]
  42. Roussat, N.; Dujet, C.; Mehu, J. Choosing a sustainable demolition waste management strategy using multicriteria decision analysis. Waste Manag. 2009, 29, 2–20. [Google Scholar] [CrossRef] [PubMed]
  43. Moussiopoulos, N.; Achillas, C.; Vlachokostas, C.; Spyridi, D.; Nikolaou, K. Environmental, social and economic information management for the evaluation of sustainability in urban areas: A system of indicators for Thessaloniki, Greece. Cities 2010, 27, 377–384. [Google Scholar] [CrossRef]
  44. Falqi, I.; Alsulamy, S.; Mansour, M. Environmental performance evaluation and analysis using ISO 14031 guidelines in construction sector industries. Sustainability 2020, 12, 1774. [Google Scholar] [CrossRef] [Green Version]
  45. Ocampo, A.L.; Estanislao-Clark, E. Developing a framework for sustainable manufacturing strategies selection. DLSU Bus. Econ. Rev. 2014, 23, 115–131. [Google Scholar]
  46. Michailidou, A.V.; Vlachokostas, C.; Moussiopoulos, N. Interactions between climate change and the tourism sector: Multiple-criteria decision analysis to assess mitigation and adaptation options in tourism areas. Tour. Manag. 2016, 55, 1–12. [Google Scholar] [CrossRef]
  47. Spyridi, D.; Vlachokostas, C.; Michailidou, A.V.; Sioutas, C.; Moussiopoulos, N. Strategic planning for climate change mitigation and adaptation: The case of Greece. Int. J. Clim. Chang. Strateg. Manag. 2015, 7, 272–289. [Google Scholar] [CrossRef]
  48. Kourmpanis, B.; Papadopoulos, A.; Moustakas, K.; Kourmoussis, F.; Stylianou, M.; Loizidou, M. An integrated approach for the management of demolition waste in Cyprus. Waste Manag. Res. 2008, 26, 573–581. [Google Scholar] [CrossRef]
  49. Haralambopoulos, D.; Polatidis, H. Renewable energy projects: Structuring a multi-criteria group decision-making framework. Renew. Energy 2003, 28, 961–973. [Google Scholar] [CrossRef]
Figure 1. Distribution of ascending and descending preference of alternative environmental indicators for the application of the Baseline Scenario.
Figure 1. Distribution of ascending and descending preference of alternative environmental indicators for the application of the Baseline Scenario.
Sustainability 12 06805 g001
Table 1. Alternative indicators for the environmental assessment of investments.
Table 1. Alternative indicators for the environmental assessment of investments.
CodeThematic AreaIndicator
A01RecyclingDesign for recycling and reuse of materials, and minimization of waste
A02Gas emissionsDesign for minimization of emissions (greenhouse gases, ozone depletion gases, air pollutants, particulate matter, etc.)
A03Resource savingsDesign for energy and other resources (materials, water, etc.) savings, and reduction of non-renewable resources consumption
A04BiodiversityDesign for biodiversity and habitat conservation
A05Impact restorationDesign for prevention and remediation of adverse effects on the environment due to the company’s operations
A06Alternative energy formsProvision for the use of alternative energy sources
A07EducationProvision for environmental employees’ training
A08Health and safetyProvision for employees’ health and safety
A09Communication & public awarenessProvision for communication with the local community (information and public disclosure of environmental performance, etc.)
A10Information across supply chainProvision for information on sustainable development to suppliers and customers
A11Social actionsProvision for social initiatives (compensations, donations, funding for environmental actions, etc.)
A12Pollution preventionProvision for covering the cost of pollution prevention projects
A13Environmental accountingProvision for application of environmental accounting
A14Research, Innovation, DevelopmentProvision for research and development of high-tech and innovative products, development of green products
A15Environmental policyProvision for consideration and planning for the implementation of environmental policies and environmental controls, the use of their results in company’s operations
A16Legal frameworkCompliance with the legal framework on environmental legislative framework
A17Environmental standardsProvision for the implementation of environmental management system (e.g., EMAS, ISO 14000)
A18Corporate governanceDesign for implementation of corporate governance rules
Table 2. Assessment matrix.
Table 2. Assessment matrix.
Environment [C1]Society
Recycling [A01]8.636.696.877.08
Gas emissions [A02]8.587.236.297.44
Resources’ savings [A03]8.507.237.217.37
Biodiversity [A04]8.216.425.315.08
Impact restoration [A05]8.487.236.656.42
Alternative energy forms [A06]7.856.316.567.21
Education [A07]7.126.445.694.75
Health and safety [A08]5.878.026.294.96
Communication & public awareness [A09]5.888.045.254.10
Information across supply chain [A10]6.087.635.854.58
Social actions [A11]6.677.876.334.31
Pollution prevention [A12]7.336.107.835.62
Environmental accounting [A13]6.274.566.984.92
Research, Innovation, Development [A14]7.986.297.778.67
Environmental policy [A15]7.856.406.466.19
Legal framework [A16]7.696.946.104.52
Environmental standards [A17]7.655.695.835.08
Corporate governance [A18]5.356.065.904.75
Weighting Factor31.923.625.119.4
Preference threshold0.
Indifference threshold0.
Table 3. Modifications of weighting factors and thresholds for sensitivity analysis purposes.
Table 3. Modifications of weighting factors and thresholds for sensitivity analysis purposes.
Weighting factor for Environmental criterion [C1] 31.925255016.716.716.7500031.931.9
Weighting factor for Social criterion [C2]23.6252516.75016.716.7050023.623.6
Weighting factor for Economic criterion [C3] 25.1255016.716.75016.750505025.125.1
Weighting factor for Technological criterion [C4]19.425016.716.716.750005019.419.4
Preference thresholdpBSpBSpBSpBSpBSpBSpBSpBSpBSpBS1.5 × pBS2 × pBS
Indifference thresholdqBSqBSqBSqBSqBSqBSqBSqBSqBSqBS1.5 × qBS2 × qBS
Table 4. Final ranking of environmental indicators (alternatives) for investments’ evaluation for each scenario studied.
Table 4. Final ranking of environmental indicators (alternatives) for investments’ evaluation for each scenario studied.
S2A3A14A5A1A12A2A8 & A11A16A10A6A13A9A15A7A4A17A18
S3A3A1A14A2A5 & A8A11A12A6A10A16A9A15A4A13A17A7A18
S5A3 & A14A1A5A12A2 & A8A6 & A11A13A16A10A15A4A17A9A7A18
S6A3 &A14A1A2A5 & A8A12A6A10 & A11A16A4A9 & A15A13A17A7A18
S7A3A1A14A5A12A2A6 & A13A15A4A16A11 & A17A8A7A10A18A9
S8A3A8A5A14A11A1A10 & A18A2A12A6A16A9A15A13A7A4A17
S9A14A3A1A12A5A2A5 & A13A15A8A17A16A18A4 & A10 & A11A7A9
S10A3A14A1A8A2A5A11A12A6A16A10A15A4A9A13 & A17A7A18
S11A3A14A1A2 & A5A8A12A6A11A16A10A15A4A7A9A17A13A18

Share and Cite

MDPI and ACS Style

Ovezikoglou, P.; Aidonis, D.; Achillas, C.; Vlachokostas, C.; Bochtis, D. Sustainability Assessment of Investments Based on a Multiple Criteria Methodological Framework. Sustainability 2020, 12, 6805.

AMA Style

Ovezikoglou P, Aidonis D, Achillas C, Vlachokostas C, Bochtis D. Sustainability Assessment of Investments Based on a Multiple Criteria Methodological Framework. Sustainability. 2020; 12(17):6805.

Chicago/Turabian Style

Ovezikoglou, Paraskevi, Dimitrios Aidonis, Charisios Achillas, Christos Vlachokostas, and Dionysis Bochtis. 2020. "Sustainability Assessment of Investments Based on a Multiple Criteria Methodological Framework" Sustainability 12, no. 17: 6805.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop