A Comparative Analysis of Multi-Criteria Decision-Making Methods and Normalization Techniques in Holistic Sustainability Assessment for Engineering Applications

Abstract

The sustainability evaluation of engineering processes and structures is a multifaceted challenge requiring the integration of diverse and often conflicting criteria. To address this challenge, Multi-Criteria Decision-Making (MCDM) methods have emerged as effective tools. However, the selection of the most suitable MCDM approach for problems involving multiple criteria is critical to ensuring robust, reliable, and actionable outcomes. Equally significant is the choice of a proper normalization technique, which plays a pivotal role in determining the robustness and reliability of the results. This study investigates the impact of common MCDM tools on the decision-making process concerning diverse aspects of sustainability. It also examines how different normalization methods influence the final outcomes. Sustainability in this context is understood as a trade-off among five key dimensions: performance, environmental impact, economic impact, social impact, and circularity. The outcome of the MCDM process is represented by an aggregated metric, referred to as the Sustainability Index (SI). This index offers a comprehensive and robust framework for evaluating sustainability and facilitating decision-making when conflicting criteria are present. To assess the effects of implementing different MCDM and normalization choices on the sustainability assessment, a dataset from the aviation sector is employed. Specifically, a typical aircraft component is analyzed as a case study for holistic sustainability assessment, utilizing data that represent the various dimensions of sustainability mentioned above, for this component. Additionally, the study investigates the influence of initial data variations and weight variations within the MCDM process on the results. The results indicate that, overall, the different MCDM and normalization methods lead to similar outcomes when applied to the design alternatives. However, a deeper dive into the results reveals that the weighted sum method, when paired with min-max normalization, appears to be more appropriate, based on the use case involved for the present investigation, due to its robustness regarding small variations in the initial data and its sensitivity to large ones. This research underscores the critical importance of selecting appropriate MCDM tools and normalization methods to enhance transparency, robustness, reliability, and consistency of sustainability assessments within a holistic framework.

1. Introduction

Multi-Criteria Decision-Making (MCDM) methods have a wide range of applications across various fields, addressing specific decision-making challenges, including those related to sustainability assessments. There are numerous MCDM techniques, including the Weighted Sum Model (WSM), TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), ELECTRE (ELimination and Choice Expressing REality), PROMETHEE (Preference Ranking Optimization METHod for Enrichment Evaluation), VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje) COPRAS (Complex Proportional Assessment), AHP (Analytic Hierarchy Process), and Goal Programming (GP), among others. Additionally, many hybrid models have been developed to address specific decision-making challenges.

The varying interpretations of sustainability and its ambiguity pose challenges for researchers. Many studies overlook defining the term, leading to methodological issues and reliance on indirect analysis through social and ecological variables [1]. A common and predominant interpretation of sustainability in the aviation sector is its association with the environmental dimension, particularly focusing on carbon neutrality [2]. In their previous works, e.g., [3,4], the authors proposed a more holistic definition of sustainability that encompassed a wide range of aspects, including performance, environmental, economic, social, and circularity aspects. In this context, sustainability is understood as a trade-off among the above dimensions. This interpretation is also adopted in the present work. Sustainability, according to this perspective of the authors, is detailed in Section 2.1 and the works cited therein.

Given the diverse nature and structure of MCDM, as well as the extensive range of MCDM application sectors, the literature review in this study focuses specifically on the use of MCDM methods for sustainability assessment across various domains. Particular emphasis is placed on their application within engineering, with a dedicated focus on the aviation sector to align with the use case data analyzed in this study. In this regard, sustainability assessment represents a common application area for MCDM, spanning multiple industries and sectors, and offering a structured approach to evaluate and optimize decisions based on sustainability objectives [5]. Irrelated to the exact interpretation of sustainability, MCDM methods have been widely employed and compared in the context of sustainability assessment, showcasing their strengths and limitations in various case studies and applications. For example, a thorough literature review on the application of MCDM methods in sustainable engineering was conducted in [6]. It highlights that selecting an appropriate MCDM method is itself a multi-criteria problem, as each method has its own strengths and limitations. The choice of method largely depends on the preferences of decision makers and analysts, with no single method universally superior to others. The review also concludes that there has been a significant increase in the application of MCDM models across engineering disciplines over the past decade, with the AHP method being the most applied one, followed closely by TOPSIS. In [7], a hybrid MCDM model combining TOPSIS, DEMATEL (Decision-Making Trial and Evaluation Laboratory), ANP, and GRA (Grey Relational Analysis) is proposed as a tool for selecting green and sustainable materials. Similarly, ref. [8] emphasizes the use of MCDM in the transportation sector, highlighting the critical role of stakeholder involvement in the decision-making process. Meanwhile, ref. [9] advocates for the application of MCDM to deliver sustainable products to consumers. Additionally, ref. [10] applies MCDM to assess the sustainability of electricity production technologies, incorporating methods such as MOORA (Multi-Objective Optimization Ratio Analysis) and TOPSIS. Moreover, ref. [11] provides a comprehensive review of MCDM methods for sustainable renewable energy development, analyzing a wide range of models, including WSM, AHP, ELECTRE, PROMETHEE, and others, to identify optimal solutions. Finally, recent research has introduced new hybrid MCDM models for sustainability assessment, incorporating regret theory and fuzzy environments, as demonstrated in [12].

In the aviation sector, MCDM methods are widely used for sustainability assessments, particularly in areas such as aircraft, airport, and airline operations evaluation. The author in [13] identifies 280 articles applying MCDM methods for sustainability assessments, involving over 60 different techniques. This study highlights a lack of clarity in many cases, as no reasons are provided for why a particular MCDM method was chosen over others. Instead, justifications typically focus on why the selected MCDM interpretation method was suitable for the study, often citing capabilities such as ranking or scoring a set of alternatives based on multiple criteria—functions that almost all MCDM methods can achieve. In this context, the above work [13] emphasizes key guidelines for assessing sustainability using MCDM methods. These guidelines highlight the importance of transparency in methodological choices and the reasoning behind the decisions made. It also notes that a well-constructed Multi-Criteria Analysis article should, at a minimum, provide clear explanations for why and how alternatives and criteria were selected, how criterion scores were determined, how weights were generated, and the rationale for selecting a specific MCDM over other existing methods (or the justification for developing a new method).

In a comprehensive review of over 160 papers on MCDM in the aviation industry [14], many studies focus on assessing sustainability criteria at the aircraft, airport, and airline levels, with AHP and TOPSIS being the most widely used methods. Another study [15] highlights that MCDM is an appropriate tool for sustainability assessment in aviation, especially when compared to other decision support systems such as deep learning and neural networks.

At the aircraft level, a hybrid MCDM approach is proposed in [16] for aircraft selection based on sustainability-related aspects. SWARA (Stepwise Weight Assessment Ratio Analysis) is utilized for weighting and COPRAS for aggregation in [17], focusing on sustainability criteria in aircraft selection. A comparison of AHP and ESM (Even Swaps Method) in [18] also results in the same rankings, with sensitivity analysis also applied on the involved weights. Additionally, ref. [19] compares a novel MCDM method with the WSM and the WPM (Weighted Product Method) to select an aircraft using a fuzzy weighting approach, leading to consistent conclusions. Additionally, in [20], MCDM methods were applied to assess and compare aircraft in terms of sustainability, incorporating AHP for weighting and WSM and TOPSIS for aggregation, which resulted in consistent outcomes.

At the airport level, ref. [21] uses a hybrid MCDM approach incorporating DEMATEL, DANP (DEMATEL-based ANP), and VIKOR to assess sustainability performance in international airports. Similarly, ref. [22] proposes MCDM for selecting a new airport, comparing methods like SAW, TOPSIS, and AHP, all yielding consistent results. The importance of sensitivity analysis to assess robustness is also emphasized in this study.

Additionally, ref. [23] proposes a hybrid MCDM method for selecting sustainable aviation fuels in supply chain management. This method is compared with others, including WSM and TOPSIS, showing consistent rankings across all methods, with sensitivity analysis applied to both input data and weights. Similarly, in [24], MCDM is used to assess the sustainability of alternative aviation fuels, incorporating and comparing methods such as WSM, TOPSIS, and GRA. The study concludes that all methods provide consistent rankings, with slight deviations observed only in the WSM. Finally, ref. [25] highlights the importance of stakeholder weighting in decision-making by proposing the Best-Worst Method, a technique that accounts for the relative importance of various stakeholders in the decision-making process.

The effect of normalization techniques in the context of MCDM methods has been also extensively studied in the literature. The choice of normalization technique can significantly impact the final ranking of alternatives and the selection of the best option. As noted by Chatterjee and Chakraborty [26], the normalization method employed in a MCDM approach plays a crucial role in determining the outcome of the decision-making process. Moreover, a comprehensive analysis of various normalization techniques for MCDM methods such as WSM, TOPSIS, and COPRAS was provided in [27], highlighting the pros and cons of each method. The authors suggest that sum-based linear normalization and vector normalization methods are among the most preferred normalization techniques, although the final choice depends on the specific structure of the problem being analyzed. In the context of ship design, ref. [28] investigates the effect of normalization (sum-based and vector normalization) methods on MCDM techniques such as WSM, WPM, TOPSIS, and ELECTRE. The results indicate consistency in the outcomes based on correlation analysis. In a study on web services [29], the impact of normalization on the VIKOR method was assessed using techniques such as linear sum, linear max, min-max, and vector normalization. The findings revealed that different normalization methods resulted in distinct outcomes, emphasizing the importance of selecting the appropriate technique for the given context. Additionally, ref. [30] examined normalization techniques like max, min-max, and vector normalization in conjunction with the WSM aggregation method, leading to different rankings for each technique. Similarly, ref. [31] assessed various normalization methods for the TOPSIS model, highlighting the challenge of identifying the most suitable normalization technique for each problem. In [32], different normalization techniques for the WSM are evaluated, concluding that min-max normalization is the most appropriate compared to other methods, although the study could not offer concrete recommendations for practical applications. In [33], the authors assessed ten different normalization techniques for the PSI (Preference Selection Index) method, noting that each technique led to different results, though some could be correlated. The challenges of selecting the optimal normalization technique were also discussed in [34], where the authors pointed out that varying rankings can result from different techniques, complicating the decision process. Different normalization methods were compared in [35] in the presence of outliers, showing that each method yielded distinct results depending on the normalization approach. On the other hand, ref. [36] assessed five normalization techniques in conjunction with the MARCOS (Measurement of Alternatives and Ranking according to COmpromise Solution) aggregation method, finding relatively consistent results for all normalization methods. In the robotics sector, ref. [37] compared different normalization techniques for the Weighted Aggregated Sum Product Assessment (WASPAS), alongside other MCDM methods, favoring min-max normalization for its effectiveness. In a study focused on time and complexity criteria, ref. [38] examined various normalization techniques for the TOPSIS method. Lastly, in the aviation sector, three normalization techniques—min-max, sum, and z-score normalization—using the WSM were assessed in [39]. The study found consistent rankings for both the min-max and z-score methods. In another study for aircraft selection problems [40], different normalization techniques were applied to the weighted sum aggregation method, with results showing that vector, linear sum, and linear max normalization produced similar outcomes, while min-max normalization led to different results.

The findings regarding both the MCDM methods and the selected normalization techniques underscore the importance of carefully choosing the appropriate method based on the specific characteristics of the problem. Different techniques can yield varying results, which highlight the need for a tailored approach in decision-making. The choice of normalization method can significantly influence the final rankings and the selection of the best alternative, depending on the structure and requirements of the problem at hand. Recognizing this impact, it becomes essential to systematically investigate how different normalization methods and decision-making tools perform in practical applications.

The aim of the present work is to explore and evaluate the effects of implementing various MCDM tools and normalization methods on sustainability assessments of engineering processes and structures, focusing on an aircraft component design with multiple variations. Specifically, in the context of aircraft component design, multiple variations in configurations need to be evaluated across various sustainability dimensions, including mechanical performance, environmental impact, economic factors, social impact, and circular economy principles. However, there is limited understanding of how different MCDM methods and normalization techniques influence the robustness, reliability, and interpretation of sustainability outcomes in engineering design, particularly in complex applications like aircraft components. Data collected from an aircraft structural component, investigated in a previous study of the authors [41], are utilized to implement a Sustainability Index (SI), firstly introduced in [3], integrating key metrics across multiple sustainability-related dimensions. This research investigates how different MCDM methods, such as the WSM, TOPSIS, modified TOPSIS, VIKOR, and COPRAS, as well as various normalization techniques (min-max, median, vector, z-score, and sum normalization), affect the robustness, reliability, and the interpretation of the final outcomes. The research questions addressed in this study are as follows: (a) how do different MCDM methods (WSM, TOPSIS, modified TOPSIS, VIKOR, and COPRAS) impact the robustness and reliability of sustainability assessments for aircraft component designs? (b) What is the effect of various normalization techniques (min-max, median, vector, z-score, and sum normalization) on the outcomes of sustainability evaluations in the context of comparing different aircraft component configurations?

The remainder of the paper is organized as follows: Following the introduction section of this paper (Section 1), which includes a literature review on the use of MCDM in the context of sustainability assessment and the objectives of the present study, Section 2 presents an overview of the holistic sustainability framework, along with the dataset used for the analysis. Section 3 outlines the various MCDM and normalization methods considered in this study. Section 4 presents the results from the analysis, followed by a discussion on the outcomes of the MCDM and normalization techniques. The paper concludes in Section 5, highlighting the key findings and offering perspectives for future research.

2. Definitions and Use Case

In this section, the sustainability approach adopted by the authors is outlined (Section 2.1), while Section 2.2 provides a description of the dataset used for the case study considered in the present work.

2.1. Definition of Sustainability

A more comprehensive approach to sustainability in the design of an actual component is adopted by the authors in [41]. This is achieved by considering the five dimensions of sustainability mentioned below, as distinct yet interrelated factors, to create a sustainable design. More details are available in the relevant study; however, for the reader’s convenience, the essential data have been summarized and are presented below:

(a)

Mechanical performance is evaluated through static and dynamic analyses of the component, with key parameters, including total deformation and mode 1 eigenfrequency.

(b)

Costs are assessed based on raw material costs, manufacturing process costs, and recyclability costs of the component.

(c)

Environmental sustainability is represented by the carbon dioxide (CO₂) emissions generated over the material’s life cycle.

(d)

Circularity is expressed through the recyclability potential of the component and its ability to be part of a closed-loop process, which depends on the quality of the recycled material.

(e)

Social impact is analyzed by considering the potential effects on various stakeholder groups throughout the product’s entire life cycle, including stages such as material extraction, manufacturing, and end-of-life.

The above dimensions were quantified through relevant metrics and aggregated into a sustainability index, unique for each alternative, as described in [41]. The index is derived using normalized data for the five Pillars—Performance (P), Costs (C), Environmental Impact (E), Circularity Performance (CIRC), and Social Impact (SOC)—with normalization performed using the min-max method. These pillars are then combined using the Weighted Sum Method, where each pillar is multiplied by its respective weighting factor, determined by the user. Under each pillar of sustainability, one or more metrics or criteria can be included, depending on the specific application. The aggregated sustainability metric is then used to rank the alternative design configurations.

2.2. Case Study

This study utilizes a comprehensive dataset and associated information aligned with defined sustainability dimensions to analyze the impact of various MCDM and normalization methods on the final ranking of design alternatives. The dataset, representative of an aircraft component design process, encapsulates the conflicting aspects of sustainability, providing a robust foundation for applying MCDM methods. By leveraging this dataset, sourced from study [41], the research enables a detailed comparison of the effects of different MCDM and normalization techniques. This dataset is specifically centered on the design and evaluation of alternative configurations of a commonly used aircraft component: a hat-stiffened panel. This type of panel is a critical structural element in aerospace applications and is composed of two primary components: the skin and the stringer. A typical profile and geometry of a hat-stiffened panel are illustrated in Figure 1 [4]. Five distinct material combinations were analyzed, as detailed in

Table 1. Component’s alternative configurations/material combinations.

Component Configurations
No	Skin	Stringer
1	Aluminium 2024 T3	Aluminium 2024 T3
2	CFRP (Carbon Fiber-Reinforced Plastic)	Aluminium 2024 T3
3	Aluminium 2024 T3	CFRP
4	CFRP	CFRP
5	17-4PH Stainless Steel	CFRP

.

The dataset includes 4487 distinct design alternatives, with variations in both the material properties and the geometrical features of the panel, namely, panel thickness, stiffener thickness, and crown width. Each configuration was systematically evaluated across multiple dimensions of sustainability: mechanical performance, environmental impact, economic cost, social impact, and circularity performance. Sustainability in design, particularly in the context of aircraft structures, is often primarily associated with environmental sustainability. This focus is exemplified by approaches such as eco-design, which aims to reduce carbon emissions and improve fuel efficiency [42]. However, there is a growing call to adopt the triple bottom line—encompassing environmental, social, and economic viability—in the aviation sector [43].

3. Methodology

For the aforementioned dataset, an aggregated metric was evaluated with respect to five pillars: performance, environment, cost, circularity and society, based on five MCDM and five normalization methods. Since most MCDM methods require a weighting procedure to determine the weights or priorities for each criterion, reducing subjectivity in this process is essential. To achieve this, the AHP is employed [44]. This method involves structured pairwise comparisons, allowing decision makers to systematically assess the relative importance of each criterion. In the present analysis, three representative weighting scenarios were considered by the authors: one with equal weights assigned to each of the five sustainability dimensions, one where environmental impact and performance were prioritized (i.e., considered significantly more important compared to the other criteria), and one where cost and performance were prioritized (i.e., considered significantly more important compared to the other criteria). The pairwise comparisons for determining the weights of the sustainability dimensions were made using the Saaty scale, a key component of the AHP. The Saaty scale allows for the comparison of criteria in pairs by assigning numerical values ranging from 1 (indicating equal importance) to 9 (indicating extreme importance). These pairwise comparisons were then used to derive the priority weights for each sustainability dimension, which were applied in the aforementioned weighting scenarios.

Following the weighting process and the derivation of the aggregated sustainability metric for each MCDM method considered, a Pearson correlation analysis was performed to evaluate the consistency and agreement among the results of the different MCDM methods. Moreover, a sensitivity analysis was conducted to test the robustness of the studied methods. More specifically, the data were perturbed by a zero-mean Gaussian error, with the standard deviation being a percentage of the standard deviations of the initial variables. Based on 1000 simulated perturbed samples, the mean and standard deviation of the final rankings were computed and compared. The most robust method with respect to small departures from the initial data, as well as the most sensitive to high perturbations, was selected for further investigation, considering the normalization method and its effect on the final outcome.

3.1. MCDM Methods Considered

The impact of various MCDM methods was assessed using the approaches outlined below. These methods are recognized as some of the most widely utilized and popular techniques in the literature due to their effectiveness in handling complex decision problems where multiple conflicting criteria must be considered. By applying these methods, the study aimed to evaluate how different decision-making frameworks can influence the sustainability assessment of alternative design configurations.

3.1.1. Weighted Sum Method (WSM)

The WSM or, alternatively, Simple Additive Weighting (SAW) method, is a widely used MCDM method in practice, since it is a method that is easy to implement and interpret. The WSM was introduced by Fishburn [45] and provides a scoring model based on the weighted average principle. It assigns weights to each objective and then uses the weighted sum of them, converting multiple objectives into an aggregated scalar objective function. The weights are usually determined by the AHP method [44,46,47].

More specifically, let $x_{ij}$ be the performance of the alternative A_i against the attribute Cj, i = 1, …. I and j = 1, …, J, $y_{ij}$ the normalized rating and $w_j$ the weight of the Cj attribute, with $\sum w_{j }=1$. The normalized data are utilized, since the data should be in a common scale. Then, the overall score for the alternative A_i is equal to:

$${\mathrm{WSM}}_{\mathrm{i}}={\sum }_{j=1}^J{w_j·y}_{ij}.$$

(1)

The attribute with the highest WSM score is the best alternative.

One disadvantage of this method is that it can lead to compromises between the pillars; high performance in one criterion can compensate for low performance in another.

3.1.2. Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS)

TOPSIS was initially proposed by Hwang and Yoon [48] as a compromising model, based on the distance principle. TOPSIS proposed as an optimal solution the one that has the shortest distance from the positive ideal solution and the longest distance from the negative ideal solution. The positive ideal solution is the solution that maximizes the benefit criteria, such as performance, circularity and social impact, and minimizes the cost criteria, such as the total cost of the component production and the CO₂ emissions. On the other hand, the negative ideal solution maximizes the cost criteria and minimizes the benefit criteria. The most used distance in TOPSIS evaluation is the Euclidean distance. However, other distances, such as Manhattan, Tchebichev [49], and others [50], can be used to calculate TOPSIS.

Let $v_{ij}=w_j\times y_{ij}$, i = 1, …., I and j = 1, …, J, the weighted and normalized rating and ${\mathrm{v}}_{\mathrm{j}}^{+}, {\mathrm{v}}_{\mathrm{j}}^{-}$ be the positive and the negative ideal solution, respectively. Then, the overall score for the alternative $A_{\mathrm{i}}$ is:

$${\mathrm{V}}_{\mathrm{i}}={\displaystyle \frac{\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{\left({{\mathrm{v}}_{\mathrm{i}\mathrm{j}}-\mathrm{v}}_{\mathrm{j}}^{-}\right)}^2}}{\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{\left({{\mathrm{v}}_{\mathrm{i}\mathrm{j}}-\mathrm{v}}_{\mathrm{j}}^{-}\right)}^2}+\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{\left({{\mathrm{v}}_{\mathrm{i}\mathrm{j}}-\mathrm{v}}_{\mathrm{j}}^{+}\right)}^2}}}$$

(2)

Finally, alternatives are ranked with respect to ${\mathrm{V}}_{\mathrm{i}}$ values. A higher ${\mathrm{V}}_{\mathrm{i}}$ score value indicates a better alternative performance.

A main limitation of TOPSIS is the rank reversal phenomenon whereby, through adding or removing an alternative solution, the initial ranking of the alternatives changes. Several modifications for addressing this problem have been proposed in the literature, e.g., in [51,52]. TOPSIS with Manhattan distance does not appear to suffer from the rank reversal phenomenon [49], as it inherits this property from WSM, with both methods exhibiting similar behavior when this distance metric is used. Additionally, in this method, the interpretation of the weights is not straightforward, as the distances from the positive and negative ideal solutions are weighted by the square of the initial weights ($w_j$²). For this reason, modified TOPSIS has been proposed.

3.1.3. Modified TOPSIS

The main difference between the modified TOPSIS and the original TOPSIS is that in the modified version, the ideal solution is identified first, and then the weighted Euclidean distance is calculated [53]. Thus, the overall score for the alternative A_i is:

$${\mathrm{V}}_{\mathrm{i}}={\displaystyle \frac{\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{{\mathrm{w}}_{\mathrm{j}}\left({y_{\mathrm{i}\mathrm{j}}-y}_{\mathrm{j}}^{-}\right)}^2}}{\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{{\mathrm{w}}_{\mathrm{j}}\left({y_{\mathrm{i}\mathrm{j}}-y}_{\mathrm{j}}^{-}\right)}^2}+\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{{\mathrm{w}}_{\mathrm{j}}\left({y_{\mathrm{i}\mathrm{j}}-y}_{\mathrm{j}}^{+}\right)}^2}}}$$

(3)

The difference between the two methods is that in TOPSIS, ${\mathrm{w}}_{\mathrm{j}}^2$ is used—contrary to modified TOPSIS, where ${\mathrm{w}}_{\mathrm{j}}$ is used—while calculating the distances from the positive and the negative ideal solutions. This helps maintain a proportional and more balanced impact of each attribute on the final decision, avoiding the exaggerated influence of higher-weighted attributes seen in the traditional TOPSIS method. Yet, TOPSIS and the modified TOPSIS methods result in the same ranking when equal attribute weights are used.

3.1.4. VIKOR

VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje or, in English, Multi-criteria Optimization and Compromise Solution) has been proposed by Opricovic [54,55,56]. Unlike TOPSIS, which prioritizes solutions closest to the positive ideal solution and farthest from the negative ideal, VIKOR focuses on a compromise solution by minimizing the distance to ideal solutions. This approach appeals to risk-takers, potentially leading to more innovative or bold decisions.

Let ${\mathrm{y}}_{\mathrm{j}}^{\mathrm{*}} \mathrm{a}\mathrm{n}\mathrm{d} {\mathrm{y}}_{\mathrm{j}}^{-}$ be the best and the worst values for each criterion.

Step 1: Compute ${\mathrm{S}}_{\mathrm{i}}$ and ${\mathrm{R}}_{\mathrm{i}}$:

$${\mathrm{S}}_{\mathrm{i}}=\sum _{\mathrm{j}=1}^{\mathrm{J}}{\mathrm{w}}_{\mathrm{j}}·{\displaystyle \frac{{\mathrm{y}}_{\mathrm{j}}^{\mathrm{*}}-{\mathrm{y}}_{\mathrm{i}\mathrm{j}}}{{\mathrm{y}}_{\mathrm{j}}^{\mathrm{*}}-{\mathrm{y}}_{\mathrm{j}}^{-}}}$$

(4)

and:

$${\mathrm{R}}_{\mathrm{i}}=\underset{\mathrm{j}}{\max }\left\{{\mathrm{w}}_{\mathrm{j}}·{\displaystyle \frac{{\mathrm{y}}_{\mathrm{j}}^{\mathrm{*}}-{\mathrm{y}}_{\mathrm{i}\mathrm{j}}}{{\mathrm{y}}_{\mathrm{j}}^{\mathrm{*}}-{\mathrm{y}}_{\mathrm{j}}^{-}}}\right\}$$

(5)

Step 2: Compute:

$${\mathrm{Q}}_{\mathrm{i}}=\mathrm{v}·{\displaystyle \frac{{\mathrm{S}}_{\mathrm{i}}-{\mathrm{S}}^{\mathrm{*}}}{{\mathrm{S}}^{-}-{\mathrm{S}}^{\mathrm{*}}}}+\left(1-\mathrm{v}\right)·{\displaystyle \frac{{\mathrm{R}}_{\mathrm{i}}-{\mathrm{R}}^{\mathrm{*}}}{{\mathrm{R}}^{-}-{\mathrm{R}}^{\mathrm{*}}}}$$

(6)

where ${\mathrm{S}}^{-}=\underset{\mathrm{i}}{\max }{\mathrm{S}}_{\mathrm{i}}$, ${\mathrm{S}}^{\mathrm{*}}=\underset{\mathrm{i}}{\min }{\mathrm{S}}_{\mathrm{i}}$, ${\mathrm{R}}^{-}=\underset{\mathrm{i}}{\max }{\mathrm{R}}_{\mathrm{i}}$, ${\mathrm{R}}^{\mathrm{*}}=\underset{\mathrm{i}}{\min }{\mathrm{R}}_{\mathrm{i}}$, and $\mathrm{v}$ is a weighting reference, where 0 ≤ $\mathrm{v}$ ≤ 1.

Step 3: Rank alternatives, sorted by the values S, R, and Q in ascending order, and produce three ranking lists.

S, R, and Q ranking lists will propose the compromise solution or a set of compromise solutions. If the attribute A′ has the best score with respect to Q list, it could be considered as the best solution if it satisfies the following conditions:

C1: Q(A″) − Q(A′) ≥ (m − 1)⁻¹

(7)

where A″ is the attribute with second-best score in Q list, andm is the number of attributes, and C2: A′ also has the best score with respect to S and/or R list. If the above conditions are not satisfied, a compromise solution set results.

3.1.5. COPRAS

COPRAS (Complex Proportional Assessment) method was introduced by Zavadskas and Kaklauskas [57], and it can be described as follows:

Step 1. Normalize the initial rating $x_{ij}i=1,\dots ,I \mathrm{a}\mathrm{n}\mathrm{d} j=1,\dots ,J.$ Let $y_{ij}i=1,\dots ,I \mathrm{a}\mathrm{n}\mathrm{d} j=1,\dots ,J$ be the normalized rating;

Step 2. Weight the normalized rating $y_{ij}$ by proper weight $w_j,\mathrm{j}=1,\dots ,\mathrm{J} \mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h} \mathsf{\Sigma }{w}_j=1;$

Step 3. Calculate the ${\mathrm{P}}_{\mathrm{i}}, {\mathrm{R}}_{\mathrm{i}}, {\mathrm{Q}}_{\mathrm{i}}$;

• where R_i is the summation of the attributes that need to be minimized, and P_i is the summation of the attributes that need to be maximized:

  $$\mathrm{Q}\mathrm{i}=P_i+{\displaystyle \frac{{\sum }_{\mathrm{j}=1}^{\mathrm{J}}{R}_j}{R_i{\sum }_{\mathrm{j}=1}^{\mathrm{J}}\frac{1}{R_j}}},U_i={\displaystyle \frac{Q_i}{Q_{max}}}·100\mathrm{\%}$$

  (8)

Alternatives are ranked based on $U_{\mathrm{i}}$ values. A higher score value in U indicates a better alternative performance.

A main limitation of COPRAS is that it is less robust than other MCDM methods, even in the case of small data changes [58]. This lack of robustness means that even minor variations in the input data, such as slight changes in the values of criteria or the performance of alternatives, can lead to significant fluctuations in the final rankings or outcomes.

3.2. Normalization Methods

Normalization is a crucial step in any decision-making problem, as it transforms heterogeneous data into data on a common scale. Data normalization involves scaling the attribute values to make them lie numerically in the same scale and thus have the same importance on the final index. There exist numerous normalization methods in the literature [59], but in the current work, the following five widely used techniques were considered and compared (

Table 2. Normalization methods considered in the present study.

Method	For the Attribute That Needs to Be Maximized	For the Attribute That Need to Be Minimized
Min–Max	$x^{\prime }={\displaystyle \frac{x-min\left(x\right)}{\mathit{max}\left(x\right)-min\left(x\right)}}$	$x^{\prime }=1-{\displaystyle \frac{x-\mathit{min}\left(x\right)}{\mathit{max}\left(x\right)-min\left(x\right)}}$
Z-Score	$x^{\prime }={\displaystyle \frac{x-mean\left(x\right)}{stdev\left(x\right)}}$	$x^{\prime }=-{\displaystyle \frac{x-mean\left(x\right)}{stdev\left(x\right)}}$
Robust Scaling	$x^{\prime }={\displaystyle \frac{x-median\left(x\right)}{IQR\left(x\right)}}$	$x^{\prime }=-{\displaystyle \frac{x-median\left(x\right)}{IQR\left(x\right)}}$
L1—Norm	$x^{\prime }={\displaystyle \frac{x}{\sum \left|x_i\right|}}$	$x^{\prime }={\displaystyle \frac{\mathit{max}\left(x\right)-x+min\left(x\right)}{\sum \left|x_i\right|}}$
L2—Norm	$x^{\prime }={\displaystyle \frac{x}{\sqrt{\sum x_i^2}}}$	$x^{\prime }={\displaystyle \frac{\mathit{max}\left(x\right)-x+min\left(x\right)}{\sqrt{\sum x_i^2}}}$

).

Let x be the initial value and x′ represent the normalized value. The $min\left(x\right)$, $max\left(x\right)$, $mean\left(x\right)$, $median\left(x\right)$, $stdev\left(x\right)$, and $IQR\left(x\right)$ correspond to the minimum, the maximum, the mean, the median, the standard deviation, and the interquartile range value of the criterion, respectively.

4. Results

4.1. MCDM Impact Results

In

Table 3. Ranking of configurations using different MCDM methods across various sustainability scenarios.

Ranking No.	Material Combination	SWM	TOPSIS	Modified TOPSIS	COPRAS	VIKOR
Equal Weighting
1	AL-AL	89	90	90	89	90
2	AL-AL	54	51	51	50	50
3	AL-AL	50	91	91	85	91
4	AL-AL	90	52	52	90	89
5	AL-AL	85	50	50	54	51
6	AL-AL	94	55	55	46	55
7	AL-AL	55	95	95	51	52
8	AL-AL	123	56	56	123	92
9	AL-AL	51	92	92	94	54
10	AL-AL	128	89	89	55	85
Prioritization to Performance and Costs Terms
Ranking No.	Material Combination	SWM	TOPSIS	Mod. TOPSIS	COPRAS	VIKOR
1	AL-AL	90	92	91	89	92
2	AL-AL	91	93	92	50	93
3	AL-AL	55	91	93	85	91
4	AL-AL	95	97	52	54	52
5	AL-AL	56	98	96	90	97
6	AL-AL	94	53	53	46	53
7	AL-AL	96	52	97	51	96
8	AL-AL	51	96	57	123	90
9	AL-AL	89	57	90	94	57
10	AL-AL	92	58	56	55	98
Prioritization to Performance and Environment Terms
Ranking No.	Material Combination	SWM	TOPSIS	Mod. TOPSIS	COPRAS	VIKOR
1	AL-AL	89	90	90	89	89
2	AL-AL	54	51	91	50	50
3	AL-AL	50	91	51	90	90
4	AL-AL	90	52	52	85	54
5	AL-AL	94	50	50	51	85
6	AL-AL	85	55	55	54	51
7	AL-AL	55	95	95	91	94
8	AL-AL	123	56	56	55	55
9	AL-AL	51	92	92	94	86
10	AL-AL	128	89	89	86	91

, the first 10 ordered configurations are presented; these were obtained from the five different MCDM methods. The numbers shown in colors correspond to the configuration IDs considered in [38] and are used to track the same colors in the rest of the columns. The first column indicates the material combinations, as shown in Table 1. As described in Section 2, each design configuration, apart from the different material combinations (as shown in Table 1), also corresponds to variations in design parameters, namely panel thickness, stringer thickness, and crown width. Moreover, three different scenarios have been considered: equal weighting among the five pillars of sustainability, prioritization of costs and performance, and prioritization of environment and performance. As described in Section 3, three representative scenarios have been chosen. Based on the procedure of the Analytic Hierarchy Process (AHP) used to derive priorities, the following weight assignments were determined for the three different scenarios. In the equal weights scenario, the five dimensions of sustainability—performance, environment, costs, society, and circular economy—were each assigned a weight of 20%. In the environment- and costs-prioritized scenario, where environmental impact and costs were strongly emphasized, these two dimensions received a weight of 41% each, while the remaining three dimensions (performance, society, and circular economy) were each assigned a weight of 6%. Lastly, in the costs- and performance-prioritized scenario, where costs and performance were given greater importance, these two dimensions also received a weight of 41% each, with the other three dimensions receiving 6% each. Coloring has been applied to help identify the similar configurations obtained across the five different MCDM methods. To improve clarity, the configuration colors derived from the WSM are highlighted in the rankings of the other MCDM methods.

As already mentioned in Section 2, the data have been obtained from design configurations where five different material combinations have been considered, as shown in Table 1. In all MCDM methods and scenarios considered, the design configurations involving a combination of aluminum in both components of the hat-stiffened panel (panel + stiffener) rank first. Therefore, the top 10 configurations shown herein also concern the aluminum–aluminum (AL-AL) combination design, for which only the design geometric parameters vary.

In the equal-weighting scenario, most of the top-ranked design configurations consistently appear across all five MCDM methods, although their positions within the rankings varied. This suggests a strong consensus among the different MCDM methods, indicating that these top configurations are consistently deemed optimal or high-performing, regardless of the specific method used. More specifically, TOPSIS and modified TOPSIS yielded identical results, as mathematically proven, and their final rankings closely aligned with those of VIKOR. SWM and COPRAS produced similar ranking orders. When greater weight was assigned to cost and performance metrics compared to the others, modified TOPSIS and VIKOR exhibited nearly identical performance, though the rankings of other configurations differed. Finally, when greater weight was assigned to performance and environmental factors, the top ten ranked attributes were identical in COPRAS and VIKOR, although their exact positions differed. These two methods differed by only two attributes compared to the corresponding list from SWM.

To verify the similarity of the entire ordered lists resulting from the five different aggregation methods, a Pearson correlation analysis was conducted, considering the three different weighting schemes. The Pearson correlation coefficient describes the strength and the direction of a linear relationship between two variables, with values ranging from −1 to 1. A value of 1 indicates perfect positive correlation (all data points align in a straight line with a positive slope), and a value of −1 indicates perfect negative correlation (all data points align in a straight line with a negative slope). The correlation analysis results (

Table 4. Pearson’s correlation analysis results for the considered MCDM methods.

Aggregation Method	SWM	TOPSIS	Modified TOPSIS	COPRAS	VIKOR
Equal Weights
SWM	1	0.866	0.866	0.958	0.948
TOPSIS	0.866	1	1	0.943	0.869
Modified TOPSIS	0.866	1	1	0.943	0.869
COPRAS	0.958	0.943	0.943	1	0.976
VIKOR	0.948	0.869	0.869	0.976	1
Prioritization to Performance and Environment Terms
SWM	1	0.856	0.890	0.898	0.941
TOPSIS	0.856	1	0.932	0.912	0.886
Modified TOPSIS	0.890	0.932	1	0.976	0.853
COPRAS	0.898	0.912	0.976	1	0.854
VIKOR	0.941	0.886	0.853	0.854	1
Prioritization to Performance and Costs Terms
SWM	1	0.908	0.923	0.931	0.917
TOPSIS	0.908	1	0.999	0.826	0.987
Modified TOPSIS	0.923	0.999	1	0.843	0.988
COPRAS	0.931	0.826	0.843	1	0.809
VIKOR	0.917	0.987	0.988	0.809	1

) indicate a strong positive correlation between the alternative rankings, meaning that all methods demonstrated equivalent rankings.

To test the sensitivity of the five MCDM methods to small variations in the initial data, 1000 perturbed samples were generated. The initial values were perturbed by a Gaussian error with zero mean and a standard deviation equal to 1% and 10% of the standard deviation of the initial data. The simulated samples were then normalized and ordered. Figure 2a–f present the boxplots of the first five ordered attributes with respect to SWM for each of the considered MCDM methods.

For the scenario with equal weights, both VICOR and COPRAS exhibited higher variability, even with small perturbations (1%) on the initial data, which is undesirable in this specific context. Conversely, the WSM demonstrated minimal variation in response to small perturbations, indicating stability in the resulting order. TOPSIS and modified TOPSIS were relatively stable under small data perturbations, although they were slightly more sensitive than WSM. As the perturbation of the initial data increased (to 10%), the variation in WSM also increased, which is a desirable characteristic in this scenario. TOPSIS and modified TOPSIS showed high sensitivity to larger perturbations, although they were less sensitive than WSM. Finally, COPRAS and VIKOR exhibited extreme sensitivity to the ranking order under large data variations.

When greater weights were assigned to cost and performance metrics, the WSM exhibited low sensitivity to small data variations but higher sensitivity to significant data variations. This behavior was also observed in TOPSIS, Modified TOPSIS, and VIKOR. In contrast, COPRAS displayed very high sensitivity to both small and large data variations.

In the scenario where greater weights were assigned to environmental and performance metrics, WSM again demonstrated low sensitivity to small data variations and high sensitivity to significant data variations. However, the other methods showed sensitivity to both small and large data variations, with VIKOR and COPRAS being particularly sensitive to substantial data changes.

In conclusion, WSM stands out as a more suitable approach when compared to other methods due to its desirable balance between stability and sensitivity. This method demonstrates minimal sensitivity to small perturbations in the input data, ensuring that the ranking order of attributes remains consistent despite minor variations. Stability is a highly desirable characteristic in the context of sustainability assessment, where small fluctuations in data may occur due to uncertainties or measurement errors. Ensuring the reliability of rankings under such conditions helps maintain the credibility and robustness of the assessment results.

At the same time, SWM exhibits increased sensitivity to larger data variations, allowing it to effectively capture significant changes in the input data. This adaptability ensures that SWM can identify and reflect meaningful shifts in trends, which may indicate substantial alterations in the sustainability attributes being evaluated. This dual characteristic of SWM—stability for minor variations and responsiveness to major shifts—positions it as an ideal method for sustainability assessments, where both precision and adaptability are crucial.

Moreover, this balance is essential in practical applications. For example, in monitoring systems or policy evaluations, slight variations in input data should not lead to disproportionate changes in outcomes, which might mislead decision makers. Conversely, when there are significant changes in the data—such as changes due to new environmental policies, technological advancements, or economic shifts—SWM’s ability to reflect these changes ensures that the method remains relevant and insightful.

4.2. Normalization Impact Results

Following the analysis on the impact of the MCDM methods, the effect of normalization methods on WSM performance was assessed. For this analysis, the most robust MCDM method from the previous step, i.e., the SWM method, was considered, and the impact of different normalization methods on this method was evaluated. In

Table 5. Ranking of configurations using different normalization methods for the WSM, across various sustainability scenarios.

Ranking No.	Material	Min–Max	z-Score	Median	L2—Norm	L1—Norm
Equal Weighting
1	AL-AL	89	90	90	91	91
2	AL-AL	54	91	91	90	90
3	AL-AL	50	50	51	92	92
4	AL-AL	90	89	50	51	51
5	AL-AL	85	51	89	52	52
6	AL-AL	94	55	55	50	93
7	AL-AL	55	95	92	56	56
8	AL-AL	123	56	95	93	50
9	AL-AL	51	94	56	95	96
10	AL-AL	128	92	52	96	95
Prioritization to Performance and Costs Terms
1	AL-AL	90	92	92	92	92
2	AL-AL	91	91	91	93	93
3	AL-AL	55	93	93	91	91
4	AL-AL	95	96	96	97	97
5	AL-AL	56	90	97	52	53
6	AL-AL	94	97	57	53	52
7	AL-AL	96	56	90	96	98
8	AL-AL	51	57	52	98	96
9	AL-AL	89	52	56	57	57
10	AL-AL	92	95	95	58	58
Prioritization to Performance and Environment Terms
1	AL-AL	89	90	90	91	91
2	AL-AL	54	91	91	92	92
3	AL-AL	50	51	51	90	90
4	AL-AL	90	50	50	51	51
5	AL-AL	94	89	89	52	52
6	AL-AL	85	55	92	93	93
7	AL-AL	55	95	55	56	56
8	AL-AL	123	56	95	50	96
9	AL-AL	51	92	56	96	95
10	AL-AL	128	94	52	95	50

, the first ten design configurations are presented for the SWM method across five normalization techniques and three weighting scenarios: equal weighting, performance- and cost-prioritized, and environment- and performance-prioritized. For all weighting scenarios, the rankings were very similar across all studied normalization methods, with the L1 and L2 norms yielding nearly identical results. Color changes are used to track the same configuration in the rest of the columns.

This consistent ranking is further confirmed by the correlation analysis (

Table 6. Pearson’s correlation analysis results for the considered normalization methods methods—assuming equal weighting.

Normalization Method	Min–Max	z-Score	Median	L2—Norm	L1—Norm
Equal Weights
Min–Max	1	0.947	0.892	0.946	0.942
z-Score	0.947	1	0.990	0.996	0.996
Median	0.892	0.990	1	0.983	0.984
L2—Norm	0.946	0.996	0.983	1	1
L1—Norm	0.942	0.996	0.984	1	1

), where all correlation coefficients are very close to one, indicating a high degree of similarity between the rankings for the different normalization methods. This is also the case for all the studied weighting scenarios.

To test the sensitivity of the five normalization methods to variability in the initial data, a simulation similar to that conducted for the MCDM methods was performed. The simulated data were ordered, and the boxplots of the first five ordered attributes with respect to the min-max normalization method are depicted in Figure 3a–f for each normalization method.

In the equal-weight scenario, the L1 norm and L2 norm normalization methods appear to be sensitive to both small and large data perturbations. The ranking using min-max normalization seems to be insensitive to small data perturbations (1%) but shows sensitivity to larger data perturbations (10%). The z-score and median normalization methods demonstrate moderate sensitivity to both small and large data perturbations.

In the scenario where greater weight was assigned to both cost and performance metrics, all normalization methods exhibit moderate sensitivity to both small and large data perturbations. However, min-max appears to be more sensitive to large data perturbations compared to the other methods, which is a desirable outcome.

Finally, in the scenario where greater weights were assigned to both environment and performance, min-max normalization appears to be less sensitive to small data perturbations compared to the other methods. Additionally, z-score and median methods seem to have low sensitivity to small data perturbations. For larger data perturbations, the ranking using min-max becomes more sensitive compared to the other methods, while L1 norm and L2 norm show high sensitivity to both small and large data perturbations.

Based on the analysis above, min-max normalization proves to be the preferred method in various scenarios due to its balanced sensitivity to data variations. This normalization technique is particularly advantageous because it combines stability and responsiveness, ensuring consistent performance across a spectrum of data conditions. Specifically, min-max normalization demonstrates a high degree of stability when handling small perturbations in the input data, preventing minor fluctuations from disproportionately affecting the rankings. At the same time, it remains sufficiently responsive to larger shifts in the data, effectively capturing significant changes in trends or attributes.

In the context of sustainability assessment, these qualities are significant. Accurate ranking and sensitivity to changes are critical for evaluating sustainability metrics, which often involve a complex interplay of environmental, economic, and social factors. Min-max normalization ensures that the rankings produced are reliable and reflective of the actual state of the system, even when faced with inherent uncertainties or variability in the input data. For instance, minor inaccuracies in data collection or reporting will not compromise the integrity of the assessment, while more substantial changes, such as policy shifts or environmental disruptions, are appropriately highlighted.

5. Conclusions and Future Perspectives

This study investigates the impact of various MCDM tools on the decision-making process and examines how prominent normalization methods affect the final outcomes in the context of sustainability assessment, specifically during the design of aircraft structures. Using a dataset from the aviation sector, a typical aircraft component was selected as a case study for a holistic sustainability assessment. A detailed sensitivity analysis was conducted to explore the influence of MCDM tools, normalization methods, and weight variations on the outcome results.

The results indicate that the ranking obtained from the considered MCDM methods was consistent, as demonstrated by the correlation analysis. This finding suggests that the choice of MCDM tool does not significantly affect the ranking in this context. However, the sensitivity analysis of the impact of data variation under different weighting scenarios revealed that the WSM is more sensitive to small data variations, while exhibiting desirable sensitivity to larger data variations. This characteristic is advantageous in sustainability assessment, as it aligns with the need for robust responses to significant changes in the dataset.

When considering normalization methods, the analysis focused on five widely used techniques. The effect of normalization on the WSM was assessed, with the ranking results showing consistency, as validated by the correlation analysis. Notably, the min-max normalization method demonstrated smaller sensitivity to small data variations and greater sensitivity to larger data variations. This property makes it a particularly robust choice for this context.

WSM and min-max normalization both offer significant advantages in sustainability assessments, each bringing the desirable balance of stability and sensitivity to data variations. WSM excels in maintaining stable rankings when faced with minor perturbations, ensuring reliability in results despite uncertainties, while remaining responsive to larger shifts to capture meaningful trends. Similarly, min-max normalization provides a balanced approach by preventing small fluctuations from distorting rankings and effectively highlighting significant changes in the data. Additionally, the bounded nature of results using min-max normalization (e.g., within the range 0–1) enhances interpretability, transparency, and comprehensibility. Together, these methods offer robust frameworks for dynamic and accurate sustainability evaluations, ensuring consistency and adaptability in varying data conditions.

Overall, the weighted sum model using the min-max method is justified for calculating an aggregated metric of sustainability in the conceptual design phase due to its ability to provide a simple, transparent, and flexible framework for decision making under uncertainty. The weighted sum model’s clarity and ability to provide a first-order approximation of sustainability make it a practical tool for conceptual design.

This work also addresses the key considerations outlined in relevant studies for assessing sustainability using MCDM methods, such as [9] ensuring complete transparency in methodological choices and providing sound reasoning behind the decisions made. It offers detailed explanations of the selection process for alternatives and criteria, the definition of criterion scores, the generation of weights, and the rationale for choosing the specific MCDM method. Additionally, a sensitivity analysis is incorporated to assess the robustness of the results, providing further confidence in the reliability and consistency of the findings under different scenarios and assumptions.

Future research should address potential drawbacks of the weighted sum model, particularly its compensatory nature, where strong performance in one criterion can offset poor performance in another. Introducing minimum thresholds for critical criteria could help ensure that design alternatives failing to meet essential sustainability requirements are excluded from consideration. Moreover, the engagement of relevant stakeholders to establish a more realistic definition of priorities within the context of holistic sustainability in specific sectors would undoubtedly enhance the credibility of the assessment.