Next Article in Journal
Enterprise Architecture Best Practices in Large Corporations
Previous Article in Journal
Blind Queries Applied to JSON Document Stores

Information 2019, 10(10), 292; https://doi.org/10.3390/info10100292

Article
A Novel Multi-Criteria Decision-Making Model: Interval Rough SAW Method for Sustainable Supplier Selection
1
Faculty of Transport and Traffic Engineering Doboj, University of East Sarajevo, Vojvode Mišića 52, 74000 Doboj, Bosnia and Herzegovina
2
Faculty of Economics, University of Banja Luka, Majke Jugovića 4, 78000 Banja Luka, Bosnia and Herzegovina
3
Department of Logistics, Military Academy, University of Defence in Belgrade, Pavla Jurišića Sturma 33, 11000 Belgrade, Serbia
4
Department for Social Sciences, Institute for Scientific Research and Development Brcko district BiH, 76100 Brcko District, Bosnia and Herzegovina
*
Author to whom correspondence should be addressed.
Received: 19 August 2019 / Accepted: 19 September 2019 / Published: 22 September 2019

Abstract

:
Sustainability in a supply chain is a demand on the one hand and a challenge on the other. It is necessary to balance between these dimensions in order to fulfill the purpose of the supply chain. Therefore, in the first phase—i.e., in procurement—it is necessary to take into account its sustainability. In this paper, a sustainable supplier was selected respecting all three aspects of sustainability. The evaluation was carried out on the basis of a total of 21 criteria arranged into two levels and three groups. A new Interval Rough SAW (simple additive weighting) method, which represents a contribution to the treatment of problems belonging to the multi-criteria decision-making (MCDM), was developed. The integration of interval rough numbers with the SAW method was completed. In addition, the full consistency method (FUCOM) was applied to determine the weights of the criteria. The integrated FUCOM-Interval Rough SAW model enables treatment of multi-criteria problems while reducing subjectivity to the lowest possible level and eliminating uncertainties and ambiguities. The results obtained were determined throughout a sensitivity analysis consisting of a change in the weight of the criteria and the influence of dynamic matrices on the change in ranks. In addition, Spearman’s rank correlation coefficient (SCC) was calculated to confirm the stability of the previously obtained results.
Keywords:
interval rough SAW; sustainable supplier; procurement; MCDM

1. Introduction

Procurement of materials as a separate logistics sub-system plays an important role in achieving the efficiency of a sustainable supply chain and meeting its goals related to meeting the requirements of all participants. In the initial phase—i.e., in a procurement process—it is necessary to take into account adequate selection of suppliers, which will greatly influence the increase in competitiveness in the market. Suppliers are obliged to meet a number of requirements and be prepared to adapt their business policies to their customers at all times. In order to do this, it is necessary to consider all three aspects of sustainability: environmental, economic, and social. Accordingly, in this paper, the evaluation of suppliers has been performed considering 21 criteria from the aspect of all sustainability factors.
For this purpose, an integrated FUCOM-Interval Rough SAW model was developed. The FUCOM method was applied to determine the significance of the criteria arranged into two levels. First, the values of the main criteria, economic, social and environmental, were determined, and then the weights of all sub-criteria for each main group of criteria were calculated. Subsequently, the final values for all criteria were obtained by multiplying the values of the main criteria by the values of the sub-criteria for each group individually. The interval rough SAW method was developed and applied to rank and select suppliers from a potential set.
This paper has several aims. The first refers to the enrichment of the theory of multi-criteria problems, which is largely dominated by ambiguities and dilemmas. This aim is achieved by developing a novel interval rough SAW method. The second aim is to reduce subjectivity in group decision-making and to provide more precise determination of the preferences by decision-makers. This is achieved by integrating the FUCOM method and the developed interval rough SAW algorithm. The third aim is to optimize the procurement process—i.e., the evaluation and selection of a sustainable supplier in a lime manufacture company. The first two aims reflect the contribution of this research.
After the introduction, the paper is written throughout five more sections. The second section provides a comprehensive review of the literature, which consists of three parts: a review of the application of the FUCOM method, a review of published studies that have integrated rough set theory with MCDM methods, and a review of supplier selection over the past five years. The third section presents the model proposed in this study. The FUCOM algorithm is briefly given, as well as the significance and operations with interval rough numbers. The development of the interval rough SAW method is presented at the end of this section. The fourth section consists of the results where the calculation is shown in detail using the proposed integrated model. A sensitivity analysis that includes changing the weight of the criteria throughout a total of 21 scenarios and the impact of dynamic matrices on the rank change is performed in the fifth section. Finally, the sixth section presents concluding considerations and guidelines for future research.

2. Literature Review

2.1. Review of Applying the FUCOM Method in Various Fields

The nature of criteria is not the same for all problems, and therefore appropriate weight coefficients should be assigned to the criteria. A novel full consistency method (FUCOM) has been proposed for the accurate determination of the weight coefficients of criteria by Pamučar et al. [1]. The FUCOM method offers an advantage over existing multi-criteria decision-making methods since it requires only (n−1) comparison of criteria. The integrated FUCOM-MAIRCA model was tested in a study [2] assessing road and rail level crossings in Serbia. The evaluation of crossings is performed throughout the assessment of seven criteria established on the basis of representative literature and expert surveys. The presented model allows for the consideration of subjectivity in a process of group decision-making throughout the linguistic assessment of evaluation criteria. By Prentkovskis et al. [3], a new Delphi-FUCOM-SERVQUAL methodology was developed for improving the process of measuring the quality of express post services. The evaluation of dimensions was performed by 70 respondents, regular customers of services and customers who used services on a one-time basis. The advantage of the developed methodology is the fact that it enables accurate processing of input and output parameters and provides more objective results. Then, Nunić [4] applied the FUCOM method in estimating the weights of criteria with MABAC for the assessment of PVC joinery. Five potential manufacturers are evaluated on the basis of seven criteria. To select AGV as one important type of material handling equipment in a warehouse, Zavadskas et al. [5] used the integration of a new R-ROV model with the FUCOM method. A multi-criteria model was created on the basis of which a selection among nine AGVs was made and the FUCOM method was used to obtain the weights of seven criteria. In their paper, Fazlollahtabar et al. [6] presented the FUCOM method in group decision-making to evaluate an adequate forklift. The selection of a side-loading forklift is conditioned by the optimality of seven criteria required by working conditions to achieve the efficiency of the company’s storage system. Bozanic et al. [7] uses the FUCOM-FAZZY MABAC model to decide on the most appropriate location for a bridge, since proper location selection can prevent potential problems in the process of designing and subsequently using a single-span Bailey bridge. The advantages of the FUCOM method in comparison with the AHP method were demonstrated in a study [8] estimating Libyan airlines. The comparison was made on the basis of 16 key indicators arranged into 5 groups: reliability, management, employees, satisfaction, and tangibility. The model can be used as a decision-making support tool to improve the performance of airlines. The study [9] selected the optimal route criteria for the transport of hazardous materials using a new approach in the field of multi-criteria decision-making. The weight coefficients of these criteria were determined using the FUCOM method, and the evaluation and selection of suppliers was determined using the TOPSIS and MABAC methods. In the research paper [10], he presents the use of a multi-criteria FUCOM-WASPAS model for ranking dangerous road sections. The study covers seven traffic safety criteria that are evaluated by a linguistic scale. The same FUCOM-WASPAS model is used [11] to rank suppliers in a procurement system of a wood company. Considering that supplier selection is a key role for achieving a competitive advantage and reducing overall production costs, Matić et al. [12] apply a combined FUCOM-rough COPRAS model to estimate a sustainable supplier in a construction company. Durmić [13] also uses the FUCOM method to determine the significance of the criteria for selecting a sustainable supplier at two decision-making levels, with the first level consisting of three criteria: economic, social, and environmental, and the second level consisting of 21 sub-criteria. The results of the weight values of the criteria will be applied in this research study.

2.2. Review of Applying Multi-Criteria Methods in the Integration with Rough Numbers

More recently, rough set theory has been widely used in all fields with the integration of multi-criteria decision-making (MCDM) methods. The use of rough numbers together with MCDM methods, according to Stević et al. [14], provides good results and has numerous benefits. To ensure the satisfaction of all participants in the evaluation and selection of suppliers, Stević et al. [14] use a combination of rough numbers with MCDM methods: DEMATEL, EDAS, COPRAS, and MULTIMOORA, and compare the obtained results with rough MABAC and rough MAIRCA. In another study, Stević et al. [15] evaluate the supplier criteria in a wire manufacture company by applying the integration of rough sets with the analytic hierarchy process (AHP). The proposed approach shows that it offers the possibility of reducing costs, increasing competitiveness and achieving final-user satisfaction. In the paper by Shindpour et al. [16] for the evaluation of design concept, to address the uncertainties and ambiguities of the data in the primary stages of product design, the performance of design concepts is evaluated in relation to quantitative and qualitative criteria simultaneously using a rough set and a fuzzy set. The AHP method was used to obtain the criterion weights, while the evaluation was performed using the TOPSIS method. Tiwari et al. [17] evaluate the concept of design as a complex multi-criteria decision-making process by applying rough sets and the VIKOR method in order to reduce the imprecise content of the user evaluation process and thus improve the efficiency of product design. Zhu et al. [18] present a symmetric evaluation method by integrating an analytic hierarchy process based on rough numbers and the VIKOR method based on rough numbers to evaluate design in a subjective environment. The study [19] for solving multi-criteria problems proposes a bipolar model based on the integration of rough set theory and three-way decisions for forming a hybrid model. Karavidić and Projović [20] use rough sets in multi-criteria decision-making models to make decisions in operations of security forces. Such a model based on rough sets formulates specific decision-making rules that help decision-makers in a process of security operation planning. For the financial modeling and improvement planning of the life insurance industry, Shen et al. [21] propose a two-stage hybrid approach, where rough financial knowledge is retrieved first and then obtained core attributes are measured and synthesized using fuzzy-integral-based decision methods. Five companies were surveyed to illustrate the planning for financial performance improvement by applying this approach. When selecting suppliers in a PVC joinery company, Stojić et al. [22] use the rough AHP method to calculate the relative values of weight criteria, while the supplier assessment and ranking is conducted using the rough WASPAS method. In their paper, Zavadskas et al. [23] develop a new multi-criteria approach Rough Step-Wise Weight Assessment Ratio Analysis (R-SWARA) and demonstrate its application in logistics. To measure the performance of a Turkish airline, it is necessary to use a methodology considering quantitative and qualitative factors and their interrelationships, and for this reason the study [24] used an integral approach of the analytic hierarchy process enhanced by rough set theory. Additionally, fuzzy TOPSIS was used to obtain the final performance ranking. Kusi-Sarpong et al. [25] also use a new multi-criteria approach that integrates elements of rough set theory and fuzzy TOPSIS methodology to manage green supply chains in the mining industry. Rough sets in combination with the TOPSIS method are also applied by Song et al. [26] to analyze the modes and effects of failure in uncertain environments. The study [27] of the game evaluation model provides an integration of rough number theory with the fuzzy Delphi-AHP-TOPSIS model, where fuzzy Delphi is used to select evaluation criteria, fuzzy AHP is used to analyze the structure of criteria and determine the weights of criteria, and finally fuzzy TOPSIS is used to determine the order of evaluation. The successful integration of RST technique and the TOPSIS multi-criteria decision-making method by Khan et al. [28] led to the successful prediction of sites for the case of Pakistan Red Crescent Society (PRCS) in four districts examined. To evaluate 20 criteria divided into four sets, Vasiljević et al. [29] use AHP based on rough numbers. The proposed evaluation methodology was demonstrated throughout the application in a company that manufactures metal washers for the automotive industry. For the evaluation of university websites, Pamučar et al. [30] present the hybrid IR-AHP–MABAC (interval rough analytic hierarchy process–multiattributive border approximation area comparison) model adapted to group decision-making based on the application of a new approach to treating uncertainties using interval rough numbers.

2.3. Selection of Suppliers over the Last Five Years

Evaluating and selecting suppliers is one of the most important processes for achieving an efficient supply chain. Maintaining long-term partnerships with suppliers and using fewer reliable suppliers can help to increase the value of the supply chain [31]. Suppliers are a large part of a production process, accordingly they need to be sorted and carefully analyzed using an efficient framework [32]. In order to achieve the company’s sustainable development, the selection of a sustainable supplier in supply chains is made on the basis of an integrated AHP-VIKOR model [33]. The results show that environmental costs, quality of product, price of product, occupational health and safety systems, and environmental competencies have been ranked as the top five sustainable supplier selection criteria. Awasthi et al. [34] also use the VIKOR method for evaluating supplier performance but in integration with the fuzzy AHP method for generating criterion weights. This integrated approach considers sustainability risks from sub-suppliers (i.e., (1+n)th-tier suppliers). The study [35] proposes a FAHP-based supplier selection model for the automobile industry that often faces a heterogeneous supply environment. Azimifard et al. [36] present an integrated approach of AHP and TOPSIS methods at three levels to select sustainable supplier countries for the Iranian steel industry. In [37], the selection of suppliers for a steel manufacture company in Libya is based on the application of the CODAS method. Recognizing the importance of a supplier selection decision that significantly influences its competitive market position, the authors Rezaei et al. [38] propose an innovative methodology for selecting suppliers throughout three phases: pre-selection, selection, and aggregation. For the supplier selection phase, the authors use the best worst method (BWM). Karsak and Dursun [39] develop a multi-criteria approach using the QFD method for supplier selection. Azadnia el al. [40] present an integrated approach of weighted rule-based fuzzy method, fuzzy analytic hierarchy process and multi-objective mathematical programming to select a sustainable supplier for the case study of packaging foils in the food industry. Girubha et al. [41] compare two ISM-ANP-ELECTRE and ISM-ANP-VIKOR models for the problem of sustainable supplier selection. In order to make supplier selection in terms of sustainability, Arabsheybani et al. [32] apply the fuzzy MOORA model and use the FMEA technique to evaluate supplier risk. The authors Öztürk and Özçelik [42] model the supplier selection problem in the context of a sustainable supply chain based on the triple bottom concept. Due to the multi-criteria nature, the fuzzy TOPSIS method is applied to the sustainable supplier selection of an energy company. Mehregan et al. [43] illustrate how an integrated interpretive structural modeling (ISM) and fuzzy DEMATEL approach can be a valuable management tool for evaluating and analyzing interactions among criteria. The feasibility and effectiveness of this approach is illustrated by the Iranian Gas Engineering and Development Company (IGEDC). The paper [44] framed 12 criteria from an economic, environmental, and social perspective for evaluating suppliers using the TOPSIS method. A novel hierarchical grey-DEMATEL approach is used to improve sustainable supply chain management in the study [45]. The purpose of this study is to apply grey theory that compensates for incomplete information and hierarchical structures to identify aspects of and criteria for supplier prioritization. Moohamed et al. [46] present a sustainable supplier problem-solving framework based on the application of four methods: AHP, fuzzy TOPSIS, MOPM, and TOPSIS.

3. Methods

Figure 1 shows the proposed methodology consisting of four phases and 14 steps. The first phase represents the initial research and data collection and consists of five steps. The first step defines the necessity for research, followed by the definition of research aims in the second step. Then, an MCDM model has been formed, consisting of three main criteria that consider all three aspects of sustainability and a total of 21 sub-criteria equally classified into all three categories. The fourth step of the first phase is data collection, and the fifth step is the assessment of criteria, sub-criteria, and alternatives by a decision-making team. The second phase includes the development of methodology that represents the basic scientific contribution of this paper. After sorting and preparing the data for calculation, a novel interval rough SAW algorithm has been developed, which is presented in detail below. In addition, an integrated model has been created on the basis of applying the FUCOM method and developed interval rough SAW method. The third phase includes a case study in which a sustainable supplier has been selected. The FUCOM method has been applied to calculate the weights of criteria, after which the transformation of individual matrices into interval rough matrices is performed. At the end of this phase, the ranking of suppliers is performed using the developed interval rough SAW algorithm. In the fourth phase, a sensitivity analysis, i.e., validation of the obtained results, is performed in three steps. First, the significance of the criteria is changed and 21 scenarios are created. Subsequently, in the second step, the influence of dynamic matrices on rank reversal is tested, and at the end of the sensitivity analysis, the results obtained are compared using the interval rough COPRAS method.

3.1. Full Consistency Method (FUCOM)

One of the new methods that is based on the principles of pairwise comparison and validation of results through deviation from maximum consistency is the full consistency method (FUCOM) [1]. Benefits that are determinative for the application of FUCOM are a small number of pairwise comparisons of criteria (only n-1 comparison), the ability to validate the results by defining the deviation from maximum consistency (DMC) of comparison and appreciating transitivity in pairwise comparisons of criteria. The FUCOM model also has a subjective influence of a decision-maker on the final values of the weights of criteria. This particularly refers to the first and second steps of FUCOM in which decision-makers rank the criteria according to their personal preferences and perform pairwise comparisons of ranked criteria. However, unlike other subjective models, FUCOM has shown minor deviations in the obtained values of the weights of criteria from optimal values [1]. Additionally, the methodological procedure of FUCOM eliminates the problem of redundancy of pairwise comparisons of criteria, which exists in some subjective models for determining the weights of criteria.
Assume that there are n evaluation criteria in a multi-criteria model that are designated as wj, j = 1,2, ..., n, and that their weight coefficients need to be determined. Subjective models for determining weights based on pairwise comparison of criteria require a decision-maker to determine the degree of impact of the criterion i on the criterion j. In accordance with the defined settings, Figure 2 presents the FUCOM algorithm [1].

3.2. Interval Rough Numbers

The process of group decision-making is accompanied by a great amount of uncertainty and subjectivity, so decision-makers often have dilemmas when assigning certain values to decision attributes. In this paper, a new approach in the theory of rough sets, an approach based on interval rough numbers (IRN), is applied to process the uncertainty contained in data in group decision-making. Suppose that one decision attribute should be assigned a value presented by a qualitative scale whose values range from 1 to 5. One decision-maker (DM) may consider that the decision attribute should have a value between 3 and 4, another DM may consider that a value between 4 and 5 should be assigned, while the third DM has no dilemma about the value of the decision attribute and assigns a value of 4. The dilemmas presented are extremely common in a group decision-making process. In such situations, one of the solutions is to geometrically average two values between which individual decision-makers are in doubt. However, in such situations, the uncertainty (ambiguity) that prevailed in a decision-making process would be lost and further calculation would be reduced to crisp values. On the other hand, the use of fuzzy or grey techniques would entail predicting the existence of uncertainty and subjectively defining the interval by which uncertainty is exploited. Subjectively defined intervals in further data processing can significantly influence the final decision [47], which should definitely be avoided if we aim at impartial decision-making.
On the contrary, the approach based on interval rough numbers includes exploiting the uncertainty that exists in the data obtained. By applying the arithmetic operations that are explained in the following section, we obtain the values of attributes that fully describe the specified uncertainties without subjectively affecting their values. Thus, the uncertainties of the first DM can be described by an interval rough number IRN = [(3,3.67), (4,4.33)], of the second DM by IRN = [(3.67,4), (4.33,5)], while of the third DM by IRN = [(3.67,4), (4,4.33)]. A detailed procedure for determining IRN is explained in the following section.
Suppose that there is a set of k classes that represent the preferences of DM, R = ( J 1 , J 2 , , J k ) , provided that they belong to a sequence that satisfies the condition J 1 < J 2 < , , < J k and the second set of m classes that also represent the preferences of DM, R * = ( I 1 , I 2 , , I k ) . All objects are defined in the universe and related to the preferences of DM. In R * , each class of objects is presented in an interval I i = { I l i , I u i } , where the condition that I l i I u i ( 1 i m ) is satisfied, as well as the condition that I l i , I u i R . Then I l i represents the lower limit of the interval, while I u i represents the upper limit of the interval of i th class of objects. If both limits of the object classes (upper and lower limit) are arranged in such a way that I l 1 * < I l 2 * < , , < I l j * , I u 1 * < I u 2 * < , , < I u k * ( 1 j , k m ), respectively, then we can define two new sets that contain the lower object class R l * = ( I l 1 * , I l 2 * , , I l j * ) and upper object class R u * = ( I u 1 * , I u 2 * , , I u k * ) , respectively. Then for any class of objects I l i * R ( 1 i j ) and I u i * R ( 1 i k ), we can define the lower approximation of I l i * and I u i * as
A p r _ ( I l i * ) = { Y U / R l * ( Y ) I l i * }
A p r _ ( I u i * ) = { Y U / R u * ( Y ) I u i * }
The upper approximations of I l i * and I u i * are defined by applying the following expressions
A p r ¯ ( I l i * ) = { Y U / R l * ( Y ) I l i * }
A p r ¯ ( I u i * ) = { Y U / R u * ( Y ) I u i * }
Both classes of objects (upper and lower class of objects I l i * and I u i * ) are defined by their lower limits L i m _ ( I l i * ) and L i m _ ( I u i * ) and upper limits L i m ¯ ( I l i * ) and L i m ¯ ( I u i * ) , respectively,
L i m _ ( I l i * ) = 1 M L R l * ( Y ) | Y A p r _ ( I l i * )
L i m _ ( I u i * ) = 1 M L * R u * ( Y ) | Y A p r _ ( I u i * )
where M L and M L * represent the total number of objects contained in the lower approximation of the classes of objects I l i * and I u i * , respectively. The upper limits L i m ¯ ( I l i * ) and L i m ¯ ( I u i * ) are defined by Expressions (7) and (8)
L i m ¯ ( I l i * ) = 1 M U R l * ( Y ) | Y A p r ¯ ( I l i * )
L i m ¯ ( I u i * ) = 1 M U * R u * ( Y ) | Y A p r ¯ ( I u i * )
where M U and M U * represent the total number of objects contained in the upper approximation of the classes of objects I l i * and I u i * , respectively.
For the lower class of objects, rough boundary interval of I l i * is presented as R B ( I l i * ) and denotes the interval between the lower and upper limit
R B ( I l i * ) = L i m ¯ ( I l i * ) L i m _ ( I l i * )
Whereas for the upper class of objects, we obtain the rough boundary interval of I u i * as
R B ( I u i * ) = L i m ¯ ( I u i * ) L i m _ ( I u i * )
Then the uncertain class of objects I l i * and I u i * can be presented using their lower and upper limits
R N ( I l i * ) = [ L i m ¯ ( I l i * ) , L i m _ ( I l i * ) ]
R N ( I u i * ) = [ L i m ¯ ( I u i * ) , L i m _ ( I u i * ) ]
As we can see, each class of objects is defined by its lower and upper limits that represent the interval rough number, which is defined as
I R N ( I i * ) = [ R N ( I l i * ) , R N ( I u i * ) ]
The procedure for determining IRN will be explained by the example of determining the weight coefficient of criterion wi in which four experts participate. The experts evaluated the criteria using a scale that encompasses integer values in the interval from one to five: 1—very small impact; 2—small impact; 3—medium impact; 4—large impact; and 5—very large impact. The experts’ evaluations are shown in Table 1.
The experts’ evaluations in Table 1 are presented in the form of ordered pairs (ai;bi), where ai and bi are the values assigned by the experts to the criteria from scale 1–5. The experts who cannot confidently opt for one of the values in the scale, enter both values that they have a dilemma about (E1, E2, and E3). In our example, only the expert E4 had no dilemma and chose a unique value from the scale.
These uncertainties can be represented by trapezoidal fuzzy numbers of the form A = (a1, a2, a3, a4), where a2 and a3 represent values in which the membership function reaches its maximum value, while a1 and a4 represent the left and right limit of the fuzzy set, respectively. In our example (Table 1), we obtain four trapezoidal fuzzy numbers A (E1) = (1,2,3,4), A (E2) = (2,3,4,5), A (E3) = (3. 4,5,5), and A (E4) = (4,5,5,5). Trapezoidal fuzzy numbers are graphically shown in Figure 3, where a darker nuance indicates values in which the membership function reaches its maximum value (a2 and a3), while a light nuance indicates elements of a set that more or less belong to the fuzzy set (a1 and a4).
In addition to the fuzzy approach, the uncertainties described can also be presented by interval rough numbers. Since in the previous section, Expressions (1)–(12), it is defined that an IRN consists of two rough sequences and we distinguish two classes of objects wi and wi: w i = { 2 ; 3 ; 4 ; 5 } and w i = { 3 ; 4 ; 5 ; 5 } . Using Expressions (1)–(8), rough sequences (11) and (12) are formed for each class of objects. For the first class of objects, we obtain:
L i m _ ( 2 ) = 2 , L i m ¯ ( 2 ) = 1 4 ( 2 + 3 + 4 + 5 ) = 3.5 ; R N ( 2 ) = [ 2 , 3.5 ]
L i m _ ( 3 ) = 1 2 ( 2 + 3 ) = 2.5 , L i m ¯ ( 3 ) = 1 3 ( 3 + 4 + 5 ) = 4 ; R N ( 3 ) = [ 2.5 , 4 ]
L i m _ ( 4 ) = 1 3 ( 2 + 3 + 4 ) = 3 , L i m ¯ ( 4 ) = 1 2 ( 4 + 5 ) = 4.5 ; R N ( 4 ) = [ 3 , 4.5 ]
L i m _ ( 5 ) = 1 4 ( 2 + 3 + 4 + 5 ) = 3.5 , L i m ¯ ( 5 ) = 5 ; R N ( 5 ) = [ 3.5 , 5 ]
For the second class of objects, we obtain:
L i m _ ( 3 ) = 3 , L i m ¯ ( 3 ) = 1 4 ( 3 + 4 + 5 + 5 ) = 4.25 ; R N ( 2 ) = [ 3 , 4.5 ]
L i m _ ( 4 ) = 1 2 ( 3 + 4 ) = 3.5 , L i m ¯ ( 4 ) = 1 3 ( 4 + 5 + 5 ) = 4.67 ; R N ( 3 ) = [ 2.5 , 4 ]
L i m _ ( 5 ) = 1 4 ( 3 + 4 + 5 + 5 ) = 4.25 , L i m ¯ ( 5 ) = 5 ; R N ( 5 ) = [ 4.25 , 5 ]
On the basis of rough sequences, we obtain interval rough numbers: I R N ( E 1 ) = ( [ 2 , 3.5 ] , [ 3 , 4.25 ] ) , I R N ( E 2 ) = ( [ 2.5 , 4 ] , [ 3.5 , 4.67 ] ) , I R N ( E 3 ) = ( [ 3 , 4.5 ] , [ 4.25 , 5 ] ) and I R N ( E 4 ) = ( [ 3.5 , 5 ] , [ 4.25 , 5 ] ) .
By rational reasoning, without applying rough and fuzzy sets, we conclude that the values of the criterion wi should be between the values of 3.5 and 4.25. These values are obtained by geometrical averaging the classes of objects w i = { 2 ; 3 ; 4 ; 5 } and w i = { 3 ; 4 ; 5 ; 5 } . In Figure 3, the rational (expected) values of 3.5 and 4.25 are shown by the dashed line. From Figure 3, we can see that the expected values (3.5 and 4.25) are completely within all IRNs. On the other hand, fuzzy numbers only partially cover expected values. The affiliation function of fuzzy numbers A(E2) and A(E3) with maximum affiliation only partially covers the expected values, while the fuzzy numbers A(E1) and A(E4) cover the expected values with an affiliation degree of 0.5. On the other hand, all IRNs fully cover the expected values (3.5 and 4.25) by their intervals.
Interval rough numbers are characterized by specific arithmetic operations, which are different from arithmetic operations with classical rough numbers. Arithmetic operations between two interval rough numbers I R N ( A ) = ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) and I R N ( B ) = ( [ b 1 , b 2 ] , [ b 3 , b 4 ] ) are performed using Expressions (14), (15), (16), (17), and (18):
(1) Addition of interval rough numbers, “+”,
I R N ( A ) + I R N ( B ) = ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) + ( [ b 1 , b 2 ] , [ b 3 , b 4 ] ) = ( [ a 1 + b 1 , a 2 + b 2 ] , [ a 3 + b 3 , a 4 + b 4 ] )
(2) Subtraction of interval rough numbers, “−”,
I R N ( A ) I R N ( B ) = ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) ( [ b 1 , b 2 ] , [ b 3 , b 4 ] ) = ( [ a 1 b 4 , a 2 b 3 ] , [ a 3 b 2 , a 4 b 1 ] )
(3) Multiplication of interval rough numbers, “×”,
I R N ( A ) × I R N ( B ) = ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) × ( [ b 1 , b 2 ] , [ b 3 , b 4 ] ) = ( [ a 1 × b 1 , a 2 × b 2 ] , [ a 3 × b 3 , a 4 × b 4 ] )
(4) Division of interval rough numbers, “/”,
I R N ( A ) / I R N ( B ) = ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) / ( [ b 1 , b 2 ] , [ b 3 , b 4 ] ) = ( [ a 1 / b 4 , a 2 / b 3 ] , [ a 3 / b 2 , a 4 / b 1 ] )
(5) Scalar multiplication of interval rough numbers where k > 0
k × I R N ( A ) = k × ( [ a 1 , a 2 ] , [ a 3 , a 4 ] ) = ( [ k × a 1 , k × a 2 ] , [ k × a 3 , k × a 4 ] )
Any two interval rough numbers I R N ( α ) = ( [ α L , α U ] , [ α L , α U ] ) and I R N ( β ) = ( [ β L , β U ] , [ β L , β U ] ) are ranked according to the following rules:
(1) If the interval of interval rough number is not strictly bounded by another interval, then:
(a) If the condition that { α U > β U i α L β L } or { α U β U i α L < β L } is satisfied, then I R N ( α ) > I R N ( β ) , Figure 4a.
(b) If the condition that { α U = β U and α L = β L } is satisfied, then I R N ( α ) = I R N ( β ) , Figure 4b.
(2) If the intervals of interval rough numbers I R N ( α ) and I R N ( β ) are strictly bounded, then it is necessary to determine the intersection points I ( α ) and I ( β ) of interval rough numbers I R N ( α ) and I R N ( β ) . If the condition that β U < α U and β L > α L is satisfied, then:
(c) If the condition that I ( α ) I ( β ) is satisfied, then I R N ( α ) < I R N ( β ) , Figure 4c,d.
(d) If the condition that I ( α ) > I ( β ) is satisfied, then I R N ( α ) > I R N ( β ) , Figure 4e.
The intersection points of interval rough numbers are obtained in the following way:
μ α = R B ( α u i ) R B ( α u i ) + R B ( α l i ) ; R B ( α u i ) = α U α L ; R B ( α l i ) = α U α L
μ β = R B ( β u i ) R B ( β u i ) + R B ( β l i ) ; R B ( β u i ) = β U β L ; R B ( β l i ) = β U β L
I ( α ) = μ α α L + ( 1 μ α ) α U
I ( β ) = μ β β L + ( 1 μ β ) β U

3.3. Interval Rough SAW Method

The SAW method is a simple and easily applicable method of multi-criteria decision-making. However, using only crisp numbers it is impossible to obtain results that treat uncertainty and objectivity in an adequate way [48]. The rough SAW method was developed two years ago and presented in study [48]. This paper presenting a new approach that combines the SAW method and interval rough numbers consists of the following steps:
Step 1: Forming a multi-criteria decision-making model that consists of m alternatives and n criteria.
Step 2: Forming a team of r experts, who will make assessment of the alternatives according to all the criteria and sub-criteria.
Step 3: Transformation of individual matrices into a group interval rough matrix. In this step, it is necessary to transform each individual matrix of experts r1, r2, ..., rn into an interval rough group matrix using Equations (1)–(12)
Y = A 1 A 2 A m C 1 [ I R N ( x 11 ) I R N ( x 21 ) I R N ( x l 1 ) C 2 I R N ( x 12 ) I R N ( x 22 ) I R N ( x l 2 )   C n I R N ( x 1 n ) I R N ( x 2 n ) I R N ( x l n ) ] m × n
where m denotes a number of alternatives and n denotes a number of criteria.
Step 4: Normalization of the initial interval rough group matrix (24) using Equations (25) and (26):
Y = A 1 A 2 A m C 1 [ I R N ( n 11 ) I R N ( n 21 ) I R N ( n l 1 ) C 2 I R N ( n 12 ) I R N ( n 22 ) I R N ( n l 2 )   C n I R N ( n 1 n ) I R N ( n 2 n ) I R N ( n l n ) ] m × n
If the criterion belongs to the benefit group, then Equation (25) is used for the normalization process
I R N ( n i j ) = ( [ n i j L , n i j U , n i j L , n i j U ] ) = ( [ n i j L , n i j U , n i j L , n i j U ] ) max ( [ n i j L , n i j U , n i j L , n i j U ] )
Whereas for the criteria belonging to the cost group, Equation (26) is applied
I R N ( n i j ) = ( [ n i j L , n i j U , n i j L , n i j U ] ) = min ( [ n i j L , n i j U , n i j L , n i j U ] ) ( [ n i j L , n i j U , n i j L , n i j U ] )
Equations (25) and (26) are further broken down in Equations (27) and (28)
( [ n i j L , n i j U , n i j L , n i j U ] ) = ( [ n i j L max n i j U , n i j U max n i j L , n i j L max n i j U , n i j U max n i j L ] ) i f C n B
( [ n i j L , n i j U , n i j L , n i j U ] ) = ( [ min n i j L n i j U , min n i j U n i j L , min n i j L n i j U , min n i j u n i j L ] ) i f C n C
Step 5: Weighting the previous normalized matrix
I R N ( V i j ) = ( [ v i j L , v i j U , v i j L , v i j U ] ) m × n = ( [ n i j L × w i j L , n i j U × w i j U , n i j L × w i j L , n i j U × w i j U ] )
Step 6: Summing all of the values of the alternatives obtained (summing by rows)
I R N ( S i ) = ( [ s i L , s i U , s i L , s i U ] ) 1 × m
Step 7: Ranking the alternatives in descending order, that is, the highest value is the best alternative. In order to rank the potential solutions more easily, the rough number can be converted into a crisp number using the average value.

4. Results

The selection of a sustainable supplier depends on the precise determination and selection of appropriate criteria and their evaluation. Criteria and alternatives were evaluated by a group of employed experts according to the requirements of Carmeuse (Bosnia and Herzegovina), a lime manufacture company. This group consists of three decision-makers. The criteria for selecting a sustainable supplier are as follows:
  • Sub-criteria of the group of economic criteria C1: C11—cost / price; C12—quality; C13—flexibility; C14—productivity; C15—financial capacity; C16—partnerships; and C17—ecological innovations
  • Sub-criteria of the group of social criteria C2: C21—reputation; C22—work safety; C23—employees’ rights; C24—local community influence; C25—training of employees; C26—respect for rights and policies; and C27—release of information.
  • Sub-criteria of the group of environmental criteria C3: C31 — environmental image; C32—recycling; C33—pollution control; C34—environmental management system; C35—environmentally friendly products; C36—resource consumption; and C37—green competencies.
The complete calculation of the criteria is shown in [13], where the following values of criteria are obtained, which are further used in the calculation with the Interval Rough SAW method:
w 11 = 0.084 , w 12 = 0.135 , w 13 = 0.058 , w 14 = 0.083 , w 15 = 0.055 , w 16 = 0.060 , w 17 = 0.045 , w 21 = 0.044 , w 22 = 0.060 , w 23 = 0.031 , w 24 = 0.024 , w 25 = 0.033 , w 26 = 0.040 , w 27 = 0.037 , w 31 = 0.029 , w 32 = 0.037 , w 33 = 0.046 , w 34 = 0.027 , w 35 = 0.030 , w 36 = 0.019 , w 37 = 0.023
The evaluation of alternatives by three decision-makers has been performed using interval numbers as shown in Table 2.
Applying Equations (1)–(12), the values from Table 2 are transformed into interval rough values, which represent the initial interval rough matrix shown in Table 3.
Then, Equations (24)–(28) need to be applied to normalize the initial interval rough matrix. Criteria C11 and C36 belong to the cost group, while the other criteria need to be maximized, i.e., they belong to the group of beneficial criteria. Table 4 shows the normalized interval rough matrix.
An example of calculating a normalized matrix for the criteria belonging to the cost group is:
I R N ( n 11 ) = ( [ 0.44 , 1.03 , 0.97 , 1.8 ] ) = ( [ 1.11 2.50 , 1.55 1.50 , 1.50 1.55 , 2.00 1.11 ] ) for the alternative A1.
An example of calculating a normalized matrix for the criteria belonging to the benefit group is:
I R N ( n 37 ) = ( [ 0.81 , 1.09 , 0.92 , 1.24 ] ) = ( [ 4.45 5.50 , 4.89 4.50 , 4.50 4.89 , 5.50 4.45 ] ) for the alternative A4.
Subsequently, the normalized interval rough matrix is weighted with the criterion values obtained using the FUCOM method. The weighting is performed by applying Equation (29), while summing the values for alternatives by rows is obtained by applying Equation (30). Table 5 shows the final results of an integrated FUCOM–interval rough SAW approach.
Ranking is performed in descending order, with the highest value being the best solution and the lowest being the worst. Alternative 4 is the most acceptable solution according to the results obtained.

5. Sensitivity Analysis

5.1. Changing the Weights of Criteria

The purpose of sensitivity analysis is to evaluate the impact of the most influential criterion on the ranking performance of the proposed model. After determining the weight coefficients of the criteria using FUCOM, the “most important criterion” is identified for the purpose of sensitivity analysis. Based on the recommendations by Kirkvood [49] and Kahraman [50], Equation (31) is defined, expressing the proportionality of weights during the analysis of sensitivity
w c = ( 1 w s ) × ( w c o / W c o ) = w c o Δ x α c
where w c represents the change in the weights of the criteria in the sensitivity analysis, w s represents the weight of the most significant criterion, w c o represents the original values of the weights of the criteria, and W c o represents the sum of the original weight values of the criteria being changed. The parameter α c is defined as the weight coefficient of elasticity, which expresses the relative compensation of other weight coefficient values with respect to the given changes in the weight of the most significant criterion [49,50].
The assumptions made when conducting the sensitivity analysis are as follows [50]: (a) The value of the elasticity weight coefficient for the most significant criterion is defined as one; (b) The ratio of variable weights remains constant throughout the sensitivity analysis. The parameter Δ x (Expression (31)) represents the amount of change that is applied to a set of weight coefficients depending on their elasticity weight coefficients. The change in the weights of the most important criterion should be restricted. Otherwise, the weights may take negative values, which would violate the constraints of weight proportionality. The parameter Δ x can be (a) positive, indicating an increase in relative importance, or (b) negative, indicating a decrease in relative importance. The limits for Δ x are defined as the largest change in weight of the most significant criterion in the negative and positive direction, i.e., w s o Δ x min { w c o / α c } . After defining the limits for Δ x , the new weights of criteria are calculated according to the previously set parameters for the sensitivity analysis. A new set of criteria always satisfies the universal state of proportionality of the weight coefficients that w s + w c = 1 [50]. Based on the new criterion values obtained, the ranks of the alternatives for the observed scenario are calculated.
In this research, criterion C2 has been identified as the most influential, since it has the highest value of the weight coefficient w 2 = 0.135 . In the next step, the elasticity weight coefficient ( α s ) of the most significant criterion is determined (Table 6) and the limits of change in the weight coefficient of the most significant criterion ( Δ x ) are defined.
Thus, the boundary values of criterion C2, which are [−0.135, 0.865], are obtained. Based on the defined limits of change in the weight coefficient of the most significant criterion, the scenarios for the sensitivity analysis are determined by dividing the interval −0.135≤ Δ x ≤0.865 into a total of 21 scenarios. Thus, the new values of the weight coefficients for 21 scenarios are defined, Table 7.
The effect of the new values of weight coefficients (Table 7) on the change in the ranks of alternatives is shown in Figure 5.
The results (Figure 5) show that the rankings of considered alternatives change, thus confirming that the model is sensitive to changes in weight coefficients. By comparing the two top-ranked alternatives (A4 and A3) across the scenarios, we can conclude that the A4 alternative remained top-ranked in nine scenarios, while in the remaining scenarios it was second-ranked (in three scenarios) and third-ranked (in nine scenarios). In addition, the second-ranked alternative (A3) maintained its position in 17 scenarios, while it was ranked first in four scenarios. The other alternatives (A1 and A3) also changed rankings. In order to make a final decision regarding the robustness of the obtained solution, it is necessary to determine the statistical dependence of the changes. The statistical dependence of changes in rankings of alternatives across scenarios was determined using the Spearman’s correlation coefficient (SCC).
The results of SCC application show that there is an extremely high correlation between the initial rank and the first 12 scenarios, which is 0.950. In the remaining nine scenarios, the correlation decreases and the average value is 0.211. Since, in more than 57% of the scenarios, the top-ranked alternative (A4) retains its rank and is ranked second or third in the remaining scenarios, we can conclude that there is a satisfactory rank correlation and that the proposed rank is confirmed and credible.

5.2. Influence of Dynamic Matrices on Rank Change

Changing certain parameters of decision matrix, such as introducing a new alternative or eliminating the existing one, can lead to changes in preferences. Therefore, in the following part, several scenarios have been formed where the change in decision matrix elements is simulated. Scenarios have been formed so that the worst alternative is eliminated from subsequent considerations in each scenario. At the same time, the remaining alternatives are ranked for each scenario according to the new initial decision-making matrix. This analysis has two goals: (1) to consider the robustness of the obtained solution under uncertain conditions; and (2) to analyze the performance of the IRN SAW model under the conditions of a dynamic initial decision matrix.
The initial solution using the IRN SAW model was generated as A4 > A3 > A1 > A2. The alternative A2 was identified as the worst, so in the first scenario it was eliminated from the set. Thus, a new decision matrix was obtained with a total of three alternatives. A new solution was generated and the following preferences A4>A3>A1 were obtained. The preferences from the first scenario show that A4 is still the best alternative, while A1 is the worst alternative. In the following scenario, alterative A1 was eliminated and the following rank A4>A3 was obtained. From the results presented, we can conclude that the A4 alternative remains the best-ranked across all the scenarios, thus confirming the robustness of the obtained ranks in a dynamic environment.

6. Conclusions

In this paper, the interval rough SAW method has been developed, which represents a major contribution. With the development of this method, it is possible to treat problems with multiple criteria in a more precise way, considering the uncertainties and ambiguities that are certainly part of solving such problems. Integration with interval numbers and the application of rough set theory results in a model that satisfies various aspects of optimization. In addition, the integration with the FUCOM method has been performed, which allows for reduction of subjectivity in determining the weight of criteria.
The integrated FUCOM–interval rough SAW model was applied to evaluate a sustainable supplier based on 21 criteria arranged into a two-level hierarchical structure. The criteria that equally support all three aspects of sustainability were involved. The FUCOM method was applied to determine the significance of the criteria. First, the values of the main criteria, economic, social, and environmental, were determined, and then the weights of all sub-criteria for each main group of criteria were calculated. The interval rough SAW method was developed and applied to rank and select suppliers from a potential set. The results obtained show that the fourth supplier is the best solution. The validity of the final results was determined throughout a sensitivity analysis. In the sensitivity analysis, 21 scenarios were formed first, which involved changing the weight of the criteria. Subsequently, the influence of dynamic matrices on the ranks of alternatives was tested, and it was identified that there was no change in ranks. The results were also compared with the interval rough COPRAS method [51], which also confirmed the previous results.
Future research related to this paper refers to the integration of other MCDM methods with interval rough numbers when it comes to the methodology applied. In addition, it is possible to identify an extension of the set of criteria for the supplier evaluation from the practical aspect.

Author Contributions

Conceptualization, M.G.; Data curation, E.D.; Formal analysis, A.P. Methodology, Ž.S.; Validation, D.P.; Writing—Original Draft, Ž.S. and E.D.; Writing—Review & Editing, M.G. and D.P.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pamučar, D.; Stević, Ž.; Sremac, S. A new model for determining weight coefficients of criteria in mcdm models: Full consistency method (fucom). Symmetry 2018, 10, 393. [Google Scholar] [CrossRef]
  2. Pamučar, D.; Lukovac, V.; Božanić, D.; Komazec, N. Multi-criteria FUCOM-MAIRCA model for the evaluation of level crossings: Case study in the Republic of Serbia. Oper. Res. Eng. Sci. Theory Appl. 2018, 1, 108–129. [Google Scholar] [CrossRef]
  3. Prentkovskis, O.; Erceg, Ž.; Stević, Ž.; Tanackov, I.; Vasiljević, M.; Gavranović, M. A new methodology for improving service quality measurement: Delphi-FUCOM-SERVQUAL model. Symmetry 2018, 10, 757. [Google Scholar] [CrossRef]
  4. Nunić, Z. Evaluation and selection of Manufacturer PVC carpentry using FUCOM-MABAC model. Oper. Res. Eng. Sci. Theory Appl. 2018, 1, 13–28. [Google Scholar] [CrossRef]
  5. Zavadskas, E.K.; Nunić, Z.; Stjepanović, Ž.; Prentkovskis, O. A novel rough range of value method (R-ROV) for selecting automatically guided vehicles (AGVs). Stud. Inform. Control 2018, 27, 385–394. [Google Scholar] [CrossRef]
  6. Fazlollahtabar, H.; Smailbašić, A.; Stević, Ž. FUCOM method in group decision-making: Selection of forklift in a warehouse. Decis. Mak. Appl. Manag. Eng. 2019, 2, 49–65. [Google Scholar] [CrossRef]
  7. Bozanic, D.; Tešić, D.; Kočić, J. Multi-criteria FUCOM-Fuzzy MABAC model for the selection of location for construction of single-span bailey bridge. Decis. Mak. Appl. Manag. Eng. 2019, 2, 132–146. [Google Scholar] [CrossRef]
  8. Badi, I.; Abdulshahed, A. Ranking the Libyan airlines by using full consistency method (FUCOM) and analytical hierarchy process (AHP). Oper. Res. Eng. Sci. Theory Appl. 2019, 2, 1–14. [Google Scholar] [CrossRef]
  9. Noureddine, M.; Ristic, M. Route planning for hazardous materials transportation: Multicriteria decision making approach. Decis. Mak. Appl. Manag. Eng. 2019, 2, 66–85. [Google Scholar] [CrossRef]
  10. Nenadić, D. Ranking dangerous sections of the road using MCDM model. Decis. Mak. Appl. Manag. Eng. 2019, 2, 115–131. [Google Scholar] [CrossRef]
  11. Erceg, Ž.; Mularifović, F. Integrated MCDM model for processes optimization in supply chain management in wood company. Oper. Res. Eng. Sci. Theory Appl. 2019, 2, 37–50. [Google Scholar] [CrossRef]
  12. Matić, B.; Jovanović, S.; Das, D.K.; Zavadskas, E.K.; Stević, Ž.; Sremac, S.; Marinković, M. A New Hybrid MCDM Model: Sustainable Supplier Selection in a Construction Company. Symmetry 2019, 11, 353. [Google Scholar] [CrossRef]
  13. Durmić, E. Evaluation of criteria for sustainable supplier selection using FUCOM method. Oper. Res. Eng. Sci. Theory Appl. 2019, 2, 91–107. [Google Scholar] [CrossRef]
  14. Stević, Ž.; Pamučar, D.; Vasiljević, M.; Stojić, G.; Korica, S. Novel integrated multi-criteria model for supplier selection: Case study construction company. Symmetry 2017, 9, 279. [Google Scholar] [CrossRef]
  15. Stević, Ž.; Tanackov, I.; Vasiljević, M.; Rikalović, A. Supplier evaluation criteria: AHP rough approach. In Proceedings of the XVII International Scientific Conference on Industrial Systems, Novi Sad, Serbia, 4–6 October 2017; pp. 298–303. [Google Scholar]
  16. Shidpour, H.; Da Cunha, C.; Bernard, A. Group multi-criteria design concept evaluation using combined rough set theory and fuzzy set theory. Expert Syst. Appl. 2016, 64, 633–644. [Google Scholar] [CrossRef]
  17. Tiwari, V.; Jain, P.K.; Tandon, P. Product design concept evaluation using rough sets and VIKOR method. Adv. Eng. Inform. 2016, 30, 16–25. [Google Scholar] [CrossRef]
  18. Zhu, G.N.; Hu, J.; Qi, J.; Gu, C.C.; Peng, Y.H. An integrated AHP and VIKOR for design concept evaluation based on rough number. Adv. Eng. Inform. 2015, 29, 408–418. [Google Scholar] [CrossRef]
  19. Shen, K.Y.; Tzeng, G.H. A novel bipolar MCDM model using rough sets and three-way decisions for decision aids. In Proceedings of the 2016 Joint 8th International Conference on Soft Computing and Intelligent Systems (SCIS) and 17th International Symposium on Advanced Intelligent Systems (ISIS), Sapporo, Japan, 25–28 August 2016; pp. 53–58. [Google Scholar]
  20. Karavidić, Z.; Projović, D. A multi-criteria decision-making (MCDM) model in the security forces operations based on rough sets. Decis. Mak. Appl. Manag. Eng. 2018, 1, 97–120. [Google Scholar] [CrossRef]
  21. Shen, K.Y.; Hu, S.K.; Tzeng, G.H. Financial modeling and improvement planning for the life insurance industry by using a rough knowledge based hybrid MCDM model. Inf. Sci. 2017, 375, 296–313. [Google Scholar] [CrossRef]
  22. Stojić, G.; Stević, Ž.; Antuchevičienė, J.; Pamučar, D.; Vasiljević, M. A novel rough WASPAS approach for supplier selection in a company manufacturing PVC carpentry products. Information 2018, 9, 121. [Google Scholar] [CrossRef]
  23. Zavadskas, E.K.; Stević, Ž.; Tanackov, I.; Prentkovskis, O. A novel multicriteria approach-rough step-wise weight assessment ratio analysis method (R-SWARA) and its application in logistics. Stud. Inform. Control 2018, 27, 97–106. [Google Scholar] [CrossRef]
  24. Aydogan, E.K. Performance measurement model for Turkish aviation firms using the rough-AHP and TOPSIS methods under fuzzy environment. Expert Syst. Appl. 2011, 38, 3992–3998. [Google Scholar] [CrossRef]
  25. Kusi-Sarpong, S.; Bai, C.; Sarkis, J.; Wang, X. Green supply chain practices evaluation in the mining industry using a joint rough sets and fuzzy TOPSIS methodology. Resour. Policy 2015, 46, 86–100. [Google Scholar] [CrossRef]
  26. Song, W.; Ming, X.; Wu, Z.; Zhu, B. A rough TOPSIS approach for failure mode and effects analysis in uncertain environments. Qual. Reliab. Eng. Int. 2014, 30, 473–486. [Google Scholar] [CrossRef]
  27. Su, C.H.; Chen KT, K.; Fan, K.K. Rough set theory based fuzzy TOPSIS on serious game design evaluation framework. Math. Probl. Eng. 2013. [Google Scholar] [CrossRef]
  28. Khan, C.; Anwar, S.; Bashir, S.; Rauf, A.; Amin, A. Site selection for food distribution using rough set approach and TOPSIS method. J. Intell. Fuzzy Syst. 2015, 29, 2413–2419. [Google Scholar] [CrossRef]
  29. Vasiljević, M.; Fazlollahtabar, H.; Stević, Ž.; Vesković, S. A rough multicriteria approach for evaluation of the supplier criteria in automotive industry. Decis. Mak. Appl. Manag. Eng. 2018, 1, 82–96. [Google Scholar] [CrossRef]
  30. Pamučar, D.; Stević, Ž.; Zavadskas, E.K. Integration of interval rough AHP and interval rough MABAC methods for evaluating university web pages. Appl. Soft Comput. 2018, 67, 141–163. [Google Scholar] [CrossRef]
  31. Keshavarz Ghorabaee, M.; Amiri, M.; Zavadskas, E.K.; Antucheviciene, J. Supplier evaluation and selection in fuzzy environments: A review of MADM approaches. Econ. Res. Ekon. Istraživanja 2017, 30, 1073–1118. [Google Scholar] [CrossRef]
  32. Arabsheybani, A.; Paydar, M.M.; Safaei, A.S. An integrated fuzzy MOORA method and FMEA technique for sustainable supplier selection considering quantity discounts and supplier’s risk. J. Clean. Prod. 2018, 190, 577–591. [Google Scholar] [CrossRef]
  33. Luthra, S.; Govindan, K.; Kannan, D.; Mangla, S.K.; Garg, C.P. An integrated framework for sustainable supplier selection and evaluation in supply chains. J. Clean. Prod. 2017, 140, 1686–1698. [Google Scholar] [CrossRef]
  34. Awasthi, A.; Govindan, K.; Gold, S. Multi-tier sustainable global supplier selection using a fuzzy AHP-VIKOR based approach. Int. J. Prod. Econ. 2018, 195, 106–117. [Google Scholar] [CrossRef]
  35. Yadav, V.; Sharma, M.K. Multi-criteria decision making for supplier selection using fuzzy AHP approach. Benchmarking Int. J. 2015, 22, 1158–1174. [Google Scholar] [CrossRef]
  36. Azimifard, A.; Moosavirad, S.H.; Ariafar, S. Selecting sustainable supplier countries for Iran’s steel industry at three levels by using AHP and TOPSIS methods. Resour. Policy 2018, 57, 30–44. [Google Scholar] [CrossRef]
  37. Badi, I.; Abdulshahed, A.M.; Shetwan, A. A case study of supplier selection for a steelmaking company in Libya by using the Combinative Distance-based ASsessment (CODAS) model. Decis. Mak. Appl. Manag. Eng. 2018, 1, 1–12. [Google Scholar] [CrossRef]
  38. Rezaei, J.; Nispeling, T.; Sarkis, J.; Tavasszy, L. A supplier selection life cycle approach integrating traditional and environmental criteria using the best worst method. J. Clean. Prod. 2016, 135, 577–588. [Google Scholar] [CrossRef]
  39. Karsak, E.E.; Dursun, M. An integrated fuzzy MCDM approach for supplier evaluation and selection. Comput. Ind. Eng. 2015, 82, 82–93. [Google Scholar] [CrossRef]
  40. Azadnia, A.H.; Saman MZ, M.; Wong, K.Y. Sustainable supplier selection and order lot-sizing: An integrated multi-objective decision-making process. Int. J. Prod. Res. 2015, 53, 383–408. [Google Scholar] [CrossRef]
  41. Girubha, J.; Vinodh, S.; Kek, V. Application of interpretative structural modelling integrated multi criteria decision making methods for sustainable supplier selection. J. Model. Manag. 2016, 11, 358–388. [Google Scholar] [CrossRef]
  42. Öztürk, B.A.; Özçelik, F. Sustainable supplier selection with a fuzzy multi-criteria decision making method based on triple bottom line. Bus. Econ. Res. J. 2014, 5, 129. [Google Scholar]
  43. Mehregan, M.R.; Hashemi, S.H.; Karimi, A.; Merikhi, B. Analysis of interactions among sustainability supplier selection criteria using ISM and fuzzy DEMATEL. Int. J. Appl. Decis. Sci. 2014, 7, 270–294. [Google Scholar] [CrossRef]
  44. Jia, P.; Govindan, K.; Choi, T.M.; Rajendran, S. Supplier selection problems in fashion business operations with sustainability considerations. Sustainability 2015, 7, 1603–1619. [Google Scholar] [CrossRef]
  45. Su, C.M.; Horng, D.J.; Tseng, M.L.; Chiu, A.S.; Wu, K.J.; Chen, H.P. Improving sustainable supply chain management using a novel hierarchical grey-DEMATEL approach. J. Clean. Prod. 2016, 134, 469–481. [Google Scholar] [CrossRef]
  46. Mohammed, A.; Setchi, R.; Filip, M.; Harris, I.; Li, X. An integrated methodology for a sustainable two-stage supplier selection and order allocation problem. J. Clean. Prod. 2018, 192, 99–114. [Google Scholar] [CrossRef]
  47. Duntsch, I.; Gediga, G. The rough set engine GROBIAN. In Proceedings of the 15th IMACS World Congress, Berlin, Germany, 24–29 August 1997; pp. 613–618. [Google Scholar]
  48. Stević, Ž.; Pamučar, D.; Kazimieras Zavadskas, E.; Ćirović, G.; Prentkovskis, O. The selection of wagons for the internal transport of a logistics company: A novel approach based on rough BWM and rough SAW methods. Symmetry 2017, 9, 264. [Google Scholar] [CrossRef]
  49. Kirkwood, C.W. Strategic Decision Making: Multi-Objective Decision Analysis with Spreadsheets; Duxbury Press: Belmont, Australia, 1997. [Google Scholar]
  50. Kahraman, Y.R. Robust Sensitivity Analysis for Multi-Attribute Deterministic Hierarchical Value Models; Storming Media: Washington, DC, USA, 2002. [Google Scholar]
  51. Pamučar, D.; Božanić, D.; Lukovac, V.; Komazec, N. Normalized weighted geometric bonferroni mean operator of interval rough numbers-application in interval rough dematel-copras model. Facta Univ. Ser. Mech. Eng. 2018, 16, 171–191. [Google Scholar] [CrossRef]
Figure 1. Proposed methodology.
Figure 1. Proposed methodology.
Information 10 00292 g001
Figure 2. Algorithms of FUCOM method.
Figure 2. Algorithms of FUCOM method.
Information 10 00292 g002
Figure 3. Criterion evaluation—interval rough and fuzzy evaluation.
Figure 3. Criterion evaluation—interval rough and fuzzy evaluation.
Information 10 00292 g003
Figure 4. Ranking interval rough numbers.
Figure 4. Ranking interval rough numbers.
Information 10 00292 g004
Figure 5. Sensitivity analysis of alternative rankings across 21 scenarios.
Figure 5. Sensitivity analysis of alternative rankings across 21 scenarios.
Information 10 00292 g005
Table 1. Experts’ evaluation of criterion w i .
Table 1. Experts’ evaluation of criterion w i .
Criterion Experts
E1E2E3E4
wi(2;3)(3;4)(4;5)(5;5)
Table 2. Evaluation of alternatives by three DMs.
Table 2. Evaluation of alternatives by three DMs.
A1A2A3A4
E1E2E3E1E2E3E1E2E3E1E2E3
C11(1;1)(2;3)(1;2)(1;2)(3;4)(3;3)(1;2)(2;2)(2;2)(1;2)(3;3)(2;3)
C12(7;8)(7;7)(7;8)(7;7)(6;7)(6;7)(7;8)(7;7)(6;7)(7;7)(6;7)(6;7)
C13(5;6)(4;5)(5;6)(5;6)(4;5)(4;5)(7;8)(6;7)(5;5)(7;7)(6;7)(6;6)
C14(7;7)(6;7)(6;7)(6;6)(5;6)(6;6)(7;7)(6;7)(5;5)(7;7)(6;6)(6;7)
C15(5;5)(4;5)(5;5)(6;7)(5;6)(5;5)(6;6)(6;7)(6;6)(7;8)(7;7)(6;6)
C16(7;8)(7;8)(7;8)(6;7)(5;6)(5;6)(6;7)(6;6)(6;6)(6;7)(6;6)(7;8)
C17(6;6)(5;6)(5;5)(6;6)(5;5)(4;5)(6;7)(5;6)(5;6)(7;7)(6;7)(5;6)
C21(5;6)(5;5)(4;5)(6;6)(6;7)(5;6)(7;7)(7;7)(6;6)(7;8)(7;7)(6;6)
C22(5;5)(4;5)(5;5)(6;6)(5;5)(4;4)(7;8)(6;7)(4;5)(7;7)(6;6)(5;6)
C23(5;6)(5;5)(5;6)(6;7)(6;7)(5;5)(6;7)(6;7)(4;5)(6;6)(5;6)(4;5)
C24(3;4)(3;3)(2;3)(3;4)(3;3)(3;4)(4;4)(4;5)(3;3)(4;5)(4;4)(4;4)
C25(3;4)(3;4)(3;4)(6;6)(5;5)(4;5)(6;6)(4;5)(4;4)(6;7)(5;6)(5;6)
C26(5;5)(4;5)(4;5)(6;6)(5;6)(5;6)(6;6)(6;7)(5;5)(6;6)(6;7)(6;6)
C27(3;3)(3;3)(2;2)(5;6)(4;5)(3;4)(6;7)(6;6)(4;4)(6;7)(6;6)(4;5)
C31(4;5)(4;4)(5;6)(4;4)(4;5)(5;5)(5;6)(5;6)(4;5)(6;6)(6;6)(5;6)
C32(6;6)(6;6)(5;6)(5;6)(5;5)(5;6)(6;7)(6;7)(5;6)(6;6)(6;6)(4;5)
C33(4;4)(4;4)(6;6)(5;5)(5;6)(5;6)(6;6)(5;5)(4;4)(5;6)(6;6)(4;5)
C34(3;4)(3;3)(4;4)(4;4)(4;5)(4;4)(6;6)(5;5)(2;3)(6;6)(5;5)(3;4)
C35(5;6)(4;5)(4;5)(4;4)(3;4)(4;4)(6;6)(5;5)(3;4)(5;5)(4;5)(5;5)
C36(4;4)(5;5)(5;5)(4;5)(5;5)(5;5)(4;5)(5;5)(6;6)(3;4)(4;4)(3;4)
C37(3;4)(4;4)(3;4)(4;5)(3;4)(4;4)(5;5)(4;5)(3;4)(5;5)(5;6)(4;4)
Table 3. Initial interval rough matrix.
Table 3. Initial interval rough matrix.
A1A2A3A4
C11[1.11, 1.55]; [1.5, 2.5][1.89, 2.78]; [2.5, 3.5][1.45, 1.89]; [2, 2][1.5, 2.5]; [2.45, 2.89]
C12[7, 7]; [7.45, 7.89][6.11, 6.55]; [7, 7][6.45, 6.89]; [7.11, 7.55][6.11, 6.55]; [7, 7]
C13[4.45, 4.89]; [5.45, 5.89][4.11, 4.55]; [5.11, 5.55][5.5, 6.5]; [5.89, 7.39][6.11, 6.55]; [6.45, 6.89]
C14[6.11, 6.55]; [7, 7][5.45, 5.89]; [5.45, 5.89][5.5, 6.5]; [5.89, 6.78][6.11, 6.55]; [6.45, 6.89]
C15[4.45, 4.89]; [5, 5][5.11, 5.55]; [5.5, 6.5][6, 6]; [6.11, 6.55][6.45, 6.89]; [6.5, 7.5]
C16[7, 7]; [8, 8][5.11, 5.55]; [6.11, 6.55][6, 6]; [6.11, 6.55][6.11, 6.55]; [6.5, 7.5]
C17[5.11, 5.55]; [5.45, 5.89][4.5, 5.5]; [5.11, 5.55][5.11, 5.55]; [6.11, 6.55][5.5, 6.5]; [6.45, 6.89]
C21[4.45, 4.89]; [5.11, 5.55][5.45, 5.89]; [6.11, 6.55][6.45, 6.89]; [6.45, 6.89][6.45, 6.89]; [6.5, 7.5]
C22[4.45, 4.89]; [5, 5][4.5, 5.5]; [4.5, 5.5][4.89, 6.39]; [5.89, 7.39][5.5, 6.5]; [6.11, 6.55]
C23[5, 5]; [5.45, 5.89][5.45, 5.89]; [5.89, 6.78][4.89, 5.78]; [5.89, 6.78][4.5, 5.5]; [5.45, 5.89]
C24[2.45, 2.89]; [3.11, 3.55][3, 3]; [3.45, 3.89][3.45, 3.89]; [3.5, 4.5][4, 4]; [4.11, 4.55]
C25[3, 3]; [4, 4][4.5, 5.5]; [5.11, 5.55][4.22, 5.11]; [4.5, 5.5][5.11, 5.55]; [6.11, 6.55]
C26[4.11, 4.55]; [5, 5][5.11, 5.55]; [6, 6][5.45, 5.89]; [5.5, 6.5][6, 6]; [6.11, 6.55]
C27[2.45, 2.89]; [2.45, 2.89][3.5, 4.5]; [4.5, 5.5][4.89, 5.78]; [4.89, 6.39][4.89, 5.78]; [5.5, 6.5]
C31[4.11, 4.55]; [4.5, 5.5][4.11, 4.55]; [4.45, 4.89][4.45, 4.89]; [5.45, 5.89][5.45, 5.89]; [6, 6]
C32[5.45, 5.89]; [6, 6][5, 5]; [5.45, 5.89][5.45, 5.89]; [6.45, 6.89][4.89, 5.78]; [5.45, 5.89]
C33[4.22, 5.11]; [4.22, 5.11][5, 5]; [5.45, 5.89][4.5, 5.5]; [4.5, 5.5][4.5, 5.5]; [5.45, 5.89]
C34[3.11, 3.55]; [3.45, 3.89][4, 4]; [4.11, 4.55][3.28, 5.28]; [3.89, 5.39][3.89, 5.39]; [4.5, 5.5]
C35[4.11, 4.55]; [5.11, 5.55][3.45, 3.89]; [4, 4][3.89, 5.39]; [4.5, 5.5][4.45, 4.89]; [5, 5]
C36[4.45, 4.89]; [4.45, 4.89][4.45, 4.89]; [5, 5][4.5, 5.5]; [5.11, 5.55][3.11, 3.55]; [4, 4]
C37[3.11, 3.55]; [4, 4][3.45, 3.89]; [4.11, 4.55][3.5, 4.5]; [4.45, 4.89][4.45, 4.89]; [4.5, 5.5]
Table 4. Normalized interval rough matrix.
Table 4. Normalized interval rough matrix.
A1A2A3A4
C11[0.44, 1.03], [0.97, 1.8][0.32, 0.62], [0.84, 1.06][0.56, 0.78], [0.79, 1.38][0.38, 0.63], [0.6, 1.33]
C12[0.89, 0.94], [1.06, 1.13][0.77, 0.88], [1, 1][0.82, 0.92], [1.02, 1.08][0.77, 0.88], [1, 1]
C13[0.6, 0.76], [0.83, 0.96][0.56, 0.71], [0.78, 0.91][0.74, 1.01], [0.9, 1.21][0.83, 1.02], [0.98, 1.13]
C14[0.87, 0.94], [1.07, 1.15][0.78, 0.84], [0.83, 0.96][0.79, 0.93], [0.9, 1.11][0.87, 0.94], [0.98, 1.13]
C15[0.59, 0.75], [0.73, 0.78][0.68, 0.85], [0.8, 1.01][0.8, 0.92], [0.89, 1.02][0.86, 1.06], [0.94, 1.16]
C16[0.88, 0.88], [1.14, 1.14][0.64, 0.69], [0.87, 0.94][0.75, 0.75], [0.87, 0.94][0.76, 0.82], [0.93, 1.07]
C17[0.74, 0.86], [0.84, 1.07][0.74, 0.86], [0.79, 1.01][0.74, 0.86], [0.94, 1.19][0.8, 1.01], [0.99, 1.25]
C21[0.59, 0.75], [0.74, 0.86][0.73, 0.91], [0.89, 1.02][0.86, 1.06], [0.94, 1.07][0.86, 1.06], [0.94, 1.16]
C22[0.6, 0.8], [0.77, 0.91][0.61, 0.9], [0.69, 1][0.66, 1.05], [0.91, 1.34][0.74, 1.06], [0.94, 1.19]
C23[0.74, 0.85], [0.93, 1.08][0.8, 1], [1, 1.24][0.72, 0.98], [1, 1.24][0.66, 0.93], [0.93, 1.08]
C24[0.54, 0.73], [0.78, 0.89][0.66, 0.73], [0.86, 0.97][0.76, 0.95], [0.88, 1.13][0.88, 0.97], [1.03, 1.14]
C25[0.46, 0.49], [0.72, 0.78][0.69, 0.9], [0.92, 1.09][0.64, 0.84], [0.81, 1.08][0.78, 0.91], [1.1, 1.28]
C26[0.63, 0.74], [0.83, 0.83][0.78, 0.91], [1, 1][0.83, 0.96], [0.92, 1.08][0.92, 0.98], [1.02, 1.09]
C27[0.38, 0.53], [0.42, 0.59][0.54, 0.82], [0.78, 1.12][0.75, 1.05], [0.85, 1.31][0.75, 1.05], [0.95, 1.33]
C31[0.69, 0.76], [0.76, 1.01][0.69, 0.76], [0.76, 0.9][0.74, 0.82], [0.93, 1.08][0.91, 0.98], [1.02, 1.1]
C32[0.79, 0.91], [1.02, 1.1][0.73, 0.78], [0.93, 1.08][0.79, 0.91], [1.1, 1.26][0.71, 0.9], [0.93, 1.08]
C33[0.72, 0.94], [0.77, 1.02][0.85, 0.92], [0.99, 1.18][0.76, 1.01], [0.82, 1.1][0.76, 1.01], [0.99, 1.18]
C34[0.57, 0.79], [0.64, 0.97][0.73, 0.89], [0.76, 1.14][0.6, 1.17], [0.72, 1.35][0.71, 1.2], [0.83, 1.38]
C35[0.74, 0.89], [0.95, 1.25][0.62, 0.76], [0.74, 0.9][0.7, 1.05], [0.83, 1.24][0.8, 0.96], [0.93, 1.12]
C36[0.64, 0.8], [0.82, 0.9][0.62, 0.71], [0.82, 0.9][0.56, 0.69], [0.73, 0.89][0.78, 0.89], [1.13, 1.29]
C37[0.57, 0.79], [0.82, 0.9][0.63, 0.86], [0.84, 1.02][0.64, 1], [0.91, 1.1][0.81, 1.09], [0.92, 1.24]
Table 5. Results of sustainable supplier selection using an integrated FUCOM–interval rough SAW approach.
Table 5. Results of sustainable supplier selection using an integrated FUCOM–interval rough SAW approach.
SiAVRank
0.6800.8380.8821.0610.8653
0.6710.8210.8631.0160.8424
0.7370.9310.9011.1540.9312
0.7650.9430.9421.1590.9521
Table 6. Coefficient of elasticity for changing weights.
Table 6. Coefficient of elasticity for changing weights.
Criteria α s Criteria α s
C10.097C80.051C150.034
C21.000C90.069C160.043
C30.067C100.036C170.053
C40.096C110.028C180.031
C50.064C120.038C190.035
C60.069C130.046C200.022
C70.052C140.043C210.027
Table 7. Formed scenarios with new criteria weights.
Table 7. Formed scenarios with new criteria weights.
CriteriaS1S2S3S4S5S6S7S8S9S10S11S12S13S14S15S16S17S18S19S20S21
C10.09710.00000.06710.09600.06360.06940.05200.05090.06940.03580.02770.03820.04620.04280.03350.04280.05320.03120.03470.02200.0266
C20.09230.05000.06370.09120.06040.06590.04940.04830.06590.03400.02640.03620.04390.04060.03180.04060.05050.02970.03290.02090.0253
C30.08740.10000.06030.08640.05720.06240.04680.04580.06240.03230.02500.03430.04160.03850.03020.03850.04790.02810.03120.01980.0239
C40.08250.15000.05700.08160.05400.05900.04420.04320.05900.03050.02360.03240.03930.03640.02850.03640.04520.02650.02950.01870.0226
C50.07770.20000.05360.07680.05090.05550.04160.04070.05550.02870.02220.03050.03700.03420.02680.03420.04250.02500.02770.01760.0213
C60.07280.25000.05030.07200.04770.05200.03900.03820.05200.02690.02080.02860.03470.03210.02510.03210.03990.02340.02600.01650.0199
C70.06800.30000.04690.06720.04450.04860.03640.03560.04860.02510.01940.02670.03240.02990.02350.02990.03720.02180.02430.01540.0186
C80.06310.35000.04360.06240.04130.04510.03380.03310.04510.02330.01800.02480.03010.02780.02180.02780.03460.02030.02250.01430.0173
C90.05830.40000.04020.05760.03820.04160.03120.03050.04160.02150.01660.02290.02770.02570.02010.02570.03190.01870.02080.01320.0160
C100.05340.45000.03690.05280.03500.03820.02860.02800.03820.01970.01530.02100.02540.02350.01840.02350.02920.01720.01910.01210.0146
C110.04860.50000.03350.04800.03180.03470.02600.02540.03470.01790.01390.01910.02310.02140.01680.02140.02660.01560.01730.01100.0133
C120.04370.55000.03020.04320.02860.03120.02340.02290.03120.01610.01250.01720.02080.01920.01510.01920.02390.01400.01560.00990.0120
C130.03880.60000.02680.03840.02540.02770.02080.02030.02770.01430.01110.01530.01850.01710.01340.01710.02130.01250.01390.00880.0106
C140.03400.65000.02350.03360.02230.02430.01820.01780.02430.01250.00970.01340.01620.01500.01170.01500.01860.01090.01210.00770.0093
C150.02910.70000.02010.02880.01910.02080.01560.01530.02080.01080.00830.01140.01390.01280.01010.01280.01600.00940.01040.00660.0080
C160.02430.75000.01680.02400.01590.01730.01300.01270.01730.00900.00690.00950.01160.01070.00840.01070.01330.00780.00870.00550.0066
C170.01940.80000.01340.01920.01270.01390.01040.01020.01390.00720.00550.00760.00920.00860.00670.00860.01060.00620.00690.00440.0053
C180.01460.85000.01010.01440.00950.01040.00780.00760.01040.00540.00420.00570.00690.00640.00500.00640.00800.00470.00520.00330.0040
C190.00970.90000.00670.00960.00640.00690.00520.00510.00690.00360.00280.00380.00460.00430.00340.00430.00530.00310.00350.00220.0027
C200.00490.95000.00340.00480.00320.00350.00260.00250.00350.00180.00140.00190.00230.00210.00170.00210.00270.00160.00170.00110.0013
C210.00001.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop