Elimination Method of Multi-Criteria Decision Analysis (MCDA): A Simple Methodological Approach for Assessing Agricultural Sustainability

Abstract

In the present world context, there is a need to assess the sustainability of agricultural systems. Various methods have been proposed to assess agricultural sustainability. Like in many other fields, Multi-Criteria Decision Analysis (MCDA) has recently been used as a methodological approach for the assessment of agricultural sustainability. In this paper, an attempt is made to apply Elimination, a MCDA method, to an agricultural sustainability assessment, and to investigate its benefits and drawbacks. This article starts by explaining the importance of agricultural sustainability. Common MCDA types are discussed, with a description of the state-of-the-art method for incorporating multi-criteria and reference values for agricultural sustainability assessment. Then, a generic description of the Elimination Method is provided, and its modeling approach is applied to a case study in coastal Bangladesh. An assessment of the results is provided, and the issues that need consideration before applying Elimination to agricultural sustainability, are examined. Whilst having some limitations, the case study shows that it is applicable for agricultural sustainability assessments and for ranking the sustainability of agricultural systems. The assessment is quick compared to other assessment methods and is shown to be helpful for agricultural sustainability assessment. It is a relatively simple and straightforward analytical tool that could be widely and easily applied. However, it is suggested that appropriate care must be taken to ensure the successful use of the Elimination Method during the assessment process.

1. Introduction

Sustainability in agriculture has become an important consideration for the Sustainable Development Goals (SDGs). Target two of the SDGs (end hunger, achieve food security, improve nutrition, and promote sustainable agriculture) emphasizes agricultural sustainability, with a variety of recommendations suggested for sustainable agriculture [1]. However, to promote the concept of sustainable agricultural systems, it is important to operationalize the assessment of sustainability [2], by evaluating the sustainability of the existing practices and initiatives [3]. Sustainability assessment through the provision of relevant environmental, economic, and social information, is employed as a policy tool for planning and decision-making. According to the International Union for Conservation of Nature [4] (p. 4), “the main uses of sustainability assessment are: (1) as an input to strategic planning, decision-making, project and programme; (2) as a source of information for monitoring, evaluation and impact analysis; (3) as a source of information for sustainability reporting; and (4) as a process to raise awareness”.

A wide variety of methods have been developed to assess the sustainability of agriculture at the international, national, regional, farm, and product level. While this is an important step, there are drawbacks for many of these methods. In some methods, only one aspect of sustainability is assessed, such as cost-benefit analysis, or the carbon or ecological footprint. Other methods assess the three pillars of sustainability: environmental, economic, and social. Some methods are expert-driven (top-down), while some are expert- and stakeholder-driven (top-down and bottom-up), and some are only stakeholder-driven (bottom-up). Some assessments are based on indicators and some are based on indexes. Most of the initiatives for agricultural assessment have been undertaken by individual scholars or groups. The approaches for the assessment of agricultural sustainability are continuously evolving, because sustainability assessment frameworks are influenced by local agricultural priorities and practices [5]. There are more than 100 assessment tools used around the world [5]. Some of the most practical and useful methods are summarized in Table 1.

Table 1. Selected agricultural sustainability assessment methods/approaches.

Name of Method	Purpose	Some Advantages	Some Disadvantages
SAFA (Sustainability Assessment of Food and Agricultural Systems) [5]	It is a guideline for Sustainability Assessment of Food and Agriculture Systems. It is a general framework for assessing sustainability of food and agriculture systems and takes an umbrella approach. It builds on existing systems that facilitate transparency. It assesses performance, not improvements of the system.	It supports a sustainability management that facilitates progress towards production to processing and distribution of food and agricultural products. The guiding vision of this method is to promote sustainable agriculture systems characterised by “environmental integrity, economic resilience, social well-being and good governance” [5] (p. 1). It is a globally applicable template. It is credible because of institutional independence [5].	It is in the development process and has been applied in few studies. Not all indicators are acceptable by all farming systems of the world.
SAFE [6] (A hierarchical framework for assessing the sustainability of agricultural systems)	SAFE is a dependable and wide-ranging framework of principles, criteria and indicators and reference values structured for sustainability assessment of agricultural systems. It identifies, develops and evaluates the production systems, techniques and policies of agriculture.	The framework is capable of assessing agricultural sustainability at the parcel, farm and higher spatial levels. It is developed in a hierarchical and structured way so it is able to assess the sustainability of agricultural systems. It encompasses the three dimensions of sustainability.	It is not designed to find an answer of agricultural sustainability as a whole. It does not measure the interaction of the three SD’s pillars.
RISE [7] (Response-Inducing Sustainability Evaluation model)	A tool that allows easy assessment of sustainability at the farm level.	It offers a holistic approach by covering agricultural sustainability aspects (ecological, economic and social). It is able to quantify the sustainability level of agricultural systems. It is globally applicable.	It is based on 12 indicators only. It does not measure the interaction of the indicators.
SALSA (A Simulation Tool to Assess Ecological Sustainability of Agricultural Production) [8]	It helps to assess the ecological sustainability of a farm’s agricultural production system. It is based on life-cycle assessment methodology.	It helps in complex studies of agricultural production systems as it is able to capture the consequences of agricultural production management options.	Concentrates on environmental issues only. Used in Switzerland.
EVAS (Empirical Evaluation of Agricultural Sustainability) [9]	It aims to develop a practical methodology for evaluating the sustainability of farms by means of composite indicators.	It evaluates and aims to improve the three dimensions of farm sustainability. This assessment helps to improve current agriculture-related policies such as income, agricultural structure and rural development.	Only 16 indicators cover the three components of the sustainability concept.
IDEA (Indicateurs de Durabilité des Exploitations Agricoles or Farm Sustainability Indicators) [10]	The IDEA method is based on research work conducted since 1998 in France. It gives a practical expression to the concept of sustainable farms. This method supports farmers as well as policy makers to assess sustainable agriculture and support it. It is based on the three different scales of sustainability.	It provides an operational tool for sustainability assessment at the farm level through 41 sustainability indicators covering the three dimensions of sustainability. It can be linked with the Farm Accounting Data Network of France which opens an interesting possibility to assess the sustainability levels of different farming systems. It concentrates on economic viability, social liveability and environmental reproducibility.	There are many models of farm sustainability, therefore while using this method the indicators must be adapted to local farming. It is based on a case study in France.
SEAMLESS (Integrated assessment of agricultural systems—A component-based framework for the European Union) [11]	This framework “aims to assess, ex-ante, agricultural and agri-environmental policies and technologies across a range of scales, from field-farm to region up to the European Union, as well as some global interactions”. “It links individual model and data components and a software infrastructure that allows a flexible (re-) use and linkage of components” [11] (p. 150).	“It addresses the four identified challenges for integrated assessment tools, i.e., linking micro and macro analysis, assessing economic, environmental, social and institutional indicators, (re-)using standalone model components for field, farm and market analysis and their conceptual and technical linkage” [1] (p.150).	Based on the European context.

Apart from the above mentioned approaches, different MCDA methods, including the Multi Attribute Utility Theory (MAUT), have recently been used in agricultural sustainability assessment [12,13]. While different methods of MCDA, like MAUT, AHP, PROMETHEE, ELECTRE, and DRSA, are being used for sustainability assessment in different fields [14,15], they have not yet been tested for their ability to assess agricultural sustainability.

As agricultural sustainability is complex, including environmental, economic, and social processes, its assessment requires a range of information across all categories. MCDA can be an important and suitable framework for assessing agricultural sustainability, because of its flexibility and capacity to handle diverse information [15]. In this paper, an attempt is made to assess agricultural sustainability using the Elimination Method of MCDA, through a case study of coastal agricultural systems in Bangladesh. The main objective of this paper is to investigate the applicability of the Elimination Method for assessing agricultural sustainability. A related objective is to identify the benefits and obstacles of using Elimination as an MCDA tool for agricultural sustainability assessment.

2. Method and Data

MCDA is a method that helps decision makers to evaluate, prioritize, and select between many conflicting alternatives and criteria [16,17]. MCDA is also known as Multiple Criteria Decision Making (MCDM), Multi Criteria Decision Aiding (MCDA), Multi-Attribute Decision Analysis (MADA), Multiple Objective Decision Analysis (MODA), and Single Participant-Multiple Criteria Decision Making (SPMC) [18]). Generally, MCDA follows several phases. It starts by defining the objectives, after which the criteria are chosen to measure these objectives, and alternatives are then specified. Once the criteria and alternatives are fixed, the criteria from different scales are transformed into commensurable units, and weights are assigned to reflect the relative importance of the criteria. In the last phase, mathematical algorithms are selected and utilized for ranking the criteria, or for choosing an alternative [19,20]. In the literature, a rich variety of MCDA techniques are available for utilization, such as the Multi-Attribute Utility Theory (MAUT) [21], Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) [22,23], ELECTRE [24], and the Analytic Hierarchy Process (AHP) [25,26]. To handle uncertainty, concepts from probability [27], fuzzy sets [28], and grey numbers [23] have been incorporated into some of the MCDA methods. Because so many different types of MCDA methods are available for employment by decision makers, one must select the most suitable technique to use in a given situation. The particular MCDA technique which one should employ depends on the characteristics of the problem under study, such as the type of data that are available and the size of the problem. Many MCDA approaches are available, as decision support systems having user-friendly computer programs to allow them to be readily applied to practical problems.

The specific MCDA technique used in the case study investigated this paper is called the Elimination Method. Reasons for utilizing this technique are its simplicity in design and implementation, as well as its capability to provide meaningful findings. Moreover, the case study contains a relatively large number of criteria, which can be readily handled by the technique.

In the next subsection, the approach for applying a particular version of the Elimination Method is explained, followed by a short mathematical description. Within Section 2.2, the data for the case study are presented for the sustainability assessment of five different agricultural systems in Bangladesh.

2.1. Elimination Method of MCDA

The Elimination Method was proposed by MacCrimmon [29] and Radford [30]. It is founded on linguistic rule-based models, which “focus on expressions of preferences on criteria via some linguistic rules, mostly expressed as ‘If ..., then ...’. The advantage of this kind of preference data is that people make decisions by searching for rules that provide good justification of their choices” [31] (p. 19). This method allows the user to rank feasible alternatives, and to consider both numeric and non-numeric criteria [32].

Reference values (Reference value is also referred to as “threshold”, “fair earthshare”, “critical flow” and “sustainability standard” [33] (p. 433) or thresholds are important considerations for elimination methods. Reference values can be determined using normative and relative considerations. “Normative reference values are defined based on science or policy (Experts and stakeholders may be involved), whereas relative reference values are based on indicator values for similar systems or a reference/ideal system. Normative reference values allow comparison of a system with previously defined reference values” [33] (p. 433). To produce a sustainability assessment which is robust, comparable, and transparent among stakeholders, it is important to clarify what type of reference point is being used in the sustainability assessment, as well as how the reference points were determined and why [33]. In the current study, relative reference values are used.

The flow chart in Figure 1 explains the major steps used by the authors to rank the alternatives in their research, as illustrated by the case study in Section 2.2, Section 3 and Section 4. As can be seen, five main steps are utilized to order the alternatives according to preference, from most to least preferred. In fact, the procedures displayed in Figure 1 constitute a special case of the overall elimination approach [32], which is explained in detail by Ma et al. [32]. In Step 4 of the current case study, shown in Figure 1, the highest criterion values of the agricultural systems are considered as reference values, to which the other values of the criteria of the agricultural systems are compared. The reference value represents the highest achievable value in this data set for a given criterion. The scores of the criteria are developed in such a way that the highest value of the criteria represents a higher level of sustainability. Therefore, all of the highest scores of the criteria of different agricultural systems are considered as reference values for the respective criteria. If the criterion value is equal to the reference value, the agricultural system fulfills the criterion. This new rule can be considered as an addition to the overall Elimination Method, that makes it easier to use in sustainability assessment. The total number of criteria fulfilled for each sustainability criterion determines the rank for each agricultural system.

Figure 1. Steps in Elimination Method. Source: Based on [32,34].

To explain the process in Figure 1 in more detail, suppose that a set of alternative agricultural systems is represented by the set:

$$A= \left\{a_1, a_{2, \mathrm{..} . , }{a}_n\right\}$$

where $\left|A\right| \ge 2$.

The sustainability of each alternative can be evaluated using the set of criteria:

$$C= \left\{c_1, c_{2, . . . , }{c}_m\right\}$$

where $\left|C\right| \ge 2$.

Note, that if there were only one criterion to assess each alternative, then the alternatives could be directly ranked according to their performance, with respect to that one criterion, from most to least preferred. This could produce tied results. When there are at least two criteria, one must then determine the scores for each criterion across all of the alternatives, as indicated in Step 2 of Figure 1.

In Step 3, let $v_i$ be the maximum value of criterion $c_i$, across all of the $m$ alternatives. Here, $v_i$ is referred to as the reference value for criterion $i$. In this application, the maximum value is used, but in other situations it may be meaningful to use a reference value such as the mean or minimum value. If the value of an alternative for criterion $c_i$ is less than $v_i$, then an “X” is assigned to indicate that the alternative is below the reference value for $c_i$. As can be seen in Step 4, one does this for every criterion over all of the alternatives. In Step 5, the total number of times that an alternative fails to meet the reference value across all of the criteria, is used to rank the alternatives, where ties are allowed.

The simplified elimination method utilized in the paper possesses a number of distinct advantages. For instance, it can easily handle a large number of criteria and alternatives, for which the criteria may be quantitative or qualitative in nature. Moreover, the criteria can be compensatory, whereby a value change in one criterion can affect others, and non-compensatory criteria. The criteria may be commensurable, whereby the criteria may or may not have the same units, respectively. As can be seen from Figure 1, the evaluation process is transparent and easy to follow, as well as to apply in practice, as demonstrated by the real-world case study in Section 2.2, Section 3 and Section 4. The methodology in Figure 1 could be expanded to handle weights for the criteria, by ordering the criteria from most to least important [32]. However, when there are a large number of criteria, ordering of the criteria could be time-consuming. In addition, uncertainty could be taken into consideration by the use of probability, fuzzy set (28), or grey numbers (23). However, a disadvantage of entertaining uncertainty is that the model becomes more complicated.

2.2. Data for the Case Study

To test the Elimination method for assessing agricultural sustainability, data (Appendix A: Table A1, Table A2, Table A3, Table A4, Table A5 and Table A6) were collected from Talukder [35]. These data are associated with the sustainability of five different agricultural systems: Bagda (shrimp)-based agricultural systems (S) from Shyamnagar Upazila (Upazila is the second lowest tier of local government in Bangladesh [36]), Bagda-rice-based agricultural systems (SR) from Kalijang Upazila, rice-based agricultural systems (R) from Kalaroa Upazila, Galda-rice-vegetable-based integrated agricultural systems (I) from Dumuria Upazila, and traditional practice-based agricultural systems (T) from Bhola Sadar Upazila. These Upazilas are located in the southwest coastal zone of Bangladesh (Figure 2). Data were collected from the literature, field observations, questionnaire surveys, and key informant interviews of knowledgeable farmers, agricultural extension officers, fishery officers, livestock officers, and block supervisors. Representatives from a total of 221 households, representing five categories of farmers: (landless (<0.01 acres), marginal (0.01 ≤ 0.50 acres), small (0.50 ≤ 2.5 acres), medium (2.5 ≤ 5.0 acres) and large (>5.0 acres) [35]) were considered during data collection [35]. The data for sustainability criteria were grouped into six categories of sustainability: productivity, stability, efficiency, durability, compatibility, and equity. In brief: productivity is related to the yields of agricultural systems; stability refers to the ability to maintain a good level of productivity over an extended period of time; efficiency is the measure of the extent to which the inputs for agricultural production enhance the crop yield (expressed in energy); durability is the ability of the agricultural system to resist or recover from stress and therefore, maintain a good level of productivity over a cropping cycle; compatibility refers to the ability of an agricultural system to fit in with the bio-geophysical, human, and socio-cultural surroundings in which the system is placed; and equity promotes a good quality of life for farmers and their family members [37].

Figure 2. Location of the study areas in Bangladesh [35].

3. Results of Elimination Method

Ranking the sustainability of agricultural systems depends on all of the scores of the criteria, from all categories. Scores of the criteria vary across the agricultural systems. For example, in the productivity category, ‘I’ (Integrated agriculture system) has the highest yield and net income (Appendix A: Table A1). A comparison of results and an in-depth knowledge of on-the-ground production and community considerations, are instructive and help to interpret results. For example, the overall productivity is higher in ‘I’ due to the year-round production of many crops, including three rice harvests a year, as well as the simultaneous production of crops such as jute, oilseed, and vegetables. Among environmental criteria, the energy output and input ratio, crop richness, and biodiversity condition, are very good in ‘I’, when compared to other systems. Due to fewer crops, the energy output to input ratio and crop richness are smaller in ‘SR’ and ‘S’. The condition of biodiversity is poor in ‘S’ because shrimp farming causes biodiversity degradation [38]. Since it is near the tidal zone, the study area ‘S’ is more exposed to salt water. However, according to the local people, the soil salinity is low in ‘R’ and close to zero in ‘T’, due to the significant input of rainwater and freshwater from the upstream rivers. Among responding farmers, those in ‘I’ have a higher level of education than their counterparts in ‘S’, ‘SR’, ‘R’, and ‘T’.

Table 2 shows the reference values and scores of the criteria of the agricultural systems. Here, all of the criteria are considered important for agricultural sustainability. The results of the case study are presented in Table 3 and Figure 3, and are self-explanatory.

Table 2. Scoring of criteria and rules of reference values.

Category	Sl. No.	Criteria	Reference Values	Agricultural Systems
				S	SR	R	I	T
Productivity	1	Weighted yield of the main staple crop	6.51	2.26	4.41	5.23	6.51	2.86
	2	Net income from the agro-ecosystem	1806.04	311.15	1020.37	1585.81	1806.04	544.01
	3	Protein yield from the agro-ecosystem	552	68.42	147.23	552	373.01	318.87
Stability	4	Land exposure to natural events: cyclone	2	1	2	2	2	1
	5	Land exposure to natural events: saline water	3	1	1	3	2	3
	6	Land exposure to natural events: drought in kharif to rabi season	3.5	1.5	1.5	2	2	3.5
	7	Land exposure to natural events: river bank erosion	2	2	2	2	2	1
	8	Stability of embankment	2	1	2	1	2	2
	9	Withdrawal of upstream water	2	1	1	1	1	2
	10	Organic materials	4	4	4	2	3	2
	11	Salinity	6	1	5	6	3	6
	12	Macronutrient: N	2	2	2	2	1	2
	13	Macronutrient: P	3	3	2	3	3	3
	14	Macronutrient: K	6	6	4	3	2	4
	15	Soil pH	4	1	3	4	2	4
	16	Water salinity in surface water (quality of surface water for irrigation)	3	1	2	2	2	3
	17	Water salinity in ground water (quality of ground water for irrigation)	4	1	2	2	4	3
	18	Arsenic concentration (quality of ground water for irrigation)	4	2	2	2	2	4
Efficiency	19	Money input and output in the agro-ecosystem	6.67	1.53	2.24	2.78	6.67	2.29
	20	Overall energy efficiency	5.9	1.37	2.01	5.53	5.54	5.9
	21	Non-renewable energy efficiency	2.52	0.78	0.92	2.17	2.52	2.44
Durability	22	Chemical response to pest stress	6.54	1.78	4.17	4.24	5.45	6.54
	23	Water availability at transplanting stage of rice	0.75	0.75	0.75	0.2	0.2	0.2
	24	Water availability at flowering stage of rice	0.75	0.75	0.75	0.2	0.2	0.2
	25	Farm management (soil test, pest management, land management, soil fertility management)	1.69	0.67	0.83	1.69	1.36	0
	26	Good product price	8.44	8.44	5	4.58	4.55	3.8
	27	Availability of seeds	10	9.33	9.5	10	10	8.85
	28	Availability of market (market diversification)	10	10	9.17	8.47	10	7.69
	29	Agricultural training	2.27	1.33	1.83	0.33	2.27	1.15
	30	Climate change awareness	1.82	1.11	0.67	0.51	1.82	0
	31	Advice from agricultural extension workers or NGO	1.17	0.66	1.17	0.51	0.45	0.38
Compatibility	32	Drinking water quality (protected)	10	0	8	9	10	9
	33	Illness from drinking water	10	5	10	10	10	10
	34	Overall biodiversity condition: percentage of non-crop area	23.01	7.54	6.48	23.01	15.73	18.68
	35	Overall biodiversity condition: crop richness	17	2	6	16	10	17
	36	Overall biodiversity condition: crop rotation	5	2	3	5	4	4
	37	Ecosystem connectivity	2	1	1	2	2	2
Equity	38	Education of farmers	10	8.56	9.25	4.75	10	5
	39	Education status of farmers’ male children	13.1	10	9.49	11.2	13.1	7.45
	40	Education status of farmers’ female children	12.5	9.07	10.54	11.17	12.5	6.36
	41	Access to electronic media	10	7.78	9.17	9.39	10	3.08
	42	Farm profitability	3340.55	648.23	3340.55	1371.32	1992.39	1025.06
	43	Average wage of farm labourer ($)	1.8	1.33	1.33	1.6	1.8	1.6
	44	Livelihood diversity other than agriculture	6.92	6.22	4.33	5.93	4.55	6.92
	45	Years of economic hardship	0.91	0.73	0.73	0.91	0.82	0.64
	46	Road network [establishing farm roads and access roads]	3	2	3	3	3	1
	47	Availability of medical treatment or public health	8.14	3.51	4.76	4.07	8.14	4.29
	48	Sanitation or public health	8.73	7.69	8.73	7.59	7.41	7.08
	49	Women’s involvement in decision making about agricultural activities	6.5	3	4	5	6.5	2.5
	50	Gender-based wage differentials	0.59	0.33	0.33	0.5	0.59	0

Table 3. Evaluation results after applying rules of Elimination Method.

Category	Sl. No.	Criteria	Reference Values	Agricultural Systems
				S	SR	R	I	T
Productivity	1	Weighted yield of the main staple crop	6.51	X	X	X		X
	2	Net income from the agro-ecosystem	1806.04	X	X	X		X
	3	Protein yield from the agro-ecosystem	552	X	X		X	X
Stability	4	Land exposure to natural events: cyclone	2	X				X
	5	Land exposure to natural events: saline water	3	X	X		X
	6	Land exposure to natural events: drought in kharif to rabi season	3.5	X	X	X	X
	7	Land exposure to natural events: river bank erosion	2					X
	8	Stability of embankment	2	X		X
	9	Withdrawal of upstream water	2	X	X	X	X
	10	Organic materials	4			X	X	X
	11	Salinity	6	X	X		X
	12	Macronutrient: N	2				X
	13	Macronutrient: P	3		X
	14	Macronutrient: K	6		X	X	X	X
	15	Soil pH	4	X	X		X
	16	Water salinity in surface water (quality of surface water for irrigation)	3	X	X	X	X
	17	Water salinity in ground water (quality of ground water for irrigation)	4	X	X	X		X
	18	Arsenic concentration (quality of ground water for irrigation)	4	X	X	X	X
Efficiency	19	Money input and output in the agro-ecosystem	6.67	X	X	X		X
	20	Overall energy efficiency	5.9	X	X	X	X
	21	Non-renewable energy efficiency	2.52	X	X	X		X
Durability	22	Chemical response to pest stress	6.54	X	X	X	X
	23	Water availability at transplanting stage of rice	0.75			X	X	X
	24	Water availability at flowering stage of rice	0.75			X	X	X
	25	Farm management (soil test, pest management, land management, soil fertility management)	1.69	X	X		X	X
	26	Good product price	8.44		X	X	X	X
	27	Availability of seeds	10	X	X			X
	28	Availability of market (market diversification)	10		X	X		X
	29	Agricultural training	2.27	X	X	X		X
	30	Climate change awareness	1.82	X	X	X		X
	31	Advice from agricultural extension workers or NGO	1.17	X		X	X	X
Compatibility	32	Drinking water quality (protected)	10	X	X	X		X
	33	Illness from drinking water	10	X
	34	Overall biodiversity condition: percentage of non-crop area	23.01	X	X		X	x
	35	Overall biodiversity condition: crop richness	17	X	X	X	X
	36	Overall biodiversity condition: crop rotation	5	X	X		X	X
	37	Ecosystem connectivity	2	X	X
Equity	38	Education of farmers	10	X	X	X		X
	39	Education status of farmers’ male children	13.1	X	X	X		X
	40	Education status of farmers’ female children	12.5	X	X	X		X
	41	Access to electronic media	10	X	X	X		X
	42	Farm profitability	3340.55	X		X	X	X
	43	Average wage of farm labourer ($)	1.8	X	X	X		X
	44	Livelihood diversity other than agriculture	6.92	X	X	X	X
	45	Years of economic hardship	0.91	X	X		X	X
	46	Road network (establishing farm roads and access roads)	3	X				X
	47	Availability of medical treatment or public health	8.14	X	X	X		X
	48	Sanitation or public health	8.73	X		X	X	X
	49	Women’s involvement in decision making about agricultural activities	6.5	X	X	X		X
	50	Gender-based wage differentials	0.59	X	X	X		X

Note: Yellow, gray, blue, green and red colors represent degree of fulfilment of the reference values by the criteria in each category of ‘S’, ‘SR’, ‘R’, ‘I’, and ‘T’, respectively. X = non-fulfilment of the reference values.

Figure 3. Ranking of the agricultural systems.

The relative reference values are considered here, since it is very difficult to identify the normative reference values in the context of the coastal agriculture of Bangladesh, because there are not enough secondary data related to sustainability of the agricultural systems. This is appropriate as the determination of normative reference values is time-consuming and sometimes pointless, since agricultural sustainability is a very relative concept, that varies over time and space [19]. Table 3 presents the evaluation results after applying the rules of the Elimination Method, as described in the methodology section.

Figure 3 displays the final results, that is, the ranking of the sustainability of agricultural systems. According to the ranking of the sustainability of agricultural systems, ‘I’ is the most preferred sustainable system, in comparison to the other four systems. ‘I’ fails on 25 of the 50 criteria, meaning that, for ‘I’, the remaining 25 criteria are equivalent to the reference values. The farmers of ‘I’ also expressed their satisfaction with most of the sustainability issues, like productivity, biodiversity, social health, and economics. This finding also echoes the finding of Rahman and Barmon [39], that ‘I’-type agricultural systems are more sustainable compared to others. Among agricultural systems, ‘S’ failed in most of the reference criteria and ranked as the least preferred system. Hossain et al. [38] also expressed that shrimp-based agricultural systems are less sustainable, due to the socio-ecological effects of shrimp cultivation.

While this type of assessment is based on very simple conditional statements and is easy to calculate, it depends entirely on the calculation of the criteria’s values. Therefore, the selection of criteria and the calculation of criteria values, requires a high degree of transparency, to ensure that this type of calculation is as clear and robust as possible. While agricultural sustainability in this assessment is divided into six categories, it does not reflect the actual performance of the individual categories in the overall ranking. It is important to note that the overall rank is heavily influenced by the number of criteria in each category as the criteria are added up, and thus have a significant impact on the final outcome. For example, ‘S’ as a whole, ranked the lowest, but if we examine the performance of each category, durability is tied between ‘S’ and ‘I’ (Table 3). If we explain this result by category, we see that “I” is highlighted as the “most sustainable” agricultural system for each category: ‘I’ for productivity, efficiency, durability (tied with ‘S’), and equity, ‘T’ for stability, and ‘R’ for compatibility. Therefore, while final rankings based on all of the criteria are important for this study, it is also useful to check the individual performance of each category. This will allow a more refined consideration of the performance of different categories, and also help to suggest ways to improve the categories of agricultural systems for agricultural sustainability.

4. Discussion

There are several considerations for applying an MCDA method as an agricultural sustainability assessment tool. In general, MCDA is appropriate because it can consider many criteria, thus allowing for the complexity needed for sustainability analysis. However, when using the MCDA framework, assigning the weighting of the criteria is very subjective. To avoid this subjectivity, using reference values based on the Elimination Method is a useful approach for sustainability assessment. By using criteria scores and relative reference values, the Elimination Method offers the ability to rank the sustainability of agricultural systems [32]. The advantage of this method is that, using the highest score in each category, readily allows for the identification of the criteria that fulfill the reference values. This makes it a flexible, transparent, time-saving, and holistic process, that can handle the imprecision and subjectivity of the information associated with sustainability criteria. If the sustainability criteria fall in a regular pattern, such as higher positive values of the criteria indicating higher sustainability, it can handle large data with ease. However, having to eliminate many criteria and not consider all the criteria’s values, will lessen the actual effect of the total criteria in the overall ranking [40].

The results of the Elimination analysis reveal that shrimp-based agricultural systems perform poorly in comparison to integrated and rice-based agricultural systems. There is a significant difference in how these systems fulfill the criteria of sustainability. It should be noted that farmers consider shrimp-based agricultural systems to be profitable, but there are adverse ecological consequences, and the production of shrimp has dropped over successive years. Rice yields are very low in S and SR, which is jeopardizing the food supply. Biodiversity is also low in these systems, which suggests a trend of agricultural unsustainability. Therefore, some of the farmers interviewed by Talukder [35] reported that they are considering changing to integrated agricultural systems.

This suggested modified Elimination Method allows the user to set a threshold value in a category, as a bar below which all data are eliminated. This leaves the top value for that category. Once all top values for each category have been determined, these category values can be summed, and the results can be ranked. This case study demonstrates that Elimination is able to determine sustainability rankings for the different systems. This finding may motivate other researchers to collect more reliable criteria with which to apply the Elimination Method for sustainability assessment. The ranking of agricultural sustainability raises various questions about the sustainability performance of the agricultural systems. The Elimination Method can be offered as an option for holistically assessing agriculture [41], as it can consider criteria from all three pillars of sustainability.

5. Conclusions

Applying MCDA to agricultural sustainability assessment is complex, as many criteria need to be considered. In any MCDA-based assessment (Like MAUT, PROMETHEE), the weighting of criteria is very subjective. To avoid this, eliminating criteria based on objective reference values, defined in terms of a case study, is a useful alternative.

To the best of our knowledge, this is the first time an attempt has been made to use Elimination Method to evaluate and compare the agricultural sustainability of different systems. In this study, the process of the Elimination Method is described, and the scores of criteria and relative reference values are determined. Furthermore, the methodological process of Elimination is tested through a case study, in order to identify its advantages and pitfalls. This paper is not an “instruction manual” for using the Elimination Method for agricultural sustainability, but the framework presented here can help to simplify sustainability assessment. However, appropriate and transparent measures are needed for selecting the criteria, and their scoring and reference values. The MCDA Elimination Method is used to rank the sustainability performance of agriculture by considering economic, environmental, and social criteria. This framework allows an integrated assessment as it handles data related to the three pillars of sustainability. The Elimination approach can be an option for sustainability assessment, but there is still a lot of scope to investigate, including the applicability of other techniques of the MCDA approach, in order to identify suitable or preferred MCDA techniques for assessing agricultural sustainability.

One drawback to this is that successive elimination can cause the method to lose fundamental properties of the original criteria, as part of the overall final ranking [41]. The research and Elimination analysis reported in this thesis offer insights for future researchers as they define their categories, and collect data to test the Elimination Method in the context of agricultural and other types of sustainability assessment. Like MAUT and PROMETHEE, Elimination can also facilitate learning, debate, and consensus building among the stakeholders, for agricultural sustainability. Adopting Elimination for agricultural sustainability assessment can be a positive step in understanding and comparing multiple dimensions of sustainability.